j"ddlZddlmZmZddlmZddlmZddl m Z Gdde Z Gdde Z Gd d e Z dS) N)CallableOptional)tanh)Modulec TeZdZdZ d dedededeeffd Zd Z d d Z xZ S) RNNaAn Elman recurrent layer. The input is a sequence of shape ``NLD`` or ``LD`` where: * ``N`` is the optional batch dimension * ``L`` is the sequence length * ``D`` is the input's feature dimension Concretely, for each element along the sequence length axis, this layer applies the function: .. math:: h_{t + 1} = \text{tanh} (W_{ih}x_t + W_{hh}h_t + b) The hidden state :math:`h` has shape ``NH`` or ``H``, depending on whether the input is batched or not. Returns the hidden state at each time step, of shape ``NLH`` or ``LH``. Args: input_size (int): Dimension of the input, ``D``. hidden_size (int): Dimension of the hidden state, ``H``. bias (bool, optional): Whether to use a bias. Default: ``True``. nonlinearity (callable, optional): Non-linearity to use. If ``None``, then func:`tanh` is used. Default: ``None``. TN input_size hidden_sizebias nonlinearityct|pt|_t |jst d|ddt j|z }||_tj | |||f|_ tj | |||f|_ |r$tj | ||fnd|_dS)Nz.Nonlinearity must be callable. Current value: .?lowhighshape)super__init__rr callable ValueErrormathsqrtr mxrandomuniformWxhWhhr )selfr r r r scale __class__s a/lsinfo/ai/hellotax_ai/base_platform/venv/lib/python3.11/site-packages/mlx/nn/layers/recurrent.pyrz RNN.__init__'s  (0D)** PPPP di ,,,&9$$U; *C%  9$$U; *D%    BI  5&u[N  K K K c `d|jjdd|jd|jd|jduS)N input_dims=, hidden_size=z, nonlinearity=, bias=)rrr r r rs r" _extra_reprzRNN._extra_reprDs^ N$(.+ N N+ N N - N N6:it6K N N r#c|j&tj|j||jj}n||jjz}g}t |jdD]g}|,tj|d|ddf||jj}n |d|ddf}||}| |htj |dS)N.axis) r raddmmrTrangerrr appendstack)rxhidden all_hiddenidxs r"__call__z RNN.__call__Ks 9 Atxz22AADHJA %% & &C!!CaaaK.&$(*EE3QQQ;&&v..F   f % % % %x ,,,,r#)TNN) __name__ __module__ __qualname____doc__intboolrrrr*r8 __classcell__r!s@r"rr s>+/       x(       :   --------r#rcBeZdZdZ d dededeffd ZdZd d ZxZ S) GRUaA gated recurrent unit (GRU) RNN layer. The input has shape ``NLD`` or ``LD`` where: * ``N`` is the optional batch dimension * ``L`` is the sequence length * ``D`` is the input's feature dimension Concretely, for each element of the sequence, this layer computes: .. math:: \begin{aligned} r_t &= \sigma (W_{xr}x_t + W_{hr}h_t + b_{r}) \\ z_t &= \sigma (W_{xz}x_t + W_{hz}h_t + b_{z}) \\ n_t &= \text{tanh}(W_{xn}x_t + b_{n} + r_t \odot (W_{hn}h_t + b_{hn})) \\ h_{t + 1} &= (1 - z_t) \odot n_t + z_t \odot h_t \end{aligned} The hidden state :math:`h` has shape ``NH`` or ``H`` depending on whether the input is batched or not. Returns the hidden state at each time step of shape ``NLH`` or ``LH``. Args: input_size (int): Dimension of the input, ``D``. hidden_size (int): Dimension of the hidden state, ``H``. bias (bool): Whether to use biases or not. Default: ``True``. Tr r r ct||_dtj|z }t j| |d|z|f|_t j| |d|z|f|_ |r't j| |d|zfnd|_ |r$t j| ||fnd|_ dS)Nrr) rrr rrrrrWxWhbbhnrr r r r r!s r"rz GRU.__init__{s  &di ,,,)##U1{?J*G$  )##U1{?K*H$    BI  5&uQ_r?rr*r8r@rAs@r"rCrC]sB            6   (-(-(-(-(-(-(-(-r#rCcBeZdZdZ d dededeffd ZdZd d ZxZ S) LSTMauAn LSTM recurrent layer. The input has shape ``NLD`` or ``LD`` where: * ``N`` is the optional batch dimension * ``L`` is the sequence length * ``D`` is the input's feature dimension Concretely, for each element of the sequence, this layer computes: .. math:: \begin{aligned} i_t &= \sigma (W_{xi}x_t + W_{hi}h_t + b_{i}) \\ f_t &= \sigma (W_{xf}x_t + W_{hf}h_t + b_{f}) \\ g_t &= \text{tanh} (W_{xg}x_t + W_{hg}h_t + b_{g}) \\ o_t &= \sigma (W_{xo}x_t + W_{ho}h_t + b_{o}) \\ c_{t + 1} &= f_t \odot c_t + i_t \odot g_t \\ h_{t + 1} &= o_t \text{tanh}(c_{t + 1}) \end{aligned} The hidden state :math:`h` and cell state :math:`c` have shape ``NH`` or ``H``, depending on whether the input is batched or not. The layer returns two arrays, the hidden state and the cell state at each time step, both of shape ``NLH`` or ``LH``. Args: input_size (int): Dimension of the input, ``D``. hidden_size (int): Dimension of the hidden state, ``H``. bias (bool): Whether to use biases or not. Default: ``True``. Tr r r ct||_dtj|z }t j| |d|z|f|_t j| |d|z|f|_ |r't j| |d|zfnd|_ dS)Nrr) rrr rrrrrrFrGr rJs r"rz LSTM.__init__s &di ,,,)##U1{?J*G$  )##U1{?K*H$    BI  5&uQ_D!xfdgi88$333JAq!Q 1 A 1 A A 1 A4x!a%'1u&F OOD ! ! !   f % % % %x ,,,bhxb.I.I.IIIr#r[)NNr\rAs@r"r^r^sH            ,   JJJJJJJJr#r^)rtypingrrmlx.corecorermlx.nn.layers.activationsrmlx.nn.layers.baserrrCr^r#r"rps  %%%%%%%%******%%%%%%O-O-O-O-O-&O-O-O-dg-g-g-g-g-&g-g-g-TZJZJZJZJZJ6ZJZJZJZJZJr#