jddlZddlmZmZddlmZddlmZddl m Z m Z GddeZ GddeZ Gd d eZdS) N)AnyOptional)Module)QQLinearQuantizedLinearcTeZdZdZdededdffd ZdejdejfdZxZ S) IdentityzA placeholder identity operator that is argument-insensitive. Args: args: any argument (unused) kwargs: any keyword argument (unused) argskwargsreturnNcHtdSN)super__init__)selfr r __class__s ^/lsinfo/ai/hellotax_ai/base_platform/venv/lib/python3.11/site-packages/mlx/nn/layers/linear.pyrzIdentity.__init__s xc|Srrrs r__call__zIdentity.__call__sr) __name__ __module__ __qualname____doc__rrmxarrayr __classcell__rs@rr r scST"(rxrr c eZdZdZddedededdffd Zdefd Zd e j de j fd Z dde ede ededefdZ xZS)LinearaApplies an affine transformation to the input. Concretely: .. math:: y = x W^\top + b where: where :math:`W` has shape ``[output_dims, input_dims]`` and :math:`b` has shape ``[output_dims]``. The values are initialized from the uniform distribution :math:`\mathcal{U}(-{k}, {k})`, where :math:`k = \frac{1}{\sqrt{D_i}}` and :math:`D_i` is equal to ``input_dims``. Args: input_dims (int): The dimensionality of the input features output_dims (int): The dimensionality of the output features bias (bool, optional): If set to ``False`` then the layer will not use a bias. Default is ``True``. T input_dims output_dimsbiasr Nc$ttjd|z }tj| |||f|_|r+tj| ||f|_dSdSNg?)lowhighshape rrmathsqrtrrandomuniformweightr&)rr$r%r&scalers rrzLinear.__init__0s  # *++i'' +(     ))F"n*DIII  rc\d|jjdd|jjddd|vS)Nz input_dims=, output_dims=r, bias=r&r1r+)rs r _extra_reprzLinear._extra_repr?s<nT[.q1nnARSTAUnn^dhl^lnnnrrc~d|vr(tj|d||dj}n||djz}|S)Nr&r1)raddmmTrs rrzLinear.__call__BsA T>>fq$x.*:;;AADN$$AraffineF group_sizebitsmodequantize_inputc|r.|dvrtd|dtj||||Stj||||S)aReturn a quantized approximation of this layer. If ``quantize_input`` is ``False``, returns a :obj:`QuantizedLinear` (weights are quantized). If ``quantize_input`` is ``True``, returns a :obj:`QQLinear` (weights and activations are quantized). Args: group_size (Optional[int]): The quantization group size (see :func:`mlx.core.quantize`). Default: ``None``. bits (Optional[int]): The number of bits per parameter (see :func:`mlx.core.quantize`). Default: ``None``. mode (str): The quantization method to use (see :func:`mlx.core.quantize`). Default: ``"affine"``. quantize_input (bool): Whether to quantize input. Default: ``False``. Returns: QuantizedLinear or QQLinear: A quantized version of this layer. Notes: Quantized input is only supported for ``"nvfp4"`` and ``"mxfp8"`` modes. )nvfp4mxfp8zLQuantized activations are only supported for 'nvfp4' and 'mxfp8' modes, got .) ValueErrorr from_linearr)rr=r>r?r@s r to_quantizedzLinear.to_quantizedIsj:  F--- jcgjjj'j$EE E*4T4HHHrT)NNr<F)rrrrintboolrstrr8rrrrrGr r!s@rr#r#s*  3 S  PT      oSoooo"(rx%)"$ #I#ISM#Ism#I #I  #I#I#I#I#I#I#I#Irr#c zeZdZdZ ddededededdf fd Zdefd Zd e j d e j de j fd Z xZ S)Bilineara!Applies a bilinear transformation to the inputs. Concretely: .. math:: y_i = x_1^\top W_i x_2 + b_i where: :math:`W` has shape ``[output_dims, input1_dims, input2_dims]``, :math:`b` has shape ``[output_dims ]``, and :math:`i` indexes the output dimension. The values are initialized from the uniform distribution :math:`\mathcal{U}(-{k}, {k})`, where :math:`k = \frac{1}{\sqrt{D_1}}` and :math:`D_1` is ``input1_dims``. Args: input1_dims (int): The dimensionality of the input1 features input2_dims (int): The dimensionality of the input2 features output_dims (int): The dimensionality of the output features bias (bool, optional): If set to ``False`` then the layer will not use a bias. Default is ``True``. T input1_dims input2_dimsr%r&r Nc&ttjd|z }tj| ||||f|_|r+tj| ||f|_dSdSr(r,)rrNrOr%r&r2rs rrzBilinear.__init__s  # +,,i'' [9(     ))F"n*DIII  rc B|jj\}}}d|d|d|dd|vS)Nz input1_dims=z, input2_dims=r5r6r&r7)routin2in1s rr8zBilinear._extra_reprsQ ) S# %3 % %c % % % %dN % % rx1x2c|jj\}}}|jdd}|d|}|dd|}|j||z|}||jz}|d||dd}||z}|d}|jg||R}d|vr ||jz}|S)Nr4r&)r1r+reshaper;swapaxessqueezer&) rrUrVrRrSrTxshapewys rrzBilinear.__call__s ) S##2# ZZC  ZZAs # # K  c 3 / / H IIb#s # # , ,R 4 4 F IIaLL AI #v #s # # # T>>DI ArrH) rrrrrIrJrrKr8rrrr r!s@rrMrMos0RV-0?BJN " S    28bhrrM)r-typingrrmlx.corecorermlx.nn.layers.basermlx.nn.layers.quantizedrrr r#rMrrrres %%%%%%========     v   RIRIRIRIRIVRIRIRIjEEEEEvEEEEEr