jtZddlZddlmZddlmZddlmZddlm Z GddeZ dS)N)Optional)Module)QuantizedEmbeddingc xeZdZdZdedeffd ZdZdZdZ dd e ed e ed e de fdZ xZ S) EmbeddingavImplements a simple lookup table that maps each input integer to a high-dimensional vector. Typically used to embed discrete tokens for processing by neural networks. Args: num_embeddings (int): How many possible discrete tokens can we embed. Usually called the vocabulary size. dims (int): The dimensionality of the embeddings. num_embeddingsdimscttjd|z }tj||f||_dS)N)shapescale)super__init__mathsqrtmxrandomnormalweight)selfrr r __class__s a/lsinfo/ai/hellotax_ai/base_platform/venv/lib/python3.11/site-packages/mlx/nn/layers/embedding.pyrzEmbedding.__init__sQ  !d(##i&&nd-C5&QQ cP|jjdd|jjdS)Nrz, r )rr )rs r _extra_reprzEmbedding._extra_reprs*+#A&@@$+*;A*>@@@rc|j|S)N)rrxs r__call__zEmbedding.__call__s{1~rc ||jjzS)z Call the embedding layer as a linear layer. Use this for example when input embedding and output projection weights are tied. )rTrs r as_linearzEmbedding.as_linear"s4;=  rNaffineF group_sizebitsmodequantize_inputcR|rtdtj||||S)zPReturn a :obj:`QuantizedEmbedding` layer that approximates this embedding layer.z!Quantized input is not supported.) ValueErrorrfrom_embedding)rr$r%r&r's r to_quantizedzEmbedding.to_quantized+s3  B@AA A!0z4NNNr)NNr#F)__name__ __module__ __qualname____doc__intrrrr"rstrboolr+ __classcell__)rs@rrr s  RsR#RRRRRR AAA!!!%)"$ O OSM Osm O O  O O O O O O O Orr) rtypingrmlx.corecorermlx.nn.layers.basermlx.nn.layers.quantizedrrrrr:s %%%%%%666666*O*O*O*O*O*O*O*O*O*Or