jPDddlZddlmZmZmZddlmZedZd7dej defdZ d8d ej d ej d eej d e de dedej fdZ d9dej d ej d eej dededej f dZ d:dej d ej dedej fdZ d:dej d ej dedej fdZ d;dej d ej d e dedej f dZ dd%ej d&ej d'ej d e d(e d)e de dedej fd*Z d7dej d ej dedej fd+Z d?dej d ej d,e dedej f d-Z d7dej d ej dedej fd.Z d@d1ej d2ej d e de dedej f d3Z dAd4ej d5ej d ej d)e dedej f d6ZdS)BN)LiteralOptionalget_args)nonemeansumrloss reductionc|ttvr%tdttd|dkrtj|S|dkrtj|S|dkr|SdS)Nz"Invalid reduction. Must be one of .rrr)r Reduction ValueErrormxrr)r r s W/lsinfo/ai/hellotax_ai/base_platform/venv/lib/python3.11/site-packages/mlx/nn/losses.py_reducer s++++Thy>Q>QTTTUUUFwt}} e  vd|| f     logitstargetsweightsaxislabel_smoothingreturnc|dks|dkrtd|d|j|jk}d}|r|j|jks|s:|j||j|kr td|jd|jd|rtj||z|}n>> import mlx.core as mx >>> import mlx.nn as nn >>> >>> # Class indices as targets >>> logits = mx.array([[2.0, -1.0], [-1.0, 2.0]]) >>> targets = mx.array([0, 1]) >>> nn.losses.cross_entropy(logits, targets) array([0.0485873, 0.0485873], dtype=float32) >>> >>> # Probabilities (or one-hot vectors) as targets >>> logits = mx.array([[2.0, -1.0], [-1.0, 2.0]]) >>> targets = mx.array([[0.9, 0.1], [0.1, 0.9]]) >>> nn.losses.cross_entropy(logits, targets) array([0.348587, 0.348587], dtype=float32) rz$Label smoothing must in [0, 1), got r cht|}||t|S)N)listpoptuple)shapers r _drop_dimz cross_entropy.._drop_dimIs(U  $U||rzTargets shape z does not match logits shape rNWeights with shape + is not the same as output loss with shape ) rndimr!rrtake_along_axis expand_dimssqueeze logsumexprr)rrrrrr targets_as_probsr"scorelogsumexp_logitsadjusted_score mean_logits smoothed_lossr s r cross_entropyr1sXo22RRRRSSS|v{2  W]fl:: ;!())FL$2O2O!O!O XW] X X X X X    v'd333"62>'4+H+H$OOWW   |F666o-6kktk,, $ 6  .0=@%' =DJ & &8gm88*.*888   4 # ##rTrinputs with_logitsc|j|jkr td|jd|jd|rtjd|||zz }ndtjtj|dd}tjtjd|z dd}||zd|z |zz }|5|j|jkr td |jd |jd||z}t ||S) a Computes the binary cross entropy loss. By default, this function takes the pre-sigmoid logits, which results in a faster and more precise loss. For improved numerical stability when ``with_logits=False``, the loss calculation clips the input probabilities (in log-space) to a minimum value of ``-100``. Args: inputs (array): The predicted values. If ``with_logits`` is ``True``, then ``inputs`` are unnormalized logits. Otherwise, ``inputs`` are probabilities. targets (array): The binary target values in {0, 1}. with_logits (bool, optional): Whether ``inputs`` are logits. Default: ``True``. weights (array, optional): Optional weights for each target. Default: ``None``. reduction (str, optional): Specifies the reduction to apply to the output: ``'none'`` | ``'mean'`` | ``'sum'``. Default: ``'mean'``. Returns: array: The computed binary cross entropy loss. Examples: >>> import mlx.core as mx >>> import mlx.nn as nn >>> logits = mx.array([0.105361, 0.223144, 1.20397, 0.916291]) >>> targets = mx.array([0, 0, 1, 1]) >>> loss = nn.losses.binary_cross_entropy(logits, targets, reduction="mean") >>> loss array(0.539245, dtype=float32) >>> probs = mx.array([0.1, 0.1, 0.4, 0.4]) >>> targets = mx.array([0, 0, 1, 1]) >>> loss = nn.losses.binary_cross_entropy(probs, targets, with_logits=False, reduction="mean") >>> loss array(0.510826, dtype=float32) Inputs shape  does not match targets shape r riN)a_mina_maxrr$r%)r!rr logaddexpcliplogr)r2rrr3r r log_inputs_cliplog_inputs_inv_clips rbinary_cross_entropyr>xs2T|w}$$ XFL X X X X X   R|C((6G+;;'"&..DIII gbfQZ&8&8DQQQ?*a'k=P-PPQ =DJ & &8gm88*.*888   4 # ##r predictionsc|j|jkr td|jd|jdtj||z }t ||S)aS Computes the L1 loss. Args: predictions (array): The predicted values. targets (array): The target values. reduction (str, optional): Specifies the reduction to apply to the output: ``'none'`` | ``'mean'`` | ``'sum'``. Default: ``'mean'``. Returns: array: The computed L1 loss. Predictions shape r6r )r!rrabsrr?rr r s rl1_lossrDstGM)) .!2 . .$] . . .    6+' ( (D 4 # ##rc|j|jkr td|jd|jdtj||z }t ||S)as Computes the mean squared error loss. Args: predictions (array): The predicted values. targets (array): The target values. reduction (str, optional): Specifies the reduction to apply to the output: ``'none'`` | ``'mean'`` | ``'sum'``. Default: ``'mean'``. Returns: array: The computed mean squared error loss. rAr6r )r!rrsquarerrCs rmse_lossrGstGM)) .!2 . .$] . . .   9[7* + +D 4 # ##rctj||d|d }t||S)a Computes the negative log likelihood loss. Args: inputs (array): The predicted distribution in log space. targets (array): The target values. axis (int, optional): The distribution axis. Default: ``-1``. reduction (str, optional): Specifies the reduction to apply to the output: ``'none'`` | ``'mean'`` | ``'sum'``. Default: ``'none'``. Returns: array: The computed NLL loss. ).Nr)rr'r)rr2rrr r s rnll_lossrJs>  vwy'94 @ @ H H L L LD 4 # ##rFư>varsfullepsc|j|jkr td|jd|jd|j|jkr td|jd|jdtj||}dtj|tj||z |z zz}|r'|dt jdt jzzz }t||S)aj Computes the negative log likelihood loss for a Gaussian distribution. The loss is given by: .. math:: \frac{1}{2}\left(\log\left(\max\left(\text{vars}, \ \epsilon\right)\right) + \frac{\left(\text{inputs} - \text{targets} \right)^2} {\max\left(\text{vars}, \ \epsilon \right)}\right) + \text{const.} where ``inputs`` are the predicted means and ``vars`` are the the predicted variances. Args: inputs (array): The predicted expectation of the Gaussian distribution. targets (array): The target values (samples from the Gaussian distribution). vars (array): The predicted variance of the Gaussian distribution. full (bool, optional): Whether to include the constant term in the loss calculation. Default: ``False``. eps (float, optional): Small positive constant for numerical stability. Default: ``1e-6``. reduction (str, optional): Specifies the reduction to apply to the output: ``'none'`` | ``'mean'`` | ``'sum'``. Default: ``'none'``. Returns: array: The Gaussian NLL loss. r5r6r z does not match vars shape ?) r!rrmaximumr;rFmathpir)r2rrLrMrNr r s rgaussian_nll_lossrUsF|w}$$ XFL X X X X X   |tz!! RFL R RTZ R R R   :dC D "&,,7V+;!4g F < ( (48A;; 6D 4 # ##rr:0yE>x1x2c tj||}tj||}tj||z|tj||z|z }t ||S)a Computes the cosine similarity between the two inputs. The cosine similarity loss is given by .. math:: \frac{x_1 \cdot x_2}{\max(\|x_1\| \cdot \|x_2\|, \epsilon)} Args: x1 (mx.array): The first set of inputs. x2 (mx.array): The second set of inputs. axis (int, optional): The embedding axis. Default: ``1``. eps (float, optional): The minimum value of the denominator used for numerical stability. Default: ``1e-8``. reduction (str, optional): Specifies the reduction to apply to the output: ``'none'`` | ``'mean'`` | ``'sum'``. Default: ``'none'``. Returns: mx.array: The computed cosine similarity loss. r#)rlinalgnormrrRr)rurvrrNr x1_normx2_normr s rcosine_similarity_lossr| sp8innRdn++GinnRdn++G 6"r' % % % 7W3Dc(J(J JD 4 # ##rinputs1inputs2c |j|jcxkr |jks*ntd|jd|jd|jd||z }tjd| |z|z}t ||S)aW Calculate the margin ranking loss that loss given inputs :math:`x_1`, :math:`x_2` and a label :math:`y` (containing 1 or -1). The loss is given by: .. math:: \text{loss} = \max (0, -y * (x_1 - x_2) + \text{margin}) Where :math:`y` represents ``targets``, :math:`x_1` represents ``inputs1`` and :math:`x_2` represents ``inputs2``. Args: inputs1 (array): Scores for the first input. inputs2 (array): Scores for the second input. targets (array): Labels indicating whether samples in ``inputs1`` should be ranked higher than samples in ``inputs2``. Values should be 1 or -1. margin (float, optional): The margin by which the scores should be separated. Default: ``0.0``. reduction (str, optional): Specifies the reduction to apply to the output: ``'none'`` | ``'mean'`` | ``'sum'``. Default: ``'none'``. Returns: array: The computed margin ranking loss. Examples: >>> import mlx.core as mx >>> import mlx.nn as nn >>> targets = mx.array([1, 1, -1]) >>> inputs1 = mx.array([-0.573409, -0.765166, -0.0638]) >>> inputs2 = mx.array([0.75596, 0.225763, 0.256995]) >>> loss = nn.losses.margin_ranking_loss(inputs1, inputs2, targets) >>> loss array(0.773433, dtype=float32) zPThe shapes of the arguments do not match. The provided shapes are inputs1.shape=z, inputs2.shape=z, and targets.shape=r r)r!rrrRr)r}r~rrcr differencesr s rmargin_ranking_lossr.sT MW] ; ; ; ;gm ; ; ; ; .$] . .rDrGrJrUrXr^rgrirqrsr|rrrrs .......... ) *   "( y    #' ! ^$^$ H^$ X^$bh ^$  ^$  ^$  ^$X^$^$^$^$H#'! ?$?$ H?$ X?$bh ?$ ?$  ?$ X ?$?$?$?$FFL$$$$&H$9B$X$$$$4FL$$$$&H$9B$X$$$$4QW$$ H$!x$/2$DM$X$$$$2! 4$4$ H4$ X4$ (4$  4$  4$  4$X4$4$4$4$pQW$$ H$!x$/2$DM$X$$$$<! ,$,$,$ X,$ ,$ ,$ X ,$,$,$,$f !'$'$ X'$x'$x'$  '$  '$  '$ '$'$X'$'$'$'$VAG$$ H$!x$4=$X$$$$8! "$"$ H"$ X"$ "$ "$ X "$"$"$"$LAG$$ H$!x$4=$X$$$$F! !$!$ !$ !$ !$  !$  !$ X !$!$!$!$P! 4$4$ X4$ X4$X4$  4$  4$ X 4$4$4$4$4$4$r