|j-x^ddlmZddlZddlmZmZmZddlZddl Z ddl m Z m Z ddl m Z ddlmZmZddlmZdd lmZerdd lmZddlmZdd l mZgZd ZGd dejZGddeZGddeZGddeZ GddeZ! d%d&d$Z"dS)') annotationsN) TYPE_CHECKINGAnyLiteral)_C_ops _legacy_C_ops)check_variable_and_dtype)_create_tensor in_pir_mode) LayerHelper)in_dynamic_mode)Sequence)TensorcNt|tjtjfSN) isinstancenpndarraygeneric)vars e/lsinfo/ai/hellotax_ai/data_center/backend/venv/lib/python3.11/site-packages/paddle/metric/metrics.py _is_numpy_r(s cBJ 3 4 44ceZdZdZddZejddZejddZejdd Z ejdd Z dd Z d S)Metrica Base class for metric, encapsulates metric logic and APIs Usage: .. code-block:: text m = SomeMetric() for prediction, label in ...: m.update(prediction, label) m.accumulate() Advanced usage for :code:`compute`: Metric calculation can be accelerated by calculating metric states from model outputs and labels by built-in operators not by Python/NumPy in :code:`compute`, metric states will be fetched as NumPy array and call :code:`update` with states in NumPy format. Metric calculated as follows (operations in Model and Metric are indicated with curly brackets, while data nodes not): .. code-block:: text inputs & labels || ------------------ | || {model} || | || outputs & labels || | || tensor data {Metric.compute} || | || metric states(tensor) || | || {fetch as numpy} || ------------------ | || metric states(numpy) || numpy data | || {Metric.update} \/ ------------------ Examples: For :code:`Accuracy` metric, which takes :code:`pred` and :code:`label` as inputs, we can calculate the correct prediction matrix between :code:`pred` and :code:`label` in :code:`compute`. For examples, prediction results contains 10 classes, while :code:`pred` shape is [N, 10], :code:`label` shape is [N, 1], N is mini-batch size, and we only need to calculate accuracy of top-1 and top-5, we could calculate the correct prediction matrix of the top-5 scores of the prediction of each sample like follows, while the correct prediction matrix shape is [N, 5]. .. code-block:: python :name: code-compute-example >>> import paddle >>> def compute(pred, label): ... # sort prediction and slice the top-5 scores ... pred = paddle.argsort(pred, descending=True)[:, :5] ... # calculate whether the predictions are correct ... correct = pred == label ... return paddle.cast(correct, dtype='float32') ... With the :code:`compute`, we split some calculations to OPs (which may run on GPU devices, will be faster), and only fetch 1 tensor with shape as [N, 5] instead of 2 tensors with shapes as [N, 10] and [N, 1]. :code:`update` can be define as follows: .. code-block:: python :name: code-update-example >>> def update(self, correct): ... accs = [] ... for i, k in enumerate(self.topk): ... num_corrects = correct[:, :k].sum() ... num_samples = len(correct) ... accs.append(float(num_corrects) / num_samples) ... self.total[i] += num_corrects ... self.count[i] += num_samples ... return accs returnNonecdSrselfs r__init__zMetric.__init__s rc<td|jjd)) Reset states and result z$function 'reset' not implemented in .NotImplementedError __class____name__r!s rresetz Metric.resets) " M4>3J M M M   rargsrc<td|jjd)au Update states for metric Inputs of :code:`update` is the outputs of :code:`Metric.compute`, if :code:`compute` is not defined, the inputs of :code:`update` will be flatten arguments of **output** of mode and **label** from data: :code:`update(output1, output2, ..., label1, label2,...)` see :code:`Metric.compute` z%function 'update' not implemented in r&r'r"r,s rupdatez Metric.updates)" NDN4K N N N   rc<td|jjd)zO Accumulates statistics, computes and returns the metric value z)function 'accumulate' not implemented in r&r'r!s r accumulatezMetric.accumulates) " R8O R R R   rstrc<td|jjd)% Returns metric name z#function 'name' not implemented in r&r'r!s rnamez Metric.names) " L$.2I L L L   rc|S)aM This API is advanced usage to accelerate metric calculating, calculations from outputs of model to the states which should be updated by Metric can be defined here, where Paddle OPs is also supported. Outputs of this API will be the inputs of "Metric.update". If :code:`compute` is defined, it will be called with **outputs** of model and **labels** from data as arguments, all outputs and labels will be concatenated and flatten and each filed as a separate argument as follows: :code:`compute(output1, output2, ..., label1, label2,...)` If :code:`compute` is not defined, default behaviour is to pass input to output, so output format will be: :code:`return output1, output2, ..., label1, label2,...` see :code:`Metric.update` r r.s rcomputezMetric.computes & rNrr)r,rrr)rrrr2)r,rrr) r* __module__ __qualname____doc__r#abcabstractmethodr+r/r1r5r7r rrrr,sPPd                          rr) metaclasscpeZdZUdZded<ded< ddfd ZddZddZd dZd!dZ d"dZ d#dZ xZ S)$Accuracya Encapsulates accuracy metric logic. Args: topk (list[int]|tuple[int]): Number of top elements to look at for computing accuracy. Default is (1,). name (str|None, optional): String name of the metric instance. Default is `acc`. Examples: .. code-block:: python :name: code-standalone-example >>> import numpy as np >>> import paddle >>> x = paddle.to_tensor(np.array([ ... [0.1, 0.2, 0.3, 0.4], ... [0.1, 0.4, 0.3, 0.2], ... [0.1, 0.2, 0.4, 0.3], ... [0.1, 0.2, 0.3, 0.4]])) >>> y = paddle.to_tensor(np.array([[0], [1], [2], [3]])) >>> m = paddle.metric.Accuracy() >>> correct = m.compute(x, y) >>> m.update(correct) >>> res = m.accumulate() >>> print(res) 0.75 .. code-block:: python :name: code-model-api-example >>> # doctest: +TIMEOUT(80) >>> import paddle >>> from paddle.static import InputSpec >>> import paddle.vision.transforms as T >>> from paddle.vision.datasets import MNIST >>> input = InputSpec([None, 1, 28, 28], 'float32', 'image') >>> label = InputSpec([None, 1], 'int64', 'label') >>> transform = T.Compose([T.Transpose(), T.Normalize([127.5], [127.5])]) >>> train_dataset = MNIST(mode='train', transform=transform) >>> model = paddle.Model(paddle.vision.models.LeNet(), input, label) >>> optim = paddle.optimizer.Adam( ... learning_rate=0.001, parameters=model.parameters()) >>> model.prepare( ... optim, ... loss=paddle.nn.CrossEntropyLoss(), ... metrics=paddle.metric.Accuracy()) ... >>> model.fit(train_dataset, batch_size=64) Sequence[int]topkintmaxkNr5 str | Noner,rkwargsrrctj|i|||_t||_|||dSr)superr#rCmaxrE _init_namer+)r"rCr5r,rIr)s rr#zAccuracy.__init__sX $)&))) II   rpredrlabelctj|d}tj|t|jdz gdg|jg}t|jdks)t|jdkr'|jddkrtj|d}n(|jddkrtj|dd }|||j k}tj |d S) a Compute the top-k (maximum value in `topk`) indices. Args: pred (Tensor): The predicted value is a Tensor with dtype float32 or float64. Shape is [batch_size, d0, ..., dN]. label (Tensor): The ground truth value is Tensor with dtype int64. Shape is [batch_size, d0, ..., 1], or [batch_size, d0, ..., num_classes] in one hot representation. Return: Tensor: Correct mask, a tensor with shape [batch_size, d0, ..., topk]. T) descendingrGr)axesstartsendsr )rUrG)axiskeepdimfloat32dtype) paddleargsortslicelenshaperEreshapeargmaxastyperZcast)r"rNrOr,corrects rr7zAccuracy.compute s~dt444| DJ!+,aS {       ! !    ! !ek"o&:&: N5'22EE [_ ! !M%b$???E%,,tz222{7)4444rrdct|tjrtj|}tjtj|jdd}g}t|jD]r\}}|dd|f }| t||z |j |xx|z cc<|j |xx|z cc<st|jdkr|dn|}|S)aw Update the metrics states (correct count and total count), in order to calculate cumulative accuracy of all instances. This function also returns the accuracy of current step. Args: correct: Correct mask, a tensor with shape [batch_size, d0, ..., topk]. Return: Tensor: the accuracy of current step. NrU.rGr)rr[rrarrayprodr_ enumeraterCsumappendfloattotalcountr^)r"rdr, num_samplesaccsik num_correctss rr/zAccuracy.update+s gv} - - (hw''Ggbhw}SbS'9::;; di(( ) )DAq"37+//11L KKl++k9 : : : JqMMM\ )MMM JqMMM[ (MMMMdi..A--tAww4 rczdgt|jz|_dgt|jz|_dS)1 Resets all of the metric state. rN)r^rCrlrmr!s rr+zAccuracy.resetCs4US^^+ S3ty>>) r list[float]cg}t|j|jD]4\}}|dkrt||z nd}||5t |jdkr|dn|}|S)z> Computes and returns the accumulated metric. rrurG)ziprlrmrkrjr^rC)r"restcrs rr1zAccuracy.accumulateJsz DJ//  DAq !Aa1 3A JJqMMMMDI!++c!ff rcjpd|jdkrfd|jD|_dSg|_dS)NaccrGcg|] }d| S)_topr ).0rqr5s r z'Accuracy._init_name..Xs'>>>qT**q**>>>r)rErC_name)r"r5s `rrMzAccuracy._init_nameUsE}u 9>>>>>>DI>>>DJJJDJJJr list[str]c|jS)z1 Return name of metric instance. rr!s rr5z Accuracy.name\ zr)rFN) rCrBr5rHr,rrIrrr)rNrrOrr,rrr)rdrr,rrrr8)rrv)r5rHrr)rr) r*r:r;r<__annotations__r#r7r/r+r1rMr5 __classcell__r)s@rrArAs66p III#       5555>0****        rrAc^eZdZUdZded<ded< ddfd ZddZddZddZddZ xZ S) Precisiona7 Precision (also called positive predictive value) is the fraction of relevant instances among the retrieved instances. Refer to https://en.wikipedia.org/wiki/Evaluation_of_binary_classifiers Noted that this class manages the precision score only for binary classification task. Args: name (str, optional): String name of the metric instance. Default is `precision`. Examples: .. code-block:: python :name: code-standalone-example >>> import numpy as np >>> import paddle >>> x = np.array([0.1, 0.5, 0.6, 0.7]) >>> y = np.array([0, 1, 1, 1]) >>> m = paddle.metric.Precision() >>> m.update(x, y) >>> res = m.accumulate() >>> print(res) 1.0 .. code-block:: python :name: code-model-api-example >>> import numpy as np >>> import paddle >>> import paddle.nn as nn >>> class Data(paddle.io.Dataset): # type: ignore[type-arg] ... def __init__(self): ... super().__init__() ... self.n = 1024 ... self.x = np.random.randn(self.n, 10).astype('float32') ... self.y = np.random.randint(2, size=(self.n, 1)).astype('float32') ... ... def __getitem__(self, idx): ... return self.x[idx], self.y[idx] ... ... def __len__(self): ... return self.n ... >>> model = paddle.Model(nn.Sequential( ... nn.Linear(10, 1), ... nn.Sigmoid() ... )) >>> optim = paddle.optimizer.Adam( ... learning_rate=0.001, parameters=model.parameters()) >>> model.prepare( ... optim, ... loss=nn.BCELoss(), ... metrics=paddle.metric.Precision()) ... >>> data = Data() >>> model.fit(data, batch_size=16) rDtpfp precisionr5r2r,rrIrrcdtj|i|d|_d|_||_dSNr)rKr#rrrr"r5r,rIr)s rr#zPrecision.__init__s: $)&))) rpreds-npt.NDArray[np.float32 | np.float64] | Tensorlabels)npt.NDArray[np.int32 | np.int64] | TensorcFt|tjrtj|}nt |st dt|tjrtj|}nt |st d|jd}tj|dz d}t|D]?}||}||}|dkr'||kr|xj dz c_ /|xj dz c_ @dS)a Update the states based on the current mini-batch prediction results. Args: preds (numpy.ndarray): The prediction result, usually the output of two-class sigmoid function. It should be a vector (column vector or row vector) with data type: 'float64' or 'float32'. labels (numpy.ndarray): The ground truth (labels), the shape should keep the same as preds. The data type is 'int32' or 'int64'. .The 'preds' must be a numpy ndarray or Tensor./The 'labels' must be a numpy ndarray or Tensor.rg?int32rGN) rr[rrrfr ValueErrorr_floorrbrangerrr"rr sample_numrprNrOs rr/zPrecision.updates  eV] + + OHUOOEEE"" OMNN N ffm , , PXf%%FFF## PNOO O\!_ %%,,W55z"" ! !A8D1IEqyy5==GGqLGGGGGqLGG ! !rc"d|_d|_dSrtrN)rrr!s rr+zPrecision.resetrrkc^|j|jz}|dkrt|j|z ndS)z Calculate the final precision. Returns: A scaler float: results of the calculated precision. rru)rrrk)r"aps rr1zPrecision.accumulates2Wtw &(AgguTW~~""36rc|jSr4rr!s rr5zPrecision.namerr)rr5r2r,rrIrrrrrrrrrr8rrkr9) r*r:r;r<rr#r/r+r1r5rrs@rrrcs>>@ GGG GGG&$!$!$!$!L7777rrc\eZdZUdZded<ded<ddfd ZddZddZddZddZ xZ S)Recallal Recall (also known as sensitivity) is the fraction of relevant instances that have been retrieved over the total amount of relevant instances Refer to: https://en.wikipedia.org/wiki/Precision_and_recall Noted that this class manages the recall score only for binary classification task. Args: name (str, optional): String name of the metric instance. Default is `recall`. Examples: .. code-block:: python :name: code-standalone-example >>> import numpy as np >>> import paddle >>> x = np.array([0.1, 0.5, 0.6, 0.7]) >>> y = np.array([1, 0, 1, 1]) >>> m = paddle.metric.Recall() >>> m.update(x, y) >>> res = m.accumulate() >>> print(res) 0.6666666666666666 .. code-block:: python :name: code-model-api-example >>> import numpy as np >>> import paddle >>> import paddle.nn as nn >>> class Data(paddle.io.Dataset): # type: ignore[type-arg] ... def __init__(self): ... super().__init__() ... self.n = 1024 ... self.x = np.random.randn(self.n, 10).astype('float32') ... self.y = np.random.randint(2, size=(self.n, 1)).astype('float32') ... ... def __getitem__(self, idx): ... return self.x[idx], self.y[idx] ... ... def __len__(self): ... return self.n ... >>> model = paddle.Model(nn.Sequential( ... nn.Linear(10, 1), ... nn.Sigmoid() ... )) >>> optim = paddle.optimizer.Adam( ... learning_rate=0.001, parameters=model.parameters()) >>> model.prepare( ... optim, ... loss=nn.BCELoss(), ... metrics=[paddle.metric.Precision(), paddle.metric.Recall()]) ... >>> data = Data() >>> model.fit(data, batch_size=16) rDrfnrecallr5r2r,rrIrrcdtj|i|d|_d|_||_dSr)rKr#rrrrs rr#zRecall.__init__4s8$)&))) rrrrrc@t|tjrtj|}nt |st dt|tjrtj|}nt |st d|jd}tj| d}t|D]?}||}||}|dkr'||kr|xj dz c_ /|xj dz c_ @dS)a Update the states based on the current mini-batch prediction results. Args: preds(numpy.array): prediction results of current mini-batch, the output of two-class sigmoid function. Shape: [batch_size, 1]. Dtype: 'float64' or 'float32'. labels(numpy.array): ground truth (labels) of current mini-batch, the shape should keep the same as preds. Shape: [batch_size, 1], Dtype: 'int32' or 'int64'. rrrrrGN) rr[rrrfrrr_rintrbrrrrs rr/z Recall.update:s eV] + + OHUOOEEE"" OMNN N ffm , , PXf%%FFF## PNOO O\!_ %%g..z"" ! !A8D1IEzz5==GGqLGGGGGqLGG ! !rrkc^|j|jz}|dkrt|j|z ndS)z} Calculate the final recall. Returns: A scaler float: results of the calculated Recall. rru)rrrk)r"rs rr1zRecall.accumulate`s247"*0A++uTW~~&&3>rc"d|_d|_dSr)rrr!s rr+z Recall.resetjrrc|jSrrr!s rr5z Recall.nameqrr)rrrrr8r9) r*r:r;r<rr#r/r1r+r5rrs@rrrsAAF GGG GGG $!$!$!$!L????rrcdeZdZdZ dd fd Zd!dZed"dZd#dZd$dZ d%dZ xZ S)&AucaD The auc metric is for binary classification. Refer to https://en.wikipedia.org/wiki/Receiver_operating_characteristic#Area_under_the_curve. Please notice that the auc metric is implemented with python, which may be a little bit slow. The `auc` function creates four local variables, `true_positives`, `true_negatives`, `false_positives` and `false_negatives` that are used to compute the AUC. To discretize the AUC curve, a linearly spaced set of thresholds is used to compute pairs of recall and precision values. The area under the ROC-curve is therefore computed using the height of the recall values by the false positive rate, while the area under the PR-curve is the computed using the height of the precision values by the recall. Args: curve (str): Specifies the mode of the curve to be computed, 'ROC' or 'PR' for the Precision-Recall-curve. Default is 'ROC'. num_thresholds (int): The number of thresholds to use when discretizing the roc curve. Default is 4095. name (str, optional): String name of the metric instance. Default is `auc`. "NOTE: only implement the ROC curve type via Python now." Examples: .. code-block:: python :name: code-standalone-example >>> import numpy as np >>> import paddle >>> m = paddle.metric.Auc() >>> n = 8 >>> class0_preds = np.random.random(size = (n, 1)) >>> class1_preds = 1 - class0_preds >>> preds = np.concatenate((class0_preds, class1_preds), axis=1) >>> labels = np.random.randint(2, size = (n, 1)) >>> m.update(preds=preds, labels=labels) >>> res = m.accumulate() .. code-block:: python :name: code-model-api-example >>> import numpy as np >>> import paddle >>> import paddle.nn as nn >>> class Data(paddle.io.Dataset): # type: ignore[type-arg] ... def __init__(self): ... super().__init__() ... self.n = 1024 ... self.x = np.random.randn(self.n, 10).astype('float32') ... self.y = np.random.randint(2, size=(self.n, 1)).astype('int64') ... ... def __getitem__(self, idx): ... return self.x[idx], self.y[idx] ... ... def __len__(self): ... return self.n ... >>> model = paddle.Model(nn.Sequential( ... nn.Linear(10, 2), nn.Softmax()) ... ) >>> optim = paddle.optimizer.Adam( ... learning_rate=0.001, parameters=model.parameters()) ... >>> def loss(x, y): ... return nn.functional.nll_loss(paddle.log(x), y) ... >>> model.prepare( ... optim, ... loss=loss, ... metrics=paddle.metric.Auc()) >>> data = Data() >>> model.fit(data, batch_size=16) ROCauccurveLiteral['ROC', 'PR']num_thresholdsrDr5r2r,rrIrrctj|i|||_||_|dz}t j||_t j||_||_dS)NrG) rKr#_curve_num_thresholdsrzeros _stat_pos _stat_negr)r"rrr5r,rI_num_pred_bucketsr)s rr#z Auc.__init__sj $)&))) -*Q."344"344 rrrrrct|tjrtj|}nt |st dt|tjrtj|}nt |st dt|D]`\}}||df}t||j z}||j ksJ|r|j |xxdz cc<K|j |xxdz cc<adS)a Update the auc curve with the given predictions and labels. Args: preds (numpy.array): An numpy array in the shape of (batch_size, 2), preds[i][j] denotes the probability of classifying the instance i into the class j. labels (numpy.array): an numpy array in the shape of (batch_size, 1), labels[i] is either o or 1, representing the label of the instance i. rrrGg?N) rr[rrrfrrrhrDrrr)r"rrrplblvaluebin_idxs rr/z Auc.updates3 ffm , , PXf%%FFF## PNOO O eV] + + OHUOOEEE"" OMNN N'' / /FAs!Q$KE%$"6677Gd22222 /w'''3.''''w'''3.'''' / /rx1rkx2y1y2c8t||z ||zzdz S)Ng@)abs)rrrrs rtrapezoid_areazAuc.trapezoid_areas 27||rBw'#--rcd}d}d}|j}|dkrJ|}|}||j|z }||j|z }||||||z }|dz}|dkJ|dkr|dkr||z |z ndS)z~ Return the area (a float score) under auc curve Return: float: the area under auc curve rurrG)rrrr)r"tot_postot_negridx tot_pos_prev tot_neg_prevs rr1zAuc.accumulates"Qhh"L"L t~c* *G t~c* *G 4&&w  C 1HCQhh(/}}3C'MG # #C rc~|jdz}tj||_tj||_dS)r%rGN)rrrrr)r"rs rr+z Auc.resets:!014"344"344rc|jSrrr!s rr5zAuc.name"rr)rrr) rrrrDr5r2r,rrIrrrr) rrkrrkrrkrrkrrkrr8r9) r*r:r;r<r#r/ staticmethodrr1r+r5rrs@rrrxsMMb'," "!/!/!/!/F...\.    45555rrrGinputrrOrqrDrd Tensor | Nonerlr5rHrc(|jtjkrtj|tj}t r[|t d}|t d}tj||\}}tj |||||\}} } |Str5tj||\}}tj |||\}} } |Std it} t|dgddtj||\}}| d} || d}|| d}| d|g|g|gd | g|g|gd  | S) a accuracy layer. Refer to the https://en.wikipedia.org/wiki/Precision_and_recall This function computes the accuracy using the input and label. If the correct label occurs in top k predictions, then correct will increment by one. Note: the dtype of accuracy is determined by input. the input and label dtype can be different. Args: input(Tensor): The input of accuracy layer, which is the predictions of network. A Tensor with type float32,float64. The shape is ``[sample_number, class_dim]`` . label(Tensor): The label of dataset. Tensor with type int64 or int32. The shape is ``[sample_number, 1]`` . k(int, optional): The top k predictions for each class will be checked. Data type is int64 or int32. correct(Tensor, optional): The correct predictions count. A Tensor with type int64 or int32. total(Tensor, optional): The total entries count. A tensor with type int64 or int32. name(str|None, optional): The default value is None. Normally there is no need for user to set this property. For more information, please refer to :ref:`api_guide_Name` Returns: Tensor, the correct rate. A Tensor with type float32. Examples: .. code-block:: python >>> import paddle >>> predictions = paddle.to_tensor([[0.2, 0.1, 0.4, 0.1, 0.1], [0.2, 0.3, 0.1, 0.15, 0.25]], dtype='float32') >>> label = paddle.to_tensor([[2], [0]], dtype="int64") >>> result = paddle.metric.accuracy(input=predictions, label=label, k=1) >>> print(result) Tensor(shape=[], dtype=float32, place=Place(cpu), stop_gradient=True, 0.50000000) NrrY)rqaccuracyr)float16uint16rXfloat64rX)OutIndicesLabel)rACorrectTotal)typeinputsoutputs)r)rZr[rrcint64rr rCrrr rr localsr "create_variable_for_type_inference append_op) rrOrqrdrlr5topk_out topk_indices_acc_helperacc_outs rrr)sR {fl"" E6<00 ?$7333G ="111E!'Ua!8!8!8,"+ lE7E  a !'Ua!8!8!8,_X|UCC a  0 0vxx 0 0F wCCCZ$[!444Hl77i7HHG;;';JJ }999HH   z|nwOO yW   Nr)rGNNN)rrrOrrqrDrdrrlrr5rHrr)# __future__rr=typingrrrnumpyrr[rrbase.data_feederr base.frameworkr r base.layer_helperr frameworkrcollections.abcr numpy.typingnptr__all__rABCMetarrArrrrr rrrsN#""""" .......... ((((((((77777788888888++++++''''''(((((( 555TTTTTs{TTTTn]]]]]v]]]@GGGGGGGGTHHHHHVHHHVnnnnn&nnnh! OOOOOOOr