zjUddlmZddlZddlZddlZddlZddlmZddlZ ddl Z ddl m Z ddl mZerGddlmZmZmZddlmZmZdd lmZdd lmZed Zd ed <GddeZGddeZgZ d:d;d(ZGd)d'ZGd*d+Z Gd,d-e Z!Gd.d/e Z"Gd0d1e Z#Gd2d3e Z$Gd4d5e Z%Gd6d7e Z&Gd8d9e Z'dS)<) annotationsN) TYPE_CHECKING) try_import) ProgressBar)AnyLiteral TypedDict)IteratorSequence) TypeAlias)Modeltrainevalpredictr _CallbackModecBeZdZUded<ded<ded<ded<ded<dS) _CallbackParamsint batch_sizeepochsstepsverbose list[str]metricsN__name__ __module__ __qualname____annotations__e/lsinfo/ai/hellotax_ai/data_center/backend/venv/lib/python3.11/site-packages/paddle/hapi/callbacks.pyrr'sB   r#rc.eZdZUded<ded<ded<dS) _CallbackLogsfloatlossrrrrNrr"r#r$r&r&.s. r#r&r callbacks$Sequence[Callback] | Callback | Nonemodel Model | Noner int | Nonerrlog_freqrr save_freqsave_dir str | Nonerlist[str] | NonemodeLiteral['train', 'test']return CallbackListc T|pg} tt| ttfr| n| g} td| Ds|rt ||g| } td| Dsg| t ||} | D]} t| t r|| _td| Dsg| t} t| }| || dkr| pgng} ||||| d}| ||S)Nc3@K|]}t|tVdSN) isinstance ProgBarLogger.0ks r$ z#config_callbacks..Hs,::z!]++::::::r#)rc3@K|]}t|tVdSr:)r;ModelCheckpointr=s r$r@z#config_callbacks..Ks,<.Qs,88az![))888888r#test)rrrrr) listr;tupleanyr<rB EarlyStoppingr1rDr7 set_model set_params)r*r,rrrr/rr0r1rr4_cbkscbksr?cbk_listparamss r$config_callbacksrP7sz OEED%=11>wD ::T::: : :AwAh888@4@ <5$/////r#stepcj||d|d}||||dS)Nro _batch_beginrqrUr4r~rlrcs r$on_batch_beginzCallbackList.on_batch_beginsD 'T''' 4t$$$$$r#cj||d|d}||||dS)Nro _batch_endrqrs r$ on_batch_endzCallbackList.on_batch_endsD %T%%% 4t$$$$$r#r:)r*rRr6rS)rWrXr6rS)r6r[rOrr6rSr,rr6rS)rcrdrerr6rS)r4rr6rS)r4rrlrmr6rSNNrwr.rlrmr6rS)r4rr~r.rlrmr6rS)rrr rVrZr_rKrJrirkrsrvryr}rrr"r#r$r7r7bsg(((($$$$!!!! 3333AEAEFJ22222 FJ00000 %) %%%%% %) %%%%%%%r#ceZdZUdZded<ded<d dZd!d Zd"d Zd#d$dZd#d$dZ d#d$dZ d#d$dZ d#d$dZ d#d$dZ d#d%dZ d#d%dZ d#d&dZ d#d&dZ d#d&dZ d#d&dZ d#d&dZ d#d&dZd S)'rXad Base class used to build new callbacks. And new callbacks could also terminate training by setting `model.stop_training=True`. Examples: .. code-block:: python >>> import paddle >>> # build a simple model checkpoint callback >>> class ModelCheckpoint(paddle.callbacks.Callback): ... def __init__(self, save_freq=1, save_dir=None): ... self.save_freq = save_freq ... self.save_dir = save_dir ... ... def on_epoch_end(self, epoch, logs=None): ... if self.model is not None and epoch % self.save_freq == 0: ... path = '{}/{}'.format(self.save_dir, epoch) ... print('save checkpoint at {}'.format(path)) ... self.model.save(path) r-r,rrOr6rSc"d|_i|_dSr:)r,rOr^s r$rVzCallback.__init__s  r#c||_dS)a Set parameters, which is dict. The keys contain: - 'batch_size': an integer. Number of samples per batch. - 'epochs': an integer. Number of epochs. - 'steps': an integer. Number of steps of one epoch. - 'verbose': an integer. Verbose mode is 0, 1 or 2. 0 = silent, 1 = progress bar, 2 = one line per epoch. - 'metrics': a list of str. Names of metrics, including 'loss' and the names of paddle.metric.Metric. N)rO)rUrOs r$rKzCallback.set_paramss r#rc||_dS)z"model is instance of paddle.Model.N)r,)rUr,s r$rJzCallback.set_models  r#NrlrmcdS)zmCalled at the start of training. Args: logs (dict): The logs is a dict or None. Nr"rUrls r$on_train_beginzCallback.on_train_beginr#cdS)zCalled at the end of training. Args: logs (dict): The logs is a dict or None. The keys of logs passed by paddle.Model contains 'loss', metric names and `batch_size`. Nr"rs r$ on_train_endzCallback.on_train_endrr#cdS)aCalled at the start of evaluation. Args: logs (dict): The logs is a dict or None. The keys of logs passed by paddle.Model contains 'steps' and 'metrics', The `steps` is number of total steps of validation dataset. The `metrics` is a list of str including 'loss' and the names of paddle.metric.Metric. Nr"rs r$ on_eval_beginzCallback.on_eval_beginrr#cdS)aCalled at the end of evaluation. Args: logs (dict): The logs is a dict or None. The `logs` passed by paddle.Model is a dict contains 'loss', metrics and 'batch_size' of last batch of validation dataset. Nr"rs r$ on_eval_endzCallback.on_eval_endrr#cdS)zpCalled at the beginning of predict. Args: logs (dict): The logs is a dict or None. Nr"rs r$on_predict_beginzCallback.on_predict_beginrr#cdS)zjCalled at the end of predict. Args: logs (dict): The logs is a dict or None. Nr"rs r$on_predict_endzCallback.on_predict_end rr#rwrcdS)zCalled at the beginning of each epoch. Args: epoch (int): The index of epoch. logs (dict): The logs is a dict or None. The `logs` passed by paddle.Model is None. Nr"r{s r$ryzCallback.on_epoch_beginrr#cdS)a Called at the end of each epoch. Args: epoch (int): The index of epoch. logs (dict): The logs is a dict or None. The `logs` passed by paddle.Model is a dict, contains 'loss', metrics and 'batch_size' of last batch. Nr"r{s r$r}zCallback.on_epoch_endrr#r~cdS)zCalled at the beginning of each batch in training. Args: step (int): The index of step (or iteration). logs (dict): The logs is a dict or None. The `logs` passed by paddle.Model is empty. Nr"rUr~rls r$on_train_batch_beginzCallback.on_train_batch_begin*rr#cdS)a<Called at the end of each batch in training. Args: step (int): The index of step (or iteration). logs (dict): The logs is a dict or None. The `logs` passed by paddle.Model is a dict, contains 'loss', metrics and 'batch_size' of current batch. Nr"rs r$on_train_batch_endzCallback.on_train_batch_end5rr#cdS)zCalled at the beginning of each batch in evaluation. Args: step (int): The index of step (or iteration). logs (dict): The logs is a dict or None. The `logs` passed by paddle.Model is empty. Nr"rs r$on_eval_batch_beginzCallback.on_eval_batch_beginArr#cdS)a>Called at the end of each batch in evaluation. Args: step (int): The index of step (or iteration). logs (dict): The logs is a dict or None. The `logs` passed by paddle.Model is a dict, contains 'loss', metrics and 'batch_size' of current batch. Nr"rs r$on_eval_batch_endzCallback.on_eval_batch_endLrr#cdS)zCalled at the beginning of each batch in predict. Args: step (int): The index of step (or iteration). logs (dict): The logs is a dict or None. Nr"rs r$on_predict_batch_beginzCallback.on_predict_batch_beginXrr#cdS)zCalled at the end of each batch in predict. Args: step (int): The index of step (or iteration). logs (dict): The logs is a dict or None. Nr"rs r$on_predict_batch_endzCallback.on_predict_batch_endbrr#r6rSrrr:rlrmr6rSrwrrlrmr6rSr~rrlrmr6rS)rrr __doc__r!rVrKrJrrrrrrryr}rrrrrrr"r#r$rXrXs0                                   8<      8<      7;      7;      7;      7;      7;     7;       r#rXceZdZUdZded<ded<ded<ded<ded <d%d&dZdZd'd(dZ d)d*dZd+dZ d'd,dZ d'd,dZ d'd-dZ d'd(dZ d'd,dZ d'd,dZd'd(d Z d'd,d!Z d'd,d"Zd'd(d#Zd'd(d$ZdS).r<a Logger callback function to print loss and metrics to stdout. It supports silent mode (not print), progress bar or one line per each printing, see arguments for more detailed. Args: log_freq (int): The frequency, in number of steps, the logs such as loss, metrics are printed. Default: 1. verbose (int): The verbosity mode, should be 0, 1, or 2. 0 = silent, 1 = progress bar, 2 = one line each printing, 3 = 2 + time counter, such as average reader cost, samples per second. Default: 2. Examples: .. code-block:: python >>> import paddle >>> import paddle.vision.transforms as T >>> from paddle.vision.datasets import MNIST >>> from paddle.static import InputSpec >>> inputs = [InputSpec([-1, 1, 28, 28], 'float32', 'image')] >>> labels = [InputSpec([None, 1], 'int64', 'label')] >>> transform = T.Compose([ ... T.Transpose(), ... T.Normalize([127.5], [127.5]) ... ]) >>> train_dataset = MNIST(mode='train', transform=transform) >>> lenet = paddle.vision.models.LeNet() >>> model = paddle.Model(lenet, ... inputs, labels) >>> optim = paddle.optimizer.Adam(0.001, parameters=lenet.parameters()) >>> model.prepare(optimizer=optim, ... loss=paddle.nn.CrossEntropyLoss(), ... metrics=paddle.metric.Accuracy()) >>> callback = paddle.callbacks.ProgBarLogger(log_freq=10) >>> model.fit(train_dataset, batch_size=64, callbacks=callback) r.rrzProgressBar | Noneprogbarrrr/rr)r6rScLd|_d|_d|_||_||_dSr:)rrrrr/)rUr/rs r$rVzProgBarLogger.__init__s)      r#c^|jo&tjjdkSNr)rpaddle distributed ParallelEnv local_rankr^s r$ _is_printzProgBarLogger._is_prints&|P 2 > > @ @ Kq PPr#Nrlrmc|jd|_|jsJ|jd|_|jsJddddd|_|rt ddSdS)Nrrr data_time batch_timecountsampleszmThe loss value printed in the log is the current step, and the metric is the average value of previous steps.)rOr train_metrics _train_timerrprintrs r$rzProgBarLogger.on_train_beginsk(+ {{![3!!!!    >>           r#rwc,|jd|_||_d|_|jr1|rt d|dzd|jt|j|j|_ tj |j d<dS)NrrEpoch r/numrbatch_start_time) rOrrw train_steprrrrr train_progbartimerr{s r$ryzProgBarLogger.on_epoch_begins[)   ; 64>>++ 6 451944t{44 5 5 5(TZNNN04 ,---r#r4rcg}t||d}t||d}t||d}|D]#}||vr||||f$|jdkrt|d|drt|d|d}|ddkr|dnd } |d dkr|d nd } |d d |d | z zf|dd |d| z zf|dd| |d |dzz zfd|d<d|d <d|d <d|d<|||dS)N_metrics_progbar_step__timerrr?ravg_reader_costz%.5f secravg_batch_costripsz%.5f samples/secg)rgrZrhasattrupdate) rUrlr4valuesrrrr?timercntrs r$_updateszProgBarLogger._updatess$4 1 1 122$4 1 1 122nnn-- , ,ADyy q$q'l+++ <1  /?4/?/?/?!@!@ D"2d"2"2"233E$)'NQ$6$6%..CC*/ *:Q*>*>eI&&CG MM"J% 2Ds2J$KL    MM!:|1Ds1J#KL    MM&% "4u\7J"JKM   E'N E) !$E+ "%E, uf%%%%%r#r~ctj|jd<|jdxx|jd|jdz z cc<dSNbatch_data_end_timerr)rrrs r$rz"ProgBarLogger.on_train_batch_beginsZ489;;/0 +&&&  3 4 23 4 &&&&&r#c|pi}|xjdz c_|jdxxtj|jdz z cc<|jdxxdz cc<|dd}|jdxx|z cc<|r@|j|jzdkr-|j|j|jkr||dtj|jd <dS) Nrrrrrrrrr)rrrgetrr/rrrUr~rlrs r$rz ProgBarLogger.on_train_batch_endszr 1 ,''' IKK$+,AB B ''' '"""a'"""((<++ )$$$/$$$ >>   -$- ?1 D Dz!T_tz%A%A dG,,,04 ,---r#cx|pi}|r|j||ddSdSdS)Nr)rrrr{s r$r}zProgBarLogger.on_epoch_endsPzr >>   )!7 MM$ ( ( ( ( ( ) )!7!7r#cb|dd|_|dg|_d|_d|_ddddd|_t |j|j|_| rtdtj |jd<dS)Nrrrrrz Eval begin...r) r eval_steps eval_metrics eval_stepevaled_samples _eval_timerrr eval_progbarrrrrs r$rzProgBarLogger.on_eval_begin s((7D11 HHY33   (    >>   # / " " "/3y{{+,,,r#ctj|jd<|jdxx|jd|jdz z cc<dSr)rrrs r$rz!ProgBarLogger.on_eval_batch_beginZ37)++./ %%%  2 312 3 %%%%%r#c@|pi}|xjdz c_|dd}|xj|z c_|jdxxt j|jdz z cc<|jdxxdz cc<|dd}|jdxx|z cc<|r@|j|jzdkr-|j|j|jkr||dt j|jd <dS) Nrrrrrrrrr) rrrrrrr/rrrs r$rzProgBarLogger.on_eval_batch_end's6zr !((<++ w& &&& IKK$*+@A A &&& !!!Q&!!!((<++ ###w.### >>   , >! C C&$.4?*J*J dF+++/3y{{+,,,r#cb|dd|_|dg|_d|_d|_ddddd|_t |j|j|_| rtdtj |jd<dS)NrrrrrzPredict begin...r) r test_steps test_metrics test_steptested_samples _test_timerrr test_progbarrrrrs r$rzProgBarLogger.on_predict_begin<s((7D11 HHY33   (    >>   & $ % % %/3y{{+,,,r#ctj|jd<|jdxx|jd|jdz z cc<dSr)rrrs r$rz$ProgBarLogger.on_predict_batch_beginQrr#c@|pi}|xjdz c_|dd}|xj|z c_|jdxxt j|jdz z cc<|jdxxdz cc<|dd}|jdxx|z cc<|j|jzdkrA|r-|j|j|jkr||dt j|jd <dS) NrrrrrrrrEr) rrrrrr/rrrrs r$rz"ProgBarLogger.on_predict_batch_endZs4zr !((<++ w& &&& IKK$*+@A A &&& !!!Q&!!!((<++ ###w.### >DM )Q . .4>>3C3C .&$.4?*J*J dF+++/3y{{+,,,r#c|pi}|r6|j1||dtd|jdSdSdS)NrzEval samples: )rrrrrrs r$rzProgBarLogger.on_eval_endoshzr >>   :!< MM$ ' ' ' 84#688 9 9 9 9 9 : :!>   =~ -22dla6G6G dF+++ ;d&9;; < < < < < = =r#)rr))r/rrrr6rSr:rrrrlrmr4rr6rSrr)rrr rr!rVrrryrrrr}rrrrrrrrr"r#r$r<r<ms))VLLLMMM!!!!!QQQ$FJ < < < < < & & & &F7;     7;<<<<<$8<)))));;;;;,7;     7;;;;;;*;;;;;,7;     7;;;;;;*::::: =======r#r<cDeZdZdZddd Z dddZdZ dddZdddZdS)rBac Model checkpoint callback function to save model weights and optimizer state during training in conjunction with model.fit(). Currently, ModelCheckpoint only supports saving after a fixed number of epochs. Args: save_freq(int): The frequency, in number of epochs, the model checkpoint are saved. Default: 1. save_dir(str|None): The directory to save checkpoint during training. If None, will not save checkpoint. Default: None. Examples: .. code-block:: python >>> import paddle >>> import paddle.vision.transforms as T >>> from paddle.vision.datasets import MNIST >>> from paddle.static import InputSpec >>> inputs = [InputSpec([-1, 1, 28, 28], 'float32', 'image')] >>> labels = [InputSpec([None, 1], 'int64', 'label')] >>> transform = T.Compose([ ... T.Transpose(), ... T.Normalize([127.5], [127.5]) ... ]) >>> train_dataset = MNIST(mode='train', transform=transform) >>> lenet = paddle.vision.models.LeNet() >>> model = paddle.Model(lenet, ... inputs, labels) >>> optim = paddle.optimizer.Adam(0.001, parameters=lenet.parameters()) >>> model.prepare(optimizer=optim, ... loss=paddle.nn.CrossEntropyLoss(), ... metrics=paddle.metric.Accuracy()) >>> callback = paddle.callbacks.ModelCheckpoint(save_dir='./temp') >>> model.fit(train_dataset, batch_size=64, callbacks=callback) rNr0rr1r2r6rSc"||_||_dSr:)r0r1)rUr0r1s r$rVzModelCheckpoint.__init__s"  r#rwr.rlrmc||_dSr:)rwr{s r$ryzModelCheckpoint.on_epoch_begins r#cl|jo-|jo&tjjdkSr)r,r1rrrrr^s r$_is_savezModelCheckpoint._is_saves7 J A  A"..00;q@ r#c|rj|j|jzdkrY|jd|}t dt j||j |dSdSdS)Nrrsave checkpoint at ) rrwr0r1rospathabspathr,save)rUrwrlrs r$r}zModelCheckpoint.on_epoch_ends ==?? "tzDN:a??m--e--D ?(=(=?? @ @ @ JOOD ! ! ! ! ! " "??r#c|rU|jd}tdtj||j|dSdS)Nz/finalr)rr1rrrrr,r)rUrlrs r$rzModelCheckpoint.on_train_endsk ==?? "m+++D ?(=(=?? @ @ @ JOOD ! ! ! ! ! " "r#)rN)r0rr1r2r6rSrrr:rr) rrr rrVryrr}rr"r#r$rBrB}s''R!!!!! FJ    8<""""""""""""r#rBc4eZdZdZddd Z dddZ dddZd S)rDaj Lr scheduler callback function Args: by_step(bool, optional): whether to update learning rate scheduler by step. Default: True. by_epoch(bool, optional): whether to update learning rate scheduler by epoch. Default: False. Examples: .. code-block:: python >>> import paddle >>> import paddle.vision.transforms as T >>> from paddle.static import InputSpec >>> inputs = [InputSpec([-1, 1, 28, 28], 'float32', 'image')] >>> labels = [InputSpec([None, 1], 'int64', 'label')] >>> transform = T.Compose([ ... T.Transpose(), ... T.Normalize([127.5], [127.5]) ... ]) >>> train_dataset = paddle.vision.datasets.MNIST(mode='train', transform=transform) >>> lenet = paddle.vision.models.LeNet() >>> model = paddle.Model(lenet, ... inputs, labels) >>> base_lr = 1e-3 >>> boundaries = [5, 8] >>> wamup_steps = 4 >>> def make_optimizer(parameters=None): ... momentum = 0.9 ... weight_decay = 5e-4 ... values = [base_lr * (0.1**i) for i in range(len(boundaries) + 1)] ... learning_rate = paddle.optimizer.lr.PiecewiseDecay( ... boundaries=boundaries, values=values) ... learning_rate = paddle.optimizer.lr.LinearWarmup( ... learning_rate=learning_rate, ... warmup_steps=wamup_steps, ... start_lr=base_lr / 5., ... end_lr=base_lr, ... verbose=True) ... optimizer = paddle.optimizer.Momentum( ... learning_rate=learning_rate, ... weight_decay=weight_decay, ... momentum=momentum, ... parameters=parameters) ... return optimizer >>> optim = make_optimizer(parameters=lenet.parameters()) >>> model.prepare(optimizer=optim, ... loss=paddle.nn.CrossEntropyLoss(), ... metrics=paddle.metric.Accuracy()) >>> # if LRScheduler callback not set, an instance LRScheduler update by step >>> # will be created auto. >>> model.fit(train_dataset, batch_size=64) >>> # create a learning rate scheduler update by epoch >>> callback = paddle.callbacks.LRScheduler(by_step=False, by_epoch=True) >>> model.fit(train_dataset, batch_size=64, callbacks=callback) TFby_stepboolby_epochr6rScH|r|rtd||_||_dS)Nz2by_step option is mutually exclusive with by_epoch) ValueErrorrr )rUrr s r$rVzLRScheduler.__init__s:  x D    r#Nrwrrlrmc|jr~|jjrtt|jjdr\t |jjjt jjj r+|jjj dSdSdSdSdSN_learning_rate) r r, _optimizerrr;rr optimizerlrrDr~r{s r$r}zLRScheduler.on_epoch_ends = < % <DJ13CDD <J)8$'3 < %499;;;;; < < < < < < < >> import paddle >>> from paddle import Model >>> from paddle.static import InputSpec >>> from paddle.vision.models import LeNet >>> from paddle.vision.datasets import MNIST >>> from paddle.metric import Accuracy >>> from paddle.nn import CrossEntropyLoss >>> import paddle.vision.transforms as T >>> device = paddle.set_device('cpu') >>> sample_num = 200 >>> save_dir = './best_model_checkpoint' >>> transform = T.Compose( ... [T.Transpose(), T.Normalize([127.5], [127.5])]) >>> train_dataset = MNIST(mode='train', transform=transform) >>> val_dataset = MNIST(mode='test', transform=transform) >>> net = LeNet() >>> optim = paddle.optimizer.Adam( ... learning_rate=0.001, parameters=net.parameters()) >>> inputs = [InputSpec([None, 1, 28, 28], 'float32', 'x')] >>> labels = [InputSpec([None, 1], 'int64', 'label')] >>> model = Model(net, inputs=inputs, labels=labels) >>> model.prepare( ... optim, ... loss=CrossEntropyLoss(reduction="sum"), ... metrics=[Accuracy()]) >>> callbacks = paddle.callbacks.EarlyStopping( ... 'loss', ... mode='min', ... patience=1, ... verbose=1, ... min_delta=0, ... baseline=None, ... save_best_model=True) >>> model.fit(train_dataset, ... val_dataset, ... batch_size=64, ... log_freq=200, ... save_freq=10, ... save_dir=save_dir, ... epochs=20, ... callbacks=[callbacks]) r(autorrNTmonitorrdr4Literal['auto', 'min', 'max']patiencerr min_deltar'baseline float | Nonesave_best_modelrr6rScPt||_||_||_||_t ||_d|_d|_ d|_ ||_ d|_ |dvrtjd|dd}|dkrtj|_nD|dkrtj|_n,d|jvrtj|_ntj|_|jtjkr|xjd zc_dS|xjd zc_dS) NrrminmaxzEarlyStopping mode # is unknown, fallback to auto mode.rrraccr)superrVrrrrabsr wait_epoch best_weights stopped_epochrr1warningswarnnpless monitor_opgreater) rUrr4rrrrr __class__s r$rVzEarlyStopping.__init__xs!       Y .$( - - - MOdOOO   D 5== gDOO U]] jDOO $$"$*"$' ?bj ( ( NNa NNNN NNb NNNNr#rlrmcd|_|j|j|_dS|jtjkr tjn tj |_d|_dSr)r%r best_valuer,r*r+infr&rs r$rzEarlyStopping.on_train_beginsK = $"mDOOO(,27(B(BbffDO $D   r#c | |j|vrtjddS||j}t|tt fr |d}nt|t jr|}ndS|||j z |j r\||_ d|_ |j rF|j ?tj|j d}|j|n|xj dz c_ |j |jkrd|j_|jdkrt-d|jdzd|j ri|j bt-dtjtj|j d|xjdz c_dS) Nz7Monitor of EarlyStopping should be loss or metric name.r best_modelrTrz: Early stopping.z$Best checkpoint has been saved at {})rr(r)r;rFrGnumbersNumberr,rr0r%rr1rrjoinr,rr stop_trainingrrr'formatr)rUrlcurrentrs r$rzEarlyStopping.on_eval_ends <4EEGOO " T]L I I ar#)r(rrrrNT)rrdr4rrrrrrr'rrrrr6rSr:r)rrr rrVrr __classcell__r.s@r$rIrI.sGGV.4!% $)!)!)!)!)!)!)!V%%%%%" " " " " " " " " r#rIcjeZdZdZddZddZddd Z dddZddZ dd dZ dddZ dddZ dddZ d S)!VisualDLa VisualDL callback class. After storing the loss values and evaluation metrics in a log file during the training time , the panel is launched to view the visual results. Args: log_dir (str): The directory to save visualdl log file. Examples: .. code-block:: python >>> import paddle >>> import paddle.vision.transforms as T >>> from paddle.static import InputSpec >>> inputs = [InputSpec([-1, 1, 28, 28], 'float32', 'image')] >>> labels = [InputSpec([None, 1], 'int64', 'label')] >>> transform = T.Compose([ ... T.Transpose(), ... T.Normalize([127.5], [127.5]) ... ]) >>> train_dataset = paddle.vision.datasets.MNIST(mode='train', transform=transform) >>> eval_dataset = paddle.vision.datasets.MNIST(mode='test', transform=transform) >>> net = paddle.vision.models.LeNet() >>> model = paddle.Model(net, inputs, labels) >>> optim = paddle.optimizer.Adam(0.001, parameters=net.parameters()) >>> model.prepare(optimizer=optim, ... loss=paddle.nn.CrossEntropyLoss(), ... metrics=paddle.metric.Accuracy()) >>> ## uncomment following lines to fit model with visualdl callback function >>> # callback = paddle.callbacks.VisualDL(log_dir='visualdl_log_dir') >>> # model.fit(train_dataset, eval_dataset, batch_size=64, callbacks=callback) log_dirrdr6rSc>||_d|_d|_d|_dSr)r>rrrw)rUr>s r$rVzVisualDL.__init__s"    r#rcPtjjdkSrrrrrr^s r$ _is_writezVisualDL._is_write!--//:a??r#Nrlrmc|jd|_|jsJ|jd|_|jsJd|_d|_dS)NrrTr)rOrr_is_fitrrs r$rzVisualDL.on_train_beginsMk(+ {{![3!!!! r#rwr.c8|jd|_||_dSNrrOrrwr{s r$ryzVisualDL.on_epoch_begin[)  r#rr4rc,|sdSt|ds.td}||j|_t ||d}t ||d}|dkr|}n|j}|D]}||vr|dz|z}t||ttfr||d} n*t||tj r ||} ni|j ||| dS) Nwritervisualdlrrrrr)tagr~value)rBrr LogWriterr>rKrgrwr;rFrGr4r5 add_scalar) rUrlr4rLr current_step total_stepr?temp_tag temp_values r$rzVisualDL._updates s8~~  FtX&& ;!*--H",,T\::DK$4 1 1 122t^^^44 7??%JJJ  ADyy#:>d1ge}55!%aJJQ88!%aJJ && z'  r#r~c|pi}|xjdz c_|r||ddSdSNrrrrBrrs r$rzVisualDL.on_train_batch_end+Szr 1 >>   ) MM$ ( ( ( ( ( ) )r#c|dd|_|dg|_d|_d|_dSNrrrrrrrrrs r$rzVisualDL.on_eval_begin4B((7D11 HHY33r#c|t|dr+|jt|ddSdS)NrK)rrKclosedelattrrs r$rzVisualDL.on_train_end:sH 4 " " $ K      D( # # # # # $ $r#c|ra||dt|ds=t|dr/|jt |ddSdSdSdS)NrrErK)rBrrrKr^r_rs r$rzVisualDL.on_eval_end?s >>   ( MM$ ' ' 'D),, ('$2I2I ( !!###h'''''  ( ( ( ( ( (r#)r>rdr6rSr6rr:rrr)rlrr4rr6rSr) rrr rrVrBrryrrrrrr"r#r$r=r=s##J @@@@FJ >7;)))))     $$$$$ (((((((r#r=ceZdZdZ dd dZdZedZd!d"dZ d!d#dZ d$dZ d!d%dZ d!d"dZ d!d"dZ d!d"dZdS)& WandbCallbackaATrack your training and system metrics using `Weights and Biases `_. **Installation and set-up** Install with pip and log in to your W&B account: .. code-block:: bash pip install wandb wandb login Args: project(str|None, optional): Name of the project. Default: uncategorized entity(str|None, optional): Name of the team/user creating the run. Default: Logged in user name(str|None, optional): Name of the run. Default: randomly generated by wandb dir(str|None, optional): Directory in which all the metadata is stored. Default: `wandb` mode(str|None, optional): Can be "online", "offline" or "disabled". Default: "online". job_type(str|None, optional): the type of run, for grouping runs together. Default: None Examples: .. code-block:: python >>> import paddle >>> import paddle.vision.transforms as T >>> from paddle.static import InputSpec >>> inputs = [InputSpec([-1, 1, 28, 28], 'float32', 'image')] >>> labels = [InputSpec([None, 1], 'int64', 'label')] >>> transform = T.Compose([ ... T.Transpose(), ... T.Normalize([127.5], [127.5]) ... ]) >>> train_dataset = paddle.vision.datasets.MNIST(mode='train', transform=transform) >>> eval_dataset = paddle.vision.datasets.MNIST(mode='test', transform=transform) >>> net = paddle.vision.models.LeNet() >>> model = paddle.Model(net, inputs, labels) >>> optim = paddle.optimizer.Adam(0.001, parameters=net.parameters()) >>> model.prepare(optimizer=optim, ... loss=paddle.nn.CrossEntropyLoss(), ... metrics=paddle.metric.Accuracy()) >>> ## uncomment following lines to fit model with wandb callback function >>> # callback = paddle.callbacks.WandbCallback(project='paddle_mnist') >>> # model.fit(train_dataset, eval_dataset, batch_size=64, callbacks=callback) Nprojectr2entityrcdirr4/Literal['online', 'offline', 'disabled'] | Nonejob_typekwargsrr6rSc tdd|_||||||d|_d|_|jjdi||j}dS)NwandbzVYou want to use `wandb` which is not installed yet install it with `pip install wandb`)rdrcrerfr4rhr")rrk wandb_args_runrrun) rUrdrercrfr4rhrirs r$rVzWandbCallback.__init__{sl  d       ((((( Hr#cPtjjdkSrrAr^s r$rBzWandbCallback._is_writerCr#c|rU|jN|jj&t jd|jj|_n|jjdi|j|_|jS)NzThere is a wandb run already in progress and newly created instances of `WandbCallback` will reuse this run. If this is not desired , call `wandb.finish()` before instantiating `WandbCallback`.r")rBrmrkrnr(r)initrlr^s r$rnzWandbCallback.runsu >>   Cy :>-MY !% DII /  B B$/ B BDIyr#rlrmc|jd|_|jsJ|jd|_|jsJd|_d|_|rn|jd|jdd|jd|jd ddSdS) NrrTr train/stepztrain/*) step_metricrwzeval/*)rOrrrErrBrn define_metricrs r$rzWandbCallback.on_train_beginsk(+ {{![3!!!!  >>   B H " "< 0 0 0 H " "9, " G G G H " "7 + + + H " "8 " A A A A A  B Br#rwrc8|jd|_||_dSrGrHr{s r$ryzWandbCallback.on_epoch_beginrIr#rc^|sdSt||d}t||d}i}|dkr|}|d|in|j}|d|i|D]}||vr|dz|z}t ||t t fr$||||diTt ||tjr||||i|j |dS)Nrrrrsrwrr) rBrgrrwr;rFrGr4r5rnlog) rUrlr4rrQrrRr?rSs r$rzWandbCallback._updatessL~~  F$4 1 1 122t^^^44  7??%J OO\:6 7 7 7 7J OOWj1 2 2 2  ADyy#:>d1ge}55OOXtAwqz$:;;;;Q88OOXtAw$78888  Xr#r~c|pi}|xjdz c_|r||ddSdSrVrWrs r$rz WandbCallback.on_train_batch_endrXr#c|dd|_|dg|_d|_d|_dSrZr[rs r$rzWandbCallback.on_eval_beginr\r#cd|r|jdSdSr:)rBrnfinishrs r$rzWandbCallback.on_train_ends6 >>    HOO       r#c|ra||dt|ds=t|dr/|jt |ddSdSdSdS)NrrErn)rBrrrnr|r_rs r$rzWandbCallback.on_eval_ends >>   % MM$ ' ' 'D),, %'$2F2F %!!!e$$$$$  % % % % % %r#)NNNNNN)rdr2rer2rcr2rfr2r4rgrhr2rirr6rSr:rrrr)rrr rrVrBpropertyrnrryrrrrrr"r#r$rcrcHs200h#!@D#:@@@  X  B B B B B 8< <7;)))))     %%%%%%%r#rcceZdZUdZded<ded<ded<ded<d ed <ded <ded <ded < d d!fd Zd"dZd#d$dZd#d$dZd%dZ xZ S)&ReduceLROnPlateaua Reduce learning rate when a metric of evaluation has stopped improving. Models often benefit from reducing the learning rate by a factor of 2-10 once learning stagnates. This callback monitors a quantity and if no improvement is seen for a 'patience' number of epochs, the learning rate is reduced. Args: monitor(str, optional): Quantity to be monitored. Default: 'loss'. factor(float, optional): factor by which the learning rate will be reduced. `new_lr = lr * factor`. Default: 0.1. patience(int, optional): Number of epochs with no improvement after which learning rate will be reduced. Default: 10. verbose(int, optional): The verbosity mode. 0: quiet, 1: update messages. Default: 1. mode(str, optional): one of `{'auto', 'min', 'max'}`. In `'min'` mode, the learning rate will be reduced when the quantity monitored has stopped decreasing. In 'max' mode, learning rate will reduce until monitored quantity stops increasing. In 'auto' mode, exact mode can be inferred by the name of monitor. If 'acc' in monitor, the mode will be considered as 'max', otherwise the mode will be set to 'min'. Default: 'auto'. min_delta(int|float, optional): threshold for measuring the new optimum, to only focus on significant changes. Default: 0. cooldown(int, optional): number of epochs to wait before resuming normal operation after lr has been reduced. Default: 0. min_lr(float, optional): lower bound on the learning rate. Default: 0. Examples: .. code-block:: python >>> import paddle >>> from paddle import Model >>> from paddle.static import InputSpec >>> from paddle.vision.models import LeNet >>> from paddle.vision.datasets import MNIST >>> from paddle.metric import Accuracy >>> from paddle.nn.layer.loss import CrossEntropyLoss >>> import paddle.vision.transforms as T >>> sample_num = 200 >>> transform = T.Compose( ... [T.Transpose(), T.Normalize([127.5], [127.5])]) >>> train_dataset = MNIST(mode='train', transform=transform) >>> val_dataset = MNIST(mode='test', transform=transform) >>> net = LeNet() >>> optim = paddle.optimizer.Adam( ... learning_rate=0.001, parameters=net.parameters()) >>> inputs = [InputSpec([None, 1, 28, 28], 'float32', 'x')] >>> labels = [InputSpec([None, 1], 'int64', 'label')] >>> model = Model(net, inputs=inputs, labels=labels) >>> model.prepare( ... optim, ... loss=CrossEntropyLoss(), ... metrics=[Accuracy()]) >>> callbacks = paddle.callbacks.ReduceLROnPlateau(patience=3, verbose=1) >>> model.fit(train_dataset, ... val_dataset, ... batch_size=64, ... log_freq=200, ... save_freq=10, ... epochs=20, ... callbacks=[callbacks]) rdrr'factorrrrrr4rcooldownmin_lrr(皙? rr-C6?rr6rSc Pt||_|dkrtd||_||_||_||_||_||_ d|_ d|_ d|_ ||_ d|_d|_|dS)Nrz3ReduceLROnPlateau does not support a factor >= 1.0.r)r#rVrr rrrrrrcooldown_counterwaitbestr4r,rw_reset) rUrrrrr4rrrr.s r$rVzReduceLROnPlateau.__init__Cs  S==E   "     !     r#c,jdvr$tjdjdd_jdksjdkr%djvrfd_t j_nfd_t j _d _d _ d S) z)Resets wait counter and cooldown counter.rzLearning rate reduction mode r rrr!c>tj||jz Sr:)r*r+rabrUs r$z*ReduceLROnPlateau._reset..os271a$.6H+I+Ir#c>tj||jzSr:)r*r-rrs r$rz*ReduceLROnPlateau._reset..rs2:aT^9K+L+Lr#rN) r4r(r)rr,r*r1rrrr^s`r$rzReduceLROnPlateau._resetds 92 2 2 M) )))   DI 9   I  E$=$=IIIIDODIILLLLDODI ! r#Nrlrmc.|dSr:)rrs r$rz ReduceLROnPlateau.on_train_beginws r#c| |j|vrtjddS |jjj}t |ts'tjdt|ddSn0#t$r#}tjd|dYd}~dSd}~wwxYw||j}t |ttfr |d}nt |tj r|}ndS|r|xjdzc_d|_|||jr||_d|_n|s|xjdz c_|j|jkr|jj}|t+j|jkr||jz}t3||j}||jj_|jdkrEt6jjdkrt?d|j dzd|d|j!|_d|_|xj dz c_ dS) Nz;Monitor of ReduceLROnPlateau should be loss or metric name.z)Expected learning_rate be float, bug got .zAThere are something wrong when get learning_rate from optimizer: rrz Epoch z.: ReduceLROnPlateau reducing learning rate to )"rr(r)r,rrr;r'type ExceptionrFrGr4r5 in_cooldownrrr,rrget_lrr*float32rrrrrrrrrrwr)rUrlrer9old_lrnew_lrs r$rzReduceLROnPlateau.on_eval_endzs <4>@LLLMMMMMMLLL''''MMMMMM.4B&55555n))))))))r#r) NNNNNr)r)rNNr)r*r+r,r-rr.rr.rr.r/rrrr0rr1r2rr3r4r5r6r7)( __future__rr4rrr(typingrnumpyr*r paddle.utilsr progressbarrrr r collection.abcr r typing_extensionsr r,rrr!rr&__all__rPr7rXr<rBrDrIr=rcrr"r#r$rsn#""""""   ######$$$$$$..........11111111++++++&'ABMBBBB)  7;! $%,(((((VL%L%L%L%L%L%L%L%^y y y y y y y y xM=M=M=M=M=HM=M=M=`F"F"F"F"F"hF"F"F"Re<e<e<e<e<(e<e<e