vjEddlZddlmZddlmZmZmZddlmZm Z ddl Z ddl Z ddl m Z ddlmZddlmZde eeeffd ZeGd d ZeGd d ZeGddZedGddeeeZdZdede eeedffdZdS)N)deepcopy) dataclassfieldfields)ListUnion)CliArgumentParser)Config)DEFAULT_DATASET_NAMESPACEvaluesct|tr|dn|}i}|pgD]R}t|dkr(|d\}}t |||<S|S)N,r=) isinstancestrsplitlenstrip parse_value)r pairs_paramskvkeyvalues q/lsinfo/ai/hellotax_ai/data_center/backend/venv/lib/python3.11/site-packages/modelscope/trainers/training_args.pyset_flatten_valuers!+FC!8!8 DFLL   fEGkr** rxxzz??a   XXc]] U"5)) NcxeZdZUedddiZeed<edddiZeed<edddiZeed <eddd iZ eed <eddd iZ eed <edddiZ eed<ee ddiZ eed<ee ddiZeed<edddiZeed<dS) DatasetArgsNhelpzNThe dataset name used for training, can be an id in the datahub or a local dirdefaultmetadatatrain_dataset_namezOThe subset name used for evaluating, can be an id in the datahub or a local dirval_dataset_namez.The subset name used for training, can be Nonetrain_subset_namez0The subset name used for evaluating, can be Noneval_subset_namezThe split of train dataset train_splitzThe split of val dataset val_splitz'The dataset namespace used for trainingtrain_dataset_namespacez)The dataset namespace used for evaluatingval_dataset_namespaceaThe json file to parse all datasets from, used in a complex dataset scenario,the json format should be like: [ { "dataset": { # All args used in the MsDataset.load function "dataset_name": "xxx", ... }, # All columns used, mapping the column names in each dataset in same names. "column_mapping": { "text1": "sequence1", "text2": "sequence2", "label": "label", }, # float or str, float means to split the dataset into train/val, # or just str(train/val) "split": 0.8, } ] dataset_json_file)__name__ __module__ __qualname__rr$r__annotations__r%r&r'r(r)r r*r+r,rrrrs $e  \       "E  ]    c   #U D    s   !5 F    OS   u 0    K   U .    Is   $)5) = $ $ $ S   "') ? " " " 3   #U      s     rrceZdZUeddddZeed<edddiZeed<eddd iZeed <edd d dZ eed <dS) ModelArgsNzThe task code to be usedtaskr cfg_noder!r zA model id or model dirmodelzthe revision of modelmodel_revisionzJThe mode type, if load_model_config is False, user need to fill this fieldz model.type model_type) r-r.r/rr4rr0r7r8r9r1rrr3r3js.     D#    -    E3    % +    NC   e Y$     J     rr3ceZdZUedddiZeed<edddd Zeed <ed d d d Zeed<edddd Z e ed<edddd Z e ed<edddd Z eed<ed ddd Z eed<edddd Ze ed<eddd d Ze ed!<ed"d#d$d Zeed%<ed&d'd(d Zeed)<ed*d+d,d Zeed-<ed.d/d0d Zeed1<ed2d3d4d Zeed5<ed6d7d8ed9Zeed:<ed6d;dd?d@gdAdBZeedC<ed ddDiZeedE<ed"dFdGdHZeedI<ed>dJdKgdAdLZeedM<edNdOdPdHZeedQ<ed6dRdSdHZeedT<ed>dUdVgdAdLZ eedW<edNdXdYdHZ!eedZ<edd[d\dHZ"e ed]<ed6d^d_dHZ#eed`<edadbdcdHZ$eedd<ed6dedfdHZ%eedg<edNdhdidHZ&eedj<eddkdldHZ'e edm<ed6dndodHZ(eedp<ed6dqdrdHZ)eeds<edtdudvdHZ*e edw<edxdydzdHZ+eed{<eddkd|dHZ,e ed}<ed6dnd~dHZ-eed<ed6dqddHZ.eed<edtduddHZ/e ed<edxdyddHZ0eed<d6S) TrainArgs*r zThe random seedr!seedz#train.dataloader.batch_size_per_gpuz:The `batch_size_per_gpu` argument for the train dataloader)r6r per_device_train_batch_sizerz train.dataloader.workers_per_gpuz7The `workers_per_gpu` argument for the train dataloadertrain_data_workerFztrain.dataloader.shufflez/The `shuffle` argument for the train dataloader train_shuffleztrain.dataloader.drop_lastz1The `drop_last` argument for the train dataloadertrain_drop_lastz(evaluation.dataloader.batch_size_per_gpuz9The `batch_size_per_gpu` argument for the eval dataloaderper_device_eval_batch_sizez%evaluation.dataloader.workers_per_gpuz6The `workers_per_gpu` argument for the eval dataloadereval_data_workerzevaluation.dataloader.shufflez.The `shuffle` argument for the eval dataloader eval_shufflezevaluation.dataloader.drop_lastz0The `drop_last` argument for the eval dataloadereval_drop_lastztrain.max_epochszThe training epochs max_epochsz./train_targetztrain.work_dirz%The directory to save models and logswork_dirg-C6 ?ztrain.optimizer.lrz"The learning rate of the optimizerlrLinearLRztrain.lr_scheduler.typezThe lr_scheduler type in torch lr_schedulerAdamWztrain.optimizer.typez+The optimizer type in PyTorch, like `AdamW` optimizerNztrain.optimizerzThe optimizer params)r6r cfg_setteroptimizer_paramsztrain.lr_schedulerzThe lr scheduler paramslr_scheduler_paramsby_epochz&train.lr_scheduler.options.lr_strategyzThe lr decay strategy)rRby_stepno)r6r choices lr_strategyzThe local rank local_rankz+The interval of iter of logging informationztrain.logging.intervalr5logging_intervalz5Eval strategy, can be `by_epoch` or `by_step` or `no`zevaluation.period.eval_strategy)r r6rU eval_strategyz Eval intervalzevaluation.period.interval eval_intervalzThe metric name for evaluationzevaluation.metrics eval_metricsz>Checkpointing strategy, can be `by_epoch` or `by_step` or `no`z%train.checkpoint.period.save_strategy save_strategyz9The interval of epoch or iter of saving checkpoint periodz train.checkpoint.period.interval save_intervalz;Save the checkpoint(if it's the best) after the evaluation.ztrain.checkpoint.best.save_bestsave_best_checkpointz%The metric used to measure the model.z train.checkpoint.best.metric_keymetric_for_best_modelmaxzDThe rule to measure the model with the metric, can be `max` or `min`ztrain.checkpoint.best.rulemetric_rule_for_best_modelzBThe max number of checkpoints to keep, older ones will be deleted.z*train.checkpoint.period.max_checkpoint_nummax_checkpoint_numzGThe max number of best checkpoints to keep, worse ones will be deleted.z(train.checkpoint.best.max_checkpoint_nummax_checkpoint_num_bestz$Push to hub after each checkpointingz#train.checkpoint.period.push_to_hub push_to_hubz     L#   #U:;     d    K;     L#    % h9     NC   #U-;     d   #U/<     s     rr;F)initcTeZdZUedddiZeed<dZd dZd d Z d Z dS) TrainingArgsFr zeUse the configuration of the model, default will only use the parameters in the CLI and the dataclassr!use_model_configc t||_t|D],}|j|vr!t ||j||j-i|_dSN)listkeys manual_argsrnamesetattr _unknown_args)selfkwargsfs r__init__zTrainingArgs.__init__se .. 6 6AvaffQVn555rNc,t|}||\}}d|D}i}tdt|dD]7}t ||dz|||dd<8t |}|xj|jz c_|j |t| D](\}} |!t||rt||| )|S)zcConstruct a TrainingArg class by the parameters of CLI. Returns: Self c"g|] }|dvd|v | S))\ z --local-rank=r1).0items r z*TrainingArgs.parse_cli..s7   <''O4,G,G ,G,G,GrrrZ-)r parse_known_argsrangerrreplacevarsr{r~updateritemshasattrr}) r parser_argsparserargsunknown_unknowni args_dictrrs r parse_clizTrainingArgs.parse_clis, #4((// << g  $   q#g,,** P PA4?A4O4OHWQZ''R00 1 1JJ  F.. !!(+++"9--3355 * *JC74#5#5c5))) rc $t}tj}||j}t |D]}|jd}|jdpd}|a|j|jvs|sPt|tr|g}|D]5}| ||t||ji6t||j||j<| |j ||fS)zyConvert the TrainingArgs to the `Config` Returns: The Config, and extra parameters in dict. Nr6rOc|Srxr1xs rz(TrainingArgs.to_config..sAr)r addictDictrvrr#getr|r{rrmerge_from_dictgetattrr~)rignore_default_configcfgrrr6rO_nodes r to_configzTrainingArgs.to_configs- hhKMM ($($9 ! : :Az~~j11H 55F++J#6T---5J-!(C00.$,:!)HH++"JJwtQV/D/D$E$EFHHHH%,D!&$9$9 !&!! D.///I~rcLt|}|D]}|j|kr|cSdSrx)rr|)rr_fieldsrs r get_metadatazTrainingArgs.get_metadatas:,,  Av}}trrx) r-r.r/rrvrqr0rrrrr1rrrurus#U  P    d      .8rrucddlm}g}g}t|d5}tj|}|D]}|jdi|d}|j}|dfd|D} ddlm } ddlm } dd lm } fd |j D} i}| D]I}t|d | r | |d j||d<8|d ||d<J| |}|d | | |d}|d}t|t$r:|dvsJ|dkr||Y||pt|t(rd|cxkrd ksnJ||}||d||d dddn #1swxYwYddlm}||||fS)aW The filename format: [ { "dataset": { "dataset_name": "xxx", ... }, "column_mapping": { "text1": "sequence1", "text2": "sequence2", "label": "label", } "usage": 0.8, } ] r) MsDatasetrdatasetcolumn_mappingcg|]}|v| Sr1r1)rcolumn keep_columnss rrz+build_dataset_from_file..s*!F,4N4N4N4N4Nr)Features)Value) ClassLabelc(g|]}|dv |S)rr1)rrrs rrz+build_dataset_from_file..s.qt|7K7K7K7K7KrrZc|Srxr1rs rrz)build_dataset_from_file.."s!r)remove_columnsfeaturesusage)trainvalr) train_sizetestN)concatenate_datasetsr1) modelscoperopenjsonload to_hf_dataset column_namesrzdatasetsrrrrrrdtypemaprename_columnsrappendrrtrain_test_splitr)filenamer train_seteval_setrds_jsondsr all_columnsrrrrr new_featuresrds_dictrrs @rbuild_dataset_from_filers>$%$$$$$IH h  %1)A,,# 1# 1B$in55r)}55CCEEG!.K./4466L%0N * ) ) ) ) ) & & & & & & + + + + + +"+1133HL . .adJ//.).qtz):):L1&&)*1L1&&#8L11Lkk -%"''(6~b9I6J'K'K wKE%%% 1 00000G##$$W----OOG,,,,!%//AAMMMMMMMMMA!22e2DD  !12220000G# 1%1%1%1%1%1%1%1%1%1%1%1%1%1%1%1N.-----   * *,@,@,J,J JJsG;H$$H(+H(rreturnc&dddddddd}||vr||Sd|vsd|vr*|ddddStjd|rt|Stjd|rt |S|S) NTF)TruetrueFalsefalseNonenonenull"'rz^\d+$z4[+-]?(?=\d*[.eE])(?=\.?\d)\d*\.?\d*(?:[eE][+-]?\d+)?)rrematchrprr)r const_maps rrr6sI  }}S"%%--c2666 (E " "5zz I  U|| r)rcopyr dataclassesrrrtypingrrrr'modelscope.trainers.cli_argument_parserr modelscope.utils.configr modelscope.utils.constantr rrrr3r;rurrrrqrr1rrrs 0000000000 EEEEEE******??????eCcN3 L L L L L L L  L ^          6 ^ ^ ^ ^ ^ ^ ^  ^ B  IIIII; 9IIIX>K>K>KBsuS%t%;<r