o di<@sdZddlZddlZddlZddlZddlZddlmZmZm Z m Z m Z ddl m Z ddlmZddlmZddlmZddlmZmZmZmZd d lmZd dl mZeeZGd d d eZdS) zConfiguration base class.N)AnyDictListTupleUnion)version)PretrainedConfig) __version__)custom_object_save) cached_file download_urlextract_commit_hash is_remote_url)loggingc @seZdZdZdZdZeddZddee e j fde fdd Z ed ee d e fd d Zedee e j fd eee efee efffddZedee e j fd eee efee efffddZedee efd dfddZd ee effddZdS) BaseConfigz Base class for configuration classes that need to respect the same API than PretrainedConfig but with a different configuration file name. z config.jsoncCst|jdddS)N.rz (.*)\.json)recompileFULL_CONFIGURATION_FILEsplit)clsri/lsinfo/ai/hellotax_ai/llm_service/venv_embed/lib/python3.10/site-packages/optimum/configuration_utils.py_re_configuration_file.sz!BaseConfig._re_configuration_fileFsave_directory push_to_hubc Kstj|rtd|dtj|dd|r\|dd}|d|tjjd}|j|fi|}| d d}| d d}|durWt d t |durQt d ||d <|}||}|jdurht|||d tj||j} |j| ddtd| |r|j|||||ddSdS)a4 Save a configuration object to the directory `save_directory`, so that it can be re-loaded using the [`~PretrainedConfig.from_pretrained`] class method. Args: save_directory (`str` or `os.PathLike`): Directory where the configuration JSON file will be saved (will be created if it does not exist). push_to_hub (`bool`, *optional*, defaults to `False`): Whether or not to push your model to the Hugging Face model hub after saving it. You can specify the repository you want to push to with `repo_id` (will default to the name of `save_directory` in your namespace). kwargs: Additional key word arguments passed along to the [`~utils.PushToHubMixin.push_to_hub`] method. zProvided path (z#) should be a directory, not a fileT)exist_okcommit_messageNrepo_iduse_auth_tokentokennThe `use_auth_token` argument is deprecated and will be removed soon. Please use the `token` argument instead.LYou cannot use both `use_auth_token` and `token` arguments at the same time.)config)use_diffzConfiguration saved in )rr")ospathisfileAssertionErrormakedirspoprsep _create_repogetwarningswarn FutureWarning ValueError_get_files_timestamps _auto_classr join CONFIG_NAME to_json_fileloggerinfo_upload_modified_files) selfrrkwargsrrr!r"files_timestampsoutput_config_filerrrsave_pretrained3s:        zBaseConfig.save_pretrainedconfiguration_filesreturnc Csi}|}|D]}||}|dur|d}|||<qt|}|j}tt} |D]}t|| kr<||}q.|S|S)a Get the configuration file to use for this version of transformers. Args: configuration_files (`List[str]`): The list of available configuration files. Returns: `str`: The configuration file to use. Nr) rsearchgroupssortedkeysr7rparser ) rrAconfiguration_files_mapr file_namerCvavailable_versionsconfiguration_fileoptimum_versionrrrget_configuration_filems       z!BaseConfig.get_configuration_filepretrained_model_name_or_pathcKslt|}|j|fi|\}}d|vr|d|d<d|vr2||d}|j|fd|i|\}}||fS)a From a `pretrained_model_name_or_path`, resolve to a dictionary of parameters, to be used for instantiating a [`PretrainedConfig`] using `from_dict`. Parameters: pretrained_model_name_or_path (`str` or `os.PathLike`): The identifier of the pre-trained checkpoint from which we want the dictionary of parameters. Returns: `Tuple[Dict, Dict]`: The dictionary(ies) that will be used to instantiate the configuration object. _commit_hashrA_configuration_file)copydeepcopy_get_config_dictrN)rrOr=original_kwargs config_dictrLrrrget_config_dicts   zBaseConfig.get_config_dictcKs |dd}|dd}|dd}|dd}|dd}|dd}|dd} |d d} |d d} |d d } |d d} |dd}|dd}|durbtdt|dur`td|}| durktdd|d}| durx| |d<t|}tj |}tj tj | |r|}d}nFt |r|}t|}n;|d|j}zt||||||| ||| | |d }t||}Wntytytd|d|d|dwz||}Wntjtfytd|dw|rtd|||fStd|d |||fS)!N cache_dirforce_downloadFresume_downloadproxiesr!r"local_files_onlyrevisiontrust_remote_code subfolder_from_pipeline _from_autorPr#r$TzgThe argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored.r%) file_typefrom_auto_classusing_pipelinerQ) rXrYr[rZr\r" user_agentr]r_rPz!Can't load the configuration of 'z'. If you were trying to load it from 'https://huggingface.co/models', make sure you don't have a local directory with the same name. Otherwise, make sure 'z2' is the correct path to a directory containing a z filez"It looks like the config file at 'z' is not a valid JSON file.zloading configuration file z from cache at )r,r0r1r2r3r9warningstrr'r(isdirr)r6rr r7r r EnvironmentError Exception_dict_from_json_filejsonJSONDecodeErrorUnicodeDecodeErrorr:)rrOr=rXrYrZr[r!r"r\r]r^r_ from_pipelinerd commit_hashrfis_localresolved_config_filerLrVrrrrTs                    zBaseConfig._get_config_dictrVrc Ks2|dd}|dd|ddd|vr d|vr |d|d<|di|}t|dr7dd |jD|_d |vrbd |vrb|d }|d durM|d ng}t||krbtd |d |d dg}|D]\}} t||rt||| |dkr||qh|D]}||dqt ||r||fS|S)ao Instantiates a [`PretrainedConfig`] from a Python dictionary of parameters. Args: config_dict (`Dict[str, Any]`): Dictionary that will be used to instantiate the configuration object. Such a dictionary can be retrieved from a pretrained checkpoint by leveraging the [`~PretrainedConfig.get_config_dict`] method. kwargs (`Dict[str, Any]`): Additional parameters from which to initialize the configuration object. Returns: [`PretrainedConfig`]: The configuration object instantiated from those parameters. return_unused_kwargsFrbNrarP pruned_headscSsi|] \}}t||qSr)int).0keyvaluerrr *sz(BaseConfig.from_dict.. num_labelsid2labelzYou passed along `num_labels=z(` with an incompatible id to label map: zX. Since those arguments are inconsistent with each other, you should remove one of them. torch_dtyper) r,hasattrruitemslenr3setattrappendr9r:) rrVr=rtr%r{r| to_removerxryrrr from_dicts<          zBaseConfig.from_dictcCs^t|j}t|jdr|jj|d<d|vr|d=d|vr |d=t|d<t|d<|||S)z Serializes this instance to a Python dictionary. Returns: `Dict[str, Any]`: Dictionary of all the attributes that make up this configuration instance. model_typer5rPtransformers_versionrM) rRrS__dict__r~ __class__rtransformers_version_strr dict_torch_dtype_to_str)r<outputrrrto_dictFs    zBaseConfig.to_dictN)F)__name__ __module__ __qualname____doc__r7r classmethodrrrhr'PathLikeboolr@rrNrrrrWrTrrrrrrr%s0 :     `6r)rrRrmr'rr0typingrrrrr packagingr transformersrr r!transformers.dynamic_module_utilsr transformers.utilsr r r rutilsr get_loggerrr9rrrrrs