o
di C @ s d Z ddlZddlZddlZddlZddlmZmZ ddlm Z ddl
m Z m Z m
Z
mZ ddlmZ ddlmZ ddlmZmZmZ dd lmZ d
d lmZ d
d lm Z e rrdd
lm Z m Z m Z m Z m!Z!m"Z" ee!e e e f Z#ed Z$e%e&Z'dZ(G dd deZ G dd de Z)dS )z2Base class for optimized model inference wrapping. N)ABCabstractmethod)Path)
TYPE_CHECKINGListOptionalUnion)HfApi)HUGGINGFACE_HUB_CACHE)
AutoConfigPretrainedConfigadd_start_docstrings)http_user_agent ) TasksManager) CONFIG_NAME)FeatureExtractionMixinImageProcessingMixinPreTrainedModelProcessorMixinSpecialTokensMixinTFPreTrainedModel)r r a
Instantiate a pretrained model from a pre-trained model configuration.
Args:
model_id (`Union[str, Path]`):
Can be either:
- A string, the *model id* of a pretrained model hosted inside a model repo on huggingface.co.
Valid model ids can be located at the root-level, like `bert-base-uncased`, or namespaced under a
user or organization name, like `dbmdz/bert-base-german-cased`.
- A path to a *directory* containing a model saved using [`~OptimizedModel.save_pretrained`],
e.g., `./my_model_directory/`.
export (`bool`, defaults to `False`):
Defines whether the provided `model_id` needs to be exported to the targeted format.
force_download (`bool`, defaults to `True`):
Whether or not to force the (re-)download of the model weights and configuration files, overriding the
cached versions if they exist.
use_auth_token (`Optional[Union[bool,str]]`, defaults to `None`):
Deprecated. Please use the `token` argument instead.
token (`Optional[Union[bool,str]]`, defaults to `None`):
The token to use as HTTP bearer authorization for remote files. If `True`, will use the token generated
when running `huggingface-cli login` (stored in `huggingface_hub.constants.HF_TOKEN_PATH`).
cache_dir (`Optional[str]`, defaults to `None`):
Path to a directory in which a downloaded pretrained model configuration should be cached if the
standard cache should not be used.
subfolder (`str`, defaults to `""`):
In case the relevant files are located inside a subfolder of the model repo either locally or on huggingface.co, you can
specify the folder name here.
config (`Optional[transformers.PretrainedConfig]`, defaults to `None`):
The model configuration.
local_files_only (`Optional[bool]`, defaults to `False`):
Whether or not to only look at local files (i.e., do not try to download the model).
trust_remote_code (`bool`, defaults to `False`):
Whether or not to allow for custom code defined on the Hub in their own modeling. This option should only be set
to `True` for repositories you trust and in which you have read the code, as it will execute code present on
the Hub on your local machine.
revision (`Optional[str]`, defaults to `None`):
The specific model version to use. It can be a branch name, a tag name, or a commit id, since we use a
git-based system for storing models and other artifacts on huggingface.co, so `revision` can be any
identifier allowed by git.
c @ s e Zd ZdS )r N)__name__
__module__ __qualname__ r r c/lsinfo/ai/hellotax_ai/llm_service/venv_embed/lib/python3.10/site-packages/optimum/modeling_base.pyr ^ s r c @ sV e Zd ZeZdZeZ d6ded ddde e
d fd d
Z d d Z e
d
d Z d7deeejf defddZe
dd Zdd Z d8dedede e de eeef de eeef d ef d d Zd9d ed efd!d"Zed#d$dddedfd%eeejf d&ed'ed(ed)ed*ed+ede eeef d efd,d-Zed#d$dddedfd.eee f d&ed'ed(ed)ed*ed+ede eeef d d fd/d0Z ed#d$dddedfd.eee f ded&ed'ed(ed)ed*ed+ede eeef d d fd1d2Z ee e ddd#d$dddedf d.eee f de e d3ed&ed'ed(ed)ed*ed+ede eeef d d fd4d5Z!dS ):OptimizedModeloptimized_modelNmodelModelTconfigr
preprocessors
PreprocessorTc C s || _ || _|p g | _d S N)r r! r" )selfr r! r" r r r __init__g s zOptimizedModel.__init__c O s | j |i |S r$ )forwardr% argskwargsr r r __call__n s zOptimizedModel.__call__c O t )zE
Forward pass of the model, needs to be overwritten.
NotImplementedErrorr( r r r r' q s zOptimizedModel.forwardFsave_directory push_to_hubc K st t j|rtd| d dS t j|dd | | | jD ]}|| q | | |r8| j
|fi |S dS )aX
Saves a model and its configuration file to a directory, so that it can be re-loaded using the
[`from_pretrained`] class method.
Args:
save_directory (`Union[str, os.PathLike]`):
Directory to which to save. Will be created if it doesn't exist.
push_to_hub (`bool`, *optional*, defaults to `False`):
Whether or not to push your model to the Hugging Face model hub after saving it.
Using `push_to_hub=True` will synchronize the repository you are pushing to with `save_directory`,
which requires `save_directory` to be a local clone of the repo you are pushing to if it's an existing
folder. Pass along `temp_dir=True` to use a temporary directory instead.
zProvided path (z#) should be a directory, not a fileNT)exist_ok) ospathisfileloggererrormakedirs _save_configr" save_pretrained_save_pretrainedr0 )r% r/ r0 r* preprocessorr r r r9 x s
z OptimizedModel.save_pretrainedc C r, )z
Saves a model weights into a directory, so that it can be re-loaded using the
[`from_pretrained`] class method.
r- r% r/ r r r r: s z OptimizedModel._save_pretrainedc C s | j | dS )z
Saves a model configuration into a directory, so that it can be re-loaded using the
[`from_pretrained`] class method.
N)r! r9 r< r r r r8 s zOptimizedModel._save_config
repository_idprivateuse_auth_tokentokenreturnc
C s |d urt dt |d urtd|}tt |d}|j||d|d t|D ]=\}}} | D ]5}
tj
||
} tj | \} }
z|j || tj
t
| |
d W q0 ty] Y q0 tye Y q0w q)d S )NznThe `use_auth_token` argument is deprecated and will be removed soon. Please use the `token` argument instead.zLYou cannot use both `use_auth_token` and `token` arguments at the same time.)
user_agentr@ T)repo_idr> r1 r@ )r@ rC path_or_fileobj path_in_repo)warningswarn
FutureWarning
ValueErrorr r create_repor2 walkr3 joinsplit upload_filegetcwdKeyError NameError)r% r/ r= r> r? r@ hf_apir3 subdirsfilesnamelocal_file_path_
hub_file_pathr r r r0 sB
zOptimizedModel.push_to_hubgit_user git_emailc
C s z0|durt jdddd|gt jt jddd |dur.t jdddd |gt jt jddd W dS W dS t jyB } zt|jd}~ww )
zI
Sets git user name and email (only in the current repo)
Ngitr! z--globalz user.nameTzutf-8)stderrstdoutcheckencodingz
user.email)
subprocessrunPIPECalledProcessErrorEnvironmentErrorr\ )r% rY rZ excr r r git_config_username_and_email s,
z,OptimizedModel.git_config_username_and_email mainconfig_name_or_path subfolderrevisionforce_downloadlocal_files_onlytrust_remote_code cache_dirc C s zt j|||||||d} W | S ty? }
z$|dkr0t j||||||d} td| d nt|
W Y d }
~
| S d }
~
ww )N) pretrained_model_name_or_pathrk ro rl r@ rj rn rg )rp rk ro rl r@ rn z1config.json not found in the specified subfolder z". Using the top level config.json.)r from_pretrainedOSErrorr5 info) clsri rj rk rl rm rn ro r@ r! er r r _load_config s:
zOptimizedModel._load_configmodel_idc
K t d )zROverwrite this method in subclass to define how to load your model from pretrainedr- )
rt rw rj rk rl rm rn ro r@ r* r r r _from_pretrained s z OptimizedModel._from_pretrainedc
K rx )zbOverwrite this method in subclass to define how to load your model from vanilla hugging face modelr- ) rt rw r! rj rk rl rm rn ro r@ r* r r r _export* s zOptimizedModel._exportexportc K s t |tr | }t|ddkr;td|dd d |dur4td| d|dd |d\}}tj||| ||
d \} }
|}| j | vr`t
| j d
| d | j d d
}tj |||| |
d}|dkrxt j
|||| |
d}|du rtjtj||r| j tkrtttj||v rtj
tj|||d}n)td| d| j||| |
|||d}nt |ttjfr| j||| |
|||d}|rt| dr| j}nt| dr| j}nt d| j }|d|||| ||
|||d | S )zT
Returns:
`OptimizedModel`: The loaded optimized model.
@ z Specifying the `revision` as @r zX is deprecated and will be removed in v1.23, please use the `revision` argument instead.Nz#The argument `revision` was set to z but will be ignored for )rj ro rk r@ z& not found in the specified subfolder z. Using the top level .rg )rj rk ro r@ timm)rn zconfig.json not found in z
local folder)rk ro r@ rl rj rn _from_transformersrz z[The `export` argument is set to `True`, but the class does not implement `_export` methods.) rw r! rk ro rl r@ rj rm rn r )
isinstancer as_posixlenrM r5 warningr get_model_files config_namers infer_library_from_modelr rq r2 r3 isdirrL r listdirr rr rv strPathLikehasattrr rz rI ry )rt rw r! r{ rj rk rl rm rn ro r@ r* all_filesrW
config_folder library_namefrom_pretrained_methodr r r rq > s
z OptimizedModel.from_pretrainedr$ )F)NNN)NN)"r r r r config_classbase_model_prefixr r r r r r& r+ r r' r r2 r boolr9 r: r8 r0 rf classmethodr
r rv r ry rz r
FROM_PRETRAINED_START_DOCSTRINGrq r r r r r b s<
&
+
(
r )*__doc__loggingr2 r` rF abcr r pathlibr typingr r r r huggingface_hubr huggingface_hub.constantsr
transformersr r r
transformers.utilsr exportersr utilsr r r r r r r r# r getLoggerr r5 r r r r r r s,
,