o di @sddlZddlZddlmZddlmZmZddlZddlm Z ddl m Z ddl m Z mZddlmZdd lmZmZmZmZmZd e d eejd ed e de f ddZdededefddZdS)N)partial)CallableList) GraphModule) AutoConfig)ConfigParallelExecutionCtx)build_parallel_pass_pipeline)MetaAwareMethodsPatcherdownload_model_from_hfinitialize_parameter_metamove_model_to_devicetry_collect_weight_map graph_moduleexample_inputsctxconfigreturncCs2||_t}||||d}|jd7_||_|S)N)rrrr)rr compile_timeslast_optimized_graph_module)rrrr pass_pipelinerl/lsinfo/ai/hellotax_ai/llm_service/venv_embed/lib/python3.10/site-packages/optimum/fx/parallelization/api.pyparallelize_backend#s rmodel parallel_ctxcOs|dd}|dd}|dd}|dd}t}t|D]\} } | |jvr5t|| | || q!tj|} | sHt |||||d} n|} t j | f|d d d |\} }| j }t td |d }|sot||| |_|d dd}}|dur||}t|| g|Ri|}|Wdn1swY|durt|t||jdt|tt||d}tj|d |d}|S)a API for automatic model parallelism through Pytorch FX. Args: model (`str`): Model to parallelize, a model id on the Huggingface Hub or path to a local directory containing config and weights of the model. parallel_ctx (`ParallelExecutionCtx`): Parallel execution context containing process groups the current process belongs to. *model_args (`Any`): Additional postional arguments for intializing the model if a model id is passed. revision (`str`, defaults to `main`): Model revision for weights downloading if a model id is passed. cache_dir (`Optional[str]`, defaults to `None`): Cache directory to store downloaded weights. Defaults to None. local_files_only (`bool`, defaults to `False`): Whether to use local files only, will avoid downloading from remote if set to `True`. skip_load_weights (`bool`, defaults to `False`): Whether to skip loading weights from disk to model. **kwargs (`Dict[str, Any]`): Addtional keyword arguments for overriding fields in parallel config, model config and `Model.__init__`. revisionmain cache_dirNlocal_files_onlyFskip_load_weights)model_name_or_pathrrr skip_download_weightsT)rr return_unused_kwargs transformersr torch_dtype)device)rr) fullgraphbackend)poprdictitems__dict__setattrospathisdirr rfrom_pretrained architecturesgetattr importlib import_moduler weight_map_set_default_torch_dtyper evaltorchset_default_dtypercurrent_devicer rrcompile)rr model_argskwargsrrr r!parallel_configkvis_local hf_folder model_config model_arch model_clsr& dtype_origr)rrrparallelize_model.sZ            rI)r5r/ functoolsrtypingrrr:torch.fxrr%rcorerr passesr utilsr r r rrTensorrstrrIrrrrs8