o :/i*@s8UddlmZddlmZmZmZmZmZddlm Z m Z m Z m Z ddl mZddlmZddlmZddlmZeGdd d Zee d d dGd d d eZee d d dGdddeZee d d dGdddeZeGdddeddZedZedZeeefZeed< eGdddZ dS))Mapping)AnyLiteral TypeAlias TypedDictfinal) ConfigDictFieldfield_validatormodel_validator) dataclass)config) safe_hash)AttentionBackendEnumc@s&eZdZUdZedddZeed<dS)BaseDummyOptionsz8Base options for generating dummy data during profiling.r)gecountN)__name__ __module__ __qualname____doc__r rint__annotations__rrc/lsinfo/ai/hellotax_ai/llm_service/venv_vllm/lib/python3.10/site-packages/vllm/config/multimodal.pyrs rforbid)extrac@sZeZdZUdZedddZedBed<edddZedBed<edddZ edBed<dS)VideoDummyOptionsz9Options for generating dummy video data during profiling.Nrgt num_frameswidthheight) rrrrr r!rrr"r#rrrrrs rc@sBeZdZUdZedddZedBed<edddZedBed<dS)ImageDummyOptionsz9Options for generating dummy image data during profiling.Nrrr"r#) rrrrr r"rrr#rrrrr$s r$c@s*eZdZUdZedddZedBed<dS)AudioDummyOptionsz9Options for generating dummy audio data during profiling.Nrrlength)rrrrr r&rrrrrrr%'s r%c@s.eZdZUdZeed< eed< eed<dS)MultiModalDummyOptionsBuiltinsz7Type annotations for modality types predefined by vLLM.imagevideoaudioN)rrrrr$rrr%rrrrr'.s r'F)total)weightsdata)shmlruMMDummyOptionsc @seZdZUdZdZeed< eedZ e ed< dZ eed< eedZ ee ee effed< dZee efdBed < ed d d Zeed < dZeed< edd d Zeed< dZeed< dZeed< dZedBed< dZeed< dZeed< eddddZedBed< eddde dee eee efBfde fd d!Z!eddde de eBdBdedBfd"d#Z"e#d$dd%d&Z$de fd'd(Z%d)e defd*d+Z&d,e'e efdee effd-d.Z(d/d0Z)dS)1MultiModalConfigz+Controls the behavior of multimodal models.Flanguage_model_only)default_factorylimit_per_promptenable_mm_embedsmedia_io_kwargsNmm_processor_kwargsr)defaultrmm_processor_cache_gbr/mm_processor_cache_typemm_shm_cache_max_object_size_mbmm_encoder_onlyr,mm_encoder_tp_modemm_encoder_attn_backendinterleave_mm_stringsskip_mm_profilinggg?)r9rltvideo_pruning_ratebefore)modevaluereturncCsi}|D]@\}}t|trd|i}|dkr!tdi|||<q|dkr/tdi|||<q|dkr=tdi|||<qtdi|||<q|S)Nrr)r(r*r)items isinstancerrr$r%r)clsrGoutkvrrr_validate_limit_per_prompts z+MultiModalConfig._validate_limit_per_promptcCsRt|tr|dkrtd|dust|tr|St|ts#Jdt|S)NXFORMERSzwAttention backend 'XFORMERS' has been removed (See PR #29262 for details). Please select a supported attention backend.zDmm_encoder_attn_backend must be a string or an AttentionBackendEnum.)rJstrupper ValueErrorr)rKrGrrr!_validate_mm_encoder_attn_backends  z2MultiModalConfig._validate_mm_encoder_attn_backendaftercCs"|jdkr|jtjkrtd|S)Nr.z]'mm_shm_cache_max_object_size_mb' should only be set when 'mm_processor_cache_type' is 'shm'.)r;r=r1rSselfrrr_validate_multimodal_configs z,MultiModalConfig._validate_multimodal_configcCs8|jdur |jjnd|jg}tt|dd}|S)a WARNING: Whenever a new field is added to this config, ensure that it is included in the factors list if it affects the computation graph. Provide a hash that uniquely identifies all the configs that affect the structure of the computation graph from input ids/embeddings to the final hidden states, excluding anything before input ids/embeddings and after the final hidden states. NF)usedforsecurity)r@namer?rrQencode hexdigest)rWfactorshash_strrrr compute_hashs zMultiModalConfig.compute_hashmodalitycCs(|jrdS|j|}|durdS|jS)z Get the maximum number of input items allowed per prompt for the given modality (backward compatible). rNr)r2r4getr)rWr` limit_datarrrget_limit_per_prompts  z%MultiModalConfig.get_limit_per_promptinference_kwargscCs|jpi}|t|BS)z Get the keyword arguments to pass to the multi-modal processor according to the extra arguments passed during inference. )r7dict)rWrdkwargsrrrmerge_mm_processor_kwargs s  z*MultiModalConfig.merge_mm_processor_kwargscCs|jduo |jdkS)Nr)rDrVrrris_multimodal_pruning_enabledsz.MultiModalConfig.is_multimodal_pruning_enabled)*rrrrr2boolrr rer4r0r5r6rQrr7objectr:floatr; MMCacheTyper=rr>r?MMEncoderTPModer@rrArBrDr classmethodrOrTr rXr_rcrrgrhrrrrr1Gsl   "               r1N)!collections.abcrtypingrrrrrpydanticrr r r pydantic.dataclassesr vllm.config.utilsr vllm.utils.hashingr#vllm.v1.attention.backends.registryrrrr$r%r'rmrlrerQr0rr1rrrrs.