o :/iC @sddlmZmZddlmZmZmZmZddlm Z ddl m Z m Z m Z ddlmZddlmZddlmZddlmZerIdd lZdd lmZn ed ed ZeZeeZed eed Zddde e!efd Bde e!efd Bde"edfde e!effddZ#ddde e!efd Bde e!efd Bde"edfde e!effddZ$eddGdddZ%eddGdddZ&d S)) dataclassfield) TYPE_CHECKINGAnyLiteralTypeVar)VLLMValidationError) EmbedsPrompt TextPrompt TokensPrompt) init_logger)merge_media_io_kwargs) TokenizerLike) LazyLoaderN)ChatTemplateContentFormatOptiontorch_Sz torch.Tensor)Nauto unset_valuesdefaults overridesr.returncs2|duri}|dur i}|fdd|DBS)Ncsi|] \}}|vr||qSr).0kvrrb/lsinfo/ai/hellotax_ai/llm_service/venv_vllm/lib/python3.10/site-packages/vllm/renderers/params.py (sz merge_kwargs..)items)rrrrrr merge_kwargss r cCs|duri}|dur i}t|}|D])\}}||vrq||vr9t||tr9t|tr9t||||d||<q|||<q|S)Nr)dictr isinstancerecursively_merge_kwargs)rrrmergedrrrrrr#+s   r#T)frozenc @seZdZUdZdZedBed< dZded< ee dZ e ee fed< dZ e ee ee ffdBed < dZ e ee fdBed <   dd e ee fdBd e ee ee ffdBd e ee fdBfddZde ee ffddZdS) ChatParamsz4Configuration to control how to parse chat messages.N chat_templaterrchat_template_content_format)default_factorychat_template_kwargsmedia_io_kwargsmm_processor_kwargsdefault_chat_template_kwargsdefault_media_io_kwargsdefault_mm_processor_kwargscCs>|s|s|s|St|j|jt||jt||jt||jdS)N)r'r(r*r+r,) r&r'r(r r*r r+r#r,)selfr-r.r/rrr with_defaultsZs.zChatParams.with_defaultsrcCst|jt|jddS)z9The arguments to pass to `tokenizer.apply_chat_template`.F)r' return_dict)r r*r!r'r0rrrget_apply_chat_template_kwargsxs z)ChatParams.get_apply_chat_template_kwargs)NNN)__name__ __module__ __qualname____doc__r'str__annotations__r(rr!r*rr+r,r1r4rrrrr&Gs.    r&c@seZdZUdZedBed< dZeed< dZedBed< dZedBed< dZ e ddBed < d Z e ed < d Z e ed < d Ze ed< dZeed< dZeed< dZeed< ededBfddZd2ddZdefddZdeeeffddZdedBdedefddZdedBdedefd d!ZdedBdedefd"d#ZdedBd$edefd%d&ZdedBd'e de fd(d)Z!dedBd'e de fd*d+Z"dedBd'e de fd,d-Z#dedBd'e de fd.d/Z$dedBd$e%e&Bde%e&Bfd0d1Z'dS)3TokenizeParamsz3Configuration to control how prompts are tokenized.Nmax_total_tokensrmax_output_tokenspad_prompt_tokenstruncate_prompt_tokens)leftrighttruncation_sideF do_lower_caseTadd_special_tokensneeds_detokenizationmax_total_tokens_parammax_output_tokens_paramtruncate_prompt_tokens_paramrcCs|jdurdS|j|jS)z'Maximum allowed number of input tokens.N)r<r=r3rrrmax_input_tokenss  zTokenizeParams.max_input_tokensc Cs|j}|j}|j}|j}|dur.|dur.||kr.t|jd|d|jd|d|j|d|durT|durV||krXt|jd|d|jd|jd|d |j|ddSdSdS) N=zcannot be greater than z=max_total_tokens=z%. Please request fewer output tokens. parametervaluez cannot be greater than z - z = z+. Please request a smaller truncation size.)r<r=rIr?rrGrFrH)r0r<r=rIr?rrr __post_init__sB zTokenizeParams.__post_init__tokenization_kwargsc Ks|d|j}|d|j}|d|j}|d|j}|d|j}|d|j}|dd}rD|dkr9|}n |dvr@d}n||d<|d d} r^| d vrS|}n | d vrZd}n| |d <|rftd ||j } t | | duss|durud n| ||||j |||dS)N max_lengthr>r?rCrDrEpadding)F do_not_pad truncation)T longest_first)Fdo_not_truncatez_The following tokenization arguments are not supported by vLLM Renderer and will be ignored: %sr)r<r=r>r?rBrCrDrE) poprIr>r?rCrDrEloggerwarningr<r;rB) r0rOrPr>r?rCrDrErQrSr<rrr with_kwargssXzTokenizeParams.with_kwargscCsf|j}|dur|dkr|j}n|dur|jdur|jd}|jdkr)td|jdSt|du||jdS)z,The arguments to pass to `tokenizer.encode`.Nrr@F)rSrD)rSrPrD)r?rIrBr!rD)r0rPrrrget_encode_kwargs s  z TokenizeParams.get_encode_kwargs tokenizertextc Csx|j}|dur |S|jdur:|dur:||j}t||kr:td|jd|jdt|d|d|d dt|d |S) z0Apply length checks to prompt text if necessary.N'This model's maximum context length is tokens. However, you requested ( output tokens and your prompt contains z characters (more than z* characters, which is the upper bound for zf input tokens). Please reduce the length of the input prompt or the number of requested output tokens. input_textrK)rIr?max_chars_per_tokenlenrr<r=)r0r\r]rImax_input_charsrrr_text_len_check:s,   zTokenizeParams._text_len_checkcCs|jr|S|S)z,Apply lowercase to prompt text if necessary.)rClower)r0r\r]rrr_text_lowercaseUszTokenizeParams._text_lowercasecCs |j|jfD]}|||}q|S)z$Apply all validators to prompt text.)rerg)r0r\r] validatorrrr_validate_textYs  zTokenizeParams._validate_textpromptcCs|||d|d<|S)z Ensure that the prompt meets the requirements set out by this config. If that is not possible, raise a `VLLMValidationError`. This method is run before tokenization occurs. rj)rir0r\rjrrrapply_pre_tokenizationcs z%TokenizeParams.apply_pre_tokenizationtokenscCsn|j}|dur|dkr|j}|dus|t|kr|S|dur"tdt|ts+td||jg|t|S)z,Apply padding to prompt tokens if necessary.Nrz1Cannot pad tokens when `skip_tokenizer_init=True`z&Cannot pad tokens for embedding inputs)r>rIrc ValueErrorr"list pad_token_id)r0r\rm pad_lengthrrr_token_paddingrs zTokenizeParams._token_paddingcCs|j}|dur|dkr|j}|dus|t|kr|S|dkr$|ddS|jp/|dur.|jnd}|dkr;|| dS|d|S)z/Apply truncation to prompt tokens if necessary.Nrr@)r?rIrcrB)r0r\rmrPsiderrr_token_truncations  z TokenizeParams._token_truncationc Cs||j}|dur |St||krr?rBrrCboolrDrErFr9rGrHpropertyrIrNrrYr!r[rrergrir rlrrrrtrzr{r r r~rrrrr;sd          ";   r;)' dataclassesrrtypingrrrrvllm.exceptionsr vllm.inputsr r r vllm.loggerr vllm.multimodal.media.connectorr vllm.tokenizersrvllm.utils.import_utilsrrvllm.entrypoints.chat_utilsrglobalsobjectr5rWrorrr!r9tupler r#r&r;rrrrsN           8