o ~ri4B @sdZddlZddlZddlZddlmZddlmZddlm Z e e Z Gdddej ZGd d d Zd d ZGd ddZ    ddedBdedBdedBdedBfddZddZdS)zACollection of utils to be used by backbones and their components.N) repo_exists)logging)maybe_install_capturing_hooksc@seZdZdZdZdS) BackboneTypetimm transformersN)__name__ __module__ __qualname__TIMM TRANSFORMERSrri/lsinfo/ai/hellotax_ai/llm_service/venv_embed/lib/python3.10/site-packages/transformers/backbone_utils.pyrsrcseZdZdZdedBdedBfddZddZed d Zej dee fd d Zed d Z e j de e dfee Bfdd Z fddZZS)BackboneConfigMixinzv A Mixin to support handling the `out_features` and `out_indices` attributes for the backbone configurations. out_featuresN out_indicescs|_t|tr t|n|_jj}}|dur1|dur1tjdg}jdg}n#|durC|durCfdd|D}n|durT|durTfdd|D}||__dS)ap Sets output indices and features to new values and aligns them with the given `stage_names`. If one of the inputs is not given, find the corresponding `out_features` or `out_indices` for the given `stage_names`. Args: out_features (`list[str]`, *optional*): The names of the features for the backbone to output. Defaults to `config._out_features` if not provided. out_indices (`list[int]` or `tuple[int]`, *optional*): The indices of the features for the backbone to output. Defaults to `config._out_indices` if not provided. Nrcsg|]}j|qSr) stage_namesindex).0layerselfrr DszJBackboneConfigMixin.set_output_features_output_indices..cg|]}j|qSrrridxrrrrF) _out_features isinstancetuplelist _out_indicesverify_out_features_out_indiceslenr)rrrrrr"set_output_features_output_indices(s z6BackboneConfigMixin.set_output_features_output_indicescs,jdur tdjduretjtfstdtjtfddjDr7tdjdjtjttjkrKtdjjfd d jD}kretd |djj durtj tsztd tj t fd dj D}tfdd|Drtdjdj t|tt|krdj }||j krd|dnd7}t||t t |krdd t t |j dddD}td|dj jdurj durtjtj krtdjfdd j DkrtddSdSdS)z_ Verify that out_indices and out_features are valid for the given stage_names. Nz2Stage_names must be set for transformers backbonesz out_features must be a list got c3s|]}|jvVqdSNrrfeatrrr VszFBackboneConfigMixin.verify_out_features_out_indices..z.out_features must be a subset of stage_names: z got z2out_features must not contain any duplicates, got csg|] }|jvr|qSrr r)rrrr]szGBackboneConfigMixin.verify_out_features_out_indices..z@out_features must be in the same order as stage_names, expected z out_indices must be a list, got c3s*|]}|dkr|tjn|VqdS)rN)r&rrrrrr+gs(c3s&|]}|ttjvr|VqdSr()ranger&rrrrrr+hs$z2out_indices must be valid indices for stage_names z, got z1out_indices must not contain any duplicates, got z(equivalent to z))cSsg|]\}}|qSrr)r_rrrrrqscSs|dS)Nrr)xrrrrszEBackboneConfigMixin.verify_out_features_out_indices..)keyz?out_indices must be in the same order as stage_names, expected zHout_features and out_indices should have the same length if both are setcrrrrrrrr{rzQout_features and out_indices should correspond to the same stages if both are set) r ValueErrorr r!r#typeanyr&setr$r"sortedzip)r sorted_featspositive_indicesmsgsorted_negativerrrr%LsV     z3BackboneConfigMixin.verify_out_features_out_indicescC|jSr(r,rrrrr~z BackboneConfigMixin.out_featurescCs|j|dddS) Set the out_features attribute. This will also update the out_indices attribute to match the new out_features. Nrr)r'rrrrrrscCr=r()r$rrrrrr>zBackboneConfigMixin.out_indices.cCs&|durt|n|}|jd|ddS) Set the out_indices attribute. This will also update the out_features attribute to match the new out_indices. Nr@)r#r'rrrrrrscs.t}|dd|d<|dd|d<|S)z Serializes this instance to a Python dictionary. Override the default `to_dict()` from `PreTrainedConfig` to include the `out_features` and `out_indices` attributes. r Nrr$r)superto_dictpop)routput __class__rrrEs zBackboneConfigMixin.to_dict)r r r __doc__r#r'r%propertyrsetterstrrr"intrE __classcell__rrrHrr#s" $2   rcstfdd}|S)ak Wrapper to filer out `hidden_states` as backbones tend to always use them to get their feature maps, i.e. they also always output `hidden_states`. This controls for user-defined behavior again. NOTE: We assume a `can_return_tuple` decorator to be applied before so that we always expect a dict like object to remove the hidden states. csX|dt|jdd}|g|Ri|}|s*dd|D}t|di|}|S)Noutput_hidden_statesFcSsi|] \}}|dvr||qS) hidden_statesrrkvrrr z@filter_output_hidden_states..wrapper..r)getgetattrconfigitemsr4)rargskwargsrPrGfiltered_output_dataforward_functionrrwrappers z,filter_output_hidden_states..wrapper) functoolswraps)r_r`rr^rfilter_output_hidden_statess rccseZdZUdZedBed<dZeed<d!fdd Zfdd Z d!d d Z d!d d Z e ddZ e jdeefddZ e ddZejdeeeeBfddZe ddZe ddZddZ   d"dedBdedBdedBfdd ZZS)# BackboneMixinN backbone_typeThas_attentionsreturncs~tj|i||dd}|durtj|_ntj|_|jtjkr*|j|ddS|jtjkr6|dSt d|jd)z Method to initialize the backbone. This method is called by the constructor of the base class after the pretrained model weights have been loaded. timm_backboneN)backbonezbackbone_type z not supported.) rD__init__rFrr rer _init_timm_backbone_init_transformers_backboner3)rr[r\rhrHrrrjs     zBackboneMixin.__init__cstt|dS)a Override `post_init` to always install capturing hooks, as backbone will ALWAYS capture outputs. We need to do it in `post_init`, as modules need to be already instantiated. It avoids some mixups with `torch.compile`, as the first hook installation will need/create a graph break, which can clash with external user call such as `model = torch.compile(model...)`. N)rD post_initrrrHrrrms zBackboneMixin.post_initcCst|jdd}t|jdd}dd|jjD|_dd|jjD|_t|jj}|j}|dur=||kr=t d|d|durN||jkrNt d |d |j|j_|j ||dS) zj Initialize the backbone model from timm. The backbone must already be loaded to backbone rNrcSg|]}|dqS)modulerrstagerrrrz5BackboneMixin._init_timm_backbone..cSrn)num_chsrrprrrrrrz!Config has `out_features` set to z which doesn't match `out_features` from backbone's feature_info. Please check if your checkpoint has correct out features/indices saved.z Config has `stage_names` set to z which doesn't match `stage_names` from backbone's feature_info. Please check if your checkpoint has correct `stage_names` saved.) rXrY feature_infoinfor num_featuresr#r module_namer3r')rriout_features_from_configstage_names_from_configrrrrrrks      z!BackboneMixin._init_timm_backbonecCs|jj|_|jd|_dSr()rYrr%rvrrrrrls   z)BackboneMixin._init_transformers_backbonecC|jjSr()rYr rrrrrzBackboneMixin.out_featuresrcC ||j_dS)r?N)rYrrArrrr cCrzr()rYr$rrrrr r{zBackboneMixin.out_indicesrcCr|)rBN)rYrrCrrrrr}csfddtjDS)Ncsi|] \}}|j|qSr)rv)rirqrrrrUsz6BackboneMixin.out_feature_channels..) enumeraterrrrrout_feature_channelssz"BackboneMixin.out_feature_channelscsfddjDS)Ncrr)r)rnamerrrr rz*BackboneMixin.channels..)rrrrrchannelsszBackboneMixin.channelscsT|js |dd|jtjkr#tt|jj fdd| D}||i|S)Noutput_attentionscsi|] \}}|vr||qSrrrR signaturerrrU'rVz>BackboneMixin.forward_with_filtered_kwargs..) rfrFrerr dictinspectrforward parametersrZ)rr[r\rrrforward_with_filtered_kwargs"s   z*BackboneMixin.forward_with_filtered_kwargsrPr return_dictcCstd)Nz7This method should be implemented by the derived class.)NotImplementedError)r pixel_valuesrPrrrrrr*szBackboneMixin.forward)rgN)NNN)r r r rer__annotations__rfboolrjrmrkrlrKrrLr#rMrr"rNrrrrrOrrrHrrds<    !     rddefault_backbonedefault_config_typedefault_config_kwargstimm_default_kwargscKsxddlm}ddlm}|dd}|di} |ddur$|dn|} |dd|durK|rK| durK|durK| sK|d d d| i|}||fS| dur|durt| rt|| \} } || d } | | | d i| }||fS|d d d| i| }||fS|dur|durt d |d |pi}||d i|}||fSt |t r|d }||} | |}||fS)Nr)PreTrainedConfig)CONFIG_MAPPINGuse_timm_backboneTbackbone_kwargsriuse_pretrained_backbonerh model_typezG`backbone_config` is `None`. Initializing the config with the default `z` vision config.r)configuration_utilsr models.autorrFrWrget_config_dictupdateloggerrur!r from_dict)backbone_configrrrrr\rrrrri config_dictr/ config_classbackbone_model_typerrr%consolidate_backbone_kwargs_to_config4sH          rcCs@ddlm}t|dd}|dur|j|d}|S|j|d}|S)a> Loads the backbone model from a config object. If the config is from the backbone model itself, then we return a backbone model with randomly initialized weights. If the config is from the parent model of the backbone model itself, then we load the pretrained backbone weights if specified. r) AutoBackbonerN)rY)rrrX from_config)rYrrrirrr load_backbonefs   r)NNNN)rJenumrarhuggingface_hubrutilsrutils.output_capturingr get_loggerr rEnumrrrcrdrMrrrrrrrs6    } 2