o di @sJddlZddlZddlZddlZddlZddlZddlZddlm Z ddl m Z m Z m Z mZmZmZddlZddlmZmZmZmZmZmZdZde fdd Zd d Zd d ZddZddZddZddZ ddZ!ddZ"ddZ#ddZ$ddZ%d d!Z& " # d-d$e e'ee fd%e(d&e(d'ee egefd(ef d)d*Z)d+d,Z*dS).N)MutableMapping)AnyCallableDictIterableOptionalTuple)is_accelerate_availableis_auto_gptq_availableis_datasets_availableis_diffusers_available"is_sentence_transformers_availableis_timm_available__DUMMY_OPTIMUM_USER__ dictionarycCsNg}|D]\}}|}t|tr|t|q|||fqt|S)z= Flatten a nested dictionaries as a flat dictionary. )items isinstancerextend flatten_dictappenddict)rrkvnew_keyri/lsinfo/ai/hellotax_ai/llm_service/venv_embed/lib/python3.10/site-packages/optimum/utils/testing_utils.pyr*s rcCttd|S)zu Decorator marking a test that requires accelerate. These tests are skipped when accelerate isn't installed. ztest requires accelerate)unittest skipUnlessr  test_caserrrrequire_accelerate8r"cCr)zs Decorator marking a test that requires auto-gptq. These tests are skipped when auto-gptq isn't installed. ztest requires auto-gptq)rrr r rrrrequire_auto_gptq?r#r$cCs&tjrdnd}t|dkd|S)z8Decorator marking a test that requires CUDA and PyTorch.cudacpuztest requires CUDA)torchr% is_availablerr)r! torch_devicerrrrequire_torch_gpuFsr*cCs(ddl}|}td|dkd|S)zNDecorator marking a test that requires ROCMExecutionProvider for ONNX Runtime.rNROCMExecutionProviderz#test requires ROCMExecutionProvider) onnxruntimeget_available_providersrr)r!ort providersrrrrequire_ort_rocmMs r0cCs(tjdd}|durtd|S|S)zG Decorator marking a test that requires huggingface hub token. HF_AUTH_TOKENNz>test requires hf token as `HF_AUTH_TOKEN` environment variableosenvirongetrskip)r!hf_tokenrrrrequire_hf_tokenXsr8cCs>tjdd}tjdd}|dus|durtd|S|S)zB Decorator marking a test that requires sigopt API token. SIGOPT_API_TOKENNSIGOPT_PROJECTzMtest requires an environment variable `SIGOPT_API_TOKEN` and `SIGOPT_PROJECT`r2)r!sigopt_api_tokenhas_sigopt_projectrrr require_sigopt_token_and_projectdsr=cCsbtjddu}tjddur-zd}tjtjddgdddWn tjy,d}Ynw|o0|S)Nzonnxruntime.training torch_ortTz-mztorch_ort.configureF)shellcheck) importlibutil find_spec subprocessrunsys executableCalledProcessError)is_ort_train_availableis_torch_ort_configuredrrris_ort_training_availablersrKcCr)z Decorator marking a test that requires onnxruntime-training and torch_ort correctly installed and configured. These tests are skipped otherwise. z:test requires torch_ort correctly installed and configured)rrrKr rrrrequire_ort_trainings rLcCr)Nztest requires diffusers)rrr r rrrrequire_diffusersrMcCr)Nztest requires timm)rrrr rrr require_timmrNrOcCr)Nz#test requires sentence-transformers)rrrr rrrrequire_sentence_transformersrNrPcCr)Nztest requires datasets)rrr r rrrrequire_datasetsrNrQFT parameters yield_dict add_test_namefilter_params_funcreturnc cstj|D]P}|dur|t|}|durqddd|D}|durFi}t|D] \}}||||<q/|durB||d<|Vq|durQ|gt|nt|} | VqdS)aV Generates an iterable over the grid of all combinations of parameters. Args: `parameters` (`Dict[str, Iterable[Any]]`): Dictionary of multiple values to generate a grid from. `yield_dict` (`bool`, defaults to `False`): If True, a dictionary with all keys, and sampled values will be returned. Otherwise, return sampled values as a list. `add_test_name` (`bool`, defaults to `True`): Whether to add the test name in the yielded list or dictionary. filter_params_func (`Optional[Callable[[Tuple], Tuple]]`, defaults to `None`): A function that can modify or exclude the current set of parameters. The function should take a tuple of the parameters and return the same. If a parameter set is to be excluded, the function should return an empty tuple. N_cSsg|]}t|qSr)str).0paramrrr sz#grid_parameters..T test_name) itertoolsproductvalueslistjoin enumeratekeys) rRrSrTrUparamsr\res_dictikey returned_listrrrgrid_parameterss" ricCsLtj|r"tj|r$tjdkrtd|dSt|dSdSdS)z Remove a directory and its content. This is a cross-platform solution to remove a directory and its content that avoids the use of `shutil.rmtree` on Windows. Reference: https://github.com/python/cpython/issues/107408 ntz rmdir /S /Q N)r3pathexistsisdirnamesystemshutilrmtree)dirpathrrrremove_directorys  rs)FTN)+importlib.utilrAr]r3rprDrFrcollections.abcrtypingrrrrrrr'r r r r rrUSERrr"r$r*r0r8r=rKrLrMrOrPrQrXboolrirsrrrrsP         '