-jddlmZddlZddlmZddlmZejr ddl m Z ddl m Z  ddlmZddlmZn#e$r ed wxYw dddZ d d!dZdS)") annotationsN) RunEvalConfigEvaluatorChain)Dataset) TestDataset)Client)LangSmithNotFoundErrorzZPlease install langsmith to use this feature. You can install it via pip install langsmithdatasetr dataset_namestr dataset_descreturnLangsmithDatasetc6t} ||}td|d|d#t$rR|||dgdg|}t d|jd |j|cYSwxYw) a Uploads a new dataset to LangSmith, converting it from a TestDataset object to a pandas DataFrame before upload. If a dataset with the specified name already exists, the function raises an error. Parameters ---------- dataset : TestDataset The dataset to be uploaded. dataset_name : str The name for the new dataset in LangSmith. dataset_desc : str, optional A description for the new dataset. The default is an empty string. Returns ------- LangsmithDataset The dataset object as stored in LangSmith after upload. Raises ------ ValueError If a dataset with the specified name already exists in LangSmith. Notes ----- The function attempts to read a dataset by the given name to check its existence. If not found, it proceeds to upload the dataset after converting it to a pandas DataFrame. This involves specifying input and output keys for the dataset being uploaded. r Dataset z already exists in langsmith. []question ground_truth)dfname input_keys output_keys descriptionzCreated a new dataset 'z'. Dataset is accessible at ) r read_dataset ValueErrorr upload_dataframe to_pandasprintrurl)r r rclients f/lsinfo/ai/hellotax_ai/base_platform/venv/lib/python3.11/site-packages/ragas/integrations/langsmith.pyupload_datasetr%sDXXF%%<%@@ N| N NG N N N    "   ))  """|'($ *    ]gl ] ]PWP[ ] ]    s,<ABBFllm_or_chain_factoryt.Anyexperiment_namet.Optional[str]metricst.Optional[list]verboseboolt.Dict[str, t.Any]c(t} ||}n!#t$rtd|dwxYw|ddlm}m}m} m} ||| | g}d|D}t|} | ||| || } | S) aP Evaluates a language model or a chain factory on a specified dataset using LangSmith, with the option to customize metrics and verbosity. Parameters ---------- dataset_name : str The name of the dataset to use for evaluation. This dataset must exist in LangSmith. llm_or_chain_factory : Any The language model or chain factory to be evaluated. This parameter is flexible and can accept a variety of objects depending on the implementation. experiment_name : Optional[str], optional The name of the experiment. This can be used to categorize or identify the evaluation run within LangSmith. The default is None. metrics : Optional[list], optional A list of custom metrics (functions or evaluators) to be used for the evaluation. If None, a default set of metrics (answer relevancy, context precision, context recall, and faithfulness) are used. The default is None. verbose : bool, optional If True, detailed progress and results will be printed during the evaluation process. The default is False. Returns ------- Dict[str, Any] A dictionary containing the results of the evaluation. Raises ------ ValueError If the specified dataset does not exist in LangSmith. See Also -------- Client.read_dataset : Method to read an existing dataset. Client.run_on_dataset : Method to run the evaluation on the specified dataset. Examples -------- >>> results = evaluate( ... dataset_name="MyDataset", ... llm_or_chain_factory=my_llm, ... experiment_name="experiment_1_with_vanila_rag", ... verbose=True ... ) >>> print(results) {'evaluation_result': ...} Notes ----- The function initializes a client to interact with LangSmith, validates the existence of the specified dataset, prepares evaluation metrics, and runs the evaluation, returning the results. Custom evaluation metrics can be specified, or a default set will be used if none are provided. rrz9 not found in langsmith, make sure it exists in langsmithNr)answer_relevancycontext_precisioncontext_recall faithfulnessc,g|]}t|Sr).0ms r$ zevaluate..s 222Q~a  222)custom_evaluators)r r& evaluationr, project_name) r rr r ragas.metricsr0r1r2r3rrun_on_dataset) r r&r(r*r,r#_r0r1r2r3 eval_configruns r$evaluaterBPsDXXF   \  : : !    ^| ^ ^ ^                $%6 nU22'222G!K   !1$   C Js 'A)r )r rr rrrrr)NNF) r rr&r'r(r)r*r+r,r-rr.) __future__rtypingtlangchain.smithrragas.integrations.langchainr TYPE_CHECKINGlangsmith.schemasrrragas.testset.generatorr langsmithr langsmith.utilsr ImportErrorr%rBr5r9r$rNs)""""""))))))777777?4======333333      6666666 +d  BD66666x(, $ ddddddds 8A