o "Æi(#ã@sjdZddlmZmZmZmZddlZddlZddlm Z ddl m Z e  e ¡ZdZGdd„dƒZeƒZdS) zR Entity Extractor using LLM Extracts entities from document text using OpenAI API é)ÚListÚDictÚAnyÚOptionalN)ÚOpenAI)Úsettingsug你是一个专业的知识图谱构建助手。请从以下文档中æå–关键实体和关系。 文档标题：{title} 文档内容： {content} 请按照以下JSONæ ¼å¼è¿”回： {{ "entities": [ {{ "name": "实体å称", "type": "PERSON|ORG|LOCATION|CONCEPT|TERM|LAW|DATE|MONEY|PERCENT", "description": "简短æ述（å¯é€‰ï¼‰", "salience": 0.0-1.0 }} ], "relations": [ {{ "source": "实体1å称", "target": "实体2å称", "type": "RELATED_TO|PART_OF|INSTANCE_OF|DEFINED_BY|REGULATES", "confidence": 0.0-1.0 }} ] }} è¦æ±‚： 1. åªæå–é‡è¦å®žä½“（至少出现2次或具有关键æ„义） 2. æ¯ä¸ªæ–‡æ¡£æå–5-20个实体 3. 关系è¦æœ‰æ˜Žç¡®è¯­ä¹‰ 4. salience表示实体在文档中的é‡è¦æ€§ï¼ˆ0-1） 5. confidence表示关系的置信度（0-1） 6. 实体类型说明： - PERSON: 人å - ORG: 组织机构 - LOCATION: 地点 - CONCEPT: 概念术语 - TERM: 专业术语 - LAW: 法律法规 - DATE: 日期时间 - MONEY: é‡‘é¢ - PERCENT: 百分比 请åªè¿”回JSON，ä¸è¦åŒ…å«å…¶ä»–文字。c@sÜeZdZdZdd„Zddd„Z  dded ed ed eee eee fffd d „Z de eee fd e eee ffdd„Z de eee fd e eee ffdd„Z dde eeefded e eee ffdd„ZdS)ÚEntityExtractorz- Entity extraction service using LLM cCsdS©N©)Úselfr r úK/lsinfo/ai/hellotax_ai/base_platform/app/services/graph/entity_extractor.pyÚ__init__BszEntityExtractor.__init__Nc Csè|rbzEddlm}m}| |¡ |¡ |jdk|jdk|jdk|jdk|j   d¡¡  ¡}|rF|j rFt |j  ¡|j jp=|j jd}||jfWSWntya}zt d|›¡WYd}~nd}~wwtjrrt tjtjd}|tjfSdS) z£ Get model configuration from database or environment Returns: Tuple of (client, model_name) or (None, None) if not available r)Ú ModelProviderÚModelÚchatTN)Úapi_keyÚbase_urlz#Failed to get model from database: )NN)Úapp.models.providerrrÚqueryÚjoinÚfilterÚtypeÚenabledÚ configuredrÚisnotÚfirstÚproviderrÚ get_api_keyrÚdefault_base_urlÚcodeÚ ExceptionÚloggerÚwarningrÚOPENAI_API_KEYÚOPENAI_BASE_URLÚGRAPH_ENTITY_EXTRACTION_MODEL)r Ú db_sessionrrÚmodelÚclientÚer r r Ú_get_model_configFs< ûú þ €€ÿþ z!EntityExtractor._get_model_configé ÚtitleÚcontentÚ max_lengthÚreturnc Csz| |¡\}}|st d¡ggdœSt|ƒ|kr*|d|…d}t d|›d¡zTtj||d}|jjj |dd d œd |d œgd d ddid}|j dj j } t  | ¡} | |  dg¡¡} | |  dg¡¡} t dt| ƒ›dt| ƒ›d|›d¡| | dœWSt jyž} zt d| ›¡ggdœWYd} ~ Sd} ~ wty¼} zt d| ›¡ggdœWYd} ~ Sd} ~ ww)ai Extract entities and relations from document text Args: title: Document title content: Document content max_length: Maximum content length to process db_session: Optional database session to get model config Returns: Dictionary with 'entities' and 'relations' lists z ÚLAWÚORGÚDATEÚTERMÚMONEYÚPERSONÚCONCEPTÚPERCENTÚLOCATIONÚnamerrWÚsalienceçà?Ú descriptionÚ)rZrr]r[©rGÚupperÚfloatrÚGRAPH_MIN_ENTITY_CONFIDENCEÚappendÚstrip)r r0Ú valid_typesÚ validatedÚentityÚ entity_typer[r r r rF³s$      üz"EntityExtractor._validate_entitiesr1cCs’hd£}g}|D]>}| d¡r| d¡r| d¡sq|d ¡}||vr&d}t| dd¡ƒ}|tjkr4q| |d ¡|d ¡||dœ¡q|S) zÑ Validate and filter relations based on confidence threshold Args: relations: Raw relation list from LLM Returns: Filtered and validated relation list >ÚPART_OFÚ REGULATESÚ DEFINED_BYÚ RELATED_TOÚ INSTANCE_OFÚsourceÚtargetrrlÚ confidencer\)rnrorrpr_)r r1rerfÚrelationÚrel_typerpr r r rHØs$      üz#EntityExtractor._validate_relationséÚ documentsÚmax_concurrentcCsPg}|D]!}|j| dd¡| dd¡d}| | d¡|d|ddœ¡q|S) zÿ Extract entities from multiple documents Args: documents: List of documents with 'title' and 'content' max_concurrent: Maximum concurrent API calls Returns: List of extraction results r,r^r-r2Úidr0r1)Ú document_idr0r1)rPrGrc)r rtruÚresultsÚdocrOr r r Úextract_entities_batchýs  þ ýz&EntityExtractor.extract_entities_batchr )r+N)rs)Ú__name__Ú __module__Ú __qualname__Ú__doc__r r*ÚstrÚintrrrrPrFrHrzr r r r r=s2 ,ûþýü ú*E*%(ýþýür)r~ÚtypingrrrrrDÚloggingÚopenairÚ app.configrÚ getLoggerr{r!r>rÚentity_extractorr r r r Ús   . `