jUddlmZmZmZmZddlZddlZddlmZddl m Z m Z ddl m Z ddlmZmZddlmZmZmZddlmZd eeefd efd Zd Zeed <e eejedddddddddddgdf deedeeedeedeededede deedeedeedeeeded ee fdZdS))IOListOptionalUnionN)add_chunking_strategy)Elementprocess_metadata) VALID_PARSERS)FileTypeadd_metadata_with_filetype) exactly_oneget_last_modified_date get_last_modified_date_from_file)partition_htmlcontentsreturncZt|tr|dS|S)Nzutf-8) isinstancebytesdecode)rs c/lsinfo/ai/hellotax_ai/base_platform/venv/lib/python3.11/site-packages/unstructured/partition/md.pyoptional_decoders+(E""(w''' OmdDETECTION_ORIGINFTautofilenamefiletexturlinclude_page_breaksinclude_metadataparsermetadata_filenamemetadata_last_modifiedchunking_strategy languagesdetect_language_per_elementc |d}t||||d} |Zt|} t|d5}t|}dddn #1swxYwYn|1t |} t|}n~||t j|}|jstd|j |j dd}| dstd |d |j }tj|d g }t||||d ||p| | | t  S)aPartitions a markdown file into its constituent elements Parameters ---------- filename A string defining the target filename path. file A file-like object using "rb" mode --> open(filename, "rb"). text The string representation of the markdown document. url The URL of a webpage to parse. Only for URLs that return a markdown document. include_page_breaks If True, the output will include page breaks if the filetype supports it. include_metadata Determines whether or not metadata is included in the output. parser The parser to use for parsing the markdown document. If None, default parser will be used. metadata_last_modified The last modified date for the document. languages User defined value for `metadata.languages` if provided. Otherwise language is detected using naive Bayesian filter via `langdetect`. Multiple languages indicates text could be in either language. Additional Parameters: detect_language_per_element Detect language per element instead of at the document level. N)rrrr utf8)encodingzURL return an error: z Content-Typez text/markdownz)Expected content type text/markdown. Got .tables) extensionsr) rr!r"r# source_formatr$r%r'r(detection_origin)r ropenrreadrrequestsgetok ValueError status_codeheaders startswithrmarkdownrr)rrrr r!r"r#r$r%r&r'r(kwargslast_modification_datefresponse content_typehtmls r partition_mdrBs^ |4SAAAA!!7!A!A (V , , , -"16688,,D - - - - - - - - - - - - - - -  !A$!G!Gtyy{{++ <$${ MKX5IKKLL L'++NB?? &&77 KLKKK }  Txj 9 9 9D  /)+5O9O$?)    s"A**A.1A.)typingrrrrr;r4unstructured.chunkingrunstructured.documents.elementsrr unstructured.documents.xmlr unstructured.file_utils.filetyper r unstructured.partition.commonr rrunstructured.partition.htmlrstrrrr__annotations__MDboolrBrrrOsa,,,,,,,,,,,,,777777EEEEEEEE444444QQQQQQQQ 766666eCJ/C #HK((" $ %! '+,0'+&,X(-UUsmU 2e9 U 3-U # U  U  U U }U%SMU }US "U"&U ']UUU)(UUUr