j!/ ddlmZmZmZmZmZmZddlZddlm Z ddl m Z m Z ddl mZddlmZddlmZddlmZdd lmZmZdd lmZmZmZmZdd lmZerdd lm Z e eej!e dddddd didddd ddd ddgd dfdee"deee#dee"dee"dee"de$de$dee"e"fde$dedee"de$dee"dee"de$dee"d eee"d!e$d"ee"d#ed$ee f*d%Z%ddd dddgd dfde"dee"deee#de$dee"dee"d eee"d!e$d"ee"d$ee fd&Z&d*d)Z'dS)+)IO TYPE_CHECKINGAnyDictListOptionalN)add_chunking_strategy)Elementprocess_metadata) HTMLDocument) VALID_PARSERS) read_txt_file)convert_file_to_html_text)FileTypeadd_metadata_with_filetype)document_to_element_list exactly_oneget_last_modified_date get_last_modified_date_from_file)apply_lang_metadata)DocumentLayoutFTautofilenamefiletexturlencodinginclude_page_breaksinclude_metadataheaders ssl_verifyparser source_formathtml_assemble_articlesmetadata_filenamemetadata_last_modifiedskip_headers_and_footerschunking_strategy languagesdetect_language_per_elementdetection_originkwargsreturnc H| |dkr|s|s|sgSt||||d}|(t|}tj|| || }n|;t |}t ||\}}tj|| | }n|'t|}tj|| | }n|tj |||}|j std|j |j d d}|d std |d tj|j| }|rt#|}t%t't)|fd|| p|| r| nd|d|||S)aiPartitions an HTML document into its constituent elements. Parameters ---------- filename A string defining the target filename path. file A file-like object using "r" mode --> open(filename, "r"). text The string representation of the HTML document. url The URL of a webpage to parse. Only for URLs that return an HTML document. encoding The encoding method used to decode the text input. If None, utf-8 will be used. include_page_breaks If True, includes page breaks at the end of each page in the document. include_metadata Optionally allows for excluding metadata from the output. Primarily intended for when partition_html is called in other partition bricks (like partition_email) headers The headers to be used in conjunction with the HTTP request if URL is set. ssl_verify If the URL parameter is set, determines whether or not partition uses SSL verification in the HTTP request. parser The parser to use for parsing the HTML document. If None, default parser will be used. source_format The source of the original html. If None we will return HTMLElements but for example partition_rst will pass a value of 'rst' so that we return Title vs HTMLTitle metadata_last_modified The last modified date for the document. skip_headers_and_footers If True, ignores any content that is within
or