uj(ddlZddlZddlZddlmZmZddlmZddlm Z m Z ddl Z GddeZ Gdd e Z Gd d e ZGd d e ZiZGddeZdS)N)ABCMetaabstractmethod)Path) GeneratorUnionc eZdZdZedefdZedefdZedede ee fddfdZ e d dede ee fd eddfd Z dS) StoragezAbstract class of storage. All backends need to implement two apis: ``read()`` and ``read_text()``. ``read()`` reads the file as a byte stream and ``read_text()`` reads the file as texts. filepathcdSNselfr s f/lsinfo/ai/hellotax_ai/data_center/backend/venv/lib/python3.11/site-packages/modelscope/fileio/file.pyreadz Storage.read cdSr r rs r read_textzStorage.read_textrrobjreturnNcdSr r rrr s rwritez Storage.writerrutf-8encodingcdSr r rrr rs r write_textzStorage.write_text!s rr) __name__ __module__ __qualname____doc__rstrrrbytesrrrrr rrr r s S   ^  #   ^   %T *: t   ^ $+   "39- ! 04   ^   rr ) metaclassc eZdZdZdeeefdefdZ d deeefdedefdZ dedeeefdd fd Z d dedeeefdedd fd Z e j deeefdeeeefd d ffd Zd S) LocalStoragezLocal hard disk storager rc~t|d5}|}dddn #1swxYwY|S)Read data from a given ``filepath`` with 'rb' mode. Args: filepath (str or Path): Path to read data. Returns: bytes: Expected bytes object. rbNopenr)rr fcontents rrzLocalStorage.read,sx(D ! ! QffhhG               s 266rrct|d|5}|}dddn #1swxYwY|S)>Read data from a given ``filepath`` with 'r' mode. Args: filepath (str or Path): Path to read data. encoding (str): The encoding format used to open the ``filepath``. Default: 'utf-8'. Returns: str: Expected text reading from ``filepath``. rrNr-)rr rr/ value_bufs rrzLocalStorage.read_text9s~(C( 3 3 3 !qI ! ! ! ! ! ! ! ! ! ! ! ! ! ! !s 488rNc.tj|}|r5tj|stj|dt |d5}||ddddS#1swxYwYdS))Write data to a given ``filepath`` with 'wb' mode. Note: ``write`` will create a directory if the directory of ``filepath`` does not exist. Args: obj (bytes): Data to be written. filepath (str or Path): Path to write data. Texist_okwbNospathdirnameexistsmakedirsr.r)rrr r>r/s rrzLocalStorage.writeJs'//(++  027>>'22 0 K$ / / / / (D ! ! Q GGCLLL                  s'B  BBc2tj|}|r5tj|stj|dt |d|5}||ddddS#1swxYwYdS)Write data to a given ``filepath`` with 'w' mode. Note: ``write_text`` will create a directory if the directory of ``filepath`` does not exist. Args: obj (str): Data to be written. filepath (str or Path): Path to write data. encoding (str): The encoding format used to open the ``filepath``. Default: 'utf-8'. Tr8wr4Nr;)rrr rr>r/s rrzLocalStorage.write_text\s '//(++  027>>'22 0 K$ / / / / (C( 3 3 3 q GGCLLL                  s)B  BBc#K|VdSz$Only for unified API and do nothing.Nr rs r as_local_pathzLocalStorage.as_local_pathss rr )r!r"r#r$rr%rr&rrrr contextlibcontextmanagerrrFr rrr)r))s[!! U39- %    #*!#t),.1"%T *:t*$+"39-!04.C !"'0c4i0@$0L&Mrr)c eZdZdZdZdZejdede e ee fddffdZ de d e ee fddfd Z dded e ee fd eddfd ZdS) HTTPStoragezHTTP and HTTPS storage.c`tj|}||jSr )requestsgetraise_for_statusr0rurlr3s rrzHTTPStorage.reads+ L   yrc`tj|}||jSr )rLrMrNtextrOs rrzHTTPStorage.read_texts* L   v rr rNc#.K tjd}|||||jVt j|jdS#t j|jwxYw)ahDownload a file from ``filepath``. ``as_local_path`` is decorated by :meth:`contextlib.contextmanager`. It can be called with ``with`` statement, and when exists from the ``with`` statement, the temporary path will be released. Args: filepath (str): Download a file from ``filepath``. Examples: >>> storage = HTTPStorage() >>> # After existing from the ``with`` clause, >>> # the path will be removed >>> with storage.get_local_path('http://path/to/file') as path: ... # do something here FdeleteNtempfileNamedTemporaryFilerrclosenamer<removerr r/s rrFzHTTPStorage.as_local_path& +5999A GGDIIh'' ( ( ( GGIII&LLL Iaf     BIaf     AA99BrrPc td)Nz&write is not supported by HTTP StorageNotImplementedError)rrrPs rrzHTTPStorage.writes!"JKKKrrrc td)Nz+write_text is not supported by HTTP Storager`)rrrPrs rrzHTTPStorage.write_texts" 9;; ;rr )r!r"r#r$rrrGrHr%rrrrFr&rrr rrrJrJ|s!!  $-eCI.>d.J$K4LLU39%5L$LLLL $+;;;c4i(;!;04;;;;;;rrJc eZdZdZddZdZddZejde de e e e fddffd Z d ede e e fddfd Z dd e de e e fd e ddfd ZdS) OSSStoragez OSS storage.Nc td)Nz3OSSStorage.__init__ to be implemented in the futurer`)ross_config_files r__init__zOSSStorage.__init__s! ACC Crc td)Nz/OSSStorage.read to be implemented in the futurer`rs rrzOSSStorage.reads! =?? ?rrc td)Nz4OSSStorage.read_text to be implemented in the futurer`)rr rs rrzOSSStorage.read_texts! BDD Drr rc#.K tjd}|||||jVt j|jdS#t j|jwxYw)agDownload a file from ``filepath``. ``as_local_path`` is decorated by :meth:`contextlib.contextmanager`. It can be called with ``with`` statement, and when exists from the ``with`` statement, the temporary path will be released. Args: filepath (str): Download a file from ``filepath``. Examples: >>> storage = OSSStorage() >>> # After existing from the ``with`` clause, >>> # the path will be removed >>> with storage.get_local_path('http://path/to/file') as path: ... # do something here FrTNrVr\s rrFzOSSStorage.as_local_pathr]r^rc td)Nz0OSSStorage.write to be implemented in the futurer`rs rrzOSSStorage.writes! >@@ @rrc td)Nz5OSSStorage.write_text to be implemented in the futurer`rs rrzOSSStorage.write_texts" CEE Err r )r!r"r#r$rgrrrGrHr%rrrrFr&rrr rrrdrds1CCCC ???DDDD$-eCI.>d.J$K4@@%T *:@t@@@@$+EEE"39-E!E04EEEEEErrdc >eZdZUeeeedZeed<e dZ e de de fdZ e ddee efde de fd Ze d e dee efdd fd Ze dd e de de dd fd Zejde deee efd d ffdZd S)File)osshttphttpslocal_prefix_to_storagect|tsJdt|d|vrd}n|d\}}|}|tjvs9Jd|dt tj|tvr!t j|t|<t|S)Nz uri should be str type, but got z://rrzUnsupported uri z, valid prefixs: ) isinstancer%typesplitrnrslistkeys G_STORAGES)uri storage_typeprefix_s r _get_storagezFile._get_storages# O O N499 N N O O   "LL %((IFA!Lt6666 6s 6 6D+002233 6 6766 z ) )'+'>|'L'N'NJ| $,''rr{rc`t|}||S)r+)rnrr)r{storages rrz File.reads)##C((||C   rrrc`t|}||S)r2)rnrr)r{rrs rrzFile.read_texts+##C((  %%%rrNcbt|}|||S)r7)rnrr)rr{rs rrz File.write#s+##C((}}S#&&&rcbt|}|||S)rB)rnrr)rr{rrs rrzFile.write_text2s-##C((!!#s+++rc#Kt|}||5}|VddddS#1swxYwYdSrE)rnrrF)r{r local_paths rrFzFile.as_local_pathCs##C((  " "3 ' ' :                     sAA Ar )r!r"r#rdrJr)rsdict__annotations__ staticmethodrr%r&rrrrrrrGrHrrFr rrrnrns   ((\(( !# !% ! ! !\ ! & &uS$Y' &3 &S & & &\ & '5 'uS$Y/ 'D ' ' '\ ',,,#,,4,,,\, 39U39-=tT-I#Jrrn)rGr<rWabcrrpathlibrtypingrrrLr r)rJrdrzobjectrnr rrrsu ''''''''########         8PPPPP7PPPf1;1;1;1;1;'1;1;1;h4E4E4E4E4E4E4E4En ^^^^^6^^^^^r