ujddlZddlZddlZddlmZddlmZmZmZddl Z ddl m Z e Z dZ GddZdS)N)Path)DictOptionalUnion) get_loggerz.ms_upload_cachec eZdZdZdeeeffdZedede de defdZ dede de de e fd Zdede de d e fd Zd Zd ZdS)UploadHashCachearPersistent local hash cache for upload_folder resume. Stores SHA256 hashes keyed by (relative_path, mtime, size) to skip re-hashing unchanged files on retry/resume. Thread-safe for concurrent put() calls from multiple upload threads. Cache is stored as JSON at {folder_path}/.ms_upload_cache, co-located with the upload source for portability. cache_pathct||_i|_tj|_|dS)z}Initialize cache. Args: cache_path: Path to the cache file (typically folder/.ms_upload_cache). N)r _cache_path_cache threadingLock_lock_load)selfr s k/lsinfo/ai/hellotax_ai/data_center/backend/venv/lib/python3.11/site-packages/modelscope/hub/upload_cache.py__init__zUploadHashCache.__init__s;  ++') ^%%  rel_pathmtimesizereturnc|d|d|S)z*Build cache lookup key from file metadata.|)rrrs r _make_keyzUploadHashCache._make_key(s ++U++T+++rc||||}|j5|j|}dddn #1swxYwY|dS||d|ddS)aFReturn cached hash info or None if not cached / stale. Args: rel_path: Relative path of the file within the upload folder. mtime: File modification time (os.stat st_mtime). size: File size in bytes. Returns: Dict with file_hash and file_size, or None. N file_hash file_size)file_path_or_objrr )rrr get)rrrrkeyentrys rr"zUploadHashCache.get-snnXud33 Z ) )KOOC((E ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) =4!){+{+   sAA  A  hash_infoc||||}|d|dd}|j5||j|<ddddS#1swxYwYdS)aStore hash info for a file. Thread-safe. Args: rel_path: Relative path of the file. mtime: File modification time. size: File size in bytes. hash_info: Dict from compute_file_hash with file_hash and file_size. rr )rr N)rrr )rrrrr%r#r$s rputzUploadHashCache.putDsnnXud33";/";/  Z % %$DK  % % % % % % % % % % % % % % % % % %s AA A c |jjdd|j5t |j}dddn #1swxYwYt jt|jjd\}} tj |dd5}tj ||dddn #1swxYwYtj |t|jn##t$rtj|wxYwt d t%|d |j|js$t d |jdSdS#t*$r0}t d |jd |Yd}~dSd}~wwxYw)zPersist cache to disk via atomic write (temp file + rename). Safe against crashes -- either the old or new file is present, never a partial write. T)parentsexist_okNz.ms_upload_cache_tmp_)dirprefixwzutf-8)encodingzHash cache saved: z entries -> z&Hash cache file not found after save: zFailed to save hash cache to z: )r parentmkdirrdictr tempfilemkstempstrosfdopenjsondumpreplace BaseExceptionunlinkloggerinfolenexistswarning Exception)rdatafdtmp_pathfes rsavezUploadHashCache.saveUs  I   # ) )$ ) F F F ) )DK(( ) ) ) ) ) ) ) ) ) ) ) ) ) ) )#+(/00.000LB Yr3999'QIdA&&&''''''''''''''' 8S)9%:%:;;;;     (###  KKNSYYNNDrAr@)rrErFs rrzUploadHashCache._loadss&&&((  F d&,, +"ill  + + + + + + + + + + + + + + + KKXc$+&6&6XXdFVXX         NNLLL M M MDKKKKKKK s:BA BABA :B C&$CCN)__name__ __module__ __qualname____doc__rr4rr staticmethodfloatintrrr1r"r'rGrrrrr r s 5d#3    ,C,,S,S,,,\, C  S Xd^    .%C%%S%T%%%%"III<     rr )r5r2rpathlibrtypingrrrr7modelscope.utils.loggerrr<UPLOAD_HASH_CACHE_FILEr rrrrWs (((((((((( ...... +mmmmmmmmmmr