#jGUddlZddlZddlZddlmZmZddlmZddlm Z ddl m Z m Z m Z mZmZddlZddlZddlmZddlmZdd lmZdd lmZmZmZdd lmZm Z m!Z!e r ddl"Z#d d l$m%Z%da&ee'e(e)d<ej*dkrdndZ+ej,dej,dej,dej,dej,dej,dej,dej,dej,dej,dej,dej,dej,dej,dgZ-eGd d!Z.d"e'e(fd#Z/d$d%d"e0fd&Z1d$d%d"e2fd'Z3d(ej4d"e2fd)Z5d*ee'e(e'e2e'ej4e'd%fd"e'e2fd+Z6dS),N) dataclassfield)BytesIO)Path) TYPE_CHECKINGAnyClassVarOptionalUnion)config)DownloadConfig) array_cast) is_local_path is_remote_urlxopen)first_non_null_valueno_op_if_value_is_nullstring_to_dict) FeatureType_IMAGE_COMPRESSION_FORMATSlittle<>z|b1|u1zu2zi2zu4zi4zf4zf8c eZdZUdZdZeeed<dZe ed<e ddZ eeed<d Z e eed <ejejejd Ze eed <e ddd Zeed<dZdeeeeeejd fdefdZddedd fdZdedeedfffdZdeejej ej!fdej fdZ" ddej de de dej fdZ#dS)Imagea=Image [`Feature`] to read image data from an image file. Input: The Image feature accepts as input: - A `str`: Absolute path to the image file (i.e. random access is allowed). - A `pathlib.Path`: path to the image file (i.e. random access is allowed). - A `dict` with the keys: - `path`: String with relative path of the image file to the archive file. - `bytes`: Bytes of the image file. This is useful for parquet or webdataset files which embed image files. - An `np.ndarray`: NumPy array representing an image. - A `PIL.Image.Image`: PIL image object. Output: The Image features output data as `PIL.Image.Image` objects. Args: mode (`str`, *optional*): The mode to convert the image to. If `None`, the native mode of the image is used. decode (`bool`, defaults to `True`): Whether to decode the image data. If `False`, returns the underlying dictionary in the format `{"path": image_path, "bytes": image_bytes}`. Examples: ```py >>> from datasets import load_dataset, Image >>> ds = load_dataset("AI-Lab-Makerere/beans", split="train") >>> ds.features["image"] Image(decode=True, id=None) >>> ds[0]["image"] >>> ds = ds.cast_column('image', Image(decode=False)) {'bytes': None, 'path': '/root/.cache/huggingface/datasets/downloads/extracted/b0a21163f78769a2cf11f58dfc767fb458fc7cea5c05dccc0144a2c0f0bc1292/train/healthy/healthy_train.85.jpg'} ``` NmodeTdecodeF)defaultrepridPIL.Image.Imagedtypebytespathpa_type)r!initr"_typec|jSN)r))selfs a/lsinfo/ai/hellotax_ai/base_platform/venv/lib/python3.11/site-packages/datasets/features/image.py__call__zImage.__call___s |valuereturnctjrddl}ntdt |t rt j|}t |tr|ddSt |tr$t| ddSt |ttfrd|dSt |t j rt|St ||jjrt!|S|d=t$j|drd|ddS|d|d+|d|ddSt+d|d ) aEncode example into a format for Arrow. Args: value (`str`, `np.ndarray`, `PIL.Image.Image` or `dict`): Data passed as input to Image feature. Returns: `dict` with "path" and "bytes" fields rN4To support encoding images, please install 'Pillow'.r(r'r(r&r'zUAn image sample should have one of 'path' or 'bytes' but they are missing or None in .)r PIL_AVAILABLE PIL.Image ImportError isinstancelistnparraystrrabsoluter' bytearrayndarrayencode_np_arrayrencode_pil_imagegetosr(isfile ValueError)r.r2PILs r/encode_examplezImage.encode_examplebs   V     TUU U eT " " $HUOOE eS ! ! !D11 1 t $ $  0 011DAA A y1 2 2  511 1 rz * * "5)) ) sy / / #E** * YYv   *rw~~eFm/L/L *!599V+<+<== = YYw   +uyy/@/@/L"YYw//69J9JKK Kphmppp r1c|jstdtjr ddl}ddl}nt d|i}|d|d}}|7|td|dt|r|j |}n| d d }| tj r tjn tj}t!||} | || d nd} t%| } t'|d | 5} t)| }dddn #1swxYwY|j |}n'|j t)|}|||j jjj|j|}|jr*|j|jkr||j}|S)aqDecode example image file into image data. Args: value (`str` or `dict`): A string with the absolute image file path, a dictionary with keys: - `path`: String with absolute or relative image file path. - `bytes`: The bytes of the image file. token_per_repo_id (`dict`, *optional*): To access and decode image files from private repositories on the Hub, you can pass a dictionary repo_id (`str`) -> token (`bool` or `str`). Returns: `PIL.Image.Image` zMDecoding is disabled for this feature. Please use Image(decode=True) instead.rNz4To support decoding images, please install 'Pillow'.r(r'zCAn image should have one of 'path' or 'bytes' but both are None in r7::repo_idtokenrbdownload_config)r RuntimeErrorr r8r9 PIL.ImageOpsr:rHrropensplit startswith HF_ENDPOINTHUB_DATASETS_URLHUB_DATASETS_HFFS_URLrrErrrreadloadgetexifExifTagsBase OrientationImageOpsexif_transposerconvert) r.r2token_per_repo_idrIr(bytes_image source_urlpatternsource_url_fieldsrPrSfs r/decode_examplezImage.decode_examplesn${ pnoo o   V         TUU U  $ " V}eGnf >| !ogl!o!o!oppp &&3INN400EE!%D!1!1"!5J&001CDD://#9 )7z7(K(K%O`Ol)--.? .JKKKrv'55&A&A&AOtT?KKK3q!(!2!2333333333333333INN622EEINN76??33E ==??  sy16B C C OL//66E 9 -ej00MM$),,E s5"E##E'*E'rcNddlm}|jr|n|d|ddS)zfIf in the decodable state, return the feature itself, otherwise flatten the feature into a dictionary.r)Valuebinarystringr&)featuresrnr )r.rns r/flattenz Image.flattensK######{ DDxh r1storagectj|jrR |tj}n*#tj$r}td||d}~wwxYwtj|jrrtj dgt|ztj }tj ||gddg|}nZtj|jrt!|tj }tj dgt|ztj}tj ||gddg|}ntj|jrrtj dgt|ztj}tj ||gddg|}n tj|jr|jddkr|d}n8tj dgt|ztj }|jddkr|d}n8tj dgt|ztj}tj ||gddg|}ntj|jrtj d|Dtj }tj dgt|ztj}tj ||gddg|}t!||jS) aCast an Arrow array to the Image arrow storage type. The Arrow types that can be converted to the Image pyarrow storage type are: - `pa.string()` - it must contain the "path" data - `pa.large_string()` - it must contain the "path" data (will be cast to string if possible) - `pa.binary()` - it must contain the image bytes - `pa.struct({"bytes": pa.binary()})` - `pa.struct({"path": pa.string()})` - `pa.struct({"bytes": pa.binary(), "path": pa.string()})` - order doesn't matter - `pa.list(*)` - it must contain the image array data Args: storage (`Union[pa.StringArray, pa.StructArray, pa.ListArray]`): PyArrow array to cast. Returns: `pa.StructArray`: Array in the Image arrow storage type, that is `pa.struct({"bytes": pa.binary(), "path": pa.string()})`. zvFailed to cast large_string to string for Image feature. This can happen if string values exceed 2GB. Original error: Ntyper'r(maskrcdg|]-}|'ttj|dnd.S)Nr')rCr=r>).0arrs r/ z&Image.cast_storage.. s8uuuZ]CO#//88QUuuur1)patypesis_large_stringrvcastrp ArrowInvalidrH is_stringr>lenro StructArray from_arraysis_nullis_large_binaryr is_binary is_structget_field_indexris_list to_pylistr))r.rse bytes_array path_arrays r/ cast_storagezImage.cast_storages( 8 # #GL 1 1  !,,ry{{33?    +'(++   8  gl + + (D6CLL#8ry{{KKKKn00+w1G'SYIZahapaparar0ssGG X % %gl 3 3  G4&3w<<"7bikkJJJJn00':1FRXHY`g`o`o`q`q0rrGG X   - - 4&3w<<"7bikkJJJJn00':1FRXHY`g`o`o`q`q0rrGG X   - - |++G4499%mmG44  hvG '<29;;OOO |++F33q88$]]622 Xtfs7||&;")++NNN n00+z1JWV\L]dkdsdsdudu0vvGG X  gl + + (uuahararatatuuuY[[K4&3w<<"7bikkJJJJn00j)GV+<;CVCVCXCX1G'4<000s&A A4A//A4 local_files remote_filescitfdtjfd|Dtj}tjfd|dDtj}tj||gddg| }t||j S) a7Embed image files into the Arrow array. Args: storage (`pa.StructArray`): PyArrow array to embed. token_per_repo_id (`dict`, optional): Dictionary repo_id -> token to fetch the files bytes. local_files (`bool`, defaults to `True`) Whether to embed local files data in the array remote_files (`bool`, defaults to `True`) Whether to embed remote files data in the array. E.g. files with paths that start with hf:// or https:// Returns: `pa.StructArray`: Array in the Image arrow storage type, that is `pa.struct({"bytes": pa.binary(), "path": pa.string()})`. Nc|dd}|tjr tjn tj}t ||}||dnd}t|}t|d|5}| cdddS#1swxYwYdS)NrLrMrNrOrQrR) rWrXr rYrZr[rrErrr\)r(rhrirjrPrSrkres r/ path_to_bytesz*Image.embed_storage..path_to_bytes.s D))"-J+5+@+@AS+T+Tv''Z`Zv !/z7 C C K\Kh%))*;I*FGGGnrE,5999OtT?CCC qvvxx                  s%CC  C cg|]U}|O|d?rt|dsr&t|dr|dn |dndVS)Nr'r()rr)rzxrrrs r/r|z'Image.embed_storage..;s   = z)%**7& *B*B*HT*YfghiogpYqYq*"M!F),,,7   r1rucg|]I}|Crt|sr.t|rtj|n|ndJSr-)rrrFr(basename)rzr(rrs r/r|z'Image.embed_storage..Is   #$(5d(;(;AMR_`dReReBG$$T***   r1r(r'rw) rr}r>rrorrprrrrr))r.rsrerrrrrs ``` @r/ embed_storagezImage.embed_storagesI0  $ "     h      !**,,       X     $MM&11;;==       .,,k:-FRXHY`g`o`o`q`q,rr'4<000r1r-)NTT)$__name__ __module__ __qualname____doc__rr r?__annotations__r boolrr#r%r r}structrorpr)rr+r0r r'rAdictr=rBrJrlrr StringArrayr ListArrayrrr1r/rr.s%%ND(3-FDd777B 777,E8C=,,,&RYibikk'R'RSSGXc]SSSwU???E3???'E#uirzSd*d$e'jn''''R;;D;EV;;;;z  }d3 3E.FFG     <1E".".",*V$W<1\^\j<1<1<1<1~osC1C1~C1LPC1gkC1 C1C1C1C1C1C1r1rr3cXtjrddl}ntdt}|jtt|jj t|jj zatS)Nrr5) r r8r9r:rrr*r<setOPENkeysSAVE)rIs r/list_image_compression_formatsrYs RPQQQ") %)#cin.A.A.C.C*D*Ds39>K^K^K`K`GaGa*a%b%b" %%r1rgr$ct}|jtvr|j}n |jdvrdnd}||||S)zmConvert a PIL Image object to bytes using native compression if possible, otherwise use PNG/TIFF compression.)1LLARGBRGBAPNGTIFF)format)rrrrsavegetvalue)rgbufferrs r/image_to_bytesrfsd YYF |57777*(GGGV JJvfJ%%% ??  r1cpt|dr|jdkr |jddSdt|dS)Nfilenamer6)hasattrrr)rgs r/rDrDqsFuj!!>en&:&:666~e'<'<===r1r>ctjrddl}ntd|j}|jdkr|jnt }|j}|j}d}|j ddrP|dvrtd|d|dtjd }||krtj d |d |d n|tvr|}n|d krk||zt|z}tj|tvr0tj|}tj d |d |d n |dz}|d kk|td|dt|j||}dt'|dS)Nrr5=r )uizUnsupported array dtype z for image encoding. Only z' is supported for multi-channel arrays.rzDowncasting array dtype z to z to be compatible with 'Pillow'rzCannot downcast dtype z- to a valid image dtype. Valid image dtypes: r6)r r8r9r:r% byteorder_NATIVE_BYTEORDERkinditemsizeshape TypeErrorr=warningswarn_VALID_IMAGE_ARRAY_DTPYESr?r fromarrayastyper) r>rIr%dtype_byteorder dtype_kinddtype_itemsize dest_dtype dtype_strrgs r/rCrCxs RPQQQ KE).C)?)?eooEVOJ^NJ {122 Z ' '5J Xe__ J   MkUkk kkk l l l + + + !!'*4s>7J7JJIx ""&???Xi00  oooJoooppp1$!!  xxx]vxx  I   Z 8 8 9 9E>%#8#8 9 99r1objsctjrddl}ntd|rt |\}}t |t r d|DSt |tjr"ttfd|DSt ||j j r"ttfd|DS|S|S)zmEncode a list of objects into a format suitable for creating an extension array of type `ImageExtensionType`.rNr5c g|] }||ddnd S)Nr6r)rzobjs r/r|z2objects_to_list_of_image_dicts..s*^^^RUCOS4000^^^r1c&g|] }|Srrrzrobj_to_image_dict_funcs r/r|z2objects_to_list_of_image_dicts..%@@@C**3//@@@r1c&g|] }|Srrrs r/r|z2objects_to_list_of_image_dicts..rr1) r r8r9r:rr;r?r=rBrrCrrD)rrI_rrs @r/objects_to_list_of_image_dictsrsRPQQQ %d++3 c3   _^^Y]^^^ ^ c2: & & %;O%L%L "@@@@4@@@ @ SY_ - - %;rsH ((((((((@@@@@@@@@@@@@@555555BBBBBBBBBBYYYYYYYYYY&%%%%%%37HT#Y/666=H44CC# BHUOO BHUOO BHUOO BHUOO BHUOO BHUOO BHUOO BHUOO BHUOO BHUOO BHUOO BHUOO BHUOO BHUOO$ g1g1g1g1g1g1g1 g1T &S & & & &+>->$>>>>(:2:(:$(:(:(:(:V S 4:tBJ'7>O9PP Q $Zr1