o Q5i @s`ddlZddlZddlmZmZddlmZddlmZm Z m Z GdddZ GdddZ dS) N)Optional Generator)Cache)EmbeddingReturnTypePrioritizedQueueItemQueueItemInnerc @s\eZdZdZdddZddZdeefd d Z dd e de de ee ddffddZ dS)CustomFIFOQueuez%Class which defines a custom orderingreturnNcCst|_g|_t|_dS)N) threadingLock_lock_queue_event_queueEvent _sync_eventselfrj/lsinfo/ai/hellotax_ai/llm_service/venv_embed/lib/python3.10/site-packages/infinity_emb/inference/queue.py__init__s zCustomFIFOQueue.__init__cCs t|jSN)lenrrrrr__len__s zCustomFIFOQueue.__len__itemscCs@|j|j|Wdn1swY|jdSr)r rextendrset)rrrrrrszCustomFIFOQueue.extend皙?size max_n_batchesc ks|js |j|s dS||}|j|jd|}|j|d|_|js+|jWdn1s5wYt||krD|dd|D}tdt||D] }||||VqSdS)a pop batch `up to size` + `continuous (sorted)` from queue Args: size (int): max size of batch max_n_batches: number of batches to be popped and sorted. timeout (float, optional): timeout until None is returned. Defaults to 0.2. latest_first (bool, optional): guarantees processing of oldest item in list. As latest first requires getting argmin of created timestamps, which is slow. Defaults to False. returns: None: if there is not a single item in self._queue after timeout else: list[EmbeddingInner] with len(1<=size) NcSsg|] }|jjs|jqSr)itemfuturedone).0mirrr Gs  z7CustomFIFOQueue.pop_optimal_batches..r)rrwaitr clearrsortrange) rrrtimeoutkwargs size_batches new_items_l new_itemsirrrpop_optimal_batches#s(   z#CustomFIFOQueue.pop_optimal_batches)r N)rr) __name__ __module__ __qualname____doc__rrlistrrintrrr0rrrrrs rc@s>eZdZd deeddfddZddZdedefd d Z dS) ResultKVStoreFutureNcacher cCs ||_dS)zholds instance of CacheN)_cache)rr8rrrrPs zResultKVStoreFuture.__init__cCsdS) deprecatedrrrrrrrTszResultKVStoreFuture.__len__r cs&|jr t|j||jIdHS)zwait for future to returnN)r9asyncio create_task aget_completer!)rr rrrwait_for_responseXs z%ResultKVStoreFuture.wait_for_responser) r1r2r3rrrrrrr>rrrrr7Osr7) r;r typingrr$infinity_emb.inference.caching_layerrinfinity_emb.primitivesrrrrr7rrrrs ?