o :/Êiöã@sdZddlZddlZddlZddlZddlZzddlZddlmZddl m Z mZddlm Z Wney<edƒd‚wddlZddlmZddlmZdd lmZdd lmZddlmZddlmZdd lmZeeƒZ dej!fdd„Z"dd„Z#edkr„e#ƒdSdS)aO vLLM gRPC Server Starts a gRPC server backed by AsyncLLM, using the VllmEngineServicer from the smg-grpc-servicer package. Usage: python -m vllm.entrypoints.grpc_server --model Example: python -m vllm.entrypoints.grpc_server --model meta-llama/Llama-2-7b-hf --host 0.0.0.0 --port 50051 éN)Ú reflection)Úvllm_engine_pb2Úvllm_engine_pb2_grpc)ÚVllmEngineServicerzTsmg-grpc-servicer is required for gRPC mode. Install it with: pip install vllm[grpc])ÚAsyncEngineArgs)Úlog_version_and_model)Úinit_logger)ÚUsageContext)ÚFlexibleArgumentParser)ÚAsyncLLM)Ú__version__Úargsc ƒs$ttt|jƒt d|¡t ¡}t |¡}|jt j d}tj|t j |j |jd}t||ƒ}tjjgd¢d}t ||¡tjjdjtjf}t ||¡|jpRd}|›d|j›} | | ¡z| ¡Id Ht d | ¡t d¡t !¡} t "¡‰‡fdd „}t#j$t#j%fD]}| &||¡q‡z ˆ '¡Id HWnt(y§t d¡Yn%wWt d¡|j)ddId Ht d¡| *¡t d¡t d¡d SWt d¡|j)ddId Ht d¡| *¡t d¡t d¡d St d¡|j)ddId Ht d¡| *¡t d¡t d¡w)z\ Main gRPC serving function. Args: args: Parsed command line arguments zvLLM gRPC server args: %s)Ú usage_context)Úvllm_configrÚenable_log_requestsÚdisable_log_stats))zgrpc.max_send_message_lengthéÿÿÿÿ)zgrpc.max_receive_message_lengthr)z1grpc.http2.min_recv_ping_interval_without_data_msi')z#grpc.keepalive_permit_without_callsT)ÚoptionsÚ VllmEngineú0.0.0.0ú:NzvLLM gRPC server started on %sz"Server is ready to accept requestscst d¡ˆ ¡dS)NzReceived shutdown signal)ÚloggerÚinfoÚset©©Ú stop_eventrúi/lsinfo/ai/hellotax_ai/llm_service/venv_vllm/lib/python3.10/site-packages/vllm/entrypoints/grpc_server.pyÚsignal_handlerxs z"serve_grpc..signal_handlerzInterrupted by userz!Shutting down vLLM gRPC server...g@)ÚgracezgRPC server stoppedzAsyncLLM engine stoppedzShutdown complete)+rrÚVLLM_VERSIONÚmodelrÚtimerÚ from_cli_argsÚcreate_engine_configr ÚOPENAI_API_SERVERrÚfrom_vllm_configrrrÚgrpcÚaioÚserverrÚ add_VllmEngineServicer_to_serverrÚ DESCRIPTORÚservices_by_nameÚ full_namerÚSERVICE_NAMEÚenable_server_reflectionÚhostÚportÚadd_insecure_portÚstartÚasyncioÚget_running_loopÚEventÚsignalÚSIGTERMÚSIGINTÚadd_signal_handlerÚwaitÚKeyboardInterruptÚstopÚshutdown) r Ú start_timeÚengine_argsrÚ async_llmÚservicerr)Ú service_namesr0ÚaddressÚlooprÚsigrrrÚ serve_grpc3sx€ ÿü ÿ þ ÿÿ ù û rGc Cs’tdd}|jdtddd|jdtdd dt |¡}| ¡}z t t |ƒ¡WdSt yH}zt d |¡t d¡WYd}~dSd}~ww) z

s@ÿýÿY ÿ