o :/iu @sddlZddlZddlZddlmZddlmZddlZddlm Z ddl m Z ddl m Z ddlmZmZddlmZdd lmZdd lmZeeZ dd e d ejdBdedefddZdejde fddZdejde fddZdS)N)partial)Any)FastAPI)envs) EngineClient)H11_MAX_HEADER_COUNT_DEFAULT%H11_MAX_INCOMPLETE_EVENT_SIZE_DEFAULT)SSLCertRefresher) init_logger)find_process_using_portFappsockenable_ssl_refreshuvicorn_kwargsc sztdjD]!}t|dd}t|dd}|dus|dur q td|d|q jD])}t|dd}t|dd}t|dd}|dusN|dusN|durOq.td||jq.|d d}|d d} |durjt}| durpt} t j fi|} || _ | | _ | t | j_ttjjj|r|gndd |sdn t| j| j| j| jd tdfdd } ddd} tj| tj | dfdd } | }zJzIdH| WW|!!Stj"y3|d}t#|}|durt$d||d|%td&YW|!!Sw|!!w)z Start a FastAPI app using Uvicorn, with support for custom Uvicorn config options. Supports http header limits via h11_max_incomplete_event_size and h11_max_header_count. zAvailable routes are:methodsNpathzRoute: %s, Methods: %sz, endpointzRoute: %s, Endpoint: %sh11_max_incomplete_event_sizeh11_max_header_count)sockets) ssl_contextkey_path cert_pathca_pathreturncs dSN)set)shutdown_eventrf/lsinfo/ai/hellotax_ai/llm_service/venv_vllm/lib/python3.10/site-packages/vllm/entrypoints/launcher.pysignal_handleras z"serve_http..signal_handlercsdSrrrrrrdummy_shutdowndsz"serve_http..dummy_shutdowncsfIdHjj}|jj}dt|j|dIdHd_  r1 dSdS)N)timeoutT) waitstate engine_client vllm_configshutdown_timeoutrun_in_executorrshutdown should_exitcancelstop)r%r"r loopserver server_taskrssl_cert_refresher watchdog_taskrrhandle_shutdownjs  z#serve_http..handle_shutdownportz7port %s is used by process %s launched with command: %s z"Shutting down FastAPI HTTP server.)rN)'loggerinforoutesgetattrjoin__name__poprruvicornConfigrrloadServerr$r/asyncioget_running_loop create_task watchdog_loopr%server ssl ssl_keyfile ssl_certfile ssl_ca_certsEventadd_signal_handlersignalSIGINTSIGTERMr+CancelledErrorr warningcmdliner))r r rrrouterrrrrconfigr r!r3 shutdown_taskr4processrr-r serve_https                    rVr/enginecs$d} t|IdHt||q)z # Watchdog task that runs in the background, checking # for error state in the engine. Needed to trigger shutdown # if an exception arises is StreamingResponse() generator. g@TN)rAsleepterminate_if_errored)r/rWVLLM_WATCHDOG_TIME_SrrrrDs  rDcCs*|jo|j }tjs|rd|_dSdSdS)a See discussions here on shutting down a uvicorn server https://github.com/encode/uvicorn/discussions/1103 In this case we cannot await the server shutdown here because handler must first return to close the connection for this request. TN)errored is_runningrVLLM_KEEP_ALIVE_ON_ENGINE_DEATHr*)r/rWengine_erroredrrrrYs  rY)F)rArLsocket functoolsrtypingrr=fastapirvllmrvllm.engine.protocolrvllm.entrypoints.constantsrrvllm.entrypoints.sslr vllm.loggerr vllm.utils.network_utilsr r;r6boolrVr@rDrYrrrrs4         v