o h?id@sTdZddlmZddlmZddlmZddlmZee Z dZ GdddeZ d S) un ServiceGuardMiddleware — blocks inference requests while the LLM service is in training or switching mode. )BaseHTTPMiddleware)Request) JSONResponse) get_logger)z /api/v1/chatz/api/v1/completionsc@seZdZdZdefddZdS)ServiceGuardMiddlewarezQReturn 503 for inference endpoints when the LLM service is not in INFERENCE mode.requestc s|jjtfddtDruzJddlm}m}|}|}||jkr?| }dd|d}t dd t d |d WS||j krYd d d}t ddt d |d WSWntyt} zt d| WYd} ~ nd} ~ ww||IdHS)Nc3s|]}|VqdS)N) startswith).0prefixpathD/lsinfo/ai/hellotax_ai/base_platform/app/middleware/service_guard.py sz2ServiceGuardMiddleware.dispatch..r)get_status_manager ServiceModezDLLM service is currently in training mode. Inference is unavailable.training)detailmode training_infozServiceGuard blocked z: training mode activei) status_codecontentz5LLM service is switching modes. Please retry shortly. switching)rrz: switching mode activez-ServiceGuardMiddleware error (failing open): )urlr anyGUARDED_PREFIXES'app.services.llm.service_status_managerrrget_current_modeTRAININGget_training_infologgerinfor SWITCHING Exceptionerror) selfr call_nextrrmanagerrrbodyer r rdispatchs6  zServiceGuardMiddleware.dispatchN)__name__ __module__ __qualname____doc__rr*r r r rrsrN) r.starlette.middleware.baserstarlette.requestsrstarlette.responsesrapp.core.loggingrr+r rrr r r rs