o :/i> @sUddlZddlZddlmZddlmZmZddlmZee Z da e e d<ej dded ed ed fd d Zejd ed fddZdaee d<dddZded dfddZdS)N) Generator)CompilationMode VllmConfig) init_loggergtorch_compile_start_time"torch.compile took %.2f s in total vllm_configmessagereturn)NNNccs.ta|j}d}|}|jtjkr2|r2ddl}|j dddt d|| | }|zEzdVWnty@wtt}|jtjkrUt j||ddW|durvz |dddWdStyut jddd YdSwdS|durz |dddWwtyt jddd Ywww) zContext manager that times torch.compile and manages depyf debugging. On normal exit: logs the compile time and exits depyf. On exception: cleans up depyf without logging (compilation failed). NrT)parentsexist_okzDumping depyf output to %slocalscopezException during depyf cleanup.)exc_info)time perf_counterrcompilation_configcompile_debug_dump_pathmoder VLLM_COMPILEdepyfmkdirloggerdebug prepare_debugas_posix __enter__ Exception info_once__exit__warning)rr rdepyf_cmpathrtotal_compile_timer%e/lsinfo/ai/hellotax_ai/llm_service/venv_vllm/lib/python3.10/site-packages/vllm/compilation/monitor.pymonitor_torch_compilesB         r'ccsTddlm}|j}t}dVt|}|j|ks Jdtjd|dddS)zContext manager that times the initial profiling run. Asserts that no backend compilation occurs during the profiling run (all compilation should have completed before this point). r)compilation_counterNzbackend compilation occurred during the initial profiling run; all compilation should be complete before the profiling run starts.z(Initial profiling/warmup run took %.2f sr r)vllm.compilation.counterr(num_backend_compilationsrrrr)r(backend_compilations_beforestartelapsedr%r%r&monitor_profiling_run9s    r.Tcudagraph_capturing_enabledcCststddS)Nz]CUDA graph capturing detected at an inappropriate time. This operation is currently disabled.)r/ RuntimeErrorr%r%r%r&$validate_cudagraph_capturing_enabledVs r1enabledcCs|adS)N)r/)r2r%r%r&set_cudagraph_capturing_enabledbsr3)r)r N) contextlibrcollections.abcr vllm.configrr vllm.loggerr__name__rrfloat__annotations__contextmanagerstrr'r.r/boolr1r3r%r%r%r&s*     '