o 3/i@spdZddlZddlZddlmZeeZdada ia ddZ ddZ ej dejd ejd iZd d Zd dZdS)znvMatmulHeuristics-based config selection for GEMM. Queries NVIDIA's analytic heuristic library to pick tile/cluster dims based on problem shape, then selects swap_ab by comparing estimated runtimes for both orientations. N) GemmConfigc CsttdurtSzddlm}m}||jddadaWtSty9}ztd|dadaWYd}~tSd}~ww) z3Lazily initialize the nvMatmulHeuristics interface.Nr)NvMatmulHeuristicsInterfaceNvMatmulHeuristicsTargetBSB)backend precisionTz"nvMatmulHeuristics not available: F) _nvmmh_available_ifacenvMatmulHeuristicsrrCUTLASS3 Exceptionloggerdebug)rrerb/lsinfo/ai/hellotax_ai/llm_service/venv_vllm/lib/python3.10/site-packages/quack/nvmmh_heuristic.py _get_ifaces"rc Cs|tvrt|SzEddlm}m}t}|durWdS|j|jd}||}|dur.WdS|}| |||j |j fD]}| ||q>|t|<|WSt yl}ztd|dt|<WYd}~dSd}~ww)z=Get or create a hardware descriptor for the given SM version.r)NvMatmulHeuristicsNvidiaGpuNvMatmulHeuristicsMatmulLayoutN) z&Failed to create hardware descriptor: )_hw_descriptorsr rrrH100_SXMB200getcreateHardwareDescriptorsetHardwarePredefinedGpu TN_ROW_MAJOR TN_COL_MAJORloadInternalDiscoverySetr r r) device_capacityrrifacegpu_mapgpuhwlayoutrrrr_get_hw,s6  r&rHSHSSSc Cstz/|j}||_|j||||d|d}||_|sWdS|dd} | j| j| j| j|ddfWSty9YdSw)zXQuery nvMMH for top-1 config. Returns (tile_m, tile_n, cl_m, cl_n, est_runtime) or None.)mnk matmulLayoutcounthardware_descriptorNrkernelruntime)r get_with_mnk cta_tile_m cta_tile_n cluster_m cluster_nr ) r!r$r*r+r,r%roriginal_precisionresultscfgrrr _query_top1Xs&   r:c CsNddlm}t}|durdSt|}|durdSt|j}|dur%dS|jdkr/|jdn|jd}|jd}|jd} t |||| ||j |} t ||| |||j |} | dur^| dur^dS| rd| dnt d} | rn| dnt d} | | kr| dur| dd\}}}}d }n | dd\}}}}d }|d ko|d k}t ||||||d |dS)aUUse nvMatmulHeuristics to pick a GemmConfig based on problem shape. Queries both normal (M,N,K) with row-major output and swapped (N,M,K) with col-major output, picks the orientation with lower estimated runtime. Returns None if nvMatmulHeuristics is unavailable, letting the caller fall back to the hardcoded default. r)rNinfTFr)tile_mtile_npingpongr5r6swap_abmax_swizzle_sizer )r rrr&_TORCH_DTYPE_TO_NVMMH_PRECISIONrdtypendimshaper:rrfloatr)ABr rr!r$rr*r,r+normalswapped normal_rt swapped_rtrBrCcl_mcl_nrErDrrrnvmmh_default_confignsJ    rT)__doc__loggingtorchquack.gemm_configr getLogger__name__r rr rrr&bfloat16float16float32rGr:rTrrrrs   &