jWFUddlmZddlZddlmZddlmZmZmZmZm Z ddl m Z m Z m Z ddlZddlmZerddlmZdd lmZgd Ze d Ze d Zeejd sHedejjd<edejjd<edejjd<ddlmZmZmZd0dZd1dZGddeZ GddZ!e dede"ffZ#de$d<e d2d3d)Z%e d2d4d,Z% d2d5d/Z%dS)6) annotationsN)Callable)Optionaloverload TYPE_CHECKING TypeAliasUnion) ParamSpecSelfTypeVar)Tensor) _POOL_HANDLE) _dummy_type)is_current_stream_capturinggraph_pool_handleXPUGraphgraphmake_graphed_callables_R_P_XpuStreamBase _XPUGraph_xpu_graph_pool_handle_xpu_isCurrentStreamCapturing)rrrreturnboolctS)zReturn True if XPU graph capture is underway on the current XPU stream, False otherwise. If a XPU context does not exist on the current device, returns False without initializing the context. )rZ/lsinfo/ai/hellotax_ai/base_platform/venv/lib/python3.11/site-packages/torch/xpu/graphs.pyrr)s ) * **r rcXtjtS)zBReturn an opaque token representing the id of a graph memory pool.)torchxpurrrr r!rr1s 9 ! !"8":": ; ;;r ceZdZdZddfd Zddfd Zdfd Zdfd Zdfd Zdfd Z dfd Z dfd Z d fd Z d!fd Z d!fd ZxZS)"raWrapper around a XPU graph. Arguments: keep_graph (bool, optional): If ``keep_graph=False``, the executable command graph will be instantiated on GPU at the end of ``capture_end`` and the underlying modifiable command graph will be destroyed. Note that the executable command graph will not be instantiated at the end of ``capture_end`` in this case. Instead, it will be instantiated via an explicit called to ``instantiate`` or automatically on the first call to ``replay`` if ``instantiate`` was not already called. Calling ``instantiate`` manually before ``replay`` is recommended to prevent increased latency on the first call to ``replay``. F keep_graphrrr cHt||SN)super__new__)clsr& __class__s r!r*zXPUGraph.__new__GswwsJ///r NpoolOptional[_POOL_HANDLE]NonecLt|dS)aBegin capturing XPU work on the current xpu stream. Typically, you shouldn't call ``capture_begin`` yourself. Use :class:`~torch.xpu.graph`, which call ``capture_begin`` internally. Arguments: pool (optional): Token (returned by :func:`~torch.xpu.graph_pool_handle` or :meth:`other_Graph_instance.pool()`) that hints this graph may share memory with the indicated pool. r-N)r) capture_begin)selfr-r,s r!r2zXPUGraph.capture_beginJs& 4(((((r cHtdS)aEnd XPU graph capture on the current stream. After ``capture_end``, ``replay`` may be called on this instance. Typically, you shouldn't call ``capture_end`` yourself. Use :class:`~torch.xpu.graph`, which call ``capture_end`` internally. N)r) capture_endr3r,s r!r5zXPUGraph.capture_endWs! r cHtdS)a/Instantiate the XPU graph. Will be called by ``capture_end`` if ``keep_graph=False``, or by ``replay`` if ``keep_graph=True`` and ``instantiate`` has not already been explicitly called. Does not destroy the xpu modify command graph returned by ``raw_xpu_graph``. N)r) instantiater6s r!r8zXPUGraph.instantiateas! r cHtdS)z+Replay the XPU work captured by this graph.N)r)replayr6s r!r:zXPUGraph.replayjs r cHtdS)z1Delete the graph currently held by this instance.N)r)resetr6s r!r<zXPUGraph.resetns  r rcDtS)zReturn an opaque token representing the id of this graph's memory pool. This id can optionally be passed to another graph's ``capture_begin``, which hints the other graph may share the same memory pool. )r)r-r6s r!r-z XPUGraph.poolrs ww||~~r cDtS)z.Enable debugging mode for XPUGraph.debug_dump.)r)enable_debug_moder6s r!r?zXPUGraph.enable_debug_modezsww((***r debug_pathstrcFt|S)z Arguments: debug_path (required): Path to dump the graph to. Calls a debugging function to dump the graph if the debugging is enabled via XPUGraph.enable_debug_mode() )r) debug_dump)r3r@r,s r!rCzXPUGraph.debug_dump~sww!!*---r intcDtS)zReturns the underlying xpuGraph_t. ``keep_graph`` must be True. XPU doesn't provide APIs to manipulate this object. )r) raw_xpu_graphr6s r!rFzXPUGraph.raw_xpu_graphs ww$$&&&r cDtS)aReturns the underlying xpuGraphExec_t. ``instantiate`` must have been called if ``keep_graph`` is True, or ``capture_end`` must have been called if ``keep_graph`` is False. If you call ``instantiate()`` after ``raw_xpu_graph_exec()``, the previously returned xpuGraphExec_t will be destroyed. It is your responsibility not to use this object after destruction. XPU doesn't provide APIs to manipulate this object. )r)raw_xpu_graph_execr6s r!rHzXPUGraph.raw_xpu_graph_execs ww))+++r )F)r&rrr r()r-r.rr/rr/rr)r@rArr/)rrD)__name__ __module__ __qualname____doc__r*r2r5r8r:r<r-r?rCrFrH __classcell__)r,s@r!rr6s} 0000000 ) ) ) ) ) ) )++++++......'''''',,,,,,,,,,r rc@eZdZUdZdZded< ddd Zdd ZddZdS)raContext-manager that captures XPU work into a :class:`torch.xpu.XPUGraph` object for later replay. Arguments: xpu_graph (torch.xpu.XPUGraph): Graph object used for capture. pool (optional): Opaque token (returned by a call to :func:`~torch.xpu.graph_pool_handle()` or :meth:`other_Graph_instance.pool()`) hinting this graph's capture may share memory from the specified pool. stream (torch.xpu.Stream, optional): If supplied, will be set as the current stream in the context. If not supplied, ``graph`` sets its own internal side stream as the current stream in the context. .. note:: For effective memory sharing, if you pass a ``pool`` used by a previous capture and the previous capture used an explicit ``stream`` argument, you should pass the same ``stream`` argument to this capture. NOptional[torch.xpu.Stream]default_capture_stream xpu_graphrr-r.streamc|jj(tj|j_|dn|f|_||n |jj|_|jtd|j|_||_ dS)Nrzcapture_stream must not be None) r,rRr#r$Streamr-capture_streamAssertionError stream_ctxrS)r3rSr-rTs r!__init__zgraph.__init__s > 0 849I4D4D4F4FDN 1,BBTG (FFdn.S    & !BCC C-"r rr/ctjtj|j|jj|jdSr() r#r$ synchronize empty_cacherY __enter__rSr2r-)r3s r!r^zgraph.__enter__sX   !!###$$di0000r argsobjectcV|j|jj|dSr()rSr5rY__exit__)r3r_s r!rbzgraph.__exit__s. ""$$$  $''''r )NN)rSrr-r.rTrQrI)r_r`rr/) rKrLrMrNrR__annotations__rZr^rbrr r!rrs} :>==== (,-1 #####.1111((((((r rtorch.nn.Module.r_ModuleOrCallableF callables sample_argstuple[Tensor, ...]num_warmup_itersrDallow_unused_inputr-r.cdSr(rrgrhrjrkr-s r!rrs r tuple[_ModuleOrCallable, ...]tuple[tuple[Tensor, ...], ...]cdSr(rrms r!rrs %(Cr 7Union[_ModuleOrCallable, tuple[_ModuleOrCallable, ...]]9Union[tuple[Tensor, ...], tuple[tuple[Tensor, ...], ...]]c )*tjr"tjrtdd}t |t s.d}|f}t jt tdf|f}n4t jt t tdfdf|}g)t||D]\}}t |tj j rt|j dkr0t|jdkrt|jdkstdt!d|Dstdtjjj|} )t | t!d | Dst-d d )D} d |D*)*fd t/t|D} dt/t|D} dt/t|D} |t1n|}tjtjtj5t||| D]\}}}d\}}}t/|D]}tjj||}t d|D}t|dkrRtj|t d|Dt d|Dd|}|||fD]}~ dddn #1swxYwYtjg}g}t||| D]\}}}tj ||5||}dddn #1swxYwYtjj!|\}}|t |||g}g}ttE| tE|tE| D]Z\}}}t d|D} t d|D}d}t|dkrtj ||5tj|t d|Dt d| Dd|}dddn #1swxYwYg}!d}"|D]A}#|#j#r#|!|!||"|"dz }",|!dBt |!}!|| ||!\|$|$d6d-}$g}%tK|D]\}&}|$| |&| |&*|&| |&||&| |&||&||&||& }'t |tj j r7d7d5}(|(||j&|'|j'|_'|%||%|'|r|%dSt |%S)8a Accept callables (functions or :class:`nn.Module`\ s) and returns graphed versions. Each graphed callable's forward pass runs its source callable's forward XPU work as a XPU graph inside a single autograd node. The graphed callable's forward pass also appends a backward node to the autograd graph. During backward, this node runs the callable's backward work as a XPU graph. Therefore, each graphed callable should be a drop-in replacement for its source callable in an autograd-enabled training loop. See :ref:`Partial-network capture` for detailed use and constraints. If you pass a tuple of several callables, their captures will use the same memory pool. Arguments: callables (torch.nn.Module or Python function, or tuple of these): Callable or callables to graph. If you pass a tuple of callables, their order in the tuple must be the same order they'll run in the live workload. sample_args (tuple of Tensors, or tuple of tuples of Tensors): Samples args for each callable. If a single callable was passed, ``sample_args`` must be a single tuple of argument Tensors. If a tuple of callables was passed, ``sample_args`` must be tuple of tuples of argument Tensors. num_warmup_iters (int): The number of warmup iterations. Currently, ``DataDistributedParallel`` needs 11 iterations for warm up. Default: ``3``. allow_unused_input (bool): If False, specifying inputs that were not used when computing outputs (and therefore their grad is always zero) is an error. Defaults to False. pool (optional): Token (returned by :func:`~torch.xpu.graph_pool_handle` or :meth:`other_Graph_instance.pool()`) that hints this graph may share memory with the indicated pool. .. note:: The ``requires_grad`` state of each Tensor in ``sample_args`` must match the state that's expected for the corresponding real input in the training loop. .. warning:: This API is in beta and may change in future releases. .. warning:: ``sample_args`` for each callable must contain only Tensors. Other types are not allowed. .. warning:: Returned callables do not support higher order differentiation (e.g., double backward). .. warning:: In any :class:`~torch.nn.Module` passed to :func:`~make_graphed_callables`, only parameters may be trainable. Buffers must have ``requires_grad=False``. .. warning:: After you pass a :class:`torch.nn.Module` through :func:`~make_graphed_callables`, you may not add or remove any of that Module's parameters or buffers. .. warning:: :class:`torch.nn.Module`\s passed to :func:`~torch.xpu.make_graphed_callables` must not have module hooks registered on them at the time they are passed. However, registering hooks on modules *after* passing them through :func:`~torch.xpu.make_graphed_callables` is allowed. .. warning:: When running a graphed callable, you must pass its arguments in the same order and format they appeared in that callable's ``sample_args``. .. warning:: The automatic mixed precision is supported in :func:`~torch.xpu.make_graphed_callables` only with disabled caching. The context manager `torch.amp.autocast()` must have `cache_enabled=False`. z_make_graphed_callables does not support the autocast caching. Please set `cache_enabled=False`.FT.rzModules must not have hooks registered at the time they are passed. However, registering hooks on modules after passing them through make_graphed_callables is allowed.c3(K|] }|jduVdS)FN requires_grad.0bs r! z)make_graphed_callables..Is)EEAq%/EEEEEEr zIn any :class:`~torch.nn.Module` passed to :func:`~make_graphed_callables`, only parameters may be trainable. All buffers must have ``requires_grad=False``.c3JK|]}t|tjVdSr() isinstancer#r )rxargs r!rzz)make_graphed_callables..Qs.HHS:c5<00HHHHHHr zfIn the beta API, sample_args for each callable must contain only Tensors. Other types are not allowed.c,g|]}t|Sr)len)rxr_s r! z*make_graphed_callables..Ys!L!L!L#d))!L!L!Lr cg|]D}t|tjjr!t |ndES)r)r|r#nnModuletuple parameters)rxcs r!rz*make_graphed_callables..ZsP""" ",Aux!?!?GallnnR"""r c2g|]}||zSrr)rxiflatten_sample_argsper_callable_module_paramss r!rz*make_graphed_callables..^s9***  A!;A!>>***r cJg|] }tj!Srr#r$rrx_s r!rz*make_graphed_callables..c&FFF1%)$$&&FFFr cJg|] }tj!Srrrs r!rz*make_graphed_callables..drr N)NNNc3(K|] }|j |VdSr(rurxos r!rzz)make_graphed_callables..qs)$K$K11?$KQ$K$K$K$K$K$Kr c3(K|] }|j |VdSr(rurxrs r!rzz)make_graphed_callables..us=%%"#q%%%%%%%r c3LK|]}|j tj|V dSr(rvr# empty_likers r!rzz)make_graphed_callables..xsH++45AO+!,Q//++++++r )outputsinputs grad_outputs only_inputs allow_unusedr1c3PK|]!}|jrtj|ndV"dSr(rrs r!rzz)make_graphed_callables..sJ$ $ AB1? .s)JJ1!/JQJJJJJJr c3(K|] }|j |VdSr(rurs r!rzz)make_graphed_callables..s) T TqAO T T T T T T Tr c3K|]}||V dSr(rrs r!rzz)make_graphed_callables..s"&W&WQq&W&Wr  fwd_graphr bwd_graph module_paramstuple[torch.nn.Parameter, ...] len_user_argsrDoutput_unflatten_spectorch.utils._pytree.TreeSpecstatic_input_surfaceristatic_outputsstatic_grad_outputstuple[Optional[Tensor], ...]static_grad_inputsrCallable[..., object]c t Gfddtjj d fd } | S)NceZdZed fd Zeejjjd fd Z d S) Omake_graphed_callables..make_graphed_autograd_function..Graphedctxr`rr rricxtD]Y}|||kr!|||Zt t st dt dDS)Nzstatic_outputs must be a tuplec3>K|]}|VdSr(detachrs r!rzzjmake_graphed_callables..make_graphed_autograd_function..Graphed.forward..s*@@AQXXZZ@@@@@@r )rangedata_ptrcopy_r:r|r RuntimeError)rrrrrrrs r!forwardzWmake_graphed_callables..make_graphed_autograd_function..Graphed.forwards}--AAA+A.7799VAY=O=O=Q=QQQ,Q/55fQi@@@  """!.%88I&'GHHH@@@@@@@@r gradsct|tkr/tdtdt|t|D]F\}}|?||kr||Gt tstdtdDS)Nz Expected z gradients but got z"static_grad_inputs must be a tuplec3FK|]}||n|VdSr(rrws r!rzzkmake_graphed_callables..make_graphed_autograd_function..Graphed.backward..sH#$-AHHJJJQr )rrziprrr:r|r)rrggradrrrs r!backwardzXmake_graphed_callables..make_graphed_autograd_function..Graphed.backwards u::%8!9!999&]C(;$<$<]]QTUZQ[Q[]] ##6>>**GAt}::<<4==??::GGDMMM  """!"4e<<M&'KLLL0r N)rr`rr rri)rr`rr rri) rKrLrM staticmethodrr#autogradfunctiononce_differentiabler)rrrrrrrsr!Graphedrs  A A A A A A A A\ A ^ $ 8       9 8\   r r user_argsr`rctjjj|}jt |z}tjj|Sr()r#utils_pytreearg_tree_leavesapplyrtree_unflatten)rflatten_user_argsoutrrrs r!functionalizedzVmake_graphed_callables..make_graphed_autograd_function..functionalizedsN % 3 CY O '-%(9":":]"JLC;&55c;PQQ Qr )rr`rr`)r#rFunction) rrrrrrrrrrrs ````````` @r!make_graphed_autograd_functionz>make_graphed_callables..make_graphed_autograd_functions! ! ! ! ! ! ! ! ! ! ! ! ! en-! ! ! F R R R R R R R R r funcrdgraph_training_statergraphedCallable[_P, _R]orig_fwdc dfd }|S) Nr_P.args user_kwargs _P.kwargsrrc:jkr|i|S|i|Sr()training)rrrrrrs r!new_fwdzEmake_graphed_callables..make_graphed_forward..new_fwds;}(<<<&w A[AAA'xBkBBBr )rrrrrrr)rrrrrs```` r!make_graphed_forwardz4make_graphed_callables..make_graphed_forwardsC CCCCCCCCC r )rrrrrrrrDrrrrirrirrrrirr) rrdrrrrrrrr)(r#is_autocast_enabledis_autocast_cache_enabledrr|rtypingcastr rrrr_backward_hooks_forward_hooks_forward_pre_hooksallbuffersrrrappend TypeErrorrrr$r\rTrV tree_leavesrrr tree_flattenreversedrvreverse enumeraterr)+rgrhrjrkr-just_one_callable _sample_argsrr_ flatten_argper_callable_len_user_args"per_callable_static_input_surfaces fwd_graphs bwd_graphsmempoolrr grad_inputsr outputs_gradrvper_callable_static_outputs"per_callable_output_unflatten_specr func_outputsflatten_outputsspec per_callable_static_grad_outputsper_callable_static_grad_inputsrrrrgrad_idxr}rretrrrrrs+ @@r!rrs N  "" u'F'H'H  m    i ' 'P L  E&#+$6 DDF {5vs{);S)@#A;OO y,//4 a ) ) A%&&!++())Q..,--22"aEEEEEEE "1 k)94@ ""5#5#5666HHKHHHHH ^  "M!L8K!L!L!L"""""*****s9~~&&***& GFc)nn0E0EFFFJFFc)nn0E0EFFFJ%)\!!!tG I   %)**,, - -03 |%G1 1    ,D$,2B .K,+,,  +-99$$+FF$$K$K$K$K$KKK |$$q(("'."5"5 ,$%%';%%%  &+++9@+++&&%)%7#6 # #K|[9  A ' . I#%)+&!$Y j!I!I88dI Y__YW_ 5 5 ' '4;L ' ' ' ' ' ' ' ' ' ' ' ' ' ' '!& 3 @ @ N N#**5+A+ABBB*11$7777(*$&(#;>344,--<< C C7ni $$ $ FT$ $ $   JJJJJJJ  |  q 99  #n11( T T,@ T T TTT!&&W&W2E&W&W&W!W!W $!3 2                 ' 0 0C  0[%<"))+h*?@@@A "))$////"#566(//0CDDD'../ABBBB%,,...#++---4444l$&CY''" " 400 qM qM &q ) &q ) .q 1 .q 1 ' * ,Q / +A .    dEHO , ,     0/dmWdlDL JJt     JJw    1v ::s8COOO&P88P< ?P< AV,,V0 3V0 )rrrJ)rfFN) rgrerhrirjrDrkrr-r.rre) rgrnrhrorjrDrkrr-r.rrn) rgrqrhrrrjrDrkrr-r.rrq)& __future__rrcollections.abcrrrrrr typing_extensionsr r r r#r torch.xpur_utilsr__all__rrhasattr_C__dict__torch._Crrrrrrrr`rercrrr r!rs""""""" $$$$$$FFFFFFFFFFFFFF6666666666 '&&&&&&     WT]]Yt__wux)**%0[%=%=EHk"2=+>V2W2WEH./9D'::EH56 VUUUUUUUUU++++<<<< ^,^,^,^,^,y^,^,^,B5(5(5(5(5(5(5(5(p %%6f8M%MNNNNN $#'   $#' (((( ($#' pppppppr