§ z”jŽãó¼—ddlmZddlmZmZerddlmZddlmZddl m Z ddl m Z ddl ZddlZddlmZddlmZdd lmZd d lmZmZ d d!d„Z d"d#d„ZdS)$é)Ú annotations)Ú TYPE_CHECKINGÚAny)ÚSequence)ÚTensor)Útask)ÚGroupN)Ú framework)Ústreamé)Úconvert_object_to_tensorÚconvert_tensor_to_objectTÚtensorrÚ tensor_listúSequence[Tensor] | NoneÚsrcÚintÚgroupú Group | NoneÚsync_opÚboolÚreturnú task | Nonecó2—tj|||||¦«S)ak Scatter a tensor to all participators. As shown below, one process is started with a GPU and the source of the scatter is GPU0. Through scatter operator, the data in GPU0 will be sent to all GPUs averagely. .. image:: https://githubraw.cdn.bcebos.com/PaddlePaddle/docs/develop/docs/api/paddle/distributed/img/scatter.png :width: 800 :alt: scatter :align: center Args: tensor (Tensor): The output Tensor. Its data type should be float16, float32, float64, int32, int64, int8, uint8, bool or bfloat16. tensor_list (list|tuple): A list/tuple of Tensors to scatter. Every element in the list must be a Tensor whose data type should be float16, float32, float64, int32, int64, int8, uint8, bool or bfloat16. Default value is None. src (int): The source rank id. Default value is 0. group (Group, optional): The group instance return by new_group or None for global default group. sync_op (bool, optional): Whether this op is a sync op. The default value is True. Returns: None. Examples: .. code-block:: python >>> # doctest: +REQUIRES(env: DISTRIBUTED) >>> import paddle >>> import paddle.distributed as dist >>> dist.init_parallel_env() >>> if dist.get_rank() == 0: ... data1 = paddle.to_tensor([7, 8, 9]) ... data2 = paddle.to_tensor([10, 11, 12]) ... dist.scatter(data1, src=1) >>> else: ... data1 = paddle.to_tensor([1, 2, 3]) ... data2 = paddle.to_tensor([4, 5, 6]) ... dist.scatter(data1, tensor_list=[data1, data2], src=1) >>> print(data1, data2) >>> # [1, 2, 3] [10, 11, 12] (2 GPUs, out for rank 0) >>> # [4, 5, 6] [4, 5, 6] (2 GPUs, out for rank 1) )r Úscatter)rrrrrs úx/lsinfo/ai/hellotax_ai/data_center/backend/venv/lib/python3.11/site-packages/paddle/distributed/communication/scatter.pyrr's€õb Œ>˜& +¨s°E¸7Ñ CÔ CÐCóÚout_object_listú list[Any]Úin_object_listúlist[Any] | NoneÚNonecó¼—tj¦«s Jd¦«‚tj¦«}g}g}||krQ|D]>}t |¦«\}} | |¦«| | ¦«Œ?t |¦«} ntjgd¬¦«} tj | |¦«t|   ¦«¦«} g} |D]U} |   ¦«}tj|| g¦«}tj|¦«}|  |¦«ŒVtj| gd¬¦«}t#|||kr| nd||¦«tjgd¬¦«}t#|||kr|nd||¦«| ¦«| t'||  ¦«¦«¦«dS)a Scatter picklable objects from the source to all others. Similar to scatter(), but python object can be passed in. Args: out_object_list (list): The list of objects to store the scattered objects. in_object_list (list): The list of objects to scatter. Only objects on the src rank will be scattered. src (int): The source rank in global view. group (Group): The group instance return by new_group or None for global default group. Returns: None. Warning: This API only supports the dygraph mode. Examples: .. code-block:: python >>> # doctest: +REQUIRES(env: DISTRIBUTED) >>> import paddle.distributed as dist >>> dist.init_parallel_env() >>> out_object_list = [] # type: ignore >>> if dist.get_rank() == 0: ... in_object_list = [{'foo': [1, 2, 3]}, {'foo': [4, 5, 6]}] >>> else: ... in_object_list = [{'bar': [1, 2, 3]}, {'bar': [4, 5, 6]}] >>> dist.scatter_object_list(out_object_list, in_object_list, src=1) >>> print(out_object_list) >>> # [{'bar': [1, 2, 3]}] (2 GPUs, out for rank 0) >>> # [{'bar': [4, 5, 6]}] (2 GPUs, out for rank 1) z6scatter_object_list doesn't support static graph mode.Úint64)ÚdtypeÚuint8N)r Úin_dynamic_modeÚdistÚget_rankr ÚappendÚmaxÚpaddleÚemptyr Ú broadcastrÚitemÚnumpyÚnpÚresizeÚ to_tensorrÚclearr)rr rrÚrankÚin_obj_tensorsÚ in_obj_sizesÚobjÚ obj_tensorÚobj_sizeÚmax_obj_size_tensorÚ max_obj_sizeÚin_tensor_listrÚ numpy_dataÚ in_tensorÚ out_tensorÚout_tensor_sizes rÚscatter_object_listrB[sÿ€õN Ô $Ñ &Ô &ððØ@ñôÐ &õ Œ=‰?Œ?€DØ€NØ€Là ˆs‚{€{Ø!ð *ð *ˆCÝ#;¸CÑ#@Ô#@Ñ ˆJ˜Ø × !Ò ! *Ñ -Ô -Ð -Ø × Ò  Ñ )Ô )Ð )Ð )Ý! ,Ñ/Ô/ÐÐå$œl¨2°WÐ=Ñ=Ô=ÐÝ ÔÐ(¨#Ñ.Ô.Ð.ÝÐ*×/Ò/Ñ1Ô1Ñ2Ô2€Lð€NØ ð)ð)ˆØ—\’\‘^”^ˆ Ý”Y˜z¨L¨>Ñ:Ô:ˆ ÝÔ$ ZÑ0Ô0ˆ Ø×Ò˜iÑ(Ô(Ð(Ð(Ý”˜|˜n°GÐ<Ñ<Ô<€JÝ ˆJ¨$°#ª+¨+˜˜¸4ÀÀeÑLÔLÐLå”l 2¨WÐ5Ñ5Ô5€OÝ ˆO¨T°Sª[¨[˜\˜\¸dÀCÈÑOÔOÐOà×ÒÑÔÐØ×ÒÝ  ¨_×-AÒ-AÑ-CÔ-CÑDÔDñôðððr)NrNT) rrrrrrrrrrrr)NrN) rrr r!rrrrrr")Ú __future__rÚtypingrrÚcollections.abcrr,rÚpaddle.base.corerÚ&paddle.distributed.communication.groupr r0r1Úpaddle.distributedÚ distributedr(r Ú paddle.distributed.communicationr Úserialization_utilsr rrrB©rrúrMsVðð#Ð"Ð"Ð"Ð"Ð"à%Ð%Ð%Ð%Ð%Ð%Ð%Ð%àð=Ø(Ð(Ð(Ð(Ð(Ð(àÐÐÐÐÐØ%Ð%Ð%Ð%Ð%Ð%Ø<Ð<Ð<Ð<Ð<Ð<àÐÐÐà € € € Ø!Ð!Ð!Ð!Ð!Ð!ØÐÐÐÐÐØ3Ð3Ð3Ð3Ð3Ð3ððððððððð,0ØØØð 1Dð1Dð1Dð1Dð1Dðl(,ØØð JðJðJðJðJðJðJr