WhddlmZddlmZmZddlZddlmZmZm Z ddl m Z ddl mZddlmZmZmZmZeedfZejZeedfZeeZe d d ddZe d d ddZe d d ddZeejGdd ZdS)) annotations)MappingSequenceN)safe_zip use_cpp_classcache) xla_bridge) xla_client)are_op_shardings_equalget_num_ways_dim_shardedis_op_sharding_replicatedop_sharding_to_indices.iF)max_sizetrace_context_in_keyshardingSharding global_shapeShapereturnMapping[Device, Index | None]c|||jrSt|drfd|jjDSdDS)N_internal_device_listc"i|] }|| Sr).0d global_maps W/var/www/html/movieo_spanner_bot/venv/lib/python3.11/site-packages/jax/_src/sharding.py z4_addressable_devices_indices_map..(s5 M M M z!} M M Mc^i|]*\}}|j|jk'||+Sr process_indexclient)rrinds rrz4_addressable_devices_indices_map..*sB : : :VQ _ 6 6 8 8 8 8 S 8 8 8r )devices_indices_mapis_fully_addressablehasattrraddressable_device_listitems)rrrs @r _addressable_devices_indices_mapr+!s++L99* "  X.//M M M M M3K M M MM : :z//11 : : ::r sMapping[Device, Index]c|||t|}t||t|j}t t |j|SN) shard_shape_to_xla_hlo_shardinglenr_device_assignmentdictr)r,r hlo_shardingindicess rcommon_devices_indices_mapr7-sm-- ''L(9(9::, "<#&q';#<#< > >' hq+W55 6 66r cT|t|}t|r|St|\}}t|t|ks&Jt|t|fg}t t ||D]o\}\}} t ||\} } n#t$rtwxYw| dkr"td|d|d|d|d|d|d | | pt|S) Nrz Sharding z implies that array axis z is partitioned z" times, but the dimension size is z (full shape: z , per-dimension tiling factors: z should evenly divide the shape)) r1r2r r enumeraterdivmod TypeErrorNotImplementedError ValueErrorappendtuple) selfrr5 partitions_outdimr,pquotient remainders r_common_shard_shaperH7s~**3|+<+<==,|,, *<88-*a ZC -- - - -J\ARAR/S - - - #x jAABB  kc6Aq "1aLLh     A~~  d  S     23  &  ,6      JJx ss 4CCceZdZdZed#dZed$dZed$dZed%d Zed&d Z d'dZ ed(dZ d)dZ d*dZ ejd#dZd+dZd,dZejd(dZd-dZd.d!Zd"S)/rzADescribes how a :class:`jax.Array` is laid out across devices. r set[Device]c td)zThe set of devices that this :class:`Sharding` spans. In multi-controller JAX, the set of devices is global, i.e., includes non-addressable devices from other processes. (Subclasses should implement this method.r<r@s r device_setzSharding.device_setV H I IIr boolc td)zIs this sharding fully replicated? A sharding is fully replicated if each device has a complete copy of the entire data. rLrMrNs ris_fully_replicatedzSharding.is_fully_replicated_rPr c td)zIs this sharding fully addressable? A sharding is fully addressable if the current process can address all of the devices named in the :class:`Sharding`. ``is_fully_addressable`` is equivalent to "is_local" in multi-process JAX. rLrMrNs rr'zSharding.is_fully_addressablehs H I IIr intc td)z-Number of devices that the sharding contains.rLrMrNs r num_deviceszSharding.num_devicesr H I IIr str | Nonec td)z(Returns the memory kind of the sharding.rLrMrNs r memory_kindzSharding.memory_kindwrXr kindstrc td)z?Returns a new Sharding instance with the specified memory kind.z'Subclasses should implement this methodrM)r@r\s rwith_memory_kindzSharding.with_memory_kind|s G H HHr XLADeviceAssignmentc tdNrLrMrNs rr3zSharding._device_assignments H I IIr num_dimensionsxc.HloShardingc tdrbrMr@rcs rr1zSharding._to_xla_hlo_sharding H I IIr c tdrbrMrfs r_to_sdy_shardingzSharding._to_sdy_shardingrgr c`tjdkr|jSd|jDS)zdThe set of devices in the :class:`Sharding` that are addressable by the current process. cVh|]&}|j|jk$|'Srr"rrs r z/Sharding.addressable_devices..s< < < z:Sharding._addressable_device_assignment..sLAAqOqx'='='?'???????AAr )r'r3r(r?rr)rNs r_addressable_device_assignmentz'Sharding._addressable_device_assignmentsv  %  $$t,--G 4-E F FF AAD3AAA A AAr c"t||S)zReturns the shape of the data on each device. The shard shape returned by this function is calculated from ``global_shape`` and the properties of the sharding. )rHrss rr0zSharding.shard_shapes t\ 2 22r r@otherndimc t||||o|j|jko|j|jkS#t$r ||kcYSwxYw)aCReturns ``True`` if two shardings are equivalent. Two shardings are equivalent if they place the same logical array shards on the same devices. For example, a :class:`NamedSharding` may be equivalent to a :class:`PositionalSharding` if both place the same shards of the array on the same devices. )r r1rr[r<)r@r{r|s ris_equivalent_tozSharding.is_equivalent_tos $T%>%>t%D%D%*%?%?%E%EGG4,0KK4%"335  U]sAAA+*A+N)rrJ)rrQ)rrU)rrY)r\r]rr)rr`)rcrUrrd)rcrU)rrrr)rrrr-rrrr)r@rr{rr|rUrrQ)__name__ __module__ __qualname____doc__propertyrOrSr'rWr[r_r3r1ri functoolscached_propertyrqrtr&ryr0r~rr rrrPs JJJ 8J JJJ 8J JJJ 8J JJJ 8J JJJ 8JIIII JJJ 8JJJJJJJJJ  <<<<@@@@:::: AAAA3333r )rrrrrr)r,rrrrr-r) __future__rcollections.abcrrr jax._src.utilrrrjax._srcr ro jax._src.libr xcjax._src.op_shardingsr r r rr?rUrDevicesliceIndexr`r+r7rHrrr rrs#"""""--------8888888888%%%%%%)))))) c3h  eSjv&5111 : : :21 :51117772175111210r{||||||||||r