~WhJdZddlZddlmcmZddlmZgdZ dZ dZ dZ dS)z(Keras Utilities for DTensor related API.N) dtensor_api) alphabetabias depthwise embeddingsgammakernel moving_meanmoving_variance pointwise recurrentcZfd}tjj|S)aA decorator for injecting layout information to layer.__init__. Layout will be a new param for any of the weights for all the keras layers. Adding the param to all the __init__ method will be a big/duplicated work. This decorator is design to reduce and code duplication and make it easy to add/remove the dtensor feature if needed. Sample usage: ```python class Dense(tf.keras.layer.Layer): @allow_initializer_layout def __init__(self, units, kernel_initializer='zeros', bias_initializer='zeros', **kwargs): super().__init__(**kwargs) d = Dense(units=8, kernel_layout=layout1, bias_layout=layout2) d.kernel_layout == layout1 d.bias_layout == layout2 ``` By adding this annotation, it will: 1. Filter out the kwargs based on some keywords, eg if the 'kernel_initialzer' appears in method signature, then it will try to pop the 'kernel_layout' if it presents. Same for "bias" and "recurrent_kernel", etc. This will make sure the layout related param is not passed to `BaseLayer.__init__`, which will raise error about unexpect keyword args. 2. Set the self.kernel/bias_layout attribute after the `__init__` method is called. Keras framework will use those fields to create weights down the stream. Args: init_method: the `__init__` method of the Keras layer to annotate. Returns: the annotated __init__ method. ctj}i}tD]1}|dz|jvr#||dzd}|r|||dz<2|g|Ri||D]\}}t |||dS)N _initializer_layout)inspect signatureKERAS_VARIABLE_NAMES parameterspopitemssetattr) layer_instanceargskwargsr layout_args variable_namelayoutlayout_param_name init_methods Y/var/www/html/movieo_spanner_bot/venv/lib/python3.11/site-packages/keras/dtensor/utils.py_wrap_functionz0allow_initializer_layout.._wrap_functionUs%k22  2 D DM~-1EEEMI$=tDDD=CK 9: N4T444V444*5):):)<)< ? ? % v N$5v > > > > ? ?targetdecorator_functf __internal__ decoratormake_decoratorr!r#s` r"allow_initializer_layoutr.)sFX?????$ ? $ 3 3> 4  r$cZfd}tjj|S)aInject DTensor mesh information to an object. This is useful for keras object like `Metric` and `Optimizer` which need DTensor mesh to create the weights, but doesn't want to change the current public API interface. This is for temporary usage and eventually the mesh/layout information will be public arguments in the `__init__` method. Sample usage: ```python class Accuracy(tf.keras.metrics.Metric): @inject_mesh def __init__(self, name='accuracy', dtype=None): super().__init__(**kwargs) acc = Accuracy(mesh=mesh) assert acc._mesh == mesh ``` Args: init_method: the `__init__` method of the Keras class to annotate. Returns: the annotated __init__ method. c^|dd}|||_|g|Ri|dS)Nmesh)r_mesh)instancerrr1r!s r"r#z#inject_mesh.._wrap_functionsKzz&$''  !HN H.t...v.....r$r%r(r-s` r" inject_meshr4lsE:///// ? $ 3 3> 4  r$c|rNtj|j5||i|}tj||cdddS#1swxYwY||i|S)aInvoke the function with inputs and relayout the result. Args: fn: the function to invoke. layout: if not None, the output of the fn will be relayout with this. *args: positional arguments to be called with fn. **kwargs: keyword arguments to be called with fn. Returns: The output of fn, with potential relayout with the layout specified. N)dtensorrun_onr1relayout)fnrrrresults r"call_with_layoutr;s4 ^FK ( ( 4 4R(((F#FF33 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 2t v  sAA  A ) __doc__rtensorflow.compat.v2compatv2r) keras.dtensorrr6rr.r4r;r$r"rBs/.!!!!!!!!!000000   @@@F(((Vr$