$ShvUdZddlmZddlZddlZddlZddlZddlmZm Z m Z m Z ddl m Z mZddlmZddlmZmZmZmZmZmZmZmZddlmZddlmZmZmZm Z m!Z!m"Z"dd l#m$Z$m%Z%m&Z&m'Z'm(Z(dd l)m*Z*m+Z+m,Z,er/ddl-Z-dd l.m/Z/m0Z0m1Z1dd l#m2Z2m3Z3m4Z4m5Z5m6Z6ed e7e4Z8gdZ9ej:dkrddiniZ;ej<dddddde;Gddee&Z=[;GddZ>e>Z?de@d<eZAde@d<[>erJddlBmCZCeCdZDedZEeCdZFed ZGGd!d"eeDeEeFeGfZHdd'ZIed(d)dd.ZJe dd(d)dd1ZJ dd(d)dd4ZJeIeJed5dd=ZK[J[Ie dddd>ddBZLeddCZL dddd>ddFZLddGZMejNddKZOddOZPddRZQddUZRddYZSdd[ZTdd_ZUdd`ZVdddZWddiZXddjZYddkZZddmZ[ddnZ\ddqZ]ddtZ^dduZ_ddvZ`ddzZadd}ZbddZcddZdddZeddZfe7de=e7deQeRe e$jgehe=eheSeTe!e$jieje=ejeUeVe!e$jkele=eleWeXee$jme e=e ecedee$jnee=ee[e\ee$joe e=e e]e^ee$jpe e=e eaebe!e$jqe%e=e%eeefe"e$jri Zsde@d<e=eleYeZee$jmZte=e e_e`ee$jpZudS)z&Registry for custom pytree node types.) annotationsN) OrderedDict defaultdictdeque namedtuple) itemgetter methodcaller)Lock) TYPE_CHECKINGAnyCallableClassVarGeneric NamedTupleTypeVaroverload) AutoEntry MappingEntryNamedTupleEntry PyTreeEntry SequenceEntryStructSequenceEntry) PyTreeKindStructSequenceTis_namedtuple_classis_structseq_class)safe_ziptotal_order_sortedunzip2) Collection GeneratorIterable)KTVTCustomTreeNode FlattenFunc UnflattenFuncCustomTreeNodeType)bound)register_pytree_noderegister_pytree_node_classunregister_pytree_nodedict_insertion_ordered slotsT)initrepreqfrozenceZdZUdZded<ded<ded<ejdkrd ed <eZd ed <e j Z d ed<dZ ded<dS)PyTreeNodeRegistryEntryz>A dataclass that stores the information of a pytree node type.zbuiltins.type[Collection[T]]typeFlattenFunc[T] flatten_funcUnflattenFunc[T]unflatten_funcr/zdataclasses.KW_ONLY_zbuiltins.type[PyTreeEntry]path_entry_typerkindstr namespaceN) __name__ __module__ __qualname____doc____annotations__sys version_inforr?rCUSTOMr@rCU/var/www/html/movieo_spanner_bot/venv/lib/python3.11/site-packages/optree/registry.pyr8r8@sHH&&&&    $$$$ 7""2;O;;;;!(D((((IrMr8c&eZdZUdZded<ddZdS) GlobalNamespacerLzClassVar[tuple[()]] __slots__returnrBcdS)NzrL)selfs rN__repr__zGlobalNamespace.__repr__Ws##rMN)rRrB)rDrErFrQrHrUrLrMrNrPrPTs;%'I''''$$$$$$rMrPrB__GLOBAL_NAMESPACEr __REGISTRY_LOCK) ParamSpec_P_T_GetP_GetTceZdZd dZdd Zd S)_CallableWithGetargs_P.argskwargs _P.kwargsrRrZctNNotImplementedErrorrTr_ras rN__call__z_CallableWithGet.__call__i% %rM _GetP.args _GetP.kwargsr\ctrdrergs rNgetz_CallableWithGet.getmrirMN)r_r`rarbrRrZ)r_rjrarkrRr\)rDrErFrhrmrLrMrNr^r^hs< & & & & & & & & & &rMr^rmCallable[_GetP, _GetT]rRDCallable[[Callable[_P, _T]], _CallableWithGet[_P, _T, _GetP, _GetT]]cdfd }|S)NfuncCallable[_P, _T]rR&_CallableWithGet[_P, _T, _GetP, _GetT]c|_|Srd)rm)rqrms rN decoratorz_add_get..decoratorxs rM)rqrrrRrsrL)rmrus` rN_add_getrvqs) rMrA)rCclsr9rCPyTreeNodeRegistryEntry | NonecdSrdrLrwrCs rNpytree_node_registry_getr{s &)SrMNone#dict[type, PyTreeNodeRegistryEntry]cdSrdrLrzs rNr{r{s +.#rM type | NoneDdict[type, PyTreeNodeRegistryEntry] | PyTreeNodeRegistryEntry | Nonec|turd}|0|tur'tj|st d|dt |t st d|d|t|dht5fdt D}dddn #1swxYwYtj |rt|t<t|t <|S|dkr t||f}||Stj |r |turtS|t urtSt|}||St%|rtt&St)|rttSdS)aLookup the pytree node registry. >>> register_pytree_node.get() # doctest: +IGNORE_WHITESPACE,ELLIPSIS { : PyTreeNodeRegistryEntry( type=, flatten_func=, unflatten_func=, path_entry_type=, kind=, namespace='' ), : PyTreeNodeRegistryEntry( type=, flatten_func=, unflatten_func=, path_entry_type=, kind=, namespace='' ), : PyTreeNodeRegistryEntry( type=, flatten_func=, unflatten_func=, path_entry_type=, kind=, namespace='' ), ... } >>> register_pytree_node.get(defaultdict) # doctest: +IGNORE_WHITESPACE,ELLIPSIS PyTreeNodeRegistryEntry( type=, flatten_func=, unflatten_func=, path_entry_type=, kind=, namespace='' ) >>> register_pytree_node.get(frozenset) # frozenset is considered as a leaf node None Args: cls (type or None, optional): The class of the pytree node to retrieve. If not provided, all the registered pytree nodes in the namespace are returned. namespace (str, optional): The namespace of the registry to retrieve. If not provided, the global namespace is used. Returns: If the ``cls`` is not provided, a dictionary of all the registered pytree nodes in the namespace is returned. If the ``cls`` is provided, the corresponding registry entry is returned if the ``cls`` is registered as a pytree node. Otherwise, :data:`None` is returned, i.e., the ``cls`` is represented as a leaf node. rANzExpected a class or None, got .$The namespace must be a string, got c2i|]}|jv |j|SrL)rCr9).0handler namespacess rN z,pytree_node_registry_get..s5$ 22 g222rM)rVrinspectisclass TypeError isinstancerB frozensetrW_NODETYPE_REGISTRYvalues_Cis_dict_insertion_ordered&_DICT_INSERTION_ORDERED_REGISTRY_ENTRYdict-_DEFAULTDICT_INSERTION_ORDERED_REGISTRY_ENTRYrrmrrr)rwrCregistryrrs @rNr{r{s'x&&&   z ! !$$ "AAAABBB i % %  A9 A A A    { 2//   188::H                 ' 2 2 RCHTN$QH[ !B$(()S)9::  N #I..A $;;9 9 +  @ @ $$S))G#6!%%n55532!%%j111 4s&B33B7:B7)r?r;r:r=r<type[Collection[T]]r?type[PyTreeEntry]c"tj|std|dtj|rt|tstd|d|t ur(t |tstd|d|dkrtd|t ur|}d}n||f}t5tj |||||t|||||t|<dddn #1swxYwY|S) aExtend the set of types that are considered internal nodes in pytrees. See also :func:`register_pytree_node_class` and :func:`unregister_pytree_node`. The ``namespace`` argument is used to avoid collisions that occur when different libraries register the same Python type with different behaviors. It is recommended to add a unique prefix to the namespace to avoid conflicts with other libraries. Namespaces can also be used to specify the same class in different namespaces for different use cases. .. warning:: For safety reasons, a ``namespace`` must be specified while registering a custom type. It is used to isolate the behavior of flattening and unflattening a pytree node type. This is to prevent accidental collisions between different libraries that may register the same type. Args: cls (type): A Python type to treat as an internal pytree node. flatten_func (callable): A function to be used during flattening, taking an instance of ``cls`` and returning a triple or optionally a pair, with (1) an iterable for the children to be flattened recursively, and (2) some hashable metadata to be stored in the treespec and to be passed to the ``unflatten_func``, and (3) (optional) an iterable for the tree path entries to the corresponding children. If the entries are not provided or given by :data:`None`, then `range(len(children))` will be used. unflatten_func (callable): A function taking two arguments: the metadata that was returned by ``flatten_func`` and stored in the treespec, and the unflattened children. The function should return an instance of ``cls``. path_entry_type (type, optional): The type of the path entry to be used in the treespec. (default: :class:`AutoEntry`) namespace (str): A non-empty string that uniquely identifies the namespace of the type registry. This is used to isolate the registry from other modules that might register a different custom behavior for the same type. Returns: The same type as the input ``cls``. Raises: TypeError: If the input type is not a class. TypeError: If the path entry class is not a subclass of :class:`PyTreeEntry`. TypeError: If the namespace is not a string. ValueError: If the namespace is an empty string. ValueError: If the type is already registered in the registry. Examples: >>> # Registry a Python type with lambda functions >>> register_pytree_node( ... set, ... lambda s: (sorted(s), None, None), ... lambda _, children: set(children), ... namespace='set', ... ) >>> # Register a Python type into a namespace >>> import torch >>> register_pytree_node( ... torch.Tensor, ... flatten_func=lambda tensor: ( ... (tensor.cpu().detach().numpy(),), ... {'dtype': tensor.dtype, 'device': tensor.device, 'requires_grad': tensor.requires_grad}, ... ), ... unflatten_func=lambda metadata, children: torch.tensor(children[0], **metadata), ... namespace='torch2numpy', ... ) >>> # doctest: +SKIP >>> tree = {'weight': torch.ones(size=(1, 2)).cuda(), 'bias': torch.zeros(size=(2,))} >>> tree {'weight': tensor([[1., 1.]], device='cuda:0'), 'bias': tensor([0., 0.])} >>> # Flatten without specifying the namespace >>> tree_flatten(tree) # `torch.Tensor`s are leaf nodes ([tensor([0., 0.]), tensor([[1., 1.]], device='cuda:0')], PyTreeSpec({'bias': *, 'weight': *})) >>> # Flatten with the namespace >>> tree_flatten(tree, namespace='torch2numpy') ( [array([0., 0.], dtype=float32), array([[1., 1.]], dtype=float32)], PyTreeSpec( { 'bias': CustomTreeNode(Tensor[{'dtype': torch.float32, 'device': device(type='cpu'), 'requires_grad': False}], [*]), 'weight': CustomTreeNode(Tensor[{'dtype': torch.float32, 'device': device(type='cuda', index=0), 'requires_grad': False}], [*]) }, namespace='torch2numpy' ) ) >>> # Register the same type with a different namespace for different behaviors >>> def tensor2flatparam(tensor): ... return [torch.nn.Parameter(tensor.reshape(-1))], tensor.shape, None ... ... def flatparam2tensor(metadata, children): ... return children[0].reshape(metadata) ... ... register_pytree_node( ... torch.Tensor, ... flatten_func=tensor2flatparam, ... unflatten_func=flatparam2tensor, ... namespace='tensor2flatparam', ... ) >>> # Flatten with the new namespace >>> tree_flatten(tree, namespace='tensor2flatparam') ( [ Parameter containing: tensor([0., 0.], requires_grad=True), Parameter containing: tensor([1., 1.], device='cuda:0', requires_grad=True) ], PyTreeSpec( { 'bias': CustomTreeNode(Tensor[torch.Size([2])], [*]), 'weight': CustomTreeNode(Tensor[torch.Size([1, 2])], [*]) }, namespace='tensor2flatparam' ) ) Expected a class, got r(Expected a subclass of PyTreeEntry, got rrA(The namespace cannot be an empty string.r?rCN)rrr issubclassrrVrrB ValueErrorrWr register_noder8r)rwr;r=r?rCregistration_keys rNr+r+s~ ?3  ;9999::: OO , ,YO[1Y1YYW?WWWXXX***:i3M3M*MyMMMNNNBCDDD&&& %s+             0G   + 0 0 0 +,                Js5DD Dr str | Nonetype[PyTreeEntry] | None2Callable[[CustomTreeNodeType], CustomTreeNodeType]cdSrdrLrwr?rCs rNr,r,s :=rMcdSrdrLrs rNr,r,s rMCustomTreeNodeType | str | NoneGCustomTreeNodeType | Callable[[CustomTreeNodeType], CustomTreeNodeType]c|tust|tr+td|dkrtdd|c}tdtur'ttst ddkrtd| dfd }|St j|st d |d t|d tt jrttst dd t|td|j |S)a Extend the set of types that are considered internal nodes in pytrees. See also :func:`register_pytree_node` and :func:`unregister_pytree_node`. The ``namespace`` argument is used to avoid collisions that occur when different libraries register the same Python type with different behaviors. It is recommended to add a unique prefix to the namespace to avoid conflicts with other libraries. Namespaces can also be used to specify the same class in different namespaces for different use cases. .. warning:: For safety reasons, a ``namespace`` must be specified while registering a custom type. It is used to isolate the behavior of flattening and unflattening a pytree node type. This is to prevent accidental collisions between different libraries that may register the same type. Args: cls (type, optional): A Python type to treat as an internal pytree node. path_entry_type (type, optional): The type of the path entry to be used in the treespec. (default: :class:`AutoEntry`) namespace (str, optional): A non-empty string that uniquely identifies the namespace of the type registry. This is used to isolate the registry from other modules that might register a different custom behavior for the same type. Returns: The same type as the input ``cls`` if the argument presents. Otherwise, return a decorator function that registers the class as a pytree node. Raises: TypeError: If the path entry class is not a subclass of :class:`PyTreeEntry`. TypeError: If the namespace is not a string. ValueError: If the namespace is an empty string. ValueError: If the type is already registered in the registry. This function is a thin wrapper around :func:`register_pytree_node`, and provides a class-oriented interface: .. code-block:: python @register_pytree_node_class(namespace='foo') class Special: TREE_PATH_ENTRY_TYPE = GetAttrEntry def __init__(self, x, y): self.x = x self.y = y def tree_flatten(self): return ((self.x, self.y), None, ('x', 'y')) @classmethod def tree_unflatten(cls, metadata, children): return cls(*children) @register_pytree_node_class('mylist') class MyList(UserList): TREE_PATH_ENTRY_TYPE = SequenceEntry def tree_flatten(self): return self.data, None, None @classmethod def tree_unflatten(cls, metadata, children): return cls(*children) Nz?Cannot specify `namespace` when the first argument is a string.rArz.decorator s#- /# rMrrTREE_PATH_ENTRY_TYPEr tree_flattenr)rwr)rRr))rVrrBrrrrgetattrrrrr+r tree_unflatten)rwr?rCrus `` rNr,r,sL    JsC$8$8  ^__ _ "99GHH HsYWXXX***:i3M3M*LyLLMMMBCDDD {        ?3  ;9999:::!#'=yII OO , ,YO[1Y1YYW?WWWXXX ^$$ '  JrMctj|std|d|tur(t |t std|d|dkrt d|tur|}d}n||f}t5tj ||t |cdddS#1swxYwYdS)aRemove a type from the pytree node registry. See also :func:`register_pytree_node` and :func:`register_pytree_node_class`. This function is the inverse operation of function :func:`register_pytree_node`. Args: cls (type): A Python type to remove from the pytree node registry. namespace (str): The namespace of the pytree node registry to remove the type from. Returns: The removed registry entry. Raises: TypeError: If the input type is not a class. TypeError: If the namespace is not a string. ValueError: If the namespace is an empty string. ValueError: If the type is a built-in type that cannot be unregistered. ValueError: If the type is not found in the registry. Examples: >>> # Register a Python type with lambda functions >>> register_pytree_node( ... set, ... lambda s: (sorted(s), None, None), ... lambda _, children: set(children), ... namespace='temp', ... ) >>> # Unregister the Python type >>> unregister_pytree_node(set, namespace='temp') rrrrArN) rrrrVrrBrrWrunregister_noderpop)rwrCrs rNr-r-%s3D ?3  ;9999:::***:i3M3M*MyMMMNNNBCDDD&&& %s+ 88 3 ***!%%&677888888888888888888s/CC CmodeboolGenerator[None]c#@K|tur(t|tstd|d|dkrt d|turd}t 5t j|d}t jt||dddn #1swxYwY dVt 5t j||ddddS#1swxYwYdS#t 5t j||dddw#1swxYwYwxYw)aContext manager to temporarily set the dictionary sorting mode. This context manager is used to temporarily set the dictionary sorting mode for a specific namespace. The dictionary sorting mode is used to determine whether the keys of a dictionary should be sorted or keeping the insertion order when flattening a pytree. >>> tree = {'b': (2, [3, 4]), 'a': 1, 'c': None, 'd': 5} >>> tree_flatten(tree) # doctest: +IGNORE_WHITESPACE ( [1, 2, 3, 4, 5], PyTreeSpec({'a': *, 'b': (*, [*, *]), 'c': None, 'd': *}) ) >>> with dict_insertion_ordered(True, namespace='some-namespace'): # doctest: +IGNORE_WHITESPACE ... tree_flatten(tree, namespace='some-namespace') ( [2, 3, 4, 1, 5], PyTreeSpec({'b': (*, [*, *]), 'a': *, 'c': None, 'd': *}, namespace='some-namespace') ) .. warning:: The dictionary sorting mode is a global setting and is **not thread-safe**. It is recommended to use this context manager in a single-threaded environment. Args: mode (bool): The dictionary sorting mode to set. namespace (str): The namespace to set the dictionary sorting mode for. rrrArF)inherit_global_namespaceN) rVrrBrrrWrrset_dict_insertion_orderedr)rrCprevs rNr.r.Zs:***:i3M3M*MyMMMNNNBCDDD&&& ==+IPUVVV %d4jj)<<<===============;   ; ;  )$ : : : ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ;_ ; ;  )$ : : : ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ;sT9B  B$'B$,C'7CC!C'D/D DD DD DitemsIterable[tuple[KT, VT]]list[tuple[KT, VT]]c>t|tdS)Nr)key)rr)rs rN _sorted_itemsrs eA 7 7 77rMr>tuple[tuple[()], None]cdS)N)rLNrL)r>s rN _none_flattenrs 8rMchildren Iterable[Any]c~t}tt|||urtddS)NzExpected no children.)objectnextiterr)r>rsentinels rN_none_unflattenrs=xxH DNNH%%X55011165rMtup tuple[T, ...]tuple[tuple[T, ...], None]c |dfSrdrLrs rN_tuple_flattenr 9rM Iterable[T]c t|Srd)tupler>rs rN_tuple_unflattenrs ??rMlstlist[T]tuple[list[T], None]c |dfSrdrL)rs rN _list_flattenrrrMc t|Srd)listrs rN_list_unflattenrs >>rMdct dict[KT, VT]/tuple[tuple[VT, ...], list[KT], tuple[KT, ...]]ctt|\}}|t||fSrd)r rrrrkeysrs rN _dict_flattenrs6- 4455LD& 4::t ##rMrlist[KT]r Iterable[VT]c<tt||Srdrrrrs rN_dict_unflattenr v&& ' ''rMcnt|\}}|t||fSrdr rrrs rN_dict_insertion_ordered_flattenr0#))++&&LD& 4::t ##rMc<tt||Srdrrs rN!_dict_insertion_ordered_unflattenrrrMOrderedDict[KT, VT]cnt|\}}|t||fSrdrrs rN_ordereddict_flattenrrrMc<tt||Srd)rrrs rN_ordereddict_unflattenrs xf-- . ..rMdefaultdict[KT, VT]Otuple[tuple[VT, ...], tuple[Callable[[], VT] | None, list[KT]], tuple[KT, ...]]c@t|\}}}||j|f|fSrd)rdefault_factoryrrrentriess rN_defaultdict_flattenrs-*#..FD' C'. 77rMmetadata!tuple[Callable[[], VT], list[KT]]cH|\}}t|t||Srd)rrrrrrs rN_defaultdict_unflattenrs( %OT f(E(E F FFrMc@t|\}}}||j|f|fSrd)rrrs rN&_defaultdict_insertion_ordered_flattenrs-rMtype[NamedTuple[T]]c ||SrdrLrwrs rN_namedtuple_unflattenrs 3>rMseqStructSequence[T]-tuple[tuple[T, ...], type[StructSequence[T]]]c$|t|fSrdr)rs rN_structseq_flattenr rrMtype[StructSequence[T]]c||SrdrLrs rN_structseq_unflattenr"s 3x==rM)r?r@z6dict[type | tuple[str, type], PyTreeNodeRegistryEntry]rrL)rmrnrRro)rwr9rCrBrRrxrd)rwr|rCrBrRr})rwrrCrBrRr) r;r:r=r<rwrr?rrCrBrRr)rwrr?rrCrrRr)rwr)r?rrCrBrRr))rwrr?rrCrrRr)rwr9rCrBrRr8)rrrCrBrRr)rrrRr)r>r|rRr)r>r|rrrRr|)rrrRr)r>r|rrrRr)rrrRr)r>r|rrrRr)rrrRr)rrrrrRr)rrrRr)rrrrrRr)rrrRr)rrrrrRr)rrrRr )r rrrrRr)rrrRr)rwrrrrRr)rrrRr)rwr rrrRr)vrG __future__r contextlib dataclassesrrI collectionsrrrroperatorrr threadingr typingr r r rrrrr optree._Croptree.accessorsrrrrrr optree.typingrrrrr optree.utilsrrr builtinscollections.abcr!r"r#r$r%r&r'r(r9r)__all__rJSLOTS dataclassr8rPrVrHrWtyping_extensionsrXrYrZr[r\r^rvr{r+r,r-contextmanagerr.rrrrrrrrrrrrrrrrrr rrrrr"NONErTUPLErLISTrDICT NAMEDTUPLE ORDEREDDICT DEFAULTDICTDEQUESTRUCTSEQUENCErrrrLrMrNr>s( -,,#""""" CCCCCCCCCCCC--------aaaaaaaaaaaaaaaaaaaaa`````````````==========SOOO??????????PPPPPPPPPPPPPP!!5T.=QRRR   +w66$BKDtTKKUKK     gaj  LK  $$$$$$$$*/++++++&++++++ 4B B Ig  E GG  E&&&&&72r5%#78&&&      ))))) ) . ..... .h hhhhhhV  "##*3 ]]]]]$#]@h =15 ===== =  ,0m15 mmmmmm`28282828j +;+;+;+;\88882222 $$$$ (((( $ $ $ $(((( $ $ $ $//// 8 8 8 8GGGG 8 8 8 8YYYY****  DJJ'' T # _  " " %       ! ! % _     ! ! $ _    '''  " (($  # (($  #  " " %      +++  & s@N@@@@F*A)@#%  ***&1H0G*,   111---rM