$h}0ddlZddlmZddlmZmZmZmZmZmZm Z ddl m Z m Z ddgZ GddeZd d ed ed ed ze_ dde ede ede ede ede edededededededefdZde ede ede ede edededededededefdZde ede ede ede edededededededefdZdS)N)Tensor) Optimizer_use_grad_for_differentiable_default_to_fused_or_foreach_differentiable_doc _foreach_doc _maximize_doc _view_as_real)ListOptionalRproprpropcreZdZ ddddddeeded effd Zfd Zd Zedd Z xZ S)r{Gz?g?g333333?gư>2NF)foreachmaximizedifferentiablerrrc,d|kstd|d|dcxkrdcxkr |dks#ntd|dd|dt||||||}t||dS) NgzInvalid learning rate: rg?rzInvalid eta values: z, )lretas step_sizesrrr) ValueErrordictsuper__init__) selfparamsrrrrrrdefaults __class__s Q/var/www/html/auto_sub_bot/venv/lib/python3.11/site-packages/torch/optim/rprop.pyrzRprop.__init__ sbyy;r;;<< <T!W,,,,s,,,,T!W,,,,HDGHHtAwHHII I!)     *****ct||jD]D}|dd|dd|ddEdS)NrrFr)r __setstate__ param_groups setdefault)r stategroupr#s r$r'zRprop.__setstate__%s} U###& 6 6E   Y - - -   Z / / /   -u 5 5 5 5 6 6r%cZd}|dD]}|j |tj|z}|||j}|jrt d|||j|} t| dkrd| d<tj|tj | d<|j jrX| | t|d|d| d <nC| | |d| d <|| d|| d | dxxd z cc<|S) NFr!z'Rprop does not support sparse gradientsrstep memory_formatprevr step_sizer)gradtorch is_complexappend is_sparse RuntimeErrorr*len zeros_likepreserve_formatdtypenew resize_as_fill_complex) r r+r!gradsprevsr has_complexpr2r*s r$ _init_groupzRprop._init_group,s x" " Av~ 5+A.. .K MM!   6D~ N"#LMMM LL   JqME5zzQ !f % 0U%:!!!f 7%  #D))wuT{E$K@@AA+&& --d3399%+FF+& LLv ' ' '   eK0 1 1 1 &MMMQ MMMMr%c\d}|5tj5|}dddn #1swxYwY|jD]j}g}g}g}g}|d\}} |d\} } |d} |d} ||||||}t ||||| | || | | |d| k|S)zPerforms a single optimization step. Args: closure (Callable, optional): A closure that reevaluates the model and returns the loss. Nrrrrr) step_size_min step_size_maxetaminusetaplusrrrrB)r3 enable_gradr(rDr)r closurelossr+r!r@rArrHrIrFrGrrrBs r$r-z Rprop.stepSs?  "$$ ! !wyy ! ! ! ! ! ! ! ! ! ! ! ! ! ! !&  EFEEJ %f Hg+0+> (M=I&GZ(H**5&% SSK ++!!$%56'      s /33)rrr)N) __name__ __module__ __qualname__r boolrr'rDrr- __classcell__)r#s@r$rr s   +#'$+++$++++++++466666%%%N"'''"!'''''r%a Implements the resilient backpropagation algorithm. .. math:: \begin{aligned} &\rule{110mm}{0.4pt} \\ &\textbf{input} : \theta_0 \in \mathbf{R}^d \text{ (params)},f(\theta) \text{ (objective)}, \\ &\hspace{13mm} \eta_{+/-} \text{ (etaplus, etaminus)}, \Gamma_{max/min} \text{ (step sizes)} \\ &\textbf{initialize} : g^0_{prev} \leftarrow 0, \: \eta_0 \leftarrow \text{lr (learning rate)} \\ &\rule{110mm}{0.4pt} \\ &\textbf{for} \: t=1 \: \textbf{to} \: \ldots \: \textbf{do} \\ &\hspace{5mm}g_t \leftarrow \nabla_{\theta} f_t (\theta_{t-1}) \\ &\hspace{5mm} \textbf{for} \text{ } i = 0, 1, \ldots, d-1 \: \mathbf{do} \\ &\hspace{10mm} \textbf{if} \: g^i_{prev} g^i_t > 0 \\ &\hspace{15mm} \eta^i_t \leftarrow \mathrm{min}(\eta^i_{t-1} \eta_{+}, \Gamma_{max}) \\ &\hspace{10mm} \textbf{else if} \: g^i_{prev} g^i_t < 0 \\ &\hspace{15mm} \eta^i_t \leftarrow \mathrm{max}(\eta^i_{t-1} \eta_{-}, \Gamma_{min}) \\ &\hspace{15mm} g^i_t \leftarrow 0 \\ &\hspace{10mm} \textbf{else} \: \\ &\hspace{15mm} \eta^i_t \leftarrow \eta^i_{t-1} \\ &\hspace{5mm}\theta_t \leftarrow \theta_{t-1}- \eta_t \mathrm{sign}(g_t) \\ &\hspace{5mm}g_{prev} \leftarrow g_t \\ &\rule{110mm}{0.4pt} \\[-1.ex] &\bf{return} \: \theta_t \\[-1.ex] &\rule{110mm}{0.4pt} \\[-1.ex] \end{aligned} For further details regarding the algorithm we refer to the paper `A Direct Adaptive Method for Faster Backpropagation Learning: The RPROP Algorithm `_. a Args: params (iterable): iterable of parameters to optimize or dicts defining parameter groups lr (float, optional): learning rate (default: 1e-2) etas (Tuple[float, float], optional): pair of (etaminus, etaplus), that are multiplicative increase and decrease factors (default: (0.5, 1.2)) step_sizes (Tuple[float, float], optional): a pair of minimal and maximal allowed step sizes (default: (1e-6, 50)) z z Fr!r@rArrrrrBrFrGrHrIc |t||d\} }|r-tjrt d|r&tjst } nt } | |||||| | | ||| dS)zpFunctional API that performs rprop algorithm computation. See :class:`~torch.optim.Rprop` for details. NF) use_fusedz6torch.jit.script not supported with foreach optimizers)rFrGrHrIrrrB)rr3jit is_scriptingr7_multi_tensor_rprop_single_tensor_rprop)r!r@rArrrrrBrFrGrHrI_funcs r$rrs,1&.TYZZZ 7U59))++USTTT$uy--//$"#D  ##%      r%ct|D]\} } || } |s| n| } || }|| }tj| rPtj| } tj|}tj| } tj|}| r:| |}n'| |}|||d<|||d<d|| d<| | ||| tj } d| | |<| | |d|| dS)Nrrr.value) enumerater3r4 view_as_realmulclonesigngtlteqmul_clamp_r:addcmul_copy_)r!r@rArrFrGrHrIrrrBiparamr2r0r1rbs r$rWrWsf%%5Qx#.tt$QxqM  E " " 6%d++D%d++D&u--E*955I  )88DJJLL))..00DD88D>>&&((D"TWWQZZ#TWWQZZTWWQZZ t##M=AAAzz(=z>>"#TWWX   tyy{{IR888 4;r%ct|dkrdS| r Jdtj||||g} | D]\\} } }}}| rt | | ||t j| |}|rt j|t j|| |rt j||} t j ||D]J}||| d<||| d<d|| d<Kt j |||D]}|||t| } t!t| D]&}d| ||| |<'~d| D}t j| ||ddS)Nrz#_foreach ops don't support autogradrc6g|]}|S)rb).0r2s r$ z'_multi_tensor_rprop..Js <<rs YYYYYYYYYYYYYYYYYY!!!!!!!! G qqqqqIqqqh"D           E0 r# -- L- <- <-V -d^-----------`, L, <, <,V ,  ,,,,,,,,,,^<Z L<Z <<Z <<ZV <Z  <Z<Z<Z<Z<Z<Z<Z<Z<Z<Z<Z<Zr%