~WhBdZddlmcmZddlmZddlmZddl m Z ddl m Z e dgGd d Z dd Zdd Zd ZdZe jfdZe dgde jdfdZdZdZdZdZdZdS)z$Utilities related to loss functions.N)backend) keras_tensor)tf_utils) keras_exportzkeras.losses.Reduction)v1cNeZdZdZdZdZdZdZedZ edZ dS) ReductionV2aTypes of loss reduction. Contains the following values: * `AUTO`: Indicates that the reduction option will be determined by the usage context. For almost all cases this defaults to `SUM_OVER_BATCH_SIZE`. When used with `tf.distribute.Strategy`, outside of built-in training loops such as `tf.keras` `compile` and `fit`, we expect reduction value to be `SUM` or `NONE`. Using `AUTO` in that case will raise an error. * `NONE`: No **additional** reduction is applied to the output of the wrapped loss function. When non-scalar losses are returned to Keras functions like `fit`/`evaluate`, the unreduced vector loss is passed to the optimizer but the reported loss will be a scalar value. Caution: **Verify the shape of the outputs when using** `Reduction.NONE`. The builtin loss functions wrapped by the loss classes reduce one dimension (`axis=-1`, or `axis` if specified by loss function). `Reduction.NONE` just means that no **additional** reduction is applied by the class wrapper. For categorical losses with an example input shape of `[batch, W, H, n_classes]` the `n_classes` dimension is reduced. For pointwise losses you must include a dummy axis so that `[batch, W, H, 1]` is reduced to `[batch, W, H]`. Without the dummy axis `[batch, W, H]` will be incorrectly reduced to `[batch, W]`. * `SUM`: Scalar sum of weighted losses. * `SUM_OVER_BATCH_SIZE`: Scalar `SUM` divided by number of elements in losses. This reduction type is not supported when used with `tf.distribute.Strategy` outside of built-in training loops like `tf.keras` `compile`/`fit`. You can implement 'SUM_OVER_BATCH_SIZE' using global batch size like: ``` with strategy.scope(): loss_obj = tf.keras.losses.CategoricalCrossentropy( reduction=tf.keras.losses.Reduction.NONE) .... loss = tf.reduce_sum(loss_obj(labels, predictions)) * (1. / global_batch_size) ``` Please see the [custom training guide]( https://www.tensorflow.org/tutorials/distribute/custom_training) for more details on this. autononesumsum_over_batch_sizec6|j|j|j|jfSN)AUTONONESUMSUM_OVER_BATCH_SIZE)clss ^/var/www/html/movieo_spanner_bot/venv/lib/python3.11/site-packages/keras/utils/losses_utils.pyallzReductionV2.allQs#(CGS-DEEc||vr(td|d|ddS)NzInvalid Reduction Key: z. Expected keys are "")r ValueError)rkeys rvalidatezReductionV2.validateUsM cggii  P#PPCGGIIPPP  rN) __name__ __module__ __qualname____doc__rrrr classmethodrrrrr r sm,,\ D D C/FF[F[rr ctj|pd5tjst jtjst jj}|j}j}|j}||||z }||dzkr7|jd drt j dgn?||dz kr6|jd drt j dgfcdddSt j t j z }| |jd dr2t j t j |dz|fdfd| |jd dr2t j t j |dz |fdfdfcdddS#1swxYwYdS) aHSqueeze last dim if ranks differ from expected by exactly 1. In the common case where we expect shapes to match, `expected_rank_diff` defaults to 0, and we squeeze the last dimension of the larger rank if they differ by 1. But, for example, if `labels` contains class IDs and `predictions` contains 1 probability per class, we expect `predictions` to have 1 more dimension than `labels`, so `expected_rank_diff` would be 1. In this case, we'd squeeze `labels` if `rank(predictions) - rank(labels) == 0`, and `predictions` if `rank(predictions) - rank(labels) == 2`. This will use static shape if available. Otherwise, it will add graph operations, which could result in a performance hit. Args: labels: Label values, a `Tensor` whose dimensions match `predictions`. predictions: Predicted values, a `Tensor` of arbitrary dimensions. expected_rank_diff: Expected result of `rank(predictions) - rank(labels)`. name: Name of the op. Returns: Tuple of `labels` and `predictions`, possibly with last dim squeezed. remove_squeezable_dimensionsNc0tjdgSNr&tfsqueeze predictionssrz.remove_squeezable_dimensions..s ;55rcSrr"r,srr.z.remove_squeezable_dimensions..s rc0tjdgSr(r)labelssrr.z.remove_squeezable_dimensions..s 6B400rcSrr"r1srr.z.remove_squeezable_dimensions..sr)r name_scoperis_tensor_or_extension_typer*convert_to_tensorshapendimsdimsis_compatible_withr+rankcondequal) r2r-expected_rank_diffnamepredictions_shapepredictions_rank labels_shape labels_rank rank_diffs `` rr$r$]s6  DB$B C C(#(#3K@@ <.{;;K3F;; 2)&11F'-,2| "(  #*:*F(;6I.2227H7M8  ##3!jrd;; 014449J:  ##5FRD11;&)(#(#(#(#(#(#(#(#.GK((276??:  $  "2 & 9 9! < < %'+a/;;5555####K    b ! 4 4Q 7 7 W+a/;;0000F {"Q(#(#(#(#(#(#(#(#(#(#(#(#(#(#(#(#(#(#sC8G?CG??HHcr j}|j}ˉj}|j}|+|)||z dks |ddkrt\ntjtjz fdtjdtjd fd}tjtjd|\fSj}|j} | dkrfS|F| D| |z dkrtjdgn|| z dkrtjdgfStj} | tjz fdfd fd } tjtj| dfd | fS) aSqueeze or expand last dimension if needed. 1. Squeezes last dim of `y_pred` or `y_true` if their rank differs by 1 (using `remove_squeezable_dimensions`). 2. Squeezes or expands last dim of `sample_weight` if its rank differs by 1 from the new rank of `y_pred`. If `sample_weight` is scalar, it is kept scalar. This will use static shape if available. Otherwise, it will add graph operations, which could result in a performance hit. Args: y_pred: Predicted values, a `Tensor` of arbitrary dimensions. y_true: Optional label `Tensor` whose dimensions match `y_pred`. sample_weight: Optional weight scalar or `Tensor` whose dimensions match `y_pred`. Returns: Tuple of `y_pred`, `y_true` and `sample_weight`. Each of them possibly has the last dimension squeezed, `sample_weight` could be extended by one dimension. If `sample_weight` is None, (y_pred, y_true) is returned. Nr%r&c$tSr)r$y_predy_truesrr.z.squeeze_or_expand_dimensions..s#?#O#Orc8tjfdS)Nc fSrr"rGsrr.z@squeeze_or_expand_dimensions....s ff5Er)r*r<) is_last_dim_1 squeeze_dimsrHrIsrr.z.squeeze_or_expand_dimensions..s'|-E-E-E-E-E**rrc0tjdgSr(r) sample_weightsrr.z.squeeze_or_expand_dimensions..sBJ}rd$C$Crcffd}tjtjd|fdS)Nc0tjdgSr()r* expand_dimsrOsrr.zMsqueeze_or_expand_dimensions.._maybe_expand_weights..s t!D!Drr&cSrr"rOsrr.zMsqueeze_or_expand_dimensions.._maybe_expand_weights..s]rr*r<r=)expand_weightsrDrPs r_maybe_expand_weightsz;squeeze_or_expand_dimensions.._maybe_expand_weightssBDDDDw HY # #^5J5J5J5J   rcVtjtjdS)Nr%rU)rWmaybe_squeeze_weightsrDsr_maybe_adjust_weightsz;squeeze_or_expand_dimensions.._maybe_adjust_weightss,w HY " "$9;P   rcSrr"rOsrr.z.squeeze_or_expand_dimensions..s r) r7r8r$r*r;r=r<r+rS)rHrIrP y_pred_shape y_pred_rank y_true_shape y_true_rankmaybe_squeeze_dims weights_shape weights_rankweights_rank_tensorrZrWrLrYrDrMs``` @@@@@rsqueeze_or_expand_dimensionsrds]0 + % * *J}rd;;MM < '1 , ,N=2$??Mv},,'-00#bgfoo5ICCCC             G $a((M 6= ((rcntj|}tj||dS)a:Computes a safe mean of the losses. Args: losses: `Tensor` whose elements contain individual loss measurements. num_present: The number of measurable elements in `losses`. Returns: A scalar representing the mean of `losses`. If `num_present` is zero, then zero is returned. valuer?)r* reduce_summath divide_no_nan)losses num_present total_losss r _safe_meanrns0v&&J 7 [w G GGrctjd5}tjtj|||jcdddS#1swxYwYdS)z3Computes the number of elements in `losses` tensor. num_elementsrg)dtypeN)rr4r*castsizerq)rkscopes r _num_elementsrus  N + +HuwrwvE222&,GGGHHHHHHHHHHHHHHHHHHs/AAAc|tjkr|}nAtj|}|tjkrt |t |}|S)z2Reduces the individual weighted loss measurements.)r rr*rhrrnru)weighted_losses reductionlosss rreduce_weighted_lossrzsRK$$$}_--  7 7 7dM/$B$BCCD Krz/keras.__internal__.losses.compute_weighted_losscBt||tjkr tj}|d}t j|pd5|t jj _ t|tj t jfst j|}t|tj t jfst j|}|jjs|j}t j|d}d}nd}t j||j}t'|d|\}}}t j||}t+||}|rt j||}|cdddS#1swxYwYdS)aComputes the weighted loss. Args: losses: `Tensor` of shape `[batch_size, d1, ... dN]`. sample_weight: Optional `Tensor` whose rank is either 0, or the same rank as `losses`, or be broadcastable to `losses`. reduction: (Optional) Type of `tf.keras.losses.Reduction` to apply to loss. Default value is `SUM_OVER_BATCH_SIZE`. name: Optional name for the op. Raises: ValueError: If the shape of `sample_weight` is not compatible with `losses`. Returns: Weighted loss `Tensor` of the same type as `losses`. If `reduction` is `NONE`, this has the same shape as `losses`; otherwise, it is scalar. N? weighted_lossfloat32TF)r rrrrr4r*compatrget_default_graph_last_loss_reduction isinstancer KerasTensor RaggedTensorr6rq is_floatingrrrdmultiplyrz) rkrPrxr? input_dtype input_casted_rwrys rcompute_weighted_lossr!s2###K$$$3   D3O 4 4$$AJ &&((=&<#;R_"MNN 2)&11F L4boF   @0??M|' ! ,KWVY//FLL L v|<<  )} E E   +fm<<$OY??  .74--DI$$$$$$$$$$$$$$$$$$sD6FFFchtjj}|dkr|d|z z}|S)zBScales and returns the given loss value by the number of replicas.r%r|)r* distribute get_strategynum_replicas_in_sync) loss_value num_replicass rscale_loss_for_distributionris8=--//DLacL(( rcd|D]N}|jjr0|jjjkr|jn|jhddhkrd|jjr|cSOrfd|D}|S)arCast a list of losses to a common dtype. If any loss is floating-point, they will all be casted to the most-precise floating-point loss. Otherwise the losses are not casted. We also skip casting losses if there are any complex losses. Args: losses: A list of losses. Returns: `losses`, but they have been casted to a common dtype. Nbfloat16float16r~c:g|]}tj|Sr")r*rr).0ry highest_floats r z/cast_losses_to_common_dtype..s%BBB4"'$ ..BBBr)rqrrs is_complex)rkryrs @rcast_losses_to_common_dtyperqsM   : ! *$ -:L(L(L $ *m,Y0GGG ) :      CBBBB6BBB Mrc$t|ddS)zReturns Keras mask from tensor. _keras_maskN)getattr)y_ps rget_maskrs 3 t , ,,rc|Stj||j}|5tj||j}t||\}}}||z}n|}|S)z2Applies any mask on predictions to sample weights.NrO)r*rrrqrd)rswmaskrs r apply_maskrs` wtSY'' >TZ((B6t2NNNKD!R $JBBB Irc ||tj||j}|tjtjfvrHtjtj||j}tj|}|||z z}t|||S)z;Redistribute sample weights considering only valid entries.) r*rrrqr rrrsrhr)rkrrrxtotalvalids rapply_valid_maskrs} wtV\** );+JK K KGBGDMM6<88EM$''E EEM !D fb$ ' ''r)rN)NN)r tensorflow.compat.v2rv2r*kerasr keras.enginer keras.utilsr tensorflow.python.util.tf_exportrr r$rdrnrurrzrrrrrrr"rrrs +*!!!!!!!!!%%%%%% :99999&2...=======/.=B59C#C#C#C#LY)Y)Y)Y)x H H HHHH +>    ?BGGG- DDDHGDN:---   (((((r