Vh0ddlmZmZddlZddlZddlmZddlmcm Z ddlm Z ddlm Z m Z mZmZddlmZddlmZmZGdd ejZGd d ejZdS) )OptionalMappingN)Tensor)LSTM BatchNorm1dLinear Parameter)wiener)make_filterbanks ComplexNormceZdZdZ dded ed ed ed ed eejdeejdeeffd Z dZ de de fdZ xZ S) OpenUnmixadOpenUnmix Core spectrogram based separation module. Args: nb_bins (int): Number of input time-frequency bins (Default: `4096`). nb_channels (int): Number of input audio channels (Default: `2`). hidden_size (int): Size for bottleneck layers (Default: `512`). nb_layers (int): Number of Bi-LSTM layers (Default: `3`). unidirectional (bool): Use causal model useful for realtime purpose. (Default `False`) input_mean (ndarray or None): global data mean of shape `(nb_bins, )`. Defaults to zeros(nb_bins) input_scale (ndarray or None): global data mean of shape `(nb_bins, )`. Defaults to ones(nb_bins) max_bin (int or None): Internal frequency bin threshold to reduce high frequency content. Defaults to `None` which results in `nb_bins` FNnb_bins nb_channels hidden_size nb_layersunidirectional input_mean input_scalemax_binc tt|||_|r||_n |j|_||_t |j|z|d|_t||_ |r|} n|dz} t|| || d|dkrdnd|_ |dz} t | |d|_ t||_ t ||j|zd|_t|j|z|_|5t!j|d|j }nt!j|j}|7t!jd |d|jz }nt!j|j}t+||_t+||_t+t!j|j|_t+t!j|j|_dS) NF)biasrr g?r) input_sizer num_layers bidirectional batch_firstdropout) in_features out_featuresrg?)superr__init__nb_output_binsrrrfc1rbn1rlstmfc2bn2fc3bn3torch from_numpyfloatzerosonesr rr output_scale output_mean) selfrrrrrrrrlstm_hidden_sizefc2_hiddensize __class__s U/var/www/html/movieo_spanner_bot/venv/lib/python3.11/site-packages/openunmix/model.pyr&zOpenUnmix.__init__ s; i'')))%  /"DLL.DL&$,4kNNN{++  0*  *a/ "( ,,$q==CCa     %qn;UZ[[[{++#,{:    t2[@AA  !):n n+E*EFFLLNNJJT\22J  "*3^t|^1L+LMMSSUUKK*T\22K#J//$[11%ej1D&E&E&K&K&M&MNN$UZ0C%D%D%J%J%L%LMMcj|D] }d|_ |dSNF parameters requires_gradevalr6ps r:freezezOpenUnmix.freezed7"" $ $A#AOO r;xreturnc|dddd}|jj\}}}}|}|dd|jf}||jz}||jz}|| d||jz}| |}| |||j }tj |}||}tj||dgd}|| d|jd}||}t%j|}||}||}| ||||j}||jz}||jz }t%j||z}|ddddS)a Args: x: input spectrogram of shape `(nb_samples, nb_channels, nb_bins, nb_frames)` Returns: Tensor: filtered spectrogram of shape `(nb_samples, nb_channels, nb_bins, nb_frames)` rrr r.N)permutedatashapedetachclonerrrr(reshaper)rr/tanhr*catr+r,Frelur-r.r'r4r5)r6rF nb_frames nb_samplesrrmixlstm_outs r:forwardzOpenUnmix.forwardks IIaAq ! !67fl3 :{Ghhjj   c>T\>! "     HHQYYr;#=>> ? ? HHQKK IIiT-= > > JqMM99Q<< Iq(1+& + + HHQYYr172;// 0 0 HHQKK F1II HHQKK HHQKK IIi[$:M N N T  T  F1IIOyyAq!$$$r;)rrrrFNNN)__name__ __module__ __qualname____doc__intboolrnpndarrayr&rDrrX __classcell__r9s@r:rr s($+/,0!%BNBNBNBN BN  BN  BNRZ(BNbj)BN#BNBNBNBNBNBNH;%;%F;%;%;%;%;%;%;%;%r;rceZdZdZ dd eeejfd ed e d e de dededede edeffd Z dZ dedefdZddede edefdZxZS) SeparatoraP Separator class to encapsulate all the stereo filtering as a torch Module, to enable end-to-end learning. Args: targets (dict of str: nn.Module): dictionary of target models the spectrogram models to be used by the Separator. niter (int): Number of EM steps for refining initial estimates in a post-processing stage. Zeroed if only one target is estimated. defaults to `1`. residual (bool): adds an additional residual target, obtained by subtracting the other estimated targets from the mixture, before any potential EM post-processing. Defaults to `False`. wiener_win_len (int or None): The size of the excerpts (number of frames) on which to apply filtering independently. This means assuming time varying stereo models and localization of sources. None means not batching but using the whole signal. It comes at the price of a much larger memory usage. filterbank (str): filterbank implementation method. Supported are `['torch', 'asteroid']`. `torch` is about 30% faster compared to `asteroid` on large FFT sizes such as 4096. However, asteroids stft can be exported to onnx, which makes is practical for deployment. rF@rr,r/ target_modelsnitersoftmaskresidual sample_raten_fftn_hoprwiener_win_len filterbankc tt|||_||_||_| |_t||d| |\|_|_ t|dk|_ tj ||_t|j|_|dt%j|dS)NT)rmrncentermethodrlr )monorl)r%rdr&rirkrjror stftistftr complexnormnn ModuleDictrhlen nb_targetsregister_bufferr/ as_tensor) r6rhrirjrkrlrmrnrrorpr9s r:r&zSeparator.__init__s i'')))     , 0# ! ! !  4:'K1,<=== ]=99d011 ]EOK,H,HIIIIIr;cj|D] }d|_ |dSr=r>rBs r:rDzSeparator.freezerEr;audiorGc |j}|jd}||}||}t j|j|fz|j|j}t|j D]>\}\}} | | } | |d|f<?| ddddd}| ddddd}|jr|dz }|dkr|jdkrt!d|jd} t j|j|fz|j|j} t#|D]} d}|jr|j}n| }|| kr|t j|t)| ||z}t+|d dz}t-|| |f|| |f|j|j|j | | |f<|| k|| dd dddd} || |jd }|S) aKPerforming the separation on audio input Args: audio (Tensor): [shape=(nb_samples, nb_channels, nb_timesteps)] mixture audio waveform Returns: Tensor: stacked tensor of separated waveforms shape `(nb_samples, nb_targets, nb_channels, nb_timesteps)` r)dtypedevice.rrr zoCannot use EM if only one target is estimated.Provide two targets or create an additional one with `--residual`rI)rjrk)length)r{rLrurwr/r2rr enumeraterhitemsrMrNrJrkri Exceptionrangeroarangeminr]r rj contiguousrv)r6r nb_sourcesrUmix_stftX spectrogramsj target_name target_moduletarget_spectrogramrT targets_stftsampleposro cur_frame estimatess r:rXzSeparator.forwards_ [^ 99U##   X & &{17j]#:%+VWV^___ /89K9Q9Q9S9S/T/T 6 6 +A+ ]!.qxxzz/?/?/A/A!B!B #5La $++Aq!Q:: ##Aq!Q22 =  !OJ ??tzA~~(  !&q) {8>ZM#A]e]lmmm J''  FC" +!%!4!* //!Lc)S>=Q.R.RSS )B-((1,28 !23VY./J!]!] 333 VY./  //$++Aq!Q1==HHJJ JJ|EKNJCC r;Nraggregate_dictci}t|jD]\}}|dd|df||<|jr|ddddf|d<|?i}|D]8}tjd||<||D]}||||z||<9|}|S)a'Convert estimates as stacked tensor to dictionary Args: estimates (Tensor): separated targets of shape (nb_samples, nb_targets, nb_channels, nb_timesteps) aggregate_dict (dict or None) Returns: (dict of str: Tensor): N.rIrkg)rrhrkr/tensor)r6rrestimates_dictktarget new_estimateskeys r:to_dictzSeparator.to_dict?s"4#566 : :IAv%.qqq!Sy%9N6 " " = ?)2111b#:)>N: &  %M% U U%*\#%6%6 c",S1UUF)6s);nV>T)TM#&&U*Nr;) rFFrerrfrrgr/)N)rYrZr[r\rstrrxModuler]r^r1rr&rDrrXdictrrarbs@r:rdrdsW<$(+!$J$JsBI~.$J$J $J  $J  $J$J$J$J! $J$J$J$J$J$J$JLKVKKKKKZ$SWr;rd)typingrrnumpyr_r/torch.nnrxtorch.nn.functional functionalrRrrrrr filteringr transformsr r rrrdr;r:rs4$$$$$$$$ 99999999999955555555Y%Y%Y%Y%Y% Y%Y%Y%xppppp pppppr;