o :/i+ @sddlZddlmZddlmZddlmZddlZddl mZ ddl Z ddl m Z zddlZWn ey=e dZYnwzddlmZWneyVe ddZYnwGd d d eeZeGd d d Zed ejdZeddZde jeje jBdede jeje jBfddZde jejdedede jejfddZde jejdedefddZ GdddZ!dej"de#ded ed!e#de$ej"f d"d#Z%d$ej"d%e#d&e#d'e#de#f d(d)Z&dS)*N) dataclass)Enum)Literal)PlaceholderModulelibrosascipysignalc@s eZdZdZdZdZdZdZdS)ChannelReductionz8Method to reduce multi-channel audio to target channels.meanfirstmaxsumN)__name__ __module__ __qualname____doc__MEANFIRSTMAXSUMrrb/lsinfo/ai/hellotax_ai/llm_service/venv_vllm/lib/python3.10/site-packages/vllm/multimodal/audio.pyr s r c@sPeZdZUdZdZedBed<ejZ eed<e de fddZ de fd d ZdS) AudioSpecaSpecification for target audio format. This dataclass defines the expected audio format for a model's feature extractor. It is used to normalize audio data before processing. Attributes: target_channels: Number of output channels. None means passthrough (no normalization). 1 = mono, 2 = stereo, etc. channel_reduction: Method to reduce channels when input has more channels than target. Only used when reducing channels. Ntarget_channelschannel_reductionreturncCs |jduS)z&Whether audio normalization is needed.Nrselfrrrneeds_normalization6s zAudioSpec.needs_normalizationcCs&|jdurdSd|jd|jjdS)NzAudioSpec(passthrough)zAudioSpec(channels=z , reduction=))rrvaluerrrr__repr__;s  zAudioSpec.__repr__)rrrrrint__annotations__r rrpropertyboolr strr#rrrrr%s  rr)rrraudiospecrcCs|js|S|jdkr|jdkr|Std|jd|jdkr(td|jd|jd|jdkr>t|tjr;|jn|j}|jd}||jkrJ|S||jkrZtd|d |jt|tj}|jdkr|j t j kr~|rvtj |dd }|S|j dd }|S|j t j kr|d}|S|j t jkr|rtj|dd }|S|jdd j}|S|j t jkr|rtj|dd }|S|jdd }|Std |j |d |jS)aKNormalize audio to the specified format. This function handles channel reduction for multi-channel audio, supporting both numpy arrays and torch tensors. Args: audio: Input audio data. Can be: - 1D array/tensor: (time,) - already mono - 2D array/tensor: (channels, time) - standard format from torchaudio - 2D array/tensor: (time, channels) - format from soundfile (will be auto-detected and transposed if time > channels) spec: AudioSpec defining the target format. Returns: Normalized audio in the same type as input (numpy or torch). For mono output (target_channels=1), returns 1D array/tensor. Raises: ValueError: If audio has unsupported dimensions or channel expansion is requested (e.g., mono to stereo). rzCannot expand mono audio to z channelszUnsupported audio shape: z. Expected 1D or 2D.rzCannot expand z channels to )axis)dimzUnknown reduction method: N)r ndimr ValueErrorshape isinstancenpndarrayTrr rr rrr valuesrr )r)r* num_channelsis_numpyresultrrrnormalize_audioIsN              r9orig_sr target_srcCstj|||dS)Nr:r;)rresampler)r:r;rrrresample_audio_librosasr?cCs8||kr t|d||S||krt|||dS|S)Nr) scipy_signal resample_polyr>rrrresample_audio_scipys rBc@sVeZdZdZ  ddedBdedfddZd eje j d ed eje j fd d Z dS)AudioResamplerz,Resample audio data to a target sample rate.Nrr;method)rrcCs||_||_dS)N)r;rD)rr;rDrrr__init__s zAudioResampler.__init__r)r:rcCsx|jdur tdtjt|t|jdddr|S|jdkr&t|||jdS|jdkr3t|||jdStd|jd ) NzBAudio resampling is not supported when `target_sr` is not providedggư>)rel_tolabs_tolrr<rzInvalid resampling method: z.. Supported methods are 'librosa' and 'scipy'.) r; RuntimeErrormathisclosefloatrDr?rBr/)rr)r:rrrr=s,    zAudioResampler.resample)Nr) rrrrrKrrEnptNDArrayr2floatingr=rrrrrCs    rC audio_data sample_ratemax_clip_duration_soverlap_duration_smin_energy_window_sizec Cst||}t||}g}d}||jdkrY|||jdkr.||d|df |S|||} t|||jd} t|| | |} ||d|| f| }||jdks|S)aSplit audio into chunks with intelligent split points. Splits long audio into smaller chunks at low-energy regions to minimize cutting through speech. Uses overlapping windows to find quiet moments for splitting. Args: audio_data: Audio array to split. Can be 1D (mono) or multi-dimensional. Splits along the last dimension (time axis). sample_rate: Sample rate of the audio in Hz. max_clip_duration_s: Maximum duration of each chunk in seconds. overlap_duration_s: Overlap duration in seconds between consecutive chunks. Used to search for optimal split points. min_energy_window_size: Window size in samples for finding low-energy regions. Returns: List of audio chunks. Each chunk is a numpy array with the same shape as the input except for the last (time) dimension. Example: >>> audio = np.random.randn(1040000) # 65 seconds at 16kHz >>> chunks = split_audio( ... audio_data=audio, ... sample_rate=16000, ... max_clip_duration_s=30.0, ... overlap_duration_s=1.0, ... min_energy_window_size=1600, ... ) >>> len(chunks) 3 r.N)r$r0appendminfind_split_point) rOrPrQrRrS chunk_size overlap_sizechunksi search_start search_end split_pointrrr split_audios$ &  r_wav start_idxend_idxmin_energy_windowc Csh|||}tj}d}tdt|||D]}||||}|dd} | |kr1||}| }q|S)aFind the best point to split audio by looking for silence or low amplitude. Searches for the quietest region within a specified range by calculating RMS energy in sliding windows. Args: wav: Audio array. Can be 1D or multi-dimensional. start_idx: Start index of search region (inclusive). end_idx: End index of search region (exclusive). min_energy_window: Window size in samples for energy calculation. Returns: Index of the quietest point within the search region. This is the recommended split point to minimize audio artifacts. Example: >>> audio = np.random.randn(32000) >>> # Insert quiet region >>> audio[16000:17600] = 0.01 >>> split_idx = find_split_point( ... wav=audio, ... start_idx=0, ... end_idx=32000, ... min_energy_window=1600, ... ) >>> 16000 <= split_idx <= 17600 True rr+g?)rIinfrangelenr ) r`rarbrcsegment min_energy quietest_idxr[windowenergyrrrrW!s "rW)'rI dataclassesrenumrtypingrnumpyr2 numpy.typingrLtorchvllm.utils.import_utilsrr ImportError scipy.signalrr@placeholder_attrr(r rrMONO_AUDIO_SPECPASSTHROUGH_AUDIO_SPECrMrNTensorr9rKr?rBrCr3r$listr_rWrrrrs            S      0 ?