fix(voice): RMS-amplify STT input for weak voice
Condition the handset capture by RMS (not peak) before whisper: the voice body is ~0.5-1.5% FS and a transient peak defeated peak-normalise, leaving it inaudible. RMS-normalise to ~20% FS + clip. Recovers weak captures whisper returned '' for (verified: -> 'C'est parti !').
This commit was merged in pull request #168.
This commit is contained in:
+15
-14
@@ -1941,17 +1941,18 @@ def _build_wav(pcm: bytes, sr: int, ch: int, sw: int) -> bytes:
|
||||
return out.getvalue()
|
||||
|
||||
|
||||
def _normalize_wav_for_stt(wav_bytes: bytes, target_peak: float = 0.7,
|
||||
max_gain: float = 40.0, hp_cutoff_hz: float = 110.0) -> bytes:
|
||||
"""Condition a PLIP handset capture for Kyutai STT: high-pass then normalise.
|
||||
def _normalize_wav_for_stt(wav_bytes: bytes, target_rms: float = 0.20,
|
||||
max_gain: float = 50.0, hp_cutoff_hz: float = 110.0) -> bytes:
|
||||
"""Condition a PLIP handset capture for the STT: high-pass then RMS-amplify.
|
||||
|
||||
Two problems with the SLIC handset path: (1) it injects low-frequency rumble
|
||||
/ DC drift that swamps the speech and makes Kyutai output nothing — a
|
||||
high-pass (~110 Hz, box moving-average subtraction) removes it; (2) the mic
|
||||
comes in very quiet (~2-5 % FS even at +24 dB PGA), so peak-normalise after
|
||||
filtering (gain capped so silence isn't blown up). WITHOUT the high-pass the
|
||||
exact same capture transcribes empty; WITH it, clean French — verified on the
|
||||
bench."""
|
||||
Two problems with the SLIC handset path: (1) low-frequency rumble/DC drift
|
||||
that swamps the speech — a high-pass (~110 Hz, box moving-average subtraction)
|
||||
removes it; (2) the voice body comes in VERY quiet (~0.5-1.5 % FS RMS). We
|
||||
amplify by RMS (NOT peak — a transient peak defeats peak-normalisation and
|
||||
leaves the body inaudible) to ~20 % FS, then hard-clip the rare transient.
|
||||
Verified: weak captures whisper returned '' for transcribe correctly once
|
||||
RMS-boosted (e.g. → "C'est parti !"). Silence gets blown up to noise but the
|
||||
whisper hallucination filter drops the result."""
|
||||
if _np is None:
|
||||
return wav_bytes
|
||||
parsed = _wav_pcm(wav_bytes)
|
||||
@@ -1967,11 +1968,11 @@ def _normalize_wav_for_stt(wav_bytes: bytes, target_peak: float = 0.7,
|
||||
if arr.size > k:
|
||||
lp = _np.convolve(arr, _np.ones(k, dtype=_np.float32) / k, mode="same")
|
||||
arr = arr - lp
|
||||
peak = float(_np.max(_np.abs(arr))) or 1.0
|
||||
gain = min((target_peak * 32767.0) / peak, max_gain)
|
||||
rms = float(_np.sqrt(_np.mean(arr * arr))) or 1.0
|
||||
gain = min((target_rms * 32767.0) / rms, max_gain)
|
||||
arr = _np.clip(arr * gain, -32768, 32767).astype(_np.int16)
|
||||
logging.info("STT conditioning: high-pass %.0f Hz, peak %.1f%% FS → gain x%.1f",
|
||||
hp_cutoff_hz, 100 * peak / 32768, gain)
|
||||
logging.info("STT conditioning: high-pass %.0f Hz, RMS %.2f%% FS → gain x%.1f",
|
||||
hp_cutoff_hz, 100 * rms / 32768, gain)
|
||||
return _build_wav(arr.tobytes(), sr, ch, sw)
|
||||
|
||||
|
||||
|
||||
Reference in New Issue
Block a user