Liquid AI has released LiquidCodec, the audio codec behind its LFM3-Audio-2B and LFM3-Omni-2B fashions. The LiquidCodec audio codec is a ~74M-parameter streaming speech tokenizer. It turns speech into tokens a language mannequin can predict, then converts these tokens again into audio. Liquid AI had 2 objectives: easy-to-predict tokens and quick conversion again to speech. The weights are on Hugging Face.
TL;DR
- Measurement: ~74M parameters (73.6M on the model card); 24 kHz audio, 12.5 Hz frames, 12 codebooks.
- Runs on: a single CPU thread (examined on an Apple M4 Max laptop computer, FP32); CUDA GPUs by way of Transformers; Q4_0, Q4_K_M and Q8_0 GGUF builds for llama.cpp.
- Efficiency: finest ViSQOL and lowest mel distance on all 8 take a look at units versus X-Codec 2, NanoCodec and Mimi.
- Finest: ~361x real-time encoding on 1 CPU thread (encode RTF 0.0028 vs Mimi’s 0.0361).
What’s LiquidCodec?
LiquidCodec is a Liquid AI speech codec that compresses 24 kHz audio into 12 token streams. Every 80 ms body turns into 12 discrete tokens, per the LFM3-Audio-2B model card. It replaces Kyutai’s Mimi, which Liquid AI utilized in LFM2-Audio-1.5B. Mimi has 96.2M parameters.
How does LiquidCodec work?
The pipeline is encode, quantize, decode. Liquid AI makes use of blocks from its LFM3 structure in each the encoder and decoder. The two halves are symmetric, and the encoder is absolutely causal, which fits streaming.
Quantization makes use of grouped finite scalar quantization (GFSQ) as a substitute of residual vector quantization (RVQ). In RVQ, every codebook encodes what the earlier one missed. GFSQ teams are unbiased, so all 12 codebooks will be predicted in parallel by MaskGIT-style transformers.
The decoder predicts spectral coefficients straight. This avoids realized transposed-convolution upsampling, a typical price in waveform decoders.
How was it educated?
Coaching ran in 2 levels. Stage 1 used 264,000 hours of speech throughout 107 languages. It utilized waveform and spectral losses, with out adversarial losses. Stage 2 fine-tuned on 55,000 hours of full-band speech (24 kHz or increased) with adversarial critics. Liquid AI says this eliminated metallic artifacts left after stage 1. A ultimate decoder-only quantization-aware coaching (QAT) step targets edge inference.
How briskly is LiquidCodec?
Liquid AI measured pace in FP32 on 1 CPU thread of an Apple M4 Max MacBook Professional. LiquidCodec ran at ~361x actual time for encoding and ~87x for decoding. These are self-reported figures.
Actual-time issue (RTF) is processing time divided by audio size, so decrease is best. LiquidCodec’s encode RTF is 0.0028, towards 0.0361 for Mimi, 0.2145 for NanoCodec and 0.7359 for X-Codec 2. Decode RTF is 0.0115, towards 0.0329, 0.2543 and 0.0379. That’s about 13x decrease encode RTF and a couple of.9x decrease decode RTF than Mimi.
How does LiquidCodec examine on high quality benchmarks?
Liquid AI used 8 take a look at units. These had been LibriSpeech test-clean, Seed-TTS English and Mandarin, and 5 FLEURS languages (German, French, Mandarin, Japanese, Korean). Baselines had been X-Codec 2, NanoCodec and Mimi at 12 codebooks.
- Reconstruction: LiquidCodec had one of the best ViSQOL and lowest mel distance on all 8 units. On LibriSpeech, ViSQOL was 4.319 versus 4.219 (NanoCodec), 4.119 (X-Codec 2) and 4.031 (Mimi).
- Textual content-to-speech: Liquid AI educated similar TTS fashions on every codec’s tokens. Information (101,000 hours, 6 languages), structure, compute and batch order had been fastened. LiquidCodec scored the best UTMOSv2 on all 8 units and one of the best WER/CER on 4 of 8. LibriSpeech WER was 3.15%, towards 6.27% for Mimi. Outcomes held with a Qwen3-1.7B spine in addition to LFM3-1.7B.
- The place it trails: NanoCodec scored increased PESQ (WB) on 7 of 8 units. It additionally led on LibriSpeech speaker similarity (SECS 0.836 vs 0.785).
(operate(){var f=doc.getElementById(“mtp-lc-frame”);window.addEventListener(“message”,operate(e){if(e.supply===f.contentWindow&&e.knowledge&&e.knowledge.mtpLcHeight){f.fashion.peak=e.knowledge.mtpLcHeight+”px”;}});})();
LiquidCodec vs Mimi vs NanoCodec vs X-Codec 2
| Function | LiquidCodec | Mimi | NanoCodec 1.78 kbps | X-Codec 2 |
|---|---|---|---|---|
| Developer | Liquid AI | Kyutai | NVIDIA | HKUST Audio |
| Parameters | 73.6M | 96.2M | 62M | 0.8B |
| Pattern charge | 24 kHz | 24 kHz (repo) | 22.05 kHz | 16 kHz |
| Body charge | 12.5 Hz | 12.5 Hz | 12.5 fps | 50 tokens/s |
| Codebooks | 12 | 8 at 1.1 kbps (blog) | 13 | 1 (docs) |
| Quantizer | GFSQ | RVQ (docs) | FSQ | Single-layer FSQ (docs) |
| Bitrate | Not disclosed | 1.1 kbps | 1.78 kbps | Not disclosed |
| Causality | Causal encoder; decoder not disclosed | Absolutely causal (Kyutai) | Non-causal encoder, causal decoder | Not disclosed |
| License | “other” on HF; weblog hyperlinks LFM License | CC-BY 4.0 | NVIDIA Open Mannequin License | CC-BY-NC 4.0 |
| Encode / decode RTF, M4 Max CPU* | 0.0028 / 0.0115 | 0.0361 / 0.0329 | 0.2145 / 0.2543 | 0.7359 / 0.0379 |
| LibriSpeech mel distance* (decrease is best) | 0.120 | 0.160 | 0.122 | 0.123 |
| LibriSpeech PESQ (WB)* | 2.640 | 2.615 | 2.942 | 2.434 |
| LibriSpeech TTS UTMOSv2* | 3.31 | 2.89 | 3.19 | 3.27 |
*Measured by Liquid AI in its LiquidCodec blog. Mimi measured at 12 codebooks.
How can builders deploy LiquidCodec?
The codec masses by Hugging Face Transformers with trust_remote_code=True. Name encode() to get audio codes and decode() to get a waveform. A local LiquidCodecStreamingCache helps chunk-by-chunk decoding, per the model card. The cardboard’s examples use CUDA with bfloat16.
For edge use, Liquid AI lists Q4_0, Q4_K_M and Q8_0 GGUF builds for llama.cpp. GGUF repos ship with LFM3-Audio-2B and LFM3-Omni-2B. Q4_0 with decoder-only QAT shrinks the decoder footprint by 3.5x. LibriSpeech ViSQOL dips solely from 4.319 to 4.292.
Key Takeaways
- ~74M-parameter streaming codec, 12.5 Hz, 12 codebooks, 24 kHz.
- ~361x real-time encode, ~87x decode on 1 CPU thread.
- Finest ViSQOL, mel distance and TTS UTMOSv2 on all 8 take a look at units.
- NanoCodec nonetheless wins PESQ (WB) on 7 of 8 units.
- Q4_0 decoder is 3.5x smaller with minimal high quality loss.
Try the technical blog, the model weights on Hugging Face, and LFM3-Audio-2B and LFM3-Omni-2B. Additionally, be happy to observe us on Twitter and don’t neglect to hitch our 150k+ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well.
Have to associate with us for selling your GitHub Repo OR Hugging Face Web page OR Product Launch OR Webinar and so on.? Connect with us
The put up Liquid AI Releases LiquidCodec, a 74M-Parameter Streaming Audio Codec appeared first on MarkTechPost.

