Close Menu
  • AI
  • Content Creation
  • Tech
  • Robotics
AI-trends.todayAI-trends.today
  • AI
  • Content Creation
  • Tech
  • Robotics
Trending
  • Tesla Rolls Out FSD V15 with Actuality View and Imaginative and prescient Upgrades – Unite.AI
  • Potential Cognition AI Spherical Would Elevate $2B-Plus at $75B Valuation – Unite.AI
  • OpenRouter Cuts Three Suppliers After AI Immediate Publicity Findings – Unite.AI
  • A Group of Researchers from Columbia and CertiK Introduce TensorLean for Formal Verification of GPU Kernels
  • Collectively AI Discloses Immediate Leak After Coding Agent Uncovered Credential – Unite.AI
  • Dragos Finds Immediate Leaks and Stolen Keys in AI Provide Chains
  • Lean and Mathlib Communities Certify Claude-Assisted Riemann Proof – Unite.AI
  • AWS Routes HyperPod Inference Throughout Areas From One Endpoint – Unite.AI
AI-trends.todayAI-trends.today
Home»Tech»Alibaba Qwen Releases Qwen-Audio-3.1-Realtime: A Full-Duplex Voice Mannequin Educated to Suppose, Act, and Resolve When to Communicate

Alibaba Qwen Releases Qwen-Audio-3.1-Realtime: A Full-Duplex Voice Mannequin Educated to Suppose, Act, and Resolve When to Communicate

Tech By Gavin Wallace29/09/20265 Mins Read
Facebook Twitter LinkedIn Email
A Coding Implementation to Build an AI Agent with Live
A Coding Implementation to Build an AI Agent with Live
Share
Facebook Twitter LinkedIn Email

Alibaba’s Qwen group has released Qwen-Audio-3.1, a 5-model audio stack spanning ASR, TTS and realtime interplay. The primary mannequin is Qwen-Audio-3.1-Realtime, a full-duplex speech mannequin constructed for voice brokers that decision instruments. Qwen additionally cut prices: about 85% on Realtime, about 70% on TTS and as much as 95% on ASR.

Is it deployable? Sure, as a managed API. qwen-audio-3.1-realtime-plus is dwell on QwenCloud over WebSocket. No open weights had been introduced.

What Ships on QwenCloud

The model page lists textual content and audio as each enter and output. Context is 262K tokens, with 245K max enter and 16K max output. Default limits are 60 requests and 100K tokens per minute. Pricing is $6.4 per 1M audio enter tokens and $0.8 per 1M textual content enter tokens. Textual content and audio output prices $24 per 1M tokens, with output textual content not charged. Key options embody perform calling, net search, structured outputs, context cache and fine-tuning.

A companion mannequin, Qwen-Audio-3.1-ASR-Flash-Filetrans, targets offline long-audio transcription. It helps scorching phrases, speaker separation, punctuation and multilingual plus Chinese language dialect recognition. It prices $0.15 enter and $0.47 output per 1M tokens.

Structure: 2 Fashions Behind 1 Voice

The system runs 2 fashions with the identical Audio Encoder and LLM design. A full-duplex choice mannequin predicts whether or not to maintain listening, communicate, cease or resume. A speech-to-text mannequin writes the response content material as textual content. A context-aware voice renderer then turns that textual content into streaming speech. It circumstances on dialog historical past, voice cues and acoustic context.

Coaching is organized into 3 layers: Suppose, Act, and Communicate and Coordinate.

Suppose: M²-OPD

Core-Cocktail SFT re-anchors the audio mannequin to its supply textual content LLM utilizing million-hour-scale paired knowledge. Multimodality OPD follows. A Textual content Instructor and a frozen Audio Reference rating every token of the scholar’s personal trajectory. That is on-policy distillation, not imitation of pre-written solutions. Area consultants for empathy, pragmatic intent and acoustic scenes are then skilled with GRPO. Multi-Instructor OPD merges them into 1 deployable mannequin.

Act: Executable Environments

Every coaching area bundles a software pool, a stateful JSON database and a natural-language enterprise coverage. Domains are seeded from open-source software and MCP server definitions. Each job defines 1 of three outcomes: a write, a justified refusal, or an unsupported request. Scoring checks terminal state, then permitted writes, then behavioral assertions. A fluent reply can’t rescue a failed state verify.

GRPO receives rewards at dialogue, milestone and switch stage. Search coaching penalizes redundant queries with rquestion=qmin⁡(1,nrefnpred)r_{textual content{question}} = q min left( 1, frac{n_{textual content{ref}}}{n_{textual content{pred}}} proper). Imply queries per search name fell from 4.37 to 1.05. Set off F1 slipped from 60.87% to 58.61%.

Communicate and Coordinate

This layer decides whether or not, when and methods to communicate. On Full-Duplex-Bench v1.5, replies to folks speaking to another person fell from 0.13 to 0.03. On v3.0, the filler fee dropped from 0.7590 to 0.2960. There are trade-offs. After interruptions, the undesirable resume fee rose from 0.035 to 0.130. Interruption cease latency is 1.116 seconds, versus 0.383 for GPT-Realtime-2.

Interactive Explainer

Discover the Suppose, Act, Communicate loop, duplex selections, a scored coaching episode and the search reward.

‘+bar(‘3.0′,x[1],’f30’,x[1].toFixed(3).substitute(/0+$/,”).substitute(/.$/,”))+bar(‘3.1′,x[2],’f31’,x[2].toFixed(4).substitute(/0+$/,”).substitute(/.$/,”))}
$(‘qaBars’).innerHTML=h;anim($(‘qaBars’));submit()}
perform bar(lab,v,cls,txt){return ‘

‘+lab+’‘+(txt===”?’0′:txt)+’

‘}
perform anim(root){var f=root.querySelectorAll(‘.fill’);setTimeout(perform(){for(var i=0;iSK2859 and Aug 20 fast, almost in one breath.’},
{t:’Model calls search_flights(SFO, NRT, 2026-08-20) to confirm Flex seats remain.’},
{t:’Model reads back flight, date, Flex fare, $990.0 and expiry, then gets explicit confirmation.’,c:1},
{t:’Model calls place_hold(..., fare_class="flex"). Tool writes to the private DB copy.’,w:1},
{t:’Model tells Emily the reference HD482902 and expiry 2026-08-15.’},
{t:’Scorer runs 3 checks in order, using the tool trace, not the dialogue.’,s:1}];
var ei=-1;
var DB0='”HD482901″: {n “flight_no”: “SK2859”,n “status”: “active”,n “expiry_date”: “2026-08-08″ // expiredn}’;
var DB1=DB0+’,n”HD482902”: {n “flight_no”: “SK2859”,n “date”: “2026-08-20”,n “fare_class”: “flex”,n “amount”: 990.0,n “status”: “active”,n “created_date”: “2026-08-12”,n “expiry_date”: “2026-08-15″n}’;
perform epRender(){var ul=$(‘qaSteps’),skip=$(‘qaSkip’).checked,h=””;for(var i=0;i‘+(i+1)+’‘+(EP[i].c&&skip&&iSkipped: no read-back, no affirmation.’:EP[i].t)+’‘}ul.innerHTML=h;
$(‘qaDb’).textContent=ei>=3?DB1:DB0;
var s=ei>=5;perform set(id,okay){var e=$(id);e.className=s?(okay?’okay’:’dangerous’):’pend’;e.textContent=s?(okay?’PASS’:’FAIL’):’pending’}
set(‘qaC1’,true);set(‘qaC2’,true);set(‘qaC3’,!skip);var v=$(‘qaVerdict’);
if(s){v.innerHTML=skip?’Episode FAILS. Right state can’t excuse a damaged interplay rule.’:’Episode PASSES. Legitimate success feeds GRPO.’}else v.textContent=”Verdict appears after scoring”;submit()}
$(‘qaEpNext’).onclick=perform(){if(ei=r?’ over’:”)+'”>question ‘+(i+1)+”;$(‘qaChips’).innerHTML=h;submit()}
$(‘qaQ’).oninput=rw;$(‘qaR’).oninput=rw;$(‘qaP’).oninput=rw;rw();

/* SLIDE 5 */
var M=[
{k:’τ-Voice overall (%)’,g:43.8,a:78.4,b:82.0,d:1,mx:100,low:false,n:’Half-duplex speech-to-text adaptation by the Qwen team; not comparable to official full-duplex τ-Voice results.’},
{k:’Audio MultiChallenge (%)’,g:50.33,a:47.12,b:52.21,d:2,mx:100,low:false,n:’GPT-4o-mini judge; official AMC uses o4-mini, so scores are not directly comparable to the leaderboard.’},
{k:’Multilingual BBA avg (%)’,g:82.1,a:81.7,b:88.1,d:1,mx:100,low:false,n:’In-house 14-language extension of Big Bench Audio. Largest gains on Arabic, Thai and Vietnamese.’},
{k:’Multi-turn attack, zh (%)’,g:42.0,a:80.5,b:26.0,d:1,mx:100,low:true,n:’In-house, 200 sessions. Attack success rate: lower is better.’},
{k:’Multi-party sessions passed’,g:0.07,a:0.08,b:0.96,d:2,mx:1,low:false,n:’In-house Qwen-Audio-FDB, 100 sessions; a session passes only if every point passes.’},
{k:’Human red-team pass (%)’,g:96.0,a:null,b:92.0,d:1,mx:100,low:false,n:’5 testers, 50 sessions, automatic judge. GPT-Realtime-2 leads here.’}];
var curMet=0,mb=$(‘qaMet’);
for(var m=0;m

Qwen-Audio-3.0-Realtimenot reported

alibaba audio x
Share. Facebook Twitter LinkedIn Email
Avatar
Gavin Wallace

Related Posts

A Group of Researchers from Columbia and CertiK Introduce TensorLean for Formal Verification of GPU Kernels

22/11/2026

OpenAI Ships GPT-Stay-1.1 for Voice Brokers, Cuts Idle Value to $0.02/Min

21/11/2026

Meet Pondering Machines Inkling 2 Mini: 26B Looped MoE Runs on a 32GB Mac

21/11/2026

Baseten Releases Bison-1 and Groundhog-1, Topping Choice Mannequin Index

21/11/2026
Top News

The ‘Pain in the Ass’ Math That May Save Vibe Coding From Itself

Pope Leo teaches the Tech Bros about Tolkien

Anthropic claims that Claude has its own set of emotions

Mistral’s at the right place and time

The Army is burning through its AI tokens

Load More
AI-Trends.Today

Your daily source of AI news and trends. Stay up to date with everything AI and automation!

X (Twitter) Instagram
Top Insights

Why Your Biomedical RAG Is Hiding Contradictions From You – Unite.AI

31/07/2026

Starcloud Raises $250M Series A Extension at $2.3B Valuation – Unite.AI

21/08/2026
Latest News

Tesla Rolls Out FSD V15 with Actuality View and Imaginative and prescient Upgrades – Unite.AI

22/11/2026

Potential Cognition AI Spherical Would Elevate $2B-Plus at $75B Valuation – Unite.AI

22/11/2026
X (Twitter) Instagram
  • Privacy Policy
  • Contact Us
  • Terms and Conditions
© 2026 AI-Trends.Today

Type above and press Enter to search. Press Esc to cancel.