Close Menu
  • AI
  • Content Creation
  • Tech
  • Robotics
AI-trends.todayAI-trends.today
  • AI
  • Content Creation
  • Tech
  • Robotics
Trending
  • Potential Cognition AI Spherical Would Elevate $2B-Plus at $75B Valuation – Unite.AI
  • OpenRouter Cuts Three Suppliers After AI Immediate Publicity Findings – Unite.AI
  • A Group of Researchers from Columbia and CertiK Introduce TensorLean for Formal Verification of GPU Kernels
  • Collectively AI Discloses Immediate Leak After Coding Agent Uncovered Credential – Unite.AI
  • Dragos Finds Immediate Leaks and Stolen Keys in AI Provide Chains
  • Lean and Mathlib Communities Certify Claude-Assisted Riemann Proof – Unite.AI
  • AWS Routes HyperPod Inference Throughout Areas From One Endpoint – Unite.AI
  • DailyAI Briefing: A Lean-checked Riemann proof, plus 4 extra
AI-trends.todayAI-trends.today
Home»Tech»Meet the Underdog Saluki 27B: A 2-bit Qwen3.8-27B That Beats the Unique at Software Calling

Meet the Underdog Saluki 27B: A 2-bit Qwen3.8-27B That Beats the Unique at Software Calling

Tech By Gavin Wallace09/10/20264 Mins Read
Facebook Twitter LinkedIn Email
A Coding Implementation to Build an AI Agent with Live
A Coding Implementation to Build an AI Agent with Live
Share
Facebook Twitter LinkedIn Email

Underdog, the on-device assistant from Conway Analysis, has released Saluki 27B below Apache 2.0. Underdog Saluki 27B is a 2-bit GGUF of Qwen3.8-27B that matches in 7.89 GB. The total BF16 mannequin wants 54 GB. Underdog tuned the compression to guard device calling, the talent that turns a chat mannequin into an agent. For builders, which means a 27B-class agent mannequin that runs in stock llama.cpp.

TL;DR

  • Measurement: 27B dense parameters. 7.89 GB GGUF versus 54 GB for BF16.
  • Runs on: inventory llama.cpp and apps constructed on it, with full GPU offload. Optionally available 629 MB or 928 MB imaginative and prescient add-on.
  • Efficiency: 96% common retention throughout 9 benchmarks versus full Qwen3.8-27B.
  • Finest: Parallel device calls, 42 versus 35 for the total mannequin (120% retention).
  • Worst: AIME 2025, 79.2 versus 96.7 (about 82% retention).
  • Backside line:
    • Finest: beats the 54 GB unique at device calling in a sub-8 GB file.
    • Worst: competitors math and multi-step reasoning drop 12 to 18 factors.

What’s Underdog Saluki 27B?

Saluki 27B is a 2-bit, mixed-precision GGUF of Qwen3.8-27B constructed for native brokers. It stacks 3 layers of labor:

  • The bottom is Qwen3.8-27B, a dense 27B mannequin from the Qwen crew. It has 64 layers, mixes Gated DeltaNet linear consideration with gated consideration, and helps 262,144 tokens natively.
  • The second layer is ISTA-DASLab’s Qwen3.8-27B-GSQ-RCO-GGUF. GSQ learns correct low-bit scalar grids per tensor. RCO assigns a quantization sort to every tensor below a hard and fast dimension price range. ISTA’s smallest file, IQ2_XS, is 8.4 GB at 2.50 bits per weight.
  • The third layer is Underdog’s personal move. It shrank the file to 7.89 GB and focused device calling. The file is called IQ2-mix and carries an imatrix tag. Underdog has not printed the total recipe for this move.

How does Saluki carry out on benchmarks?

Underdog splits its outcomes into 2 teams.

The primary group ran each fashions in the identical harness:

  • Underdog Bench: 120 duties from BFCL v4, frozen earlier than testing. Considering off, temperature 0. Saluki scores 88, the total mannequin 84, and PrismML’s Bonsai 2 scores 70.
  • Parallel device calls: 100 BFCL v4 parallel duties with the official checker. Saluki 42, full mannequin 35.
  • SWE-bench Verified: 50 points. Saluki fixes 30, the total mannequin 33.

The second group compares Saluki with public full-size scores:

Benchmark Saluki 27B Qwen3.8-27B (public)
IFEval (prompt-loose) 93.5 91.5
IFBench (prompt-loose) 72.7 71.0
MBPP+ 78.0 83.9
MuSR 67.5 79.6
AIME 2025 (avg@4) 79.2 96.7
AIME 2026 (avg@4) 80.0 94.6

window.addEventListener(“message”,operate(e){var f=doc.getElementById(“mtp-saluki-frame”);if(f&&e.supply===f.contentWindow&&e.knowledge&&e.knowledge.mtpH){f.model.top=e.knowledge.mtpH+”px”;}});

How does Saluki evaluate with different compact Qwen3.8-27B builds?

Function Underdog Saluki 27B Qwen3.8-27B (BF16) ISTA GSQ-RCO IQ2_XS PrismML Bonsai 2 27B (PTQ1_0)
Org Underdog (Conway Analysis) Qwen crew ISTA-DASLab PrismML
Parameters 27B 27B 27B 27.36B
File dimension 7.89 GB 54 GB 8.4 GB 5.95 GB
Bits per weight Not disclosed (2-bit combine) 16 2.50 1.75
Context Not disclosed 262,144 native Not disclosed 262K
Imaginative and prescient Add-on, 629 or 928 MB Native 0.9 GB mmproj Optionally available 0.63 GB
Runtime Inventory llama.cpp Transformers, vLLM, SGLang Inventory llama.cpp, Ollama, LM Studio PrismML llama.cpp fork
Underdog Bench (of 120) 88 84 Not disclosed 70
Vendor headline 96% avg retention, 9 benchmarks Baseline 100.3% zero-shot restoration 98.2% of FP16, 14 benchmarks
License Apache 2.0 Apache 2.0 Apache 2.0 Apache 2.0

Bonsai 2 is smaller and experiences 98.2% retention throughout 14 thinking-mode benchmarks. It additionally posts stronger math, together with 95.00 on AIME25. Nonetheless, it wants PrismML’s llama.cpp fork, since inventory llama.cpp rejects its packing codecs. Saluki runs on inventory llama.cpp. Every vendor makes use of its personal harness, so cross-vendor scores will not be straight comparable.

Key Takeaways

  • Saluki 27B matches Qwen3.8-27B into 7.89 GB, down from 54 GB.
  • It beats the total mannequin on device calling: 88 versus 84.
  • Parallel device calls rise to 42 from 35.
  • Math and reasoning take the largest hit, right down to about 82 to 85%.
  • It runs in inventory llama.cpp below Apache 2.0.


Take a look at the Model Card on Hugging Face, the Underdog launch page and the announcement on X. All credit score goes to the researcher of this mission. Additionally, be at liberty to comply with us on Twitter and don’t neglect to affix our 150k+ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well.

The submit Meet the Underdog Saluki 27B: A 2-bit Qwen3.8-27B That Beats the Original at Tool Calling appeared first on MarkTechPost.

ar software uk war
Share. Facebook Twitter LinkedIn Email
Avatar
Gavin Wallace

Related Posts

A Group of Researchers from Columbia and CertiK Introduce TensorLean for Formal Verification of GPU Kernels

22/11/2026

OpenAI Ships GPT-Stay-1.1 for Voice Brokers, Cuts Idle Value to $0.02/Min

21/11/2026

Meet Pondering Machines Inkling 2 Mini: 26B Looped MoE Runs on a 32GB Mac

21/11/2026

Baseten Releases Bison-1 and Groundhog-1, Topping Choice Mannequin Index

21/11/2026
Top News

French Authorities Calls Emergency Conferences With Platforms Over Protests

OpenAI terminates an employee due to insider market trading predictions

The Pinterest users are tired of all the artificial intelligence slop

Carbon-X: Hugging Face’s Open Mannequin That Reads DNA

Dr. ChatGPT Sees You Right Now

Load More
AI-Trends.Today

Your daily source of AI news and trends. Stay up to date with everything AI and automation!

X (Twitter) Instagram
Top Insights

Python Implementation for a Secure AI Agent that has Self-Auditing guardrails, Redaction of PII, and Safe Tool access

13/10/2025

The Humanoids at China’s Robot Games Were Faster Than Usain Bolt—but I’m More Impressed by Their Tweezer Mastery

26/08/2026
Latest News

Potential Cognition AI Spherical Would Elevate $2B-Plus at $75B Valuation – Unite.AI

22/11/2026

OpenRouter Cuts Three Suppliers After AI Immediate Publicity Findings – Unite.AI

22/11/2026
X (Twitter) Instagram
  • Privacy Policy
  • Contact Us
  • Terms and Conditions
© 2026 AI-Trends.Today

Type above and press Enter to search. Press Esc to cancel.