Underdog, the on-device assistant from Conway Analysis, has released Saluki 27B below Apache 2.0. Underdog Saluki 27B is a 2-bit GGUF of Qwen3.8-27B that matches in 7.89 GB. The total BF16 mannequin wants 54 GB. Underdog tuned the compression to guard device calling, the talent that turns a chat mannequin into an agent. For builders, which means a 27B-class agent mannequin that runs in stock llama.cpp.
TL;DR
- Measurement: 27B dense parameters. 7.89 GB GGUF versus 54 GB for BF16.
- Runs on: inventory llama.cpp and apps constructed on it, with full GPU offload. Optionally available 629 MB or 928 MB imaginative and prescient add-on.
- Efficiency: 96% common retention throughout 9 benchmarks versus full Qwen3.8-27B.
- Finest: Parallel device calls, 42 versus 35 for the total mannequin (120% retention).
- Worst: AIME 2025, 79.2 versus 96.7 (about 82% retention).
- Backside line:
- Finest: beats the 54 GB unique at device calling in a sub-8 GB file.
- Worst: competitors math and multi-step reasoning drop 12 to 18 factors.
What’s Underdog Saluki 27B?
Saluki 27B is a 2-bit, mixed-precision GGUF of Qwen3.8-27B constructed for native brokers. It stacks 3 layers of labor:
- The bottom is Qwen3.8-27B, a dense 27B mannequin from the Qwen crew. It has 64 layers, mixes Gated DeltaNet linear consideration with gated consideration, and helps 262,144 tokens natively.
- The second layer is ISTA-DASLab’s Qwen3.8-27B-GSQ-RCO-GGUF. GSQ learns correct low-bit scalar grids per tensor. RCO assigns a quantization sort to every tensor below a hard and fast dimension price range. ISTA’s smallest file, IQ2_XS, is 8.4 GB at 2.50 bits per weight.
- The third layer is Underdog’s personal move. It shrank the file to 7.89 GB and focused device calling. The file is called
IQ2-mixand carries an imatrix tag. Underdog has not printed the total recipe for this move.
How does Saluki carry out on benchmarks?
Underdog splits its outcomes into 2 teams.
The primary group ran each fashions in the identical harness:
- Underdog Bench: 120 duties from BFCL v4, frozen earlier than testing. Considering off, temperature 0. Saluki scores 88, the total mannequin 84, and PrismML’s Bonsai 2 scores 70.
- Parallel device calls: 100 BFCL v4 parallel duties with the official checker. Saluki 42, full mannequin 35.
- SWE-bench Verified: 50 points. Saluki fixes 30, the total mannequin 33.
The second group compares Saluki with public full-size scores:
| Benchmark | Saluki 27B | Qwen3.8-27B (public) |
|---|---|---|
| IFEval (prompt-loose) | 93.5 | 91.5 |
| IFBench (prompt-loose) | 72.7 | 71.0 |
| MBPP+ | 78.0 | 83.9 |
| MuSR | 67.5 | 79.6 |
| AIME 2025 (avg@4) | 79.2 | 96.7 |
| AIME 2026 (avg@4) | 80.0 | 94.6 |
window.addEventListener(“message”,operate(e){var f=doc.getElementById(“mtp-saluki-frame”);if(f&&e.supply===f.contentWindow&&e.knowledge&&e.knowledge.mtpH){f.model.top=e.knowledge.mtpH+”px”;}});
How does Saluki evaluate with different compact Qwen3.8-27B builds?
| Function | Underdog Saluki 27B | Qwen3.8-27B (BF16) | ISTA GSQ-RCO IQ2_XS | PrismML Bonsai 2 27B (PTQ1_0) |
|---|---|---|---|---|
| Org | Underdog (Conway Analysis) | Qwen crew | ISTA-DASLab | PrismML |
| Parameters | 27B | 27B | 27B | 27.36B |
| File dimension | 7.89 GB | 54 GB | 8.4 GB | 5.95 GB |
| Bits per weight | Not disclosed (2-bit combine) | 16 | 2.50 | 1.75 |
| Context | Not disclosed | 262,144 native | Not disclosed | 262K |
| Imaginative and prescient | Add-on, 629 or 928 MB | Native | 0.9 GB mmproj | Optionally available 0.63 GB |
| Runtime | Inventory llama.cpp | Transformers, vLLM, SGLang | Inventory llama.cpp, Ollama, LM Studio | PrismML llama.cpp fork |
| Underdog Bench (of 120) | 88 | 84 | Not disclosed | 70 |
| Vendor headline | 96% avg retention, 9 benchmarks | Baseline | 100.3% zero-shot restoration | 98.2% of FP16, 14 benchmarks |
| License | Apache 2.0 | Apache 2.0 | Apache 2.0 | Apache 2.0 |
Bonsai 2 is smaller and experiences 98.2% retention throughout 14 thinking-mode benchmarks. It additionally posts stronger math, together with 95.00 on AIME25. Nonetheless, it wants PrismML’s llama.cpp fork, since inventory llama.cpp rejects its packing codecs. Saluki runs on inventory llama.cpp. Every vendor makes use of its personal harness, so cross-vendor scores will not be straight comparable.
Key Takeaways
- Saluki 27B matches Qwen3.8-27B into 7.89 GB, down from 54 GB.
- It beats the total mannequin on device calling: 88 versus 84.
- Parallel device calls rise to 42 from 35.
- Math and reasoning take the largest hit, right down to about 82 to 85%.
- It runs in inventory llama.cpp below Apache 2.0.
Take a look at the Model Card on Hugging Face, the Underdog launch page and the announcement on X. All credit score goes to the researcher of this mission. Additionally, be at liberty to comply with us on Twitter and don’t neglect to affix our 150k+ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well.
The submit Meet the Underdog Saluki 27B: A 2-bit Qwen3.8-27B That Beats the Original at Tool Calling appeared first on MarkTechPost.

