Close Menu
  • AI
  • Content Creation
  • Tech
  • Robotics
AI-trends.todayAI-trends.today
  • AI
  • Content Creation
  • Tech
  • Robotics
Trending
  • Tesla Rolls Out FSD V15 with Actuality View and Imaginative and prescient Upgrades – Unite.AI
  • Potential Cognition AI Spherical Would Elevate $2B-Plus at $75B Valuation – Unite.AI
  • OpenRouter Cuts Three Suppliers After AI Immediate Publicity Findings – Unite.AI
  • A Group of Researchers from Columbia and CertiK Introduce TensorLean for Formal Verification of GPU Kernels
  • Collectively AI Discloses Immediate Leak After Coding Agent Uncovered Credential – Unite.AI
  • Dragos Finds Immediate Leaks and Stolen Keys in AI Provide Chains
  • Lean and Mathlib Communities Certify Claude-Assisted Riemann Proof – Unite.AI
  • AWS Routes HyperPod Inference Throughout Areas From One Endpoint – Unite.AI
AI-trends.todayAI-trends.today
Home»Tech»PrismML releases Bonsai 27,B: 1 bit and Ternary builds of Qwen3.627B that run on Laptops, Phones, and Tablets

PrismML releases Bonsai 27,B: 1 bit and Ternary builds of Qwen3.627B that run on Laptops, Phones, and Tablets

Tech By Gavin Wallace14/07/20265 Mins Read
Facebook Twitter LinkedIn Email
DeepSeek Releases R1-0528: An Open-Source Reasoning AI Model Delivering Enhanced
DeepSeek Releases R1-0528: An Open-Source Reasoning AI Model Delivering Enhanced
Share
Facebook Twitter LinkedIn Email

” style=”width:100%;border:0;height:600px;display:block;overflow:hidden;” scrolling=”No,” loading=”lazy” title=”Bonsai 27B interactive explainer”>

The architecture below shapes the compression technique.

The Compression Process

Every weight has a unique code. There is only one FP16 scalar per 128-member group. Effective weight is w_i = s_g · t_i.

It is possible to carry a value in ternary form. log2(3) ≈ 1.585 bits. Additions of 128 weights per FP16 scale 16/128, giving ≈1.71 bits per weight. That is a ~9.4× reduction against FP16. Binary costs 1 + 16/128 = 1.125 bits, a ~14.2× reduction.

This representation is a continuous line that runs from the matrices to all of their components. These are the embeddings and attention projections. MLPs projections also appear. The normalization and scaling parameters are only marginally higher in precision.

Qwen3.627B is the true average. “4-bit” Build (Q4_K_XL), which is 5.2 bit per weight. The weight of the build (Q4_K_XL) is 5.2 bits. “2-bit” Build (IQ2_XXS), is 2.8. Bonsai is also a departure from BitNet. BitNet can only be avoided by pretraining.

It is obvious to ask what accuracy costs when compressed.

The Performance of a

PrismML evaluated fifteen benchmarks, in thinking mode using EvalScope on H100 graphics cards. Ternary Bonsai27B maintains 94.6% and 1-bit Bonsai27B retains 89.6% respectively of the FP16 benchmark.

Variant True bpw Footprint Think avg Density (in g/kg)
Qwen3.6-27B FP16 16.0 54GB 85.07 0.051
Qwen3.6-27B Q4_K_XL (“4-bit”) 5.2 17.6GB 84.99 0.155
Qwen3.6-27B IQ2_XXS (“2-bit”) 2.8 9.4GB 72.73 0.199
Ternary Bonsai 28B 1.71 5.9GB 80.49 0.400
Bonsai 1 Bit 27B 1.125 3.9GB 76.11 0.530
Categories FP16 Ternary 1-bit
It is a good idea to use the word “Me” instead of “mean”. 95.33 93.40 91.66
Coding 88.74 85.96 81.88
Understanding and Reasoning 83.15 76.96 73.39
The call of the Agentic Tool 80.00 74.01 66.03
Please follow the instructions below 78.47 71.77 65.74
Visualise the Future 72.61 65.19 59.57

The failure of conventional sub-4 bit builds is different. LiveCodeBench and AIME26 both drop IQ2_XXS to 56.5 on LiveCodeBench. The benchmarks are still 88.93 for MMLU Redux. The pattern is repeated on the second model, Gemma-4-3B Q2_K_XL.

However, scores alone do not explain release. Memory is the key.

Memory as a Binding Constrained

Storage numbers are not the only thing to consider when fitting a new phone. iOS only allows a single program to use about half of the memory. The 12GB iPhone exposes approximately 6GB.

Second budget is KV cache. Only 16 of 64 layers carry a growing full-attention cache, so FP16 costs ≈64 KiB/token. A 262K window costs ≈17.2GB, and a 4-bit KV cache cuts that to ≈4.3GB.

The tolerance is then measured. Ternary Bonsai, a FP16 KV-based company, shows output-forward-KL of 0.0011 nanometers on MATH 500. Q4_K_XL displays 0.0146.

Peaks follow. Peaks follow. The derived Q4_K_XL row needs ≈25.6GB.

After a model is found to fit, the question of throughput arises.

Throughput and DSpark Speculative decoding

The Platform A Variant tg128 pp512
M5 Max The Binary System 66.4 874
M5 Pro Ternary 26.2 393
iPhone 17 Pro Max The Binary System 11.0 111
H100 (CUDA) The Binary System 104.8 2755

Generation is memory-bandwidth-bound, so fewer bytes per step means more tokens per second. Prefill has a lower gain because it is compute bound.

PrismML also includes a DSpark drafting system that has been trained to use the Bonsai 27B. On an H100 at draft depth k=4, the binary target reaches accepted length τ=3.6. That is 143.8 tok/s, a 1.37× speedup. Verification is lossless, so output stays distribution-identical. Apple Silicon’s drafter defaults to batch size 1, so output is distribution-identical.

Run It

Demo repo default is Ternary27B. You can start the server or directly generate:

./scripts/start_llama_server.sh   # OpenAI-compatible API + chat/vision UI on :8080

./llama-cli -m ./Ternary-Bonsai-27B-gguf/Ternary-Bonsai-27B-Q2_0.gguf 
  --mmproj ./Ternary-Bonsai-27B-gguf/mmproj.gguf -c 0 
  -p "Explain KV cache growth."

mlx_lm.generate --model prism-ml/Ternary-Bonsai-27B-mlx-2bit 
  --prompt "Explain KV cache growth."

OpenAI tool calling is standard. You can also find out more about array:

curl http://localhost:8080/v1/chat/completions 
  -H "Content-Type: application/json" 
  -d '{
    "messages": [{"role": "user", "content": "What is the weather in Lisbon?"}],
    "tools": [{
      "type": "function",
      "function": {
        "name": "get_weather",
        "parameters": {"type": "object", "properties": {"city": {"type": "string"}}, "required": ["city"]}
      }
    }]
  }'

Return of the call The following are alternatives to[0].message.tool_calls. By default, the thinking mode is activated; thinking_budget_tokens It can be toggled on or off per your request.

Four deployment patterns are mapped onto this map.

Use Cases

Agents on laptops run the ternary builds for work involving full repository code over 262K tokens. The 1-bit build is run by phone-local reasoning; according to the whitepaper, 672 tokens are required for every 1% iPhone battery. By construction, privacy sensitive and offline workflows maintain prompts within the device. Combining the 4-bit KV Cache, one-GPU service fits 27B class quality onto a 24GB memory card.

What you need to know

  • Bonsai 28B converts Qwen3.6 to 27B into binary and ternary weights. This is not a pretrain.
  • The Ternary version retains 95.6% at 5.9GB, while the 1-bit version retains an 89.9% at 3.9GB.
  • PrismML says the 27B class model is first to be able to fit into a smartphone.
  • AIME LiveCodeBench & agentic tasks can cause the conventional sub-4 bit builds to fail.
  • The latest version of Apache 2.0 is shipped with llama.cpp and MLX (CUDA/Metal).

Take a look at the Technical details, GitHub Repo The following are some examples of how to get started: Model Weight. Also, feel free to follow us on Twitter Join our Facebook group! 150k+ML SubReddit Subscribe Now our Newsletter. Wait! What? now you can join us on telegram as well.

Want to promote your GitHub repo, Hugging Face page, Product release or Webinar?? Connect with us







Next articleMistral Vibe for Code vs Claude Code vs Cursor vs Codex: Four Agents Scored on One Scaffold-to-PR Task























AI ar laptop laptops phones
Share. Facebook Twitter LinkedIn Email
Avatar
Gavin Wallace

Related Posts

A Group of Researchers from Columbia and CertiK Introduce TensorLean for Formal Verification of GPU Kernels

22/11/2026

OpenAI Ships GPT-Stay-1.1 for Voice Brokers, Cuts Idle Value to $0.02/Min

21/11/2026

Meet Pondering Machines Inkling 2 Mini: 26B Looped MoE Runs on a 32GB Mac

21/11/2026

Baseten Releases Bison-1 and Groundhog-1, Topping Choice Mannequin Index

21/11/2026
Top News

The Deepfake Scams are distorting reality itself

Meet the AI Brokers Attempting to Be Actual-Life Journalists

Meta will allow job candidates to use AI during coding tests

‘Uncanny Valley’: Iran War in the AI Era, Prediction Market Ethics, and Paramount Beats Netflix

Code Metal raises $125M to Rewrite Defense Industry Code With AI

Load More
AI-Trends.Today

Your daily source of AI news and trends. Stay up to date with everything AI and automation!

X (Twitter) Instagram
Top Insights

A Calmer and More Flexible Buffer

26/03/2026

Gradium Voice Design Launches: Just Write Your Prompt and Get A Brand New Synthetic voice in Seconds

09/09/2026
Latest News

Tesla Rolls Out FSD V15 with Actuality View and Imaginative and prescient Upgrades – Unite.AI

22/11/2026

Potential Cognition AI Spherical Would Elevate $2B-Plus at $75B Valuation – Unite.AI

22/11/2026
X (Twitter) Instagram
  • Privacy Policy
  • Contact Us
  • Terms and Conditions
© 2026 AI-Trends.Today

Type above and press Enter to search. Press Esc to cancel.