Close Menu
  • AI
  • Content Creation
  • Tech
  • Robotics
AI-trends.todayAI-trends.today
  • AI
  • Content Creation
  • Tech
  • Robotics
Trending
  • OpenRouter Cuts Three Suppliers After AI Immediate Publicity Findings – Unite.AI
  • A Group of Researchers from Columbia and CertiK Introduce TensorLean for Formal Verification of GPU Kernels
  • Collectively AI Discloses Immediate Leak After Coding Agent Uncovered Credential – Unite.AI
  • Dragos Finds Immediate Leaks and Stolen Keys in AI Provide Chains
  • Lean and Mathlib Communities Certify Claude-Assisted Riemann Proof – Unite.AI
  • AWS Routes HyperPod Inference Throughout Areas From One Endpoint – Unite.AI
  • DailyAI Briefing: A Lean-checked Riemann proof, plus 4 extra
  • Anthropic Shares Drop 8.5% in Second Session Throughout AI Selloff – Unite.AI
AI-trends.todayAI-trends.today
Home»Tech»Google DeepMind releases Gemma 12B, an Encoder-Free multimodal model with Native Audio that runs on 16GB laptop

Google DeepMind releases Gemma 12B, an Encoder-Free multimodal model with Native Audio that runs on 16GB laptop

Tech By Gavin Wallace03/06/20267 Mins Read
Facebook Twitter LinkedIn Email
Can LLMs Really Judge with Reasoning? Microsoft and Tsinghua Researchers
Can LLMs Really Judge with Reasoning? Microsoft and Tsinghua Researchers
Share
Facebook Twitter LinkedIn Email

Google DeepMind has just been released Gemma 4 12BA dense, multimodal model which eliminates traditional encoders. The LLM’s backbone is able to handle both audio and video. It is possible to run agentic workflows using a laptop that has 16 GB of memory. It is released under Apache 2.0.

Model Overview & Access

Gemma 12B, a transformer with only 12-billion parameters is available. The natively-implemented software can handle text, images and audio. It does not have separate encoders for audio and vision. This decoder has the same design as Gemma 4 model 31B Dense. This bridges the difference between the E4B edge-friendly variant and the 26B mix of experts.

  • Architecture: Unified, encoder-free decoder-only transformer.
  • Modalities: Text, image, video, and native audio input — the first mid-sized Gemma with audio.
  • Hardware Requirement Runs on consumer GPU laptops and Apple Silicon Macs. Compatible with Apple Silicon Macs, consumer GPU laptops.
  • License: Apache 2.0. Weights openly available and publicly downloadable.
  • The stack of inference: Compatible with llama.cpp (and other llama-related programs), MLX (and vLLM), Ollama and Unsloth.
  • Download: Hugging Face You can also find out more about the following: Kaggle. This is the instruct variant. google/gemma-4-12B-it.
  • Integration: The OpenAI local API server, LiteRT CLI and Hugging Face Transforms are available via Litert-lm service.

Also released is a model for Multi-Token Prediction. This reduces the inference delay on local hardware.

Architecture: the Encoder Free Design

Every previous mid-sized Gemma models used separate Transformer Encoders for audio and video. These encoders increased latency and overhead. These Gemma 4 medium models are equipped with a vision encoder that has a resolution of 550M. E2B, E4B include audio encoders with 300M parameters. The 12B does not have any of this.

The embedder has 35M parameter values.): Raw images are split into 48×48 pixel patches. With a simple matrix multiplication, each patch is projected into the LLM’s hidden dimension. Each patch is treated independently. There is no layer of attention. A factorized coordinate search is used to inject spatial position: an X matrix learned and a Y matrice learned. The model adds two embeddings to create a vector of position for a patch located at (x,y). The normalization is then added, and the model adds this to patch embedding. The entire pipeline is shown.

Wave projectionEach frame contains 640 values. Each frame contains 640 values. These values are projected linearly into the same embedded space as text. The conformer layers and feature extraction are not used. RoPE, the LLM’s current Rotary Position Embedding (RoPE), handles 1-D temporal order. In the E2B & E4B, audio encoder used 12 conformer layer. This is all removed.

Importance: With the unified weight-space, you don’t have to tune separate encoders. LoRA fine tuning or full-tuning can be used to update vision, audio and text in one pass. This is already supported by Hugging Face Transformers, Unsloth and other products.

Encoder-free technology reduces latency in multimodal communication. LLM starts immediately processing. There is no need to wait for an encoder.

Capabilities & Performance

Google DeepMind hasn’t published the full benchmark results as part of its initial launch material. According to the official release notes, the 12B MoE model is close to the 26B MoE MoE on benchmarks and has less than half of the memory footprint.

https://blog.google/innovation-and-ai/technology/developers-tools/introducing-gemma-4-12b/

This model has the following capabilities:

  • Automated speech recognition Transcribing audio without external ASR pipeline.
  • Agentic reasoning Performs similar to the 26B MoE Model when running multi-step processes locally.
  • Diarization. Distinguishes speakers in audio input.
  • Video understanding. Processes video frames alongside audio. In a demo, a Google I/O 5-minute keynote segment was analyzed using 313 frames with 1 FPS and a budget for visual tokens of 70 per each frame.
  • Coding. Created a Gradio app for image processing using its own code generator, and served locally via llama.cpp.
  • Agentic multimodal workflows Gemma Skills – The Official Gemma Skills Repository github.com/google-gemma/gemma-skills Agents are pre-built with the necessary capabilities.

Google AI Edge Eloquent, Google’s app, reported a 60%+ improvement in quality with better instruction follow-through and scope adherence.

Marktechpost’s Visual Explainer

Released on June 3, 2026

Gemma 4 12B

Google DeepMind’s multimodal, unified model is encoder-free and unified.

The 12-billion-parameter transformer is a decoder only, removing the need for separate encoders. LLM is the backbone of LLM. The software runs on a laptop with 16 GB of memory under the Apache 2.0 License.

  • Encoder-free — no separate vision or audio encoders
  • First mid-sized Gemma Video and audio are now available.
  • Local-ready — 16 GB VRAM or unified memory

Overview & Access

What Ships

Inference, weights and Specs

  • Architecture — decoder-only, same structure as Gemma 4 31B Dense
  • Modalities — text, image, video, and native audio
  • Hardware — 16 GB VRAM / unified memory; GPU laptops and Apple Silicon
  • License — Apache 2.0; weights on Hugging Face and Kaggle
  • You can also Instruct a variant — google/gemma-4-12B-it
  • Fasten your seatbelts — a dedicated Multi-Token Prediction (MTP) drafter is also released

Architecture · Vision

The 35M Vision Embedder

The 550M Vision Encoder is replaced on the mid-sized models

  • Splitting raw images into 48×48 pixel patches
  • The LLM Hidden Dimension is a projection of each patch. single matrix multiplication
  • No attention layer — each patch is processed independently
  • A position is a factorized X/Y coordinate lookupNormalization is the next step
  • The entire pipeline of vision

Architecture · Audio

Audio Wave Projection

No conformer layers, no feature extraction

  • Take away the 12 conformer layers Gemma 4 E2B & E4B
  • Raw 16 kHz audio You can slice into The 40-ms frame 640 each
  • Projected frames into the wall Same embedding area as text tokens
  • There are currently two LLMs available. RoPE Handles the temporal order
  • First mid-sized Gemma with native audio ingest

Capabilities & Performance

Near-26B reasoning: half of the memory

Google’s performance is nearing 26B MoE with less than half of the memory footprint

  • ASR & diarization — native transcription, speaker separation
  • The Agentic Reasoning — multi-step workflows run locally
  • Video — demo on a 5-min I/O keynote: 313 frames at 1 FPS, token budget 70
  • Coding — built a Gradio app via gemma-skillsServe llama with this dish
  • There will be no full benchmark table published for launch

You can run it locally

Three different paths for day one

Native macOS applications plus an optional local server

  • Google AI Edge Gallery (macOS) — sandboxed Python execution loop
  • Google AI Edge Eloquent (macOS) — on-device dictation and editing
  • LiteRT CLI — Litert-lm service OpenAI compatible endpoint exposed
  • Open WebUI and Continue, Aider or OpenCode
  • Also LM Studio Transformers Unsloth vLLM SGLang MLX
  • Cloud Run GKE or Gemini Enterprise Agent Platform model Garden

The Key Takeaways

Bottom line

The encoder-free system is a great way to save money

  • No separate visual (550M), or audio (300M), encoders
  • The 35M embedded vision system with direct audio wave projector
  • In a single step, fine-tuning can update vision, text, and audio.
  • Runs on 16GB and achieves performance close to 26B at less than half of the memory.
  • Apache 2.0 comes with broad support from the start

Marktechpost
— AI research, model releases & developer tools for 1M+ practitioners.
marktechpost.com

The Key Takeaways

  • Google DeepMind launched Gemma 412B – a dense multimodal model that does not require encoders. It is licensed under Apache 2.0.
  • Vision and audio feed straight into the LLM backbone — no separate vision (550M) or audio (300M) encoders.
  • The 35M embedder is based on a factorized lookup of X/Y positions and a single matrix. Audio projects 16kHz raw frames.
  • This is the first Gemma mid-sized laptop with video and native audio. It runs on 16GB of memory.
  • The benchmark performance is comparable to the 26B MoE at less than half of the memory footprint.

Check out the Model Weights You can also find out more about the following: Technical details. Also, feel free to follow us on Twitter Join our Facebook group! 150k+ ML SubReddit Subscribe Now our Newsletter. Wait! What? now you can join us on telegram as well.

Want to promote your GitHub repo, Hugging Face page, Product release or Webinar?? Connect with us


audio code deepmind Google laptop
Share. Facebook Twitter LinkedIn Email
Avatar
Gavin Wallace

Related Posts

A Group of Researchers from Columbia and CertiK Introduce TensorLean for Formal Verification of GPU Kernels

22/11/2026

OpenAI Ships GPT-Stay-1.1 for Voice Brokers, Cuts Idle Value to $0.02/Min

21/11/2026

Meet Pondering Machines Inkling 2 Mini: 26B Looped MoE Runs on a 32GB Mac

21/11/2026

Baseten Releases Bison-1 and Groundhog-1, Topping Choice Mannequin Index

21/11/2026
Top News

Disinformation Floods Social Media After Nicolás Maduro’s Capture

I Recorded Myself for Money. Who’s The Robot Now?

Trump Drafts Executive Order to Target State AI Laws

Nvidia’s Board Has No Plans to Convey Again Jensen Huang

‘I Like My Big Rat Wife’: Meet the People Using Chatbots to Write Custom Fiction

Load More
AI-Trends.Today

Your daily source of AI news and trends. Stay up to date with everything AI and automation!

X (Twitter) Instagram
Top Insights

UK House Workplace Might Face Investigation Over Use of OpenAI Instruments

27/10/2026

YouTube to stream Oscars exclusively starting in 2029

18/12/2025
Latest News

OpenRouter Cuts Three Suppliers After AI Immediate Publicity Findings – Unite.AI

22/11/2026

A Group of Researchers from Columbia and CertiK Introduce TensorLean for Formal Verification of GPU Kernels

22/11/2026
X (Twitter) Instagram
  • Privacy Policy
  • Contact Us
  • Terms and Conditions
© 2026 AI-Trends.Today

Type above and press Enter to search. Press Esc to cancel.