Google DeepMind has just been released Gemma 4 12BA dense, multimodal model which eliminates traditional encoders. The LLM’s backbone is able to handle both audio and video. It is possible to run agentic workflows using a laptop that has 16 GB of memory. It is released under Apache 2.0.
Model Overview & Access
Gemma 12B, a transformer with only 12-billion parameters is available. The natively-implemented software can handle text, images and audio. It does not have separate encoders for audio and vision. This decoder has the same design as Gemma 4 model 31B Dense. This bridges the difference between the E4B edge-friendly variant and the 26B mix of experts.
- Architecture: Unified, encoder-free decoder-only transformer.
- Modalities: Text, image, video, and native audio input — the first mid-sized Gemma with audio.
- Hardware Requirement Runs on consumer GPU laptops and Apple Silicon Macs. Compatible with Apple Silicon Macs, consumer GPU laptops.
- License: Apache 2.0. Weights openly available and publicly downloadable.
- The stack of inference: Compatible with llama.cpp (and other llama-related programs), MLX (and vLLM), Ollama and Unsloth.
- Download: Hugging Face You can also find out more about the following: Kaggle. This is the instruct variant.
google/gemma-4-12B-it. - Integration: The OpenAI local API server, LiteRT CLI and Hugging Face Transforms are available via
Litert-lm service.
Also released is a model for Multi-Token Prediction. This reduces the inference delay on local hardware.
Architecture: the Encoder Free Design
Every previous mid-sized Gemma models used separate Transformer Encoders for audio and video. These encoders increased latency and overhead. These Gemma 4 medium models are equipped with a vision encoder that has a resolution of 550M. E2B, E4B include audio encoders with 300M parameters. The 12B does not have any of this.
The embedder has 35M parameter values.): Raw images are split into 48×48 pixel patches. With a simple matrix multiplication, each patch is projected into the LLM’s hidden dimension. Each patch is treated independently. There is no layer of attention. A factorized coordinate search is used to inject spatial position: an X matrix learned and a Y matrice learned. The model adds two embeddings to create a vector of position for a patch located at (x,y). The normalization is then added, and the model adds this to patch embedding. The entire pipeline is shown.
Wave projectionEach frame contains 640 values. Each frame contains 640 values. These values are projected linearly into the same embedded space as text. The conformer layers and feature extraction are not used. RoPE, the LLM’s current Rotary Position Embedding (RoPE), handles 1-D temporal order. In the E2B & E4B, audio encoder used 12 conformer layer. This is all removed.
Importance: With the unified weight-space, you don’t have to tune separate encoders. LoRA fine tuning or full-tuning can be used to update vision, audio and text in one pass. This is already supported by Hugging Face Transformers, Unsloth and other products.
Encoder-free technology reduces latency in multimodal communication. LLM starts immediately processing. There is no need to wait for an encoder.
Capabilities & Performance
Google DeepMind hasn’t published the full benchmark results as part of its initial launch material. According to the official release notes, the 12B MoE model is close to the 26B MoE MoE on benchmarks and has less than half of the memory footprint.
This model has the following capabilities:
- Automated speech recognition Transcribing audio without external ASR pipeline.
- Agentic reasoning Performs similar to the 26B MoE Model when running multi-step processes locally.
- Diarization. Distinguishes speakers in audio input.
- Video understanding. Processes video frames alongside audio. In a demo, a Google I/O 5-minute keynote segment was analyzed using 313 frames with 1 FPS and a budget for visual tokens of 70 per each frame.
- Coding. Created a Gradio app for image processing using its own code generator, and served locally via llama.cpp.
- Agentic multimodal workflows Gemma Skills – The Official Gemma Skills Repository github.com/google-gemma/gemma-skills Agents are pre-built with the necessary capabilities.
Google AI Edge Eloquent, Google’s app, reported a 60%+ improvement in quality with better instruction follow-through and scope adherence.
Marktechpost’s Visual Explainer
— AI research, model releases & developer tools for 1M+ practitioners.
marktechpost.com
The Key Takeaways
- Google DeepMind launched Gemma 412B – a dense multimodal model that does not require encoders. It is licensed under Apache 2.0.
- Vision and audio feed straight into the LLM backbone — no separate vision (550M) or audio (300M) encoders.
- The 35M embedder is based on a factorized lookup of X/Y positions and a single matrix. Audio projects 16kHz raw frames.
- This is the first Gemma mid-sized laptop with video and native audio. It runs on 16GB of memory.
- The benchmark performance is comparable to the 26B MoE at less than half of the memory footprint.
Check out the Model Weights You can also find out more about the following: Technical details. Also, feel free to follow us on Twitter Join our Facebook group! 150k+ ML SubReddit Subscribe Now our Newsletter. Wait! What? now you can join us on telegram as well.
Want to promote your GitHub repo, Hugging Face page, Product release or Webinar?? Connect with us

