Google Launches Gemma 4: A New Era for Local LLMs

Google has introduced the Gemma 4 family of local large language models (LLMs), which includes four distinct models: the 31B dense model, the 26B-A4B Mixture of Experts (MoE) model, and two edge models, E4B and E2B. All models are built from the Gemini 3 research and released under the Apache 2.0 license, facilitating their use in commercial and self-hosted projects. The models are designed to balance quality and speed, addressing common challenges faced by users of local LLMs.

Google's Gemma 4 isn't the smartest local LLM I've run, but it's the one I reach for most

The flagship 31B dense model achieves impressive performance metrics, scoring 89.2% on AIME 2026 and 80% on LiveCodeBench. The 26B-A4B MoE model closely follows, with scores of 88.3% on AIME and 68.2% on tau2-bench, while only activating 3.8 billion parameters per token, making it faster and more efficient. The 31B model requires approximately 17.4GB of memory in 4-bit quantization, while the 26B-A4B requires about 15.6GB, making the latter more suitable for typical consumer GPUs.

Google's Gemma 4 isn't the smartest local LLM I've run, but it's the one I reach for most

The E4B and E2B models are optimized for audio input alongside text and vision, enabling functionalities such as speech recognition and document parsing without requiring internet connectivity. These models support 128K context windows, allowing users to input larger documents seamlessly. The E2B model is particularly noted for its speed, delivering three times faster inference than the E4B while consuming 60% less battery on mobile devices.

Google's Gemma 4 isn't the smartest local LLM I've run, but it's the one I reach for most

A significant advancement in Gemma 4 is the integration of tool calling directly into the model architecture. This feature allows users to define tools using JSON schemas or Python functions, enabling the model to generate structured function call objects. The 26B-A4B model, in particular, has shown reliability in executing tool calls, making it suitable for applications like local voice assistants.

Google's Gemma 4 isn't the smartest local LLM I've run, but it's the one I reach for most

Google has outlined various deployment paths for the Gemma 4 models, including AI Edge Gallery and LiteRT-LM across multiple platforms such as Android, iOS, and desktop. The edge models are particularly suited for practical applications, allowing users to build local voice assistants that understand context and execute commands without cloud dependency.

Google's Gemma 4 isn't the smartest local LLM I've run, but it's the one I reach for most

Despite its advancements, some users have expressed frustration over Google's decision to withhold Multi-Token Prediction (MTP) heads, which could enhance inference speed. This decision has led to community-driven workarounds to achieve similar performance improvements. Critics argue that this move undermines the "open" aspect of the open-weight models.

Google's Gemma 4 isn't the smartest local LLM I've run, but it's the one I reach for most

The Gemma 4 family represents a significant step forward in local LLM technology, offering a range of models that cater to different needs. The 26B-A4B model strikes a balance between speed and quality, while the edge models expand the potential use cases for local inference. For users seeking a robust local LLM solution, Gemma 4 provides a comprehensive and versatile option.

Google's Gemma 4 isn't the smartest local LLM I've run, but it's the one I reach for most