Gemma-4 on a 2016 Xeon: Cost-effective LLM inference for indie founders

This review examines a setup for running Gemma-4 on a decade-old Xeon CPU. We detail the claimed performance, cost implications, and feasibility for indie founders seeking local, GPU-free LLM inference without significant hardware investment.

TL;DR

Best for: Indie founders, hobbyists, or small teams needing local LLM inference without GPU investment, particularly for text generation with MTP Drafters. Those with existing older server hardware. Skip if: Production-grade performance, low-latency inference, or large-scale model deployment is required. Users without existing compatible hardware seeking a general-purpose solution. Bottom line: The "10-year-old Xeon" setup offers a surprisingly viable, cost-effective path to local Gemma-4 inference for specific use cases, but it is not a general-purpose solution.

METHODOLOGY

This v0 review evaluates the claims made by cafkafk in their blog post, "A 10 year old Xeon is all you need (for 2B-A4B MTP Drafters without GPU)," published on point.free. The review focuses on a specific technical setup: running Google's Gemma-4 model (2B and 7B parameter versions, quantized to A4B) on a 2016 Intel Xeon E3-1505M v5 CPU with 32GB DDR4 RAM. The source signal, accessed on June 1, 2026, details the hardware specifications, the software stack (primarily llama.cpp and MTP Drafters), and the founder's claimed performance metrics in tokens per second, alongside a cost analysis for acquiring such hardware. This review covers the founder's own claims, public artifacts linked in the blog post, and the technical details presented. It does not include independent performance benchmarks, long-term workflow integration assessments, power consumption analyses, or evaluation of edge cases. Independent benchmarks are pending, and this review will be re-tested when claims diverge from observed behavior.

WHAT IT DOES

Local LLM inference on legacy hardware

The setup demonstrates running Google's Gemma-4 model, specifically the 2B and 7B parameter versions quantized to A4B, on a 2016 Intel Xeon E3-1505M v5 CPU. This is notable because it enables modern LLM inference without the typical requirement for dedicated GPUs, making AI capabilities accessible on older, server-grade CPUs. The core premise is to leverage existing or inexpensive legacy hardware for AI development.

Software stack for CPU-only inference

The primary software component is llama.cpp, an inference engine optimized for CPU performance. This tool is crucial for making the setup viable, as it efficiently handles the quantized Gemma-4 models. The blog post pairs llama.cpp with MTP Drafters, a tool designed for interactive text generation, suggesting a practical application beyond raw model execution. This combination targets a specific workflow of drafting and content creation.

Performance metrics for Gemma-4

cafkafk claims specific token generation speeds for the setup. For Gemma-4 2B (A4B quantization), the system achieves 12.5 tokens/second. The larger Gemma-4 7B (A4B) model runs at 4.5 tokens/second. These figures are presented as sufficient for interactive text generation, particularly for drafting tasks where immediate, high-speed output is not the absolute priority.

Cost-effective hardware utilization

A central appeal of this approach is its low cost. The blog post highlights that a 10-year-old Xeon workstation, such as the one used, can be acquired for under $200. This makes it a highly economical entry point for local LLM experimentation and development, especially when compared to the significant investment required for new GPU-equipped systems capable of similar tasks.

WHAT'S INTERESTING / WHAT'S NOT

What's interesting about this setup is its potential to democratize access to LLMs. The ability to run modern models like Gemma-4 on commodity, decade-old hardware significantly lowers the barrier to entry for local AI development and experimentation. This directly challenges the prevailing narrative that LLMs are exclusively for high-end GPUs, opening doors for indie developers and hobbyists on a tight budget. The performance figures, while modest, underscore the remarkable optimization work done in llama.cpp, making CPU-only inference a practical reality for certain use cases. The specific pairing with MTP Drafters also suggests a thoughtful application beyond raw inference, targeting interactive drafting workflows where a few tokens per second are acceptable.

What's not as compelling, or what's missing from the founder's pitch, is a broader view of performance and operational costs. While impressive for its cost, the raw inference speed (4.5 tokens/second for 7B) is slow for many applications beyond single-user, non-time-critical drafting. This setup is not suitable for production-grade environments, high-throughput serving, or applications requiring very low latency. The blog post also does not address power consumption. An older Xeon, while cheap to acquire, might have higher idle and load power draw compared to modern, more efficient hardware, impacting long-term operational costs. Furthermore, the

Pull quote: “The setup demonstrates running Google's Gemma-4 model, specifically the 2B and 7B parameter versions quantized to A4B, on a 2016 Intel Xeon E3-1505M v5 CPU.”