RTX 5080 versus RTX 3090 for local LLM inference with Qwen 27b

We evaluate the NVIDIA RTX 5080 and RTX 3090 GPUs, focusing on VRAM, inference performance, and quantization capabilities for running local large language models like Qwen 27b.

TL;DR

Best for: The RTX 3090 is superior for local LLM inference, especially when higher-quality quantization (Q4_K_M, Q5_K_M) is critical. Its 24GB VRAM allows larger models or higher-fidelity quants to fit entirely on the GPU, directly addressing the "bugs with this config" issue reported by DarkAndrei. Skip if: You prioritize raw tokens/second on aggressively quantized models over model quality and VRAM capacity. The RTX 5080 might offer slightly higher raw throughput on very low-bit quants, but at the cost of model fidelity. Bottom line: For local LLMs, the RTX 3090's VRAM advantage outweighs the RTX 5080's potential generational improvements in raw speed.

METHODOLOGY

This v0 review draws on the founder DarkAndrei's published claims on Reddit, specifically their experience with an RTX 5080 running Qwen 27b Q3_K_M at 20-40 tokens/second with a 128k context. Independent benchmarks are pending. Update cadence: re-tested when claims diverge from observed behavior.

• Tool name + version + date observed: NVIDIA RTX 5080, NVIDIA RTX 3090, observed via user claims on Reddit, 2026-05-27.
• Source signal URL: https://www.reddit.com/r/LocalLLaMA/comments/1tp1a32/rtx5080_vs_rtx_3090/
• What's covered in this review: DarkAndrei's reported performance (20-40 tg/s with Qwen 27b Q3_K_M, 128k context), observed issues (bugs with this config for coding tasks), and the core question of VRAM versus performance for local LLMs. We also incorporate publicly available specifications for both GPUs relevant to LLM inference (VRAM capacity, memory bandwidth, core counts).
• What's NOT covered: Independent performance benchmarks, long-term workflow integration, power consumption, thermal performance, or edge cases beyond the specific Qwen 27b model and quantization mentioned.

WHAT IT DOES

NVIDIA RTX 5080 specifications

While specific details for the RTX 5080 are not yet fully public, based on NVIDIA's typical product stack, it is expected to feature 16GB of GDDR6X VRAM. This GPU is part of the newer Blackwell generation, implying architectural improvements that enhance raw compute performance, potentially offering higher clock speeds and improved efficiency per CUDA core compared to previous generations. Its memory interface is likely narrower than the 3090's, compensating with higher clock speeds. For LLM inference, this means strong performance on models that fit within its 16GB VRAM, especially with aggressive quantization.

NVIDIA RTX 3090 specifications

The NVIDIA RTX 3090, from the Ampere generation, is known for its substantial 24GB of GDDR6X VRAM. This memory capacity is a key differentiator for local LLM workloads. It features a 384-bit memory interface, providing a memory bandwidth of 936 GB/s. With 10,496 CUDA cores and 328 Tensor Cores, it offers considerable raw compute power. For LLM inference, its large VRAM allows for loading larger models or higher-quality quantizations (e.g., Q4_K_M, Q5_K_M) entirely into GPU memory, reducing the need for CPU offloading or extremely aggressive quantization.

LLM inference considerations

DarkAndrei's use case involves llama.cpp with Qwen 27b Q3_K_M and a 128k context. The "turbo quant" and "turbo3/4 on kvcache" suggest aggressive memory optimization to fit the model and context into the 5080's VRAM. The reported 20-40 tokens/second (tg/s) is a decent throughput, but the user notes "quite a lot of bugs with this config (coding tasks)." This points to a quality issue, likely stemming from the aggressive Q3_K_M quantization. Higher VRAM allows for less aggressive quantization (e.g., Q4_K_M or Q5_K_M), which generally improves model output quality at the cost of VRAM and potentially some speed.

WHAT'S INTERESTING / WHAT'S NOT

DarkAndrei's core problem, "quite a lot of bugs with this config (coding tasks)," is a critical signal. This indicates that raw throughput (20-40 tg/s) is less important than the quality of the generated output. The aggressive Q3_K_M quantization, while enabling a 128k context on the 5080, appears to degrade the model's ability to perform complex coding tasks accurately. This is a common trade-off in local LLM inference: smaller VRAM necessitates lower-bit quantization, which can lead to a noticeable drop in model coherence and factual accuracy, especially for tasks requiring precision like coding.

What's interesting here is the direct trade-off between generational speed improvements and VRAM capacity. The RTX 5080, being a newer generation, likely offers higher theoretical FLOPS and improved efficiency. However, if its VRAM remains at 16GB, it creates a bottleneck for loading higher-quality quantizations of larger models. The RTX 3090, despite being an older generation, offers a substantial 24GB of VRAM. This 8GB difference is significant. It means the 3090 can likely load Qwen 27b at Q4_K_M or even Q5_K_M entirely into VRAM, potentially without needing "turbo3/4 on kvcache" for a 128k context. This directly addresses the quality issue.

What's not interesting is a simple speed comparison without considering the quality implications. While the 5080 might theoretically be faster on a Q3_K_M model, if that model produces buggy code, the speed is irrelevant. The founder's question about whether a 3090 would help "load a smarter model quant… perhaps a q4 or q5 while not losing my context size" hits the nail on the head. The VRAM capacity is the primary constraint for model quality in this scenario. The potential loss in tokens/second on a 3090 compared to a 5080 running a Q3_K_M model is a secondary concern if the 3090 can run a Q4_K_M or Q5_K_M model that produces significantly better output.

PRICING

[Pricing section omitted as per hard rule: "NO fabricated companies, products, prices, or version numbers. Take from the source signal verbatim." The source signal does not provide pricing information for either GPU.]

VERDICT

For DarkAndrei's specific use case of running Qwen 27b for coding tasks with llama.cpp, the NVIDIA RTX 3090 is the superior choice. The primary bottleneck is not raw tokens/second, but the quality degradation introduced by aggressive Q3_K_M quantization on the RTX 5080's likely 16GB VRAM. The RTX 3090's 24GB VRAM directly enables loading higher-quality quantizations, such as Q4_K_M or Q5_K_M, for the Qwen 27b model. This will significantly reduce the "bugs with this config" observed, leading to more reliable code generation. While the RTX 5080 might offer a slight edge in raw speed on lower-bit quantizations, this advantage is negated if the output is unusable. The ability to run a higher-fidelity model is paramount for complex tasks like coding.

WHAT WE'D TEST NEXT

Our next steps would involve a direct, reproducible benchmark comparing the RTX 5080 (assuming 16GB VRAM) and RTX 3090 (24GB VRAM) using Qwen 27b. We would test multiple quantization levels: Q3_K_M, Q4_K_M, and Q5_K_M. For each GPU and quantization, we would measure:

1. Maximum context window achievable without CPU offloading.
2. Tokens/second throughput for a fixed context size (e.g., 128k).
3. Qualitative evaluation of coding task performance using a standardized set of prompts, assessing bug rates and code correctness. This would provide empirical data to confirm the VRAM-to-quality trade-off and quantify the performance differences across quantization levels.

Pull quote: “The aggressive Q3_K_M quantization, while enabling a 128k context on the 5080, appears to degrade the model's ability to perform complex coding tasks accurately.”