Local LLMs fail 96% of Claude Code tasks but run 48% of steps
Tactics · Dev.to · stat: 96% fail Developer Kenimo49 tests Claude Code against a local Qwen 3.5 model, finding that 96% of full requests fail locally due to complex planning requirements. However,…
Tactics · Dev.to · stat: 96% fail
Developer Kenimo49 tests Claude Code against a local Qwen 3.5 model, finding that 96% of full requests fail locally due to complex planning requirements. However, breaking the workflows down into 200 discrete steps reveals that local models successfully execute 48% of subtasks, such as web extraction, managed by a custom YAML routing block.
Step-level routing beats the all-or-nothing approach to local LLMs Developers can cut API costs by routing simple extraction tasks locally while reserving frontier models for complex planning.
Every claim ties to a primary source. See our methodology.