We spent about three days testing a question: how much real coding-agent work can you push onto a small model running on cheap, old hardware, instead of calling a frontier model in the cloud for everything? Short answer: more than we expected. Grunt work — running tests, refactors, data pulls — turned out to be most of our token spend, and moving it onto the local model cut our monthly AI bill by roughly 70%. The card paid for itself in weeks. This is what broke on the way there and what we changed.
The setup. One GPU: an RTX 4070 Ti, 12GB of VRAM, 64GB of system RAM. About $500 used. Ollama serving qwen3-coder-30b-a3b at a 4-bit quant. Fable (Anthropic's latest) stays the planner — it writes the roadmap and breaks work into steps. The local model does the grunt work: run the tests, fix what breaks, refactor, pull analytics and write them up. Our day-to-day is now Fable for thinking, the local model for doing.
The received wisdom is that you need a 24GB card and a frontier model for any of this. We wanted to know how true that still is.
Problem: it stopped after every message
The first and worst symptom. The model would emit some reasoning, the turn would end, and nothing had run. You type continue, it does one step, stops again. Completely unusable as an agent.
Reading the raw event log (not the model's own summary, which is confidently wrong most of the time) it turned out "it stops" was several different things. The one that mattered: when you give the model a lot of tools, qwen3-coder sometimes emits its tool call as XML in the message body instead of as a real tool call:
<function=terminal>
<parameter=command>
cd /root && ls -la
</parameter>
</function>
To the harness that's just text. Nothing runs, the turn ends, the agent looks dead. It's probabilistic (fine for ten turns, then it leaks) and gets worse the more tools you expose, which for agent work is always.
The fix that stuck: we stopped trying to fix the model and fixed the pipe. A small proxy sits between the harness and Ollama, watches every response, and rewrites leaked XML into a real tool call before the harness sees it. About 120 lines. Point everything at it instead of Ollama directly and the whole problem disappears.
cleaned, calls = extract_tool_calls(content)
if calls:
msg["content"] = cleaned or None
msg["tool_calls"] = calls
choice["finish_reason"] = "tool_calls"
This is also where we made a harness decision. We tried OpenCode first and could not get it stable with a local model — even after the proxy, its handling of local tool calls kept dropping steps, and we abandoned it. Hermes behaved: with the proxy in front and its tools trimmed to what the task needs, it runs long multi-step jobs without stalling. Everything below runs on Hermes.
Problem: the model didn't fit, then kept forgetting
Two hardware-shaped problems.
First, Ollama defaults to a tiny context window (4K, 8K on some builds). An agent loop burns through that in a couple of tool calls, the context truncates, and the model "forgets" it was mid-task and stops. You have to set num_ctx explicitly in a Modelfile. This is step zero and it is not optional.
Second, why a 30B model runs on a 12GB card at all: qwen3-coder-30b-a3b is a mixture-of-experts model. 30 billion parameters total, but only about 3 billion are active for any given token. The full weights sit in system RAM and only that small active slice does work each step, so spilling most of the model to CPU barely costs you. At a 32K window it stays in VRAM and runs around 36 tokens/sec. Push it to 64K and the KV cache spills to RAM, dropping to ~29. A dense 30B offloaded like that would be unusable; the MoE shrugs. That single property is what makes cheap hardware viable.
Problem: it needed a babysitter, and it lied
A weaker model needs guardrails a frontier model doesn't. Two we leaned on.
It hallucinates its own environment constantly. During one debugging session it kept insisting it was in a Linux sandbox with no access to the machine, while its actual commands were running fine on the host. If you trust its narration you'll chase ghosts. Trust the tool output, not the prose.
And it occasionally ends a turn having only thought, with no action. Hermes can be told to verify before it stops, and for unattended jobs we wrap the session so that an empty turn gets automatically re-prompted until a check passes or it gives up cleanly. That turned "sit and watch every step" into "submit and walk away."
What a real run looks like
Not a benchmark, just a normal task. We handed it a small module with a planted bug and a failing test, and asked it to fix it. Here's the actual run, lightly trimmed (paths shortened):
> Run test_inventory.py, find the bug in inventory.py,
fix it, and re-run the test to confirm it passes.
[tool] bash: ls -la
[tool] read inventory.py
[tool] read test_inventory.py
[tool] bash: python test_inventory.py
AssertionError: got 106
[tool] edit inventory.py
- total += it["price"] + it["qty"]
+ total += it["price"] * it["qty"]
[tool] bash: python test_inventory.py
ALL TESTS PASS
Found it: total_value was adding price and quantity
instead of multiplying. Fixed line 4, tests pass.
Six tool calls, no human in the loop, correct diagnosis and fix. That is the kind of task we used to send to a paid API by reflex.
How it scored
Vibes aren't evidence, so we built a 12-task eval suite, each task with a planted flaw and an automatic grader. The tasks:
bug fix, feature-by-tests, data analysis with a hidden anomaly, SQL query optimization, multi-tenant schema design, a multi-step tool-chain, a security audit with planted vulnerabilities, a UI dashboard build, product discovery from raw feedback, an architecture review, a build-a-package-from-scratch endurance task, and a Search-Console-style analytics task.
The local 30B passed 7 of 8 of the automatically graded tasks. The one it missed on the first pass (the analytics task) passed on a re-run through the retry wrapper. For comparison, on the same suite two other local models we tried scored 3 of 8 each — the specialized coder won not because it's the biggest but because its tool-call format survived the harness. For this work, fit beats raw size.
The endurance task is the one worth calling out: build a Python package from scratch — config, CLI, tests, README — then run the tests, fix failures, and self-verify. Twenty-plus consecutive tool calls with no human. It finished in about seven minutes without stalling. That is squarely the thing people say small local models cannot do.
The point
You do not need to call a frontier model for everything, and you do not need a $4,000 GPU to run a useful agent. A $500 card and a well-chosen small model handle a large share of real day-to-day work — writing code, fixing tests, refactoring, analyzing data — once the harness around them is built properly. The frontier model earns its keep on the hard thinking; the cheap local one earns its keep on volume.
There's a second reason this matters beyond cost: nothing leaves the machine. Every file it reads and every database it touches stays on hardware you own. For regulated or privacy-sensitive work, that's not a bonus, it's the requirement, and no API tier sells it.
The catch, and we'll be honest about it: almost none of the difficulty was the model. It was the plumbing — parsers, context windows, tool schemas, session hygiene, knowing which harness to trust. That work is unglamorous and it's most of the job. Get it right and old hardware goes a very long way.
Who we are
We're Wavicle — a small team of AI-native developers. Making models genuinely useful inside real harnesses, on real hardware, against real workloads is a new and mostly-undocumented discipline, and it's most of what we do. We're actively taking on work and looking for people to build with. If you're working on something in this space, or you want a private local-agent setup like this stood up and made reliable, we'd like to hear from you.