zap runs a frontier model for design and a local small model for the actual coding — and it proposes which model fits a task, then waits for you to confirm. No silent downgrades, no cloud round-trip for the bulk of the edits.
Validated on qwen3-coder-30b, devstral-small-2, gemma-4-e4b, glm-4.7-flash-reap, and more — through the real zap TUI, objective verification, 191 passing tests. As of v0.15.18.
Frontier models are the right tool for deciding what to change. They're an expensive, slow, privacy-leaking tool for mechanically applying a change you've already specified. That mechanical majority of agentic coding can run on a 14B model on your laptop.
The frontier model is only paid for the thinking — add a field, rename a symbol, wire a route. The bulk of the edits run locally with no per-token billing.
The bulk of your code never leaves your laptop. Only the task plan (a few hundred tokens) goes to the cloud. The actual code changes happen locally.
No network hop for the execution half. Local inference on scoped tasks is fast — a 4-step structured plan completes in under 5 minutes end-to-end.
Once the plan is written, execution works on a plane or air-gapped network. The local model handles the entire execution phase without internet access.
zap never silently changes models. It proposes and waits — the frontier model writes a step-by-step plan with exact code snippets, then recommends which local model should execute it. Nothing runs until you press a key.
The optimal SLM workflow: a frontier model pre-writes a step-by-step plan with exact code snippets, and the SLM executes it mechanically — one step, one verification, one result. No improvisation, no dead ends.
| Open-ended goal | Structured plan | |
|---|---|---|
| Success rate | 50% per attempt | 100% first attempt |
| Wall-clock | 305–905s | 294s |
| Model turns | 4–8 | 2 |
| Failure mode | Wrong conditional → 13 min debugging | None — plan is unambiguous |
Test 6: Add a GET /todos REST endpoint to a Node.js Express app. The plan had 4 numbered steps with exact code in fenced blocks and a verification command per step. The model followed every step, made two correct edits in one turn, and verification passed.
Four features that make the difference between an SLM that spins for 20 minutes and one that ships correct code in 2 turns.
zap --index-only builds an AST index (tree-sitter + SQLite) so the SLM navigates with find_definition and find_references instead of reading dozens of files. For a 300-file project, this is the difference between a 2-turn execution and a 12-turn goose chase.
AGENT_TOOL_PROFILE=core sends only 6 tool schemas (file ops, shell, search). Small models struggle with large tool schemas — this profile makes tool-calling tractable without sacrificing capability.
Local models prefill big prompts for minutes. zap tolerates that: 1-hour streaming cap, idle detection at 10 minutes, and "waiting for first token" progress notices every 30 seconds. Backoff on stream drops prevents stacked prefills from grinding your machine.
If the SLM gets stuck in a verification loop (3 consecutive failed shell runs), zap injects a rethink nudge. At 6 failures, tools are withdrawn and the model must produce a structured handoff summary. No other local-first agent has this — OpenHands' StuckDetector only catches identical repeated actions.
Backed by a full multi-turn agentic eval — 3 realistic tasks, objective verification, through the real zap TUI.
| Model | Size | Score | Verdict |
|---|---|---|---|
devstral-small-2-2512 |
24B (MLX 4-bit, 14 GB) | 3/3 | 🥇 Default for long sessions — 256K ctx, 12× faster than Qwen3 at turn 17 (measured live) |
qwen3-coder-30b |
30B (MoE, ~3.3B active) | 3/3 | Faster early turns, RL-tuned for the agent loop |
gemma-4-e4b |
4B | 3/3 | Punches far above its size on scoped tasks |
glm-4.7-flash-reap |
23B (MoE) | 2/3 | Solid on fixes/renames, weaker on multi-step |
qwen2.5-coder-14b |
14B | 1/3 | ⚠️ Completion model — avoid for executor role |
Key lesson: model class beats model size. Pick an agentic-tuned model (Devstral, Qwen3-Coder) for the executor role — not a generic "coder" completion model. Scoped single-file tasks passed on every model tested.
Both pass the eval. The difference shows up in sustained agent loops where the conversation grows over many turns — measured live via LM Studio logs running the same zap workflow on each.
| Convo length | devstral-small-2 (24B dense) |
qwen3-coder-30b (MoE A3B) |
Winner |
|---|---|---|---|
| 2–3 msgs | 11–44s | 31s | mixed |
| 5–9 msgs | 24–71s | 10–19s | Qwen (faster small generations) |
| 11 msgs | 4s ⚡ | 37s | Devstral 9× |
| 13 msgs | 4s ⚡ | 14s | Devstral 3.5× |
| 15 msgs | 9s ⚡ | 62s | Devstral 7× |
| 17 msgs | 7s ⚡ | 86s | Devstral 12× |
Why: Qwen3-Coder MoE is fast per-token but prompt-eval cost grows roughly linearly and stalls long sessions. Devstral is dense (slower per token) but has aggressive prompt caching plus a chat template tuned for agent loops — once the prefix stabilizes, every subsequent turn is near-free. For zap's typical 15–30 turn workloads, Devstral wins.
All tests run through the real zap TUI with qwen3-coder-30b via LM Studio on a 32 GB Apple M-series machine. Full evidence, scripts, and raw logs in the research repo.
| Test | Scenario | Result | Time |
|---|---|---|---|
| Test 3 | Frontier-decomposed task (tool validation) | ✅ PASS | 214s |
| Test 4 r1 | Goal-level task (8 behaviors) | ❌ 7/8 | 905s |
| Test 4 r2 | Same goal, retry | ✅ 8/8 | 305s |
| Test 5 | Impossible task — escalation drill | ⚠️ partial | 10–45s |
| Test 6 | Structured plan execution | ✅ PASS | 294s |
Bottom line: Give an SLM a scoped, pre-written plan with exact verification steps → it executes reliably and fast. Give it an open-ended goal with ambiguous constraints → success is model and run dependent, but zap's watchdog bounds the failure cost to minutes. The recommended production workflow is the structured plan pattern (Test 6).
We ran Devstral cold — no pre-written plan, no frontier decomposition — just a raw open-ended task and the zap harness. The harness learned three things from watching the model fail, and tuned itself accordingly.
| Model | Task | Turns | Result |
|---|---|---|---|
qwen2.5:7b (Ollama) |
List files in empty dir | 16 | ❌ Created wrong files, never answered |
qwen2.5-coder-14b (LM Studio) |
LRU cache + tests | 1 | ⚠️ Correct implementation, wrong test assertions, never ran the tests |
devstral:latest (Ollama) |
Math expression evaluator + tests | 1 user turn | ✅ 6/6 tests passed |
Three targeted improvements — all gated on tier = "slm", zero impact on frontier model behaviour.
When an SLM replies with code in a markdown block but makes no tool call, zap injects a follow-up: "Call write_file NOW — do not write explanatory text first." A second, stronger nudge fires if the model chats again. Counter resets on every successful tool-call round so the model gets fresh nudges after each real action.
If the current turn's tool calls exactly match the previous turn's calls, the model is stuck — the results won't change. zap appends a note to the last tool result: "You called the same tools with the same arguments as last round. Draw a conclusion and act on it." This catches the 7B's 16-turn repeat loop that the watchdog (which only tracks failing shell commands) missed entirely.
The compact ~400-token SLM prompt (vs ~3 000 tokens for frontier) gained a sharper tool-use mandate: "Act immediately. Call write_file to create files, shell to run them. Never describe a tool call in a code block — make the actual call." Combined with the nudges, this shifted 14B from chat-only to tool-first without any other changes.
We checked every major open-source coding agent. The chat-without-tools nudge and write_file content rule are absent from all of them. The loop detector is the most differentiated — Gemini CLI has a broad version with known false-positive issues; Goose has a solid MD5-based version; no one else.
| Feature | OpenCode | Gemini CLI | Goose | Aider | zap |
|---|---|---|---|---|---|
| SLM auto-detection + compact prompt | No | No | No | No | Yes |
| 6-tool core profile for small models | No | No | No | No | Yes |
| Chat-without-tools nudge (``` block, no tool call) | No | No | No | No | Yes |
| Identical-call loop detector | Partial (count only) | Partial (false positives #11002) | Yes (MD5 hash) | No | Yes |
| write_file literal-content rule | No | No | No | No | Yes |
Sources: OpenCode tool-calling failures with Ollama (#1034, #1068);
Gemini CLI LoopDetectionService (#5276, #11002);
Goose MD5 fingerprinting, 20-call window;
Claude Code identical-call detection feature request #4277 — closed as not planned.
Single user turn. No plan, no hints. The model chose recursive descent parsing, installed its own missing dependencies, and self-corrected until tests passed.
| Test | Result |
|---|---|
| Basic arithmetic (+, -, *, /) | ✅ PASS |
| Operator precedence (1+2*3 = 7) | ✅ PASS |
| Unary minus (-3, 1--3, -(2+1)) | ✅ PASS |
| Nested parentheses (((9))/3) | ✅ PASS |
| Float numbers (3.5+2, 4.25/0.5) | ✅ PASS |
| Error cases (div/0, malformed, empty) | ✅ PASS |
View the code:
expr.py ↗ ·
tests_expr.py ↗ ·
lrucache.py ↗
— Devstral's code committed verbatim. Ollama on Apple Silicon, tier = "slm", v0.15.74.
The next eval: implement a thread-safe bounded task queue with a worker pool from scratch in Python — no concurrent.futures, implement the queue and threads yourself. 5 concurrent-correctness test cases. Harder than the expression evaluator.
Future with threading.Condition — textbook correct. Waits, notifies all, re-raises on exception. Worker pool — N daemon threads, queue-driven, clean try/except per task so one crash never kills the pool. Exception propagation through Future.result() — works. Shutdown sentinel pattern — right idea (push N Nones, join threads).
Devstral wrapped the already-thread-safe queue.Queue in an outer Lock. When the inner queue is full, queue.Queue.put(block=True) blocks while holding that outer lock — preventing any worker from calling get(). Deadlock. The fix: delete ThreadSafeQueue, use queue.Queue directly. 15 lines removed, 0 lines added.
| Test case | Devstral (raw) | After fix |
|---|---|---|
| 30 tasks across 3 concurrent threads — all results correct | ✅ PASS | ✅ PASS |
Exception from task propagates through Future.result() | ✅ PASS | ✅ PASS |
| maxsize backpressure — producers block when queue full | ✅ PASS | ✅ PASS |
shutdown(wait=True) drains all in-flight tasks | ⚠ HANG | ✅ PASS |
submit() after shutdown() raises RuntimeError | — not reached | ✅ PASS |
Condition-variable Future, sentinel-based shutdown, per-task exception isolation. A single wrapping mistake caused the deadlock; everything compositional was correct. This is what "good architecture, one bug" looks like from a 24B local model with no frontier hand-holding.
Devstral's Future class — written verbatim, no edits:
class Future:
def __init__(self):
self._result = None
self._exception = None
self._done = False
self._lock = threading.Lock()
self._condition = threading.Condition(self._lock)
def set_result(self, result):
with self._condition:
self._result = result
self._done = True
self._condition.notify_all()
def set_exception(self, exception):
with self._condition:
self._exception = exception
self._done = True
self._condition.notify_all()
def result(self, timeout=None):
with self._condition:
if not self._done:
self._condition.wait(timeout)
if self._exception:
raise self._exception
return self._resultCondition variable, notify_all, timeout-aware wait, exception re-raise — all correct, first attempt. The concurrency primitive knowledge is there.
View the code:
task_queue.py ↗ ·
tests.py ↗
(Devstral's implementation with the 15-line ThreadSafeQueue fix applied; Ollama on Apple Silicon, tier = "slm", v0.15.75)
Running a 24B model reliably in an agentic loop required five specific pieces of engineering. We audited every major open-source agent — OpenCode, Gemini CLI, Goose, Aider, Continue, Claude Code. None has all five. Most have none of the first four.
| Feature | OpenCode | Gemini CLI | Goose | Aider | zap |
|---|---|---|---|---|---|
| SLM auto-detection ≤13B local → switch prompt + tools |
No | No | No | No | Yes |
| Compact SLM prompt ~400 tok vs ~3 000 tok for frontier |
No | No | No | No | Yes |
| Reduced tool set 6 core tools only, not 20+ |
No | No | No | No | Yes |
| Chat-without-tools nudge Fires when model returns ``` but makes no tool call |
No | No | No | No | Yes |
| Identical-call loop detector (tool, args) fingerprint, fires on consecutive match |
Partial count only, not args |
Partial false positives on legitimate repetition |
Yes MD5 hash, 20-call window |
No | Yes |
| write_file content rule Prevents variable-name placeholders in tool content arg |
No | No | No | No | Yes |
Sources: OpenCode DOOM_LOOP_THRESHOLD=3 (issues #1034, #1068); Gemini CLI LoopDetectionService (issues #5276, #11002 — known false-positive problem); Goose MD5 fingerprinting (20-call window, 3-hit threshold); Aider --weak-model for summarization only; Claude Code identical-call detection feature request #4277 — closed as not planned.
No config needed. Hit a localhost URL with a model whose name contains a ≤13B size suffix (7b, 8b, 9b, 13b…) and zap switches to SLM mode automatically. Or set tier = "slm" in ~/.agent.toml for any model. Frontier model paths: zero changes — all five features are gated on is_slm_tier().
The frontier system prompt is ~3 000 tokens of scaffolding a strong model needs. Small models read it as noise. zap's SLM prompt strips to ~400 tokens: identity + navigation rules + tool rules + git. Every saved token is context the model can use for actual code. Context window injection (num_ctx: 8192) also happens automatically for Ollama models.
Small models hallucinate tool names and mix arguments when handed 20+ schemas. zap sends only 6 in SLM mode: read_file, edit_file, write_file, shell, search_code, list_directory. These cover the full agentic coding loop. The schema surface the model has to reason about drops by ~70%.
The most common SLM failure: the model describes what it would do — markdown code block, explanation, plan — but makes zero tool calls. No other agent catches this pattern. zap detects a ``` block with no tool_calls and fires: "Call write_file NOW — do not write explanatory text first." A second stronger nudge fires if the model chats again. Counter resets on every successful tool-call round.
When an SLM calls the same tool with the same arguments twice in a row, the results won't change — but small models keep trying anyway. zap fingerprints (tool_name, args_json) per round and fires when consecutive rounds match: "The results will not change. Draw a conclusion and act on it." This broke the 7B's 16-turn infinite loop — the shell-failure watchdog missed it entirely because no shell command was failing.
An observed failure unique to agentic small models: the model plans code in chat, names it as a variable, then passes the variable name — not the code — as the content argument to write_file. The call succeeds, the file is wrong. zap's SLM prompt explicitly rules this out: "content must be the COMPLETE, LITERAL file text. Never pass a variable name or placeholder." Discovered across three Devstral retries before we named the pattern.
Single binary, no runtime. Cold start in milliseconds.
Works with LM Studio, Ollama, or any OpenAI-compatible endpoint. Set role = "local-coder" to enable propose-and-confirm routing.
The index gives the SLM find_definition, find_references, code_map, and who_calls — it navigates directly to symbols instead of reading dozens of files.
Full docs: SLM Support Guide ↗ · Research: SLM Coding Eval ↗
Open source, MIT licensed. Zero telemetry. No cloud for the bulk of your edits.