Can a local model on a Mac handle the same agentic coding work as a cloud model? We gave five models one tightly bounded job and verified the result ourselves. Four cloud models finished in under 40 seconds. The local 27B model failed four times—even when its reading budget grew from 128,000 to one million tokens.
This comparison began with a practical question: when should a Mac owner keep work local, and when is a cloud model still the better tool? The answer cannot come from model size, a leaderboard, or a privacy slogan alone. It has to come from watching models attempt the same real job under the same rules.
The original test was run for a client-facing workshop. This MacLocalLLM version focuses on what the run says—and does not say—about local inference on Apple silicon.
A deliberately small job
Each model received a fresh copy of the same codebase and one narrow request: update a tool so it accepted two short aliases as well as its full name, write a test proving the aliases worked and an incorrect name was still rejected, run the test, and change nothing else.
This was not a trivia question. It required the model to inspect an existing project, find the relevant code, make a constrained edit, add a useful test, run it, and stop. A run passed only if the resulting files survived independent inspection. A model saying “done” did not count as completion.
Results at a glance
The final round gave every model the same one-million-token reading allowance. Four models completed the task. The local model ran for more than three minutes and stopped without a finished test.
| Model | Where it ran | Result | Time | Work produced |
|---|---|---|---|---|
| DeepSeek V4 Flash | Cloud | Passed | 7.8s | Clean, 23 lines |
| Kimi K3 | Cloud | Passed | 17.9s | Clean, 23 lines |
| GLM 5.3 Flash | Cloud | Passed | 16.7s | Correct, 31 lines; one redundant edit |
| Claude Fable 5 | Cloud | Passed | 37.9s | Clean, 23 lines |
| Qwen 3.8 27B | Local Mac Studio | Failed | 210.4s | Unfinished, 4 lines |
One run per cloud model · One-million-token final budget · 31 August 2026
Every passing model changed the two intended files, wrote the requested test, and passed all 13 checks when we ran them independently. For this task, the cloud-model differences were speed and editing discipline—not completion.
Time to finish the same job Seconds · shorter is better · the local run did not finish
The local run ended when the safety guard stopped it, not when the job was complete.
Focus mattered more than raw speed
Elapsed time is easy to understand, but total input processed tells us more about how efficiently an agent navigated the task. Three passing models stopped near 100,000 input tokens. Claude processed about twice that amount. The local Qwen run processed 351,602 tokens and still did not produce a usable result.
Total text processed Input tokens · lower is better for this fixed task
Token counts describe this harness and task. They are not a general intelligence score, and Claude’s own tooling accounts for text differently.
The local model got four chances
The local result deserved more scrutiny than a single failed attempt. Qwen 3.8 27B was running through MLX on a Mac Studio, which made it the only model in the group running on one local desktop rather than datacenter infrastructure. Its first failure looked like a reading-budget problem, so we raised the allowance repeatedly.
| Reading budget | Outcome | Actually read | Time | Left behind |
|---|---|---|---|---|
| 128,000 | Failed budget | 129,654 | 150.3s | Nothing |
| 256,000 | Failed budget | 284,841 | 184.0s | Valid work, but outside the limit |
| 384,000 | Failed steps | 352,621 | 225.9s | Unfinished; test missing |
| 1,000,000 | Failed steps | 351,602 | 210.4s | Unfinished; test missing |
Qwen 3.8 27B via MLX · Four budgets · Four failed attempts
More room did not cure the behavior. Once the reading budget stopped ending the run, the step guard did. The model reopened files it had already inspected, revisited edits, and eventually failed to leave the complete change and test requested.
What this result changes for a Mac owner
A zero-dollar local attempt is not automatically cheaper than a paid completion. Hardware, electricity, setup time, review, retries, and failed work all belong in the cost. In this test, Claude was the only run with a directly reported price: US$0.56 to US$0.64. The local model had no per-call bill, but it also had no completed result.
Private knowledge work
Drafting, summarization, extraction, classification, and question answering when the material should stay on the Mac and a person will review the answer.
Bounded tool use
Coding and multi-step workflows that must inspect files, make changes, verify the result, and stop reliably. Keep permissions and scope narrow.
Review what matters
Use the strongest suitable model—or a human reviewer—when ambiguity, risk, or the cost of a bad result is higher than the cost of another pass.
What this test does not prove
This is a field test, not a universal ranking. It covers one coding task and one attempt per cloud model. The local model received four attempts because the changing budget was itself the question. A different prompt, harness, runtime, model build, quantization, or task could produce a different result.
Two comparisons also need qualification. Claude ran through its own tooling, which counts and reuses text differently from the shared harness, so its token total is not perfectly interchangeable. And Qwen was the only model running on a single Mac Studio instead of datacenter hardware. That hardware mismatch is intentional—we wanted to test the practical local-versus-cloud choice—but it must remain visible.
How the run was controlled
- Every attempt began from the same commit in a clean copy of the repository.
- Every model received the same request and the same two-file boundary.
- Runs were sequential, so models did not compete for local resources.
- The final comparison gave every model a one-million-token reading allowance.
- Changed files, tests, repository state, and whitespace were checked independently after every run.
The bottom line
The local model did not fail because it lacked a giant context allowance. It failed because it could not manage a small agentic loop reliably enough to finish. That is a workflow limitation, not a verdict on all local models or all local tasks.
For Mac users, the practical answer is to route work by consequence. Keep sensitive, reviewable knowledge work local when the model is good enough. Use a more dependable agent for bounded tool use. Measure completion, not just tokens per second—and never confuse a model that can answer with an agent that can finish.
Source: an earlier, workshop-oriented write-up of the same run, prepared for a client audience. The run data and settings are reproduced above in full.