Local LLM vs DeepSeek Cloud in a real Hermes agent job

We gave one read-only WordPress inventory job to three models through the same Hermes agent: Qwen 27B and Muse Glimmer 30B running locally on a Mac Studio, and DeepSeek V4 Flash in the cloud. All three returned the correct answer. The cloud model reached its first action in 4.4 seconds; the slower local model took 223.6. But the timing is the least interesting thing that happened.

By ikeo1 · Runs recorded 2 September 2026 · Three Hermes sessions, one run per configuration

A deliberately boring job

The task was the smallest useful thing an operations agent does: connect to a WordPress site, call one read-only action, and report what is installed. The site was clientsite.example. The action was fluent_inventory. The correct answer was 12 Fluent and Stifli plugins.

Boring is the point. There is no reasoning to admire and no code to review. Either the agent reaches the tool, reads the result faithfully, and says so in the required format, or it does not. That makes the differences between models unusually legible.

What happened

All three runs called the same read-only action and all three returned 12. On correctness of the underlying task, this was a three-way tie. Everything that follows is about how long it took and what came back around the answer.

The compliance column is not what it looks like

Every run was asked to open with a machine-parseable line naming the site and the count. This is the part of the job an automation actually depends on: a downstream script reads that first line and never sees the prose underneath.

Here is what each model returned, verbatim.

Qwen 27B · session 20260902_222931_b1ffb30b
SITE_AGENT_OK clientsute.example 12

The domain is misspelled — clientsute.example, one vowel wrong. The response went on to correct itself in prose, return a complete and accurate 12-row table, and document that its access was read-only. The prose was good. The first line, the only part a parser reads, was wrong.

Muse Glimmer 30B · session 20260902_234152_8eb663
SITE_AGENT_OK clientsite.example 12

plugin                  version  status
fluent-security         2.1.2    active
fluent-cart             1.6.3    active
fluent-community        2.8.1    active
fluent-community-pro    2.8.1    active
fluent-crm              3.1.13   active
fluentform              6.2.13   active
fluentformpro           6.2.13   active
fluent-messaging        2.7.5    active
fluent-player-pro       1.4.0    active
fluent-player           1.4.0    active
fluent-smtp             2.3.1    active
stifli-flex-mcp         3.4.8    active

site: clientsite.example
access: read

DeepSeek V4 Flash · session 20260902_224503_0de915ff
SITE_AGENT_READY clientsite.example 12

Read those three together and the obvious ranking dissolves. Muse produced an exact sentinel and a complete table — but its prompt was the only one that explicitly asked for a table. DeepSeek returned 38 characters, which looks like admirable concision until you notice its prompt never requested a table either. And DeepSeek’s sentinel token is SITE_AGENT_READY where the other two required SITE_AGENT_OK; our own record does not state which token DeepSeek was asked for.

So we can report first-line compliance for the two runs that shared a rule, and we cannot compare the third against them. That is an unsatisfying sentence to publish. It is also the honest one.

The typo matters more than the clock

A slow agent is an inconvenience you can schedule around. An agent that emits a confidently formatted, subtly wrong identifier is a different category of problem.

clientsute.example is not a parse error or a refusal. It is a well-formed domain that happens to be the wrong one, sitting in the exact position a downstream script trusts. Had a workflow branched on that line — writing a report, opening a ticket, selecting a site to act on — it would have branched on a site that does not exist, and the model’s own graceful correction two paragraphs later would have gone unread.

This is the finding we would actually act on. It says nothing about Qwen in general; a single run cannot. It says a great deal about where to put validation: parse the sentinel, assert the domain against the profile you dispatched, and fail loudly rather than trusting well-formed output.

Why local was slow here

Not because the network was slow. Round-trip time to the Mac Studio was about a millisecond. The cost is structural, and it is worth understanding because it applies to any agent workload on Apple silicon.

Hermes hands the model roughly 12.5 KB of instructions plus 37.3 KB of tool schemas — 29 tools — before the task is even described. Every discovery step and every tool result requires another full pass over that context. Qwen took five tool turns to Muse’s and DeepSeek’s three, so it paid that cost nearly twice as often.

Two further conditions were not controlled and plausibly contributed. The endpoint held several large models resident, so requests may have queued behind other work; during setup a trivial request once took 31 seconds on the same machine. And model residency was never deliberately managed, so an unknown share of Qwen’s 223-second first action may be load time rather than inference. We cannot separate those from the measurement, which is precisely why this is a field report.

Four conclusions this run does not support

The numbers above are easy to over-read. Four readings that would be wrong:

“Local is 28× slower than cloud.” One run per configuration, with residency and endpoint contention uncontrolled. This is a single observation, not a ratio you can carry to another machine.

“DeepSeek is the most concise.” It answered a question that did not ask for a table. Thirty-eight characters is the right length for its prompt and tells you nothing about how it summarises when asked to.

“Qwen is the most thorough.” Its table was unsolicited. Volunteering extra output is not the same as following instructions, and on the instruction it was given — the first line — it was the only model to get the domain wrong.

“Qwen used the most input tokens, so it is least efficient.” These counters accumulate harness and tool context across turns; they are not unique prompt size. Qwen’s higher count largely reflects taking five turns instead of three.

What a decision-grade test would need

We are publishing the weak version because the method is the useful part. To turn this into something you could make a purchasing decision on:

One byte-identical prompt for every model, with the same output contract — same sentinel token, same table requirement. One invocation platform; here Muse ran through a CLI one-shot while Qwen and DeepSeek were recorded through Mattermost. Deliberate cold and warm states, evicting and reloading models rather than letting residency fall where it may. An isolated endpoint with no other resident models and no concurrent load. At least three cold and three warm runs per model. And medians with variance reported, not single values.

What we changed on our own stack

DeepSeek is the default for interactive agent work where somebody is waiting. Ten seconds is a pause; five minutes is an abandoned task.

Muse Glimmer at roughly 98 seconds total sits in genuinely useful territory for scheduled and background jobs — inventory sweeps, nightly checks, anything where no human is blocked. Qwen at over five minutes, on this endpoint under these conditions, is not something we would put in an interactive loop.

And regardless of model, the sentinel line is now validated rather than trusted. That change came from the typo, not from the timings.

Sessions: Qwen 20260902_222931_b1ffb30b · DeepSeek 20260902_224503_0de915ff · Muse Glimmer 20260902_234152_8eb663. Timings read from persisted Hermes session-message timestamps. Evidence package and machine-readable run record held internally as HER-5; reusable procedure as HER-6.