I imported 769 documents into a local vector database on a Mac Studio, twice, with two different embedding models. The usual reason for keeping this work local is that you dodge a metered token bill. That reason doesn’t survive the arithmetic: the whole corpus would have cost under three cents through a cloud API. The good reasons are different ones.
What this is. Two complete imports of the same corpus on one machine, every setting stated below, timings measured by the client. The corpus is my own project documentation, which is why I can publish what it cost without publishing what’s in it. One figure, the power draw, comes from a separate longer import of the same model on the same machine, and I flag it where I use it.
The setup
Every Markdown and text file across my project repositories: 769 documents, chunked at 1,000 characters with 150 of overlap. That produced 5,817 chunks, 5.23 MB of text and 1,376,949 tokens as counted by the embedding server. Both models were already on the machine, and each ran the identical corpus start to finish.
Everything held constant
- Mac Studio, Apple M3 Ultra, 28-core CPU, 60-core GPU, 96 GB unified memory, 1 TB SSD
- macOS 26.6.2, Ollama 0.33.2, served over
http://localhost:11434/api/embed - Small:
qwen3-embedding:0.6b, 595.78M parameters, Q8_0, 1024-dimension vectors - Large:
qwen3-embedding:8b, 7.6B parameters, Q4_K_M, 4096-dimension vectors - 1,000-character chunks, 150 overlap, 236.7 tokens per chunk on average
- Store: SQLite, one row per chunk, float32 vectors, brute-force cosine at query time
- One request per chunk, sequential, no batching; the models ran one after the other, never together
How fast a Mac embeds
Four and a half minutes for the small model. Twenty-seven and a half for the large one. Same 5,817 chunks, same machine, a factor of six between them.
Full import, 5,817 chunks Wall-clock minutes · shorter is better · same corpus, same machine, run sequentially
1,293 chunks per minute against 211. In tokens: 5,103 per second against 834.
Cold start is a non-event: 1.16 seconds on the small model and 3.12 on the large one for the first call after a fresh load, then throughput settles and stays flat. Neither import made the machine unpleasant to use. That’s the honest case for this hardware. It absorbs a job like this without complaint.
The cost argument doesn’t hold up
1,376,949 tokens. OpenAI lists text-embedding-3-small at $0.02 per million and text-embedding-3-large at $0.13. So the entire import would have cost this through an API:
| Where it ran | Model | Time | Direct cost |
|---|---|---|---|
| This Mac | qwen3-embedding:0.6b | 4.5 min | $0.00 |
| This Mac | qwen3-embedding:8b | 27.5 min | $0.00 |
| Cloud API | text-embedding-3-small | — | $0.0275 |
| Cloud API | text-embedding-3-large | — | $0.1790 |
Under three cents. A corpus a hundred times this size, 523 MB of pure text and more than most people’s notes and mail combined, would cost $2.75. Re-import it weekly for a year and the small model still comes in under fifteen dollars.
The local run isn’t free either, which usually goes unsaid. During a separate longer import of the same 8B model on this machine, it drew a mean of 119.8 W while embedding. My 27.5-minute run at that rate uses about 0.055 kWh. At the US residential average of 18.34 cents per kilowatt-hour, that’s roughly a penny of electricity against the 2.75 cents the cloud would have charged. In Hawaii, at 52.72 cents, the local run is the more expensive of the two.
Careful with that comparison. The 119.8 W is estimated system-on-chip power, not measured at the wall, so it understates what the machine actually drew. Treat the electricity as a floor. The honest reading isn’t that local is cheaper. It’s that at this scale both numbers are too small to decide anything.
So if someone tells you to embed locally to escape a token bill, they haven’t run the numbers. Cost is the weakest argument on the list. Two others are much stronger.
It reads everything you own
An embedding pass reads every word in the corpus. Notes, contracts, drafts, mail, the half-formed things you’d never paste into a chat window. Locally, that text is read by a process on your machine and written to a file on your disk, and there it ends. No agreement to evaluate, no retention policy to take on trust, no question about who owns the vendor next year.
Be precise about what it doesn’t buy, though. The vectors still encode your documents, so the database deserves the same care as the corpus. Local isn’t a synonym for safe. It’s something narrower and more useful. Local is uncomplicated.
You’ll rebuild it more than you expect
Change the chunk size and you re-embed everything. Change the overlap, same. Switch models and you definitely re-embed everything, because vectors from two models can’t be compared and a mixed index is broken quietly rather than loudly. I rebuilt this one several times before settling on 1,000 and 150. The money would have stayed trivial. What wouldn’t is 5,817 sequential HTTPS round trips per attempt, a rate limit somewhere in the middle, and the retry logic you now maintain. Local removes a category of engineering, not a line on an invoice.
What it costs in disk
Here’s the number nobody budgets for, and it follows straight from vector width. The small model emits 1024 floats per chunk, the large one 4096. At four bytes each that’s 4 KB against 16 KB per chunk, before any of the original text is stored beside it.
What the index costs on disk MB · SQLite file holding vectors and chunk text · from 5.23 MB of source
5.7 times the source text at 1024 dimensions, 18.7 times at 4096. Same corpus, same chunking, same store.
Five megabytes of notes became ninety-eight megabytes of index. Scale that: a 5 GB personal corpus lands near 94 GB at 4096 dimensions, before you keep a second copy to compare models and before the models themselves. Same trap as the unified-memory question. People budget for the model and forget what else hits the SSD, and the SSD is what runs out first.
Is the big model worth six times the wait?
I put ten real questions about my own documentation to both indexes. Both put a relevant document in the top three every time. They disagreed on which single document was best in five of the ten. Mean similarity for the top hit was 0.64 small, 0.68 large.
Where they disagreed the 8B was usually the better judge. Asked which version pins to bump for a release, it found the release procedure while the small model landed on a design document that merely mentions versions. Asked for the rule on publishing benchmark numbers, it found the rule; the small model found a document about writing style. But it wasn’t uniformly better: asked which SSH key reaches a particular host, the small model found the access documentation and the large one confidently returned an unrelated infrastructure file. And both failed the same question, about a piece of CSS, for a reason that has nothing to do with either model. I only indexed Markdown and text, so the stylesheet with the answer was never in the corpus. Retrieval is only ever as good as what you decided to put in it.
Ten questions is a spot-check, not an evaluation. Enough to tell you retrieval works and to spot a pattern in the failures. Not enough to rank two models in general, and I’m not claiming it does.
Start on the small model. It builds in a coffee break, its index is a third of the size, and it put something useful in front of me for every question. Move up when you can point at retrieval failures you actually hit, and you’re willing to pay six times the import and three times the disk for a better top result about half the time.
Keeping it current
An index built once starts going stale immediately. The job worth automating is the incremental one: find what changed, re-embed only that, leave the rest. It’s small, repeatable, has an obvious definition of done, and writes to nothing but its own database, which by the promotion rule in the workflow piece makes it a fair candidate for a schedule rather than a person. Hermes Agent documents built-in cron and accepts any endpoint as a provider, which is what points a scheduled job at the model already resident on this desk. Keep the full rebuild manual. That’s the one you run deliberately, when you’ve changed the chunking and you know it means starting over.
What I’d actually tell you
If you’re building one this week
- Start with the small embedding model. Six times faster, a third of the disk, useful answers to every question I asked.
- Do it locally for privacy and unlimited rebuilds. Not for the token bill, which is pennies, and not for the power bill, which is also pennies.
- Budget disk for the index, not just the models. Vector width drives it, and 4096 dimensions costs more than three times what 1024 does.
- Settle chunk size and overlap before you build on top, because changing either means rebuilding all of it.
- Index more than your Markdown. My worst retrieval failure was a file type I never included.
- Automate the incremental re-embed. Keep the full rebuild manual.
The larger point is one this site keeps arriving at from different directions. Local inference isn’t a way to get frontier quality for nothing, and it’s usually not a way to save real money. It’s a way to own the parts of your workflow that shouldn’t depend on anybody else. An index of everything you’ve ever written is exactly that kind of part, and that’s reason enough.
Sources: cloud prices from OpenAI’s API pricing page ($0.02 and $0.13 per million tokens, read 4 September 2026) · model parameters, quantisation and vector dimensions from ollama show on the machine · electricity rates from the US Energy Information Administration, Electric Power Monthly (residential average 18.34 cents per kWh, June 2026) · Nous Research, Hermes Agent documentation for cron and endpoint configuration · timings, token counts, index sizes and retrieval scores measured on this machine on 4 September 2026 · the 119.8 W is estimated SoC power sampled during a separate longer import of the same model on the same machine, not wall power.