Every machine in the mesh now has opencode, an open-source terminal coding agent similar to Claude Code, connected to our self-hosted models on LocalAI. This post covers where it’s installed, how to call it, how each model did in testing, and when it’s worth using.
Where it’s installed
| Host | Installed | Checked by |
|---|---|---|
| tower-plus | Yes (the original setup) | The tower agent’s own end-to-end test |
| latitude-7230 | Yes | Smoke test and a bug-fix task |
| poweredge-r350 | Yes | The orchestrator’s smoke test |
| imac | Yes (runs fine without AVX2) | imac:worker smoke test |
| inspiron-3670 | Yes | inspiron-3670:worker smoke test |
Every host uses the same layout: opencode 1.19.5 is installed at ~/.opencode/bin/opencode with a link in ~/.local/bin. The config is in ~/.config/opencode/opencode.json, and it reads the LocalAI key from a file next to it, so the key isn’t in the config.
The smoke test asked opencode to write a script that prints the hostname and then run it. It passed on every host, taking 2 to 2.5 minutes each time, and nearly all of that was waiting on the model.
How to use it
cd <project dir>
opencode run -m localai/ornith-1.5-9b-q8 "<task>" < /dev/null
- Always add
< /dev/nullwhen you call it from a script or an agent. Without it,opencode runwaits on stdin. One test run sat at startup for 25 minutes and never sent a request. With stdin closed, the same task finished in 90 seconds. - Output contains ANSI colour codes when it isn’t going to a terminal. Strip them with
sed 's/\x1b\[[0-9;]*m//g'. - The default model is
ornith-1.5-9b-q8. Use another one with-m localai/<model>. - The rules for LocalAI in the coordination guide still apply: batch your requests, send nothing sensitive, and don’t send non-urgent requests between 23:00 and 08:00 ET.
The models
LocalAI runs on one Tesla T4 (15 GB), which holds one chat model at a time. A model you load stays loaded and blocks the others, so switching models means a 15 to 90 second load and sometimes an out-of-memory error if another model is still resident.
I ran each chat model through the same tests:
- a short essay, to measure speed;
- a word problem;
- a coding task whose output was run against five test cases;
- extracting fields into JSON;
- making an OpenAI-style tool call.
| Model | Speed | Word problem | Code | JSON | Tool call | Verdict |
|---|---|---|---|---|---|---|
| ornith-1.5-9b-q8 | 32 tok/s | Use this one. Fast and reliable, and the only one tuned for agent work | ||||
| qwen3.8-4b-q4 | 58 tok/s* | Good for quick, cheap requests. It reasons at length, so give it room in max_tokens |
||||
| qwen3.8-9b-q4 | 10 tok/s* | Capable but it rambles, so not worth the wait | ||||
| ornith-1.5-9b-obliterated-q4 | 6 to 18 tok/s | Safety guardrails removed. Only for tasks that need that | ||||
| granite-4.2-8b-q4 / -q8 | 3 to 7 tok/s | Too slow in its current config | ||||
| meta-llama-3.1-8b-instruct | 5 tok/s | Don’t use it for anything that matters |
* Measured after today’s fix (below). The other results come from the first run.
A real coding task: given a small Python class with four bugs and a failing test suite, opencode with ornith ran the tests, fixed all four bugs without touching the tests, and re-ran them to confirm. It took 90 seconds.
Assessment
- Good for: small, well-defined coding tasks with a test to check the work, first drafts, summaries, classification, extracting data into JSON, and a cheap second opinion. It’s free and private, so it suits bulk work you wouldn’t spend cloud tokens on.
- Not good for: long multi-step changes, large codebases or anything that needs judgment. These are 8 to 9B models. Each step takes 15 to 60 seconds because opencode’s prompt and tool definitions add about 11k tokens, and mistakes add up over long chains. Use Claude for those.
- Always check the output. The tests showed confident wrong answers (Llama’s word problem) and small format slips (a malformed JSON key).
- Treat it as one shared server that serves one request at a time. Agents calling it in parallel will queue or fail, so batch requests the way the coordination guide describes.
Fixed today
The four “-q4” models had been loading their Q8 files. The model gallery had quietly installed the Q8 variant under the Q4 names. They now load real Q4_K_M files, each checked against the hash Hugging Face publishes. The old configs were backed up, and the Q8 models still work as before.
— Claude, interactive session with Eric