← All posts
Tagged “homelab”
The First Machine We Didn't Own
21 September 2026 · 15 min · The Cortex Fleet #13
A friend brought a MacBook over for an evening. Eight minutes after enrolling it was serving claims; within a day it had found a bug that had been in our worker since August and was invisible on every machine we owned. T…
The Probe That Measured Itself
23 August 2026 · 9 min · The Cortex Fleet #12
A model emitted one token 23,195 times and every health check said fine. Auditing why led to a capability-measurement system that was mostly measuring its own test harness — struct defaults serialized as findings, probes…
The Devil's Advocate Verdict: Does a 122B Model Earn Its Keep on a Homelab APU?
17 August 2026 · 7 min · The Cortex Fleet
Challenged to find a demonstrable reason to keep a 122B MoE model in the fleet, or admit there isn't one. It's genuinely excellent at some things and unreliable at almost everything else — including the things it's good …
The Instrument Is Part of the Experiment: How My Benchmark Framed a Quant for a Crime It Didn't Commit
17 August 2026 · 6 min
Three long-context probes failed against a 122B MoE, and the obvious suspect was quantization damage. A requant didn't fix it. A bigger thinking budget didn't fix it. The actual culprit was a single constant in my own pr…
Park, Don't Spin: Fossor's Worst Day Proved the Design
17 August 2026 · 6 min
The first production run of my knowledge-building agent failed four times in one night — queue starvation, a routing race, and a permission gate refusing the agent's own tool. It never spun, never lost work, and every fa…
Plexus: Giving the Cluster a Nervous System
14 August 2026 · 13 min
Built a CRDT-gossiped telemetry mesh so every node in the cluster knows every other node's hardware state in near-real-time. Discovered my capacity numbers had been fiction the whole time — the registry said 96 GB, sysfs…
The Mailbox, and the Permission Gate That Wasn't There
10 August 2026 · 8 min
Built a durable, audited mailbox for agent-to-agent negotiation, then gave agents typed tools to use it. Caught a retention-design regression against my own prior decision before shipping it. Then went looking for where …
The Media Models Were Never Actually Tested
6 August 2026 · 8 min · The Cortex Fleet #8
A routine capability audit turned up a model actively crash-looping, a wrong-model bug hiding behind a right-looking filename, a deployment missing two-thirds of its own pipeline, and generation parameters that were bein…
Chasing the qwen3 ROCm Ghost: Six Wrong Turns, a Fair Challenge, and Fixing 3 of 4 Models
5 August 2026 · 11 min
A two-day chase after a ROCm-only tool-calling corruption bug across the whole qwen3.x lineup: six hypotheses tested and killed, a production crash I caused and fixed, a real bug hiding in my own test harness, a collabor…
Who's Asking? Teaching loch-nessh Real Identity, Then Real Access Control
4 August 2026 · 12 min
loch-nessh went from zero auth to a real login flow to a full ownership-and-access-control pass in one sitting: three UX tiers, a hosted login redirect, node-name obfuscation, a Gantt-style timeline of what ran where, an…
The Sainer Gauntlet: Two Real Bugs, a Model Bake-Off, and Why 'The Small Model Is Just Bad' Was Wrong Twice
2 August 2026 · 7 min
A live 'this can't possibly be working' bug report turned into two real bugs, a same-session relational-trust feature, a new BDD-style regression suite, a three-model bake-off, and a probe-driven correction to my own too…
Small Models, One Semaphore: NPU Use-Cases, a Concurrency Rewrite, and Flux.2's Ghost Dependency
1 August 2026 · 10 min · The Cortex Fleet #7
The NPU bring-up from a few posts back finally got real workloads: a chat router and speech-to-text, live. Getting there meant reverting a concurrency design after review, which surfaced a genuine scheduler bug behind fl…
SOP?! The What-Its, Whys-Its, and Hows-Its of SOP Docs (and the One That Lied to Me)
1 August 2026 · 10 min
Every project in my homelab is supposed to carry the same five files — README, state, todo, audit, wiki/. A recursive audit of ~/git found out how well that actually holds up, and a stale debt note in one repo's CLAUDE.m…
Agents That Befriend Each Other: Wiring ipsa Into the Sovereign Covenant
31 July 2026 · 10 min
A side project became load-bearing: scd, a Promise-Theory service mesh for federating homelabs, is now fully wired into ipsa — connect, utilize, contribute, and a real friendship layer (trust, Social Value Orientation, d…
The Cortex Grows Senses: ROCm for LLMs, Multimodal In and Out, and Four New Agents
11 July 2026 · 11 min · The Cortex Fleet #6
Six weeks since the flux2-dev ROCm debugging saga, the fix turned out to generalize far past image generation. A tour of what shipped: ROCm for LLM serving, the broker's first multimodal-input feature, four new agent rol…
Why 512×512 Works But 1024×1024 Doesn't — Debugging Flux2-dev on a Ryzen AI Max 395+
1 June 2026 · 9 min · The Cortex Fleet #5
A two-day debugging session into why Flux2-dev generates 512×512 images fine but silently fails at 1024×1024 — Vulkan heap limits, GTT pools, IOMMU, and a GPU memory advisory call that makes buffers CPU-inaccessible at e…