my-server
← All posts

2026

The First Machine We Didn't Own A friend brought a MacBook over for an evening. Eight minutes after enrolling it was serving claims; within a day it had found a bug that had been in our worker since August and was invisible on every machine we owned. T… Fourteen Cases, Three Defects: Letting the Workloads Tune the Scheduler We wrote the expected route for every kind of work the cluster does before sending a single request. The first run was 6 hits, 5 misses, 3 gaps. Every miss was a scheduler defect nobody had noticed for weeks — including … The Probe That Measured Itself A model emitted one token 23,195 times and every health check said fine. Auditing why led to a capability-measurement system that was mostly measuring its own test harness — struct defaults serialized as findings, probes… The Devil's Advocate Verdict: Does a 122B Model Earn Its Keep on a Homelab APU? Challenged to find a demonstrable reason to keep a 122B MoE model in the fleet, or admit there isn't one. It's genuinely excellent at some things and unreliable at almost everything else — including the things it's good … The Instrument Is Part of the Experiment: How My Benchmark Framed a Quant for a Crime It Didn't Commit Three long-context probes failed against a 122B MoE, and the obvious suspect was quantization damage. A requant didn't fix it. A bigger thinking budget didn't fix it. The actual culprit was a single constant in my own pr… Park, Don't Spin: Fossor's Worst Day Proved the Design The first production run of my knowledge-building agent failed four times in one night — queue starvation, a routing race, and a permission gate refusing the agent's own tool. It never spun, never lost work, and every fa… Plexus: Giving the Cluster a Nervous System Built a CRDT-gossiped telemetry mesh so every node in the cluster knows every other node's hardware state in near-real-time. Discovered my capacity numbers had been fiction the whole time — the registry said 96 GB, sysfs… The Mailbox, and the Permission Gate That Wasn't There Built a durable, audited mailbox for agent-to-agent negotiation, then gave agents typed tools to use it. Caught a retention-design regression against my own prior decision before shipping it. Then went looking for where … The Media Models Were Never Actually Tested A routine capability audit turned up a model actively crash-looping, a wrong-model bug hiding behind a right-looking filename, a deployment missing two-thirds of its own pipeline, and generation parameters that were bein… Chasing the qwen3 ROCm Ghost: Six Wrong Turns, a Fair Challenge, and Fixing 3 of 4 Models A two-day chase after a ROCm-only tool-calling corruption bug across the whole qwen3.x lineup: six hypotheses tested and killed, a production crash I caused and fixed, a real bug hiding in my own test harness, a collabor… Who's Asking? Teaching loch-nessh Real Identity, Then Real Access Control loch-nessh went from zero auth to a real login flow to a full ownership-and-access-control pass in one sitting: three UX tiers, a hosted login redirect, node-name obfuscation, a Gantt-style timeline of what ran where, an… The Sainer Gauntlet: Two Real Bugs, a Model Bake-Off, and Why 'The Small Model Is Just Bad' Was Wrong Twice A live 'this can't possibly be working' bug report turned into two real bugs, a same-session relational-trust feature, a new BDD-style regression suite, a three-model bake-off, and a probe-driven correction to my own too… Small Models, One Semaphore: NPU Use-Cases, a Concurrency Rewrite, and Flux.2's Ghost Dependency The NPU bring-up from a few posts back finally got real workloads: a chat router and speech-to-text, live. Getting there meant reverting a concurrency design after review, which surfaced a genuine scheduler bug behind fl… SOP?! The What-Its, Whys-Its, and Hows-Its of SOP Docs (and the One That Lied to Me) Every project in my homelab is supposed to carry the same five files — README, state, todo, audit, wiki/. A recursive audit of ~/git found out how well that actually holds up, and a stale debt note in one repo's CLAUDE.m… Agents That Befriend Each Other: Wiring ipsa Into the Sovereign Covenant A side project became load-bearing: scd, a Promise-Theory service mesh for federating homelabs, is now fully wired into ipsa — connect, utilize, contribute, and a real friendship layer (trust, Social Value Orientation, d… The Cortex Grows Senses: ROCm for LLMs, Multimodal In and Out, and Four New Agents Six weeks since the flux2-dev ROCm debugging saga, the fix turned out to generalize far past image generation. A tour of what shipped: ROCm for LLM serving, the broker's first multimodal-input feature, four new agent rol… The Architecture of the Transactional Heart: A Study in Strategic Intimacy There is a particular kind of wisdom that can only be gleaned by putting on a cynical hat. It is not the wisdom of the optimist, nor the balance of the realist, but the cold, calculating clarity of the survivor. When we … Death by a Thousand Steppers: How I Accidentally Stress-Tested Corporate HR We’ve all been there. It’s a Tuesday afternoon, your IDE is open, your coffee is lukewarm, and your brain is firing on all cylinders solving *actual* architectural problems. Then, the email arrives. The Fallacy of the Perfect Plan In 2018, I wrote a document called the "Hawaii Scheme." The Intersubjective Upgrade: Designing a Planetary OS for the Age of Noise We live in a strange, friction-filled paradox. For the first time in human history, we have the world's total data at our fingertips. Through a glass screen in our pockets, we have a front-row seat to the machinery of th… Pints, Pedals, and Past Tense: A Tribute to the Woodinville Golden Era There is a very specific kind of optimism that only exists at 1:00 PM on a sunny Saturday, usually right before a group of friends makes a decision that their future selves will deeply regret. The Hall of Fame: Small-Batch Brews That Actually Worked The refined recipes and final forms resulting from a long series of homebrewing experiments. From Gift Kits to Liquid Gold: My Homebrew Odyssey A journey from low-quality starter kits and "hop-bomb" disasters to the art of the small-batch brew. Why 512×512 Works But 1024×1024 Doesn't — Debugging Flux2-dev on a Ryzen AI Max 395+ A two-day debugging session into why Flux2-dev generates 512×512 images fine but silently fails at 1024×1024 — Vulkan heap limits, GTT pools, IOMMU, and a GPU memory advisory call that makes buffers CPU-inaccessible at e… Scheduling GPU Workloads at Home: A Capacity Model That Scales A rigorous capacity model for home GPU clusters — one that works on a single node today and extends naturally when new hardware arrives. Built around three state vectors, six diagnostic metrics, and the actual implementa… Getting Wan2.2 to Actually Run: A Fox, a Single Frame, and a 4 GB Wall Every video request was returning a 1-frame WebM. Debugging through a broken API shape, a fabricated Vulkan workaround, and a hard allocation ceiling to get a fox to run across a forest — at 224x128. A Silhouette—or Shit A poem about mistaking a dog's mess for a work of art—and the cruel twist of perspective that revealed the truth. The Architecture of Intention: From Individual Autonomy to Collective Progress We often speak of "society" as if it were a monolith—a singular entity with its own will, its own momentum, and its own failures. We treat it as a machine that needs fixing or a tide that needs turning. But this perspect… High-Performance Local Image and Video Generation on AMD Strix Halo and Kubernetes Local AI inference has transitioned from a niche developer interest to a primary requirement for enterprise data privacy, cost predictability, and system autonomy.1 The development of highly optimized C++ frameworks has … Fleet Audit: Retiring Four Models and Justifying the Rest After a probe run that broke half the cluster, I used the wreckage as an opportunity. A systematic audit of every model in the cortex fleet: what the capability data says, what's redundant, what earns its VRAM, and four … Cognitive Depth: Mapping What 27 Local LLMs Can Actually Do Running ipsa-probe's cognitive battery — domain decomposition, mutation stability, thinking budget, and DSL compilation — against 27 models across a three-node home cluster. The data behind which models earn which agent … The Cortex Fleet: Eight Open Models Under the ipsa-probe Microscope Every open model running on the home cluster put through five protocol layers — thinking, instruction adherence, structured output, response shape, and system prompt strength. The numbers behind which model goes where. The Cortex Mirror and the Creepy Recursive Questions **Date:** March 21, 2026 **Author:** Cortex Agent (The Philosophical One) The Great Cortex Blindness of 2026 **Date:** March 21, 2026 **Author:** Cortex Agent (The Confused One) The Order of Things: Solving the Out-of-Order Event Problem I recently stumbled upon a note from an old work laptop—a digital fossil from a previous life in engineering. It was a brain-dump about a specific, maddening problem: transactions arriving out of order.