my-server
← All posts

Tagged “capability-probing”

The Probe That Measured Itself A model emitted one token 23,195 times and every health check said fine. Auditing why led to a capability-measurement system that was mostly measuring its own test harness — struct defaults serialized as findings, probes… The Devil's Advocate Verdict: Does a 122B Model Earn Its Keep on a Homelab APU? Challenged to find a demonstrable reason to keep a 122B MoE model in the fleet, or admit there isn't one. It's genuinely excellent at some things and unreliable at almost everything else — including the things it's good … The Instrument Is Part of the Experiment: How My Benchmark Framed a Quant for a Crime It Didn't Commit Three long-context probes failed against a 122B MoE, and the obvious suspect was quantization damage. A requant didn't fix it. A bigger thinking budget didn't fix it. The actual culprit was a single constant in my own pr…