The Erasure That Kept the Memory

The verdict engine behind our outreach tooling keeps its entire rule corpus in RAM: about 4.9 million clauses, compiled from a roughly 850MB facts file that a sync job rewrites on a twenty-minute cadence. Facts change; the engine can't go down; so the store is hot-swappable — each reload streams the file into a fresh generation, flips a pointer, and marks the old generation erased. Readers always see one complete generation or the other, never a torn store.

It's a clean design. It also climbed to twenty gigabytes of swap in nineteen days.

The API says freed. The kernel says mine.

The first suspect was the swap logic itself, so we audited it: every clause class in the stream is registered, old generations are abolished on commit, error paths clean up partial loads. All correct — and all beside the point, because the leak wasn't in our code's bookkeeping. It was in what "freed" means.

The runtime's abolish does erase the clauses — logically, they stop existing; queries can't reach them. But the memory they occupied goes back to the runtime's own freelists, not to the operating system. And freed clause memory turns out to be barely reusable across different clause shapes: we measured a fresh generation allocating almost entirely new segments — about +925MB of RSS per reload cycle, every twenty minutes, forever.

We verified it with a two-minute experiment instead of trusting the docs: build a 2M-clause generation, abolish it, force every garbage collector the runtime offers, measure RSS. Flat. Build another. +250MB more. The collectors weren't broken — they did exactly what they promise, which is reclaiming unreachable terms for the runtime's own reuse. Returning pages to the kernel is simply not in their contract.

Reclaiming what can be reclaimed

The fix is honest about the boundary. In-process, we now do the parts that actually work: force the collectors immediately after each generation retires (so whatever is reclaimable gets reclaimed now, not eventually), keep the health endpoint from materializing a five-million-element list per call, trim pooled worker stacks on a timer, and raise the atom-GC margin during loads so the stream doesn't pay for hundreds of full-table scans — that last one took the load itself from minutes to about fifty-five seconds.

But the true release happens one layer down. A small timer checks the process's resident set; when it crosses a threshold, the service restarts — the only mechanism that actually hands the pages back. Boot takes under a minute and reloads the corpus from disk, so the recycle is self-healing. A hard memory cap and an unconditional restart policy sit underneath as the last backstop, turning what used to be an unclean OOM into a scheduled nap.

The free bug

The audit also surfaced a latent correctness bug with the same root: one fact class was being streamed into every generation — written, stored, never reachable by any query, and never erased on retire. It couldn't be read or freed. Wiring it through the same registration the rest of the corpus uses fixed both halves at once: it's now reachable when queried and collectable when retired.

The discipline that found it

Nothing about this was visible from inside the abstraction. The logs said every commit succeeded; the API said every generation was erased; the pointer flipped exactly as designed. The only witness was VmRSS in /proc — and the honest number lived there the whole time.

That's the recurring lesson in this codebase: when a system tells you it freed something, believe it the way you'd believe a test that can't fail. Ask the layer underneath.