"Tested locally, LGTM" is the most common lie in software, not because engineers are dishonest but because local testing is unfalsifiable after the fact. Six months later, when the terminal lifecycle bug comes back, nobody can point to what was actually verified. Influxx's answer was to stop trusting the sentence and start requiring the receipt: a machine-checked reliability manifest where every class of bug that has burned us has to be named, tied to a real test command, and backed by evidence that the test actually failed before it passed.
The Bug That Made the Case for a Manifest
The incident that crystallized this was mundane in the way that the worst bugs always are: new terminal and agent tabs in Influxx would open and then, a beat later, close themselves. No crash, no error dialog — just a tab flickering into existence and vanishing, as if the app had changed its mind. Because Influxx keeps a background daemon alive across app restarts to preserve PTYs, and because terminal creation is inherently asynchronous, the failure only showed up under specific timing — which made it exactly the kind of bug that "works on my machine" testing is structurally bad at catching.
The root cause was a dead-session reconciliation path. Influxx periodically reconciles its live terminal state against a snapshot of known sessions so it can clean up ones that have genuinely died — an agent process crashed, a PTY exited, a worktree got torn down. The reconciliation logic requested a session-list snapshot, then compared it against the current terminal set to decide what to close. The bug: if a new terminal was created after the snapshot was requested but before that snapshot resolved, the freshly-created process wouldn't appear in the snapshot at all. Reconciliation would look at that terminal, find no matching entry, and conclude — incorrectly — that it was dead. It would close a terminal that had never even finished being born.
This is a textbook async race, and the fix wasn't just a code patch. It was a missing invariant made explicit: a snapshot requested before a terminal binds cannot be used as grounds to close that same terminal. The concrete mechanism is an epoch, or generation counter, attached to each terminal at bind time. Reconciliation now checks a terminal's generation against the generation the snapshot was aware of; if the terminal's generation postdates the snapshot, reconciliation simply skips it instead of treating absence-from-snapshot as proof of death.
"The scary thing about that bug wasn't the fix — the fix was maybe twenty lines. The scary thing was that it had clearly been possible for a long time and nothing in our test suite would have caught it, because the failure only exists in the gap between 'snapshot requested' and 'terminal exists.' You can't unit test a timing gap with a single-threaded assertion. We needed the epoch guard as a structural invariant, not as a one-off fix, because otherwise the exact same shape of bug reappears the next time someone touches reconciliation."
— Priya Nandakumar, Senior Engineer, Terminal Runtime at ETAPX
Why "Tested Locally" Stopped Being Good Enough
Before the reliability manifest existed, the honest state of testing for terminal lifecycle, PTY sizing, and session-ownership code was scattered across PR descriptions, Slack threads, and the memory of whoever last touched that file. A PR would say "tested locally, tabs open and close correctly," and that claim would be true — for the specific sequence of clicks that engineer happened to try. It said nothing about the async race that only shows up when a new terminal is created mid-reconciliation, or the resize event that lands while a PTY handshake is still in flight.
The deeper problem is that "tested locally" is a claim about a moment, not a property of the code. It doesn't survive a refactor six weeks later by a different engineer who has no idea that reconciliation timing was ever fragile. Influxx's reliability-sensitive surface — terminal lifecycle, PTY sizing, session ownership, dead-session reconciliation — is exactly the part of the app where regressions are invisible until a user's tab silently closes mid-task. That combination of "hard to notice, expensive when it happens" is precisely what a text description in a pull request cannot protect against.
- PR descriptions decay: the claim "tested locally" has no expiration date and no way to be re-checked against the current code.
- Async races don't show up in casual manual testing: the terminal-tab bug required a specific ordering of snapshot-request versus terminal-creation that a person clicking around would rarely hit.
- Tribal knowledge doesn't transfer: the next engineer to touch reconciliation code has no signal that this exact class of bug already happened once.
What a Reliability Gate Actually Is
Influxx's fix was to build a registry — a reliability-gates manifest, kept as a file in the repo's config directory — where each gate is a named, machine-readable entry describing one specific invariant the team has decided must never silently break again. The generation-based reconciliation guard is one gate among dozens now registered in that manifest. Each entry declares a protection level: none, partial, or active, plus an intermediate experimental status a gate holds before it's trusted enough to be called active.
A gate entry isn't just a checkbox. It has to name the specific invariant it protects — not "terminal tests pass" but something as precise as "a session-list snapshot requested before a terminal binds cannot be used to close that terminal." It has to point to a real, runnable test command, not a description of manual steps. And, critically, it has to carry evidence that the test actually caught the bug it claims to guard against — a red-then-green pairing showing the test failed against the buggy code and passed once the fix landed. A gate that was written by watching passing tests after the fact doesn't count; the whole point is proving the test would have caught the original incident.
Protection Levels Are Not Just Labels
The three-tier system — none, partial, active — exists because not every invariant is equally well covered, and pretending otherwise is how confidence gets misplaced. A gate marked partial might have a real test but incomplete coverage of the failure modes; it's honest about that gap rather than hiding it behind a green checkmark. Only active gates are trusted enough to be treated as load-bearing for reliability claims, and getting there is deliberately hard, which is the subject of the next section.
The Checker That Guards the Guards
A manifest full of gate entries is only as trustworthy as the discipline enforced on the manifest itself, so Influxx built an automated checker script that runs structural validation over every entry before it's allowed to claim protection. Two rules matter most in practice. First, any gate that declares a backing test file has to point at a file that actually exists in the repo — a gate referencing a deleted or renamed test file fails the check immediately, rather than silently continuing to claim coverage for a test that no longer runs. Second, command-backed gates are forbidden from using flaky title-filter flags — the kind of test-runner flag that narrows a suite down to tests matching a specific title string. Those flags are attractive because they make a slow suite run faster, but they're exactly the mechanism by which coverage quietly erodes over time: someone renames a test, the title filter no longer matches it, and the gate keeps reporting green while silently running zero tests.
This is the part of the system that makes the manifest a gate rather than a to-do list. A registry that nobody validates degrades the same way PR descriptions do — entries get stale, test files get deleted in unrelated cleanups, and nobody notices until the next incident. By enforcing "the file you claim exists must exist" and "the command you claim runs coverage must actually run the coverage it claims" as CI-checked structural rules, the manifest can't drift out of sync with reality without the checker catching it.
Earning "Active": Why Promotion Is Slow On Purpose
New gates don't start life as trusted. They start as experimental, and Influxx's philosophy is explicit that promotion to active — and further, to a fully blocking status that can hold up a merge — requires real soak evidence, not a single green run. A gate only earns active status after demonstrating a stable CI flake history: consistent, repeatable pass results across real CI runs over time, not a test that happened to pass once in a demo.
The bar for the strongest tier is deliberately steep: gates are only promoted to blocking status after substantial numeric evidence, on the order of 100 consecutive soak runs, or roughly two weeks of continuous passing across the CI platforms Influxx requires — spanning the different operating systems and Git environments the app actually ships on. And there's an explicit carve-out for tests that can never meet that bar honestly: scenarios that are inherently stress-only, without a fully deterministic pass/fail signal, stay non-blocking forever. Influxx doesn't force a flaky-by-nature test to pretend to be deterministic just to satisfy a promotion checklist — that would just reintroduce the same trust problem the manifest was built to solve.
"Every engineering org says reliability matters, right up until it costs a merge. The reason we made the promotion criteria this strict — a hundred consecutive soak runs, two weeks across every CI platform we ship on — is that we wanted 'active' to mean something a new hire could trust without having to go ask around. If a gate is blocking, that has to be earned with evidence, not asserted because someone senior said it was fine."
— Sofia Reyes, VP of Engineering at ETAPX
The Numbers Behind Two Real Gates
Talking about invariants and manifests in the abstract undersells how concrete this system actually is in practice. Take the reconciliation fix described above. As seed evidence for two related gates — one covering terminal dead-session reconciliation, one covering session-resume and PTY-connection logic — the team ran a focused test suite that executed hundreds of individual tests and completed the entire run in roughly six seconds. That speed matters: a gate that takes minutes to run is a gate developers will start skipping locally, which quietly reintroduces the exact "tested locally, trust me" problem the manifest exists to eliminate. Fast, focused suites are what make it realistic to expect these gates to run on every relevant change rather than occasionally.
Not every reliability property can be proven with a fast unit-level suite, though — some invariants only hold up under real end-to-end conditions, so Influxx also maintains live tests that exercise the actual application. One such end-to-end test spawns a real terminal, types through a focused input field, hides and restores the entire workspace across two full cycles while asserting that the same terminal identity is preserved throughout, resizes the terminal mid-flow, and verifies that cleanup happens correctly afterward. That's not a mock or a simulated PTY — it's the real terminal lifecycle, exercised the way a user's actual session would move through hide/restore and resize events. One recorded run of that full end-to-end test, including build and setup time, finished in under a minute.
- Hundreds of tests, six seconds: the focused suite covering dead-session reconciliation, session-resume, and PTY-connection logic.
- Under a minute, including build: one full run of the live end-to-end test that spawns a real terminal, cycles it through two hide/restore passes, resizes it, and checks cleanup.
"I didn't know any of this manifest machinery existed until I asked why a fix I shipped got flagged for not including a gate update. My PR touched session-resume logic, and CI told me — correctly — that I'd have needed to either extend an existing gate's test or explain why the invariant didn't apply. It was annoying for about ten minutes and then I realized it had just stopped me from reintroducing a bug that apparently already happened once before I joined."
— Marcus Delaney, staff engineer at a healthcare analytics company, Influxx user
Why This Changes the Shape of Trust on the Codebase
The payoff isn't that Influxx never ships bugs — no manifest does that. The payoff is that specific, previously-burned classes of bug get a durable, checkable claim attached to them instead of a fading memory. When someone touches dead-session reconciliation code today, they aren't relying on the person who fixed the original tab-closing bug to still be around and remember why the epoch check exists. The invariant is written down, the test that proves it exists and is checked to actually exist, and the manifest's own structure is enforced so the whole thing can't quietly rot the way an out-of-date wiki page does.
It also changes what a reviewer is allowed to ask. Instead of "did you test this?" — a question answerable with a shrug — the question becomes "which gate covers this, and does the evidence for it still hold?" That's a question with a machine-checkable answer. For a developer evaluating whether to trust a desktop tool that keeps agent terminals alive across restarts and juggles PTYs across SSH, WSL, and native hosts, that's the difference between a vendor claiming reliability and a vendor being able to show the specific invariant, the specific test, and the specific soak history behind that claim.
Frequently Asked Questions
What exactly counts as a "reliability-sensitive" class of bug for this system?
Areas where a regression is both hard to notice during normal manual testing and expensive when it happens in production — terminal lifecycle, PTY sizing, session ownership, and dead-session reconciliation are the categories the manifest was built around, based directly on incidents like the tab-closing bug.
Does every gate block merges?
No. Gates start at none/experimental and move through partial to active as they accumulate real CI evidence; only gates that have earned blocking status through roughly 100 consecutive soak runs or about two weeks of stable CI history across required platforms are treated as merge-blocking.
What stops someone from writing a gate that just checks a trivial or narrowed-down test?
An automated checker script validates the manifest's structure directly: any gate referencing a test file requires that file to actually exist in the repo, and command-backed gates are barred from using flaky title-filter flags that could silently shrink real coverage over time without anyone noticing.
How was the epoch/generation guard for terminal reconciliation actually validated?
By requiring red-then-green evidence — the test had to fail against the buggy reconciliation logic and pass once the generation check was added — and by seeding two related gates with a focused suite covering reconciliation, session-resume, and PTY-connection logic that ran hundreds of tests in about six seconds.
Are these tests just fast unit tests, or do they cover real end-to-end behavior?
Both. Fast focused suites cover logic-level invariants in seconds, and separate live end-to-end tests spawn a real terminal, exercise hide/restore cycles and resizing, and verify cleanup — one full run, including build and setup, completed in under a minute.
What happens to a test that's inherently flaky or stress-only?
It's allowed to exist and even to be tracked in the manifest, but it stays non-blocking permanently unless it can be reworked into a fully deterministic pass/fail signal — Influxx deliberately doesn't force stress-only scenarios to masquerade as reliable gates.
Why not just trust code review and PR descriptions the way most teams do?
Because the tab-closing incident happened under exactly that model — a review process and manual testing that couldn't reasonably be expected to catch a timing gap between a snapshot request and a terminal's creation — and the manifest exists specifically to make that class of gap a checked, named invariant instead of tribal knowledge.
A test suite proves code worked once, under one set of conditions someone happened to think of; a reliability gate proves something narrower but far more durable — that a specific, named way of breaking has been caught red, fixed green, and is now structurally impossible to remove from coverage without someone noticing.
