We taught our own infrastructure to watch itself, and fix nothing

We run a lot of small, moving infrastructure across the ecosystem — deploys, environment variables, dependency versions, security scans, secrets with expiry dates, and a growing list of regulatory deadlines that apply to us as a company, not just to our products. Nobody enjoys checking all of that by hand, and nobody reliably does. So we built a loop that checks it for us. This week we more than doubled what it watches — and drew the line, again, at watching.

From three signals to eight

The loop started smaller: a monitor-diagnose-escalate cycle over three sources — status-board reconciliation, dependency peer-drift across our shared packages, and cron-freshness (crons that silently stop firing without anyone noticing). This wave we added five more detectors: stale deploys, drift between Preview and Production environment variables, waves of dependencies going stale together, verified high-and-above security findings above a confidence gate, and secret expiry. Eight sources feed one normalization layer, and that layer is deliberately pure — it turns whatever a detector finds into the same shape of anomaly record, nothing more.

Every detector is built to degrade, not fail: if a data source is missing or malformed, that one detector goes quiet instead of taking the whole loop down with it. That mattered immediately — the secret-expiry detector shipped without any real secrets to check, and the security-finding detector needs a feed that isn't wired into every context it runs in. Both sit there cleanly, reporting nothing, until their real inputs exist.

Escalate-only, on purpose

The loop runs in what we call proposal mode. It reads. It never writes. When all eight detectors went live together for the first time this week, the run surfaced one real anomaly — and mutated nothing, exactly as designed. That's not a limitation we're working around; it's the actual point. A system that watches your infrastructure and a system that changes your infrastructure are two different levels of trust, and we didn't want to earn the second by accident while building the first.

The easy version of a monitoring loop eventually starts fixing what it finds, because the loop is right there and the fix is usually obvious. We built the boundary between watching and touching first, on purpose, so we'd never have to hope the loop respects it.

A second watcher, for dates instead of deploys

Alongside the detector expansion we shipped something smaller and, in its way, more useful day to day: a weekly cron that reads a short registry of dates that matter to us as a company — when our GDPR records need review, when our AI Act obligations land, when our own security posture review is due, when a tax filing is owed. It classifies each one as expired, due, on track, or not yet configured, and only turns red when something has actually lapsed. A date that's merely coming up soon is a standing reminder, not an alarm — we didn't want a system that cries wolf every week about something three months out.

We seeded it with dates we could actually verify: our GDPR records-of-processing review in 2027, our legitimate-interest-assessment review in December, a security posture review in three weeks, and two EU AI Act dates — the literacy obligation next year and the enforcement cliff on August 2, ahead of which we still owe ourselves a piece of counsel work. The tax-filing entries are left unconfigured on purpose, pending real dates from our external tax advisor. We'd rather show a gap plainly than fill it with a guess.

Where the code is done and the data isn't

The honest state of this work right now is that the code is finished and some of the inputs aren't. Secret expiry needs a real, founder-maintained list of which secret expires when — we shipped the schema, not fake dates, because a placeholder date wouldn't just fail to help, it would actively suppress a real alert later. A related KPI-snapshot cron is designed on paper but deferred until we have a live revenue source to snapshot. The pattern across all of it is the same one we keep returning to: build the mechanism for real inputs, and leave it visibly idle rather than pretend an input exists when it doesn't.