← Blog
ENGINEERING2 min read

Moving operational data to its own database: copy, verify, then fail closed

Nov 11, 2026DekimuAI-generated

Operational data (queues, audit records, job state) now lives in its own database, separate from application data. The move worked because the copy could be re-run safely, each key's state was recorded, and anything reading from the wrong store was made to refuse rather than guess.

Why split the data at all?

When operations data and application data share a database, every credential for one can reach the other. That is a convenience early on and a liability later: a scheduled job that only needs a waitlist count ends up holding a key that could read far more.

Splitting the stores lets each credential's reach match its job. It also lets the operations side live under stricter rules, such as who may reset its credentials, without those rules touching customer-facing code.

What made the copy safe to repeat?

The copy tool resolves both databases by name, never by position, and never overwrites a key that already exists in the destination. It keeps each key's type, value and remaining time-to-live. It also refuses to run unless a kill switch is on, so no scheduled job writes to the source mid-copy.

A ledger records a digest per copied key. On a second run, the ledger decides what is allowed: a change made on the live side wins, a source-only change is followed, and a genuine conflict is reported rather than resolved silently. Digests are computed so that sets and hashes compare equal regardless of the order the database returns their members, otherwise a harmless reordering would look like a change.

A data move is not a command you trust. It is a series of checks you can run again.

How do reads refuse the wrong store?

Jobs that read application data now go through a single helper that connects with a read-only credential. If it is handed a connection that points at the operations store, it refuses. A misconfiguration therefore stops the job instead of returning an empty, plausible-looking answer.

Every consumer of the data was inventoried first: scheduled jobs, a dashboard, a worker and two small services. Because several of them connect with their own credentials, the cutover was one coordinated switch rather than a secret swap. A script checks each new value against its database before writing anything, supports a dry run, and restores the previous configuration if a restart fails.

What did independent review change?

A reviewer with no shared context with the author attacked the plan and the scripts, and the findings shaped the final design: the ledger itself, atomic per-key rewrites, the kill-switch check following wherever the jobs actually read it, and surfacing time-to-live drift instead of ignoring it.

The cutover is done. The old copy is kept for several clean days before it is pruned, and the ledger is dropped first, so nothing can mistake a leftover for live state.

ENGINEERING

This post was drafted by an AI system from Dekimu's public engineering record and published with automated checks, without per-post human editing.

← Back to blog