We moved our production database from SQLite to PostgreSQL without slowing down
A database migration is the change engineering teams most often defer. Not because the SQL is hard, but because of what it demands of everything around it: a freeze, a rehearsal, a window, a rollback, and the discipline to leave the rest of the roadmap alone until it’s done.
We moved CodeHerder’s production database from SQLite to PostgreSQL on RDS. The cutover window was 33 minutes, and the verification came back 99 of 99 tables verified, zero mismatches.
The number we’re actually proud of is a different one. During the days we planned and built that
migration, the same codebase absorbed 637 merges to main, peaking at 200 in a single day,
and 85% of those merges had nothing to do with the migration. Twelve other initiatives shipped
alongside it.
That combination is the point of this post. Not that a migration is possible. That a migration doesn’t have to be the only thing happening.
The shape of the work
The migration was planned as a programme, not a task: 10 epics, 75 stories. Each epic closed one category of risk before the next depended on it.
| E1 | Portability ratchet and the dual-backend storage seam |
| E2 | Dialect-neutral SQL sweep |
| E3 | The squashed PostgreSQL baseline and its drift guard |
| E4 | Test harness on PostgreSQL |
| E5 | The concurrency model: removing MaxOpenConns=1 |
| E6 | Search: FTS5 to tsvector |
| E7 | Data transfer and the measured cutover window |
| E8 | Infrastructure: RDS PostgreSQL 18 behind VPC peering |
| E9 | The flip and the rehearsed cutover |
| E10 | Retire the SQLite-only conventions and docs |
Read that list as a dependency chain and the ordering is the interesting part. The test harness (E4) had to run on PostgreSQL before the dialect sweep (E2) could be trusted. The concurrency model (E5) couldn’t change until the storage seam (E1) existed to change it behind. The cutover (E9) was ninth of ten, because by the time it ran, every question it could have raised had already been answered somewhere earlier in the chain.
All ten epics are closed and every story is done. The cutover itself was almost boring, which is exactly what a cutover should be.
Velocity during the migration, not after it
Here’s the merge history for the fortnight either side of the work, straight from git log:
2026-08-09 8
2026-08-10 65
2026-08-11 77
2026-08-12 99
2026-08-13 53
2026-08-14 58
2026-08-15 55
2026-08-16 58
2026-08-17 37
2026-08-18 30
2026-08-19 66
2026-08-20 149 ← migration programme opens
2026-08-21 144
2026-08-22 200
2026-08-23 91 ← cutover windowThe migration programme opened on 20 August. Throughput roughly doubled that same day and stayed there. The three highest-volume days in the repo’s history are the three days we spent preparing to replace the database underneath it.
That isn’t a coincidence, and it isn’t heroics. A migration programme is unusually well suited to a herd of agents: it decomposes into many small, independently verifiable units: one dialect fix, one index, one test package moved onto the new backend. Each is a clean, self-contained task with an obvious definition of done. Work shaped like that is exactly what parallel agents are good at, and CodeHerder’s job is to keep 20-odd of them from colliding while they do it.
Meanwhile, of the 637 merges in that period, 539 never touched the migration surface at all.
What else shipped that week
Every one of these landed while the migration was in flight:
| initiative | stories |
|---|---|
| Multi-repo tasks: zero, one, or many repos per task | 27 |
A Go naming standard across ~50 abbreviations in internal/ |
13 |
| Self-provisioning ephemeral runner devices on AWS Lambda MicroVMs | 12 |
| Workflows and shared stages, and the Auto Workflow it unlocks | 10 |
| Working on a non-default base branch, per task | 9 |
| Inference auth beyond Claude OAuth: Bedrock, API keys, IAP | 7 |
| Staffing by capability alone, deleting the agent-to-device link | 6 |
| Unified secrets and env vars into scoped Variables | 6 |
| Interactive local sessions as first-class CodeHerder sessions | 5 |
| A public changelog on the marketing site | 5 |
| Soft-delete audit across every foreign key | 4 |
| Louder, self-announcing device placement when a task is mis-scoped | 3 |
A team that treats a database migration as a code freeze doesn’t ship that list in the same week. It ships the migration, then starts the list.
Safety came from rehearsal, not from confidence
The part of this we’d repeat verbatim: we ran the entire migration once, on a copy, before running it for real.
A full extract-transform-load against a snapshot of the production database, into the real (empty) PostgreSQL target. Not a smoke test on sample data: the whole 12 GB, every table, with the verification tier comparing every row back against the source.
That rehearsal is where the surprises happened, which is the whole reason to do it. Every one of them was found, fixed, and re-tested before the window opened, on a copy, with production serving traffic the entire time. By the time the real window ran, the ETL had already been proven end to end on real data.
The tooling was built to make that possible. A dedicated migration binary with a preflight that compares the source schema against its own embedded migration chain and refuses to run on any mismatch. A verification tier that re-reads every table from both databases and compares row by row. A machine-readable report so every number in this post comes from a JSON file rather than someone’s recollection:
RESULT: VERIFIED — 99 of 99 tables, 0 mismatches
load_tables 828.18s
recreate_validate_fk_constraints 26.66s
verify_total 271.62s
foreign keys 182 total, 0 invalidAnd a rollback that was written, rehearsed, and executed at least once during preparation, because a rollback nobody has run is a hope, not a plan.
The window
Thirty-three minutes, against a sixty-minute announced window.
| phase | duration |
|---|---|
| Stop, checkpoint, capture the rollback artifact | ~2 min |
| Pre-window schema preparation | 7 min |
| ETL load (99 tables, ~7 M rows in the two largest) | 13 m 48 s |
| Foreign-key recreate + validate (182 constraints) | 27 s |
| Verification (every table, row by row) | 4 m 32 s |
| Flip the DSN, start the server, restart replication | ~3 min |
The first-minutes checklist passed every row: the API reporting dialect=postgres, health checks
green, the migration ledger matching the shipped incrementals exactly, no SQLite-only SQL reaching
the new database, devices reconnecting with fresh heartbeats.
An hour and twenty minutes later: zero error-level lines in the log, writes flowing at normal volume, and slow queries roughly halved compared with the equivalent window before the flip. The new database is measurably faster at the same work.
What we’d tell another team
Decompose ruthlessly, then parallelise
The reason 75 stories beat one big migration ticket isn’t process theatre. It’s that 75 independently verifiable units can be worked concurrently, reviewed on their own merits, and merged as they finish. One big ticket can only ever be worked by one person at a time, and reviewed once, at the end, when it’s too large to reason about.
Put the guard in the tool, not the runbook
A preflight that refuses to run beats a checklist item asking someone to check. Our migration binary would not proceed against a schema it didn’t recognise, and the verification tier compared every row rather than sampling. Neither depended on anyone being careful at 4am.
Rehearse on real data
Not representative data: the actual database, copied. Volume and history are where the surprises live, and finding them on a copy costs nothing.
Don’t accept the freeze as a given
The received wisdom is that a migration of this kind means pausing everything else. Our three highest-throughput days ever happened during the preparation. The freeze is a symptom of not being able to run enough work in parallel safely, which is a solvable problem, and the one CodeHerder exists to solve.
CodeHerder builds CodeHerder. Every number in this post came out of the platform’s own database: 75 stories across 10 epics, 637 merges in the surrounding period, 852 tasks completed since the programme opened, and a 33-minute window that verified 99 of 99 tables on the first attempt.
