I put page-level SQLite replication inside my iPhone. Here's what broke.
Adopting liters, a Rust page replicator, to sync a 766 MB SQLite health database off an iPhone — and the WAL, checkpoint, retention and fallback bugs that surfaced once it ran in production.
A phone is a genuinely bad place to run a database replicator. The OS suspends your process whenever it feels like it. The radio comes and goes. You can get killed mid-push and nobody tells you why. Every assumption a server-side replication tool makes about “the process keeps running” is false on iOS, and most of them fail quietly rather than loudly.
I did it anyway, and it’s been in production since July 30.
Here’s the setup. I run NOOP, a fork of the open-source WHOOP client app, on my own phone, for my own health data. It writes into a local SQLite database: three streams landing at 1 Hz, about 16.3 million raw sample rows, growing around 34 MB a day. I wanted that database in the cloud, live, so MCP tools and agents could query my actual biometrics instead of whatever a vendor API felt like exposing. Agentic infrastructure for my biometrics. So the question was how to get it up there.
Why not a row protocol
My first instinct was to build a delta sync. Diff the tables, ship the changed rows, apply them server-side. I got a long way into designing that before I stopped and asked the obvious question: is this not just some manual patchwork? Has nobody solved this?
The answer is that plenty of people have solved it, and every one of them solves it by owning your schema. PowerSync stores your data as schemaless JSON in its own tables and hands your tables back to you as views. Fine trade if you’re greenfield. Not a fine trade when your store is shared with an upstream project, carries roughly three dozen hand-written migrations, and has a byte-identical Room twin on the Android side. The other candidates were dead or pointing the wrong direction: LiteFS Cloud retired in October 2024, Atlas Device Sync hit EOL in September 2025, cr-sqlite is ~21 months stale, and ElectricSQL is read-path only, which is the opposite arrow from the one I need.
The real problem with a row protocol isn’t any single table. It’s that it never ends. Every new table is a new sync implementation, every migration is a sync migration, and you are signing up to re-implement sync per table forever, in a schema you don’t control.
Page replication doesn’t care about your schema at all. It ships dirty SQLite pages. Add a table, drop a column, run a migration, it’s all the same bytes. And the measurement that settled it: 0.24% of pages change per sync at 8 syncs a day. Ordinary use is a rounding error, and page selection agrees with sqlite3_rsync to within 0.04%, which is how I know the primitive is honest. sqlite3_rsync was the strongest thing on the buy side, and I rejected it at runtime because it spawns ssh for transport, then adopted it anyway as the correctness oracle for page-apply tests.
What I adopted is liters, Kurt Mackey’s Rust library that reads and writes Litestream v0.5’s LTX format and bucket layout and exposes it to a host app through UniFFI. I published my mobile work as vishk23/liters-mobile. It’s a derivative of mrkurt/liters by Kurt Mackey, MIT-licensed and used with his permission. Kurt wrote the LTX codec, the WAL reader, the storage backends, the writer surface, the HTTP protocol and the bindings, which is the substantial majority of the code, and his commits keep their original SHAs, authorship and dates.
Then I found out what breaks.
The WAL is bigger than you think, and nothing checkpoints it
First measurement, about 20× my estimate: the WAL grows 166 MB a day. Not because of data volume. Because of commit count. The collector flushes every 30 seconds or so, which is around 2,880 commits a day, each dirtying about 14 pages. Small writes, constantly, is the worst possible shape for a WAL.
The corollary is what made it sharp. Under external checkpointing, if the replicator never runs, nothing checkpoints at all. Even the emergency truncate lives inside push(). So the failure mode of “replication stopped” isn’t “data is stale,” it’s “the phone fills up.”
Two SQLites in one process is a correctness bug, not a hazard
GRDB links Apple’s system libsqlite3. liters hardcoded rusqlite’s bundled feature, and workspace dependency features are additive, so a member crate can’t remove them. Two copies of SQLite in one process each keep their own unixInodeInfo table, and POSIX advisory locks are per-copy, so they can silently drop each other’s locks.
For most apps that’s a hazard you can live with. For liters it’s fatal, because the writer’s guarantee that no foreign checkpointer restarts the WAL underneath it is a long-running read lock. I moved the choice into a cargo feature, on by default so nothing changes for existing users, plus a linkage switch in the iOS build script. That went upstream.
Related, in the replica applier: F_SETLKW with no timeout. A reader holding a SQLite transaction wedges replication forever, and cancel() sets a flag that a blocked fcntl will never read. I replaced it with deadline-bounded, cancellable F_SETLK polling plus a retryable LockBusy error. That one arrived via Kurt’s own “Mobile hardening” commit, so it’s upstream’s bug, and it went back to him as a PR.
Every checkpoint mode restarts the WAL, including the polite one
This is the one I’d want someone else to read before they lose a week.
Every single push was a full upload. 640,736,289 bytes, every time. liters told me exactly why, in the telemetry I’d added for precisely this: snapshotReason: "wal truncated by another process".
The other process was me. The fallback upload path exported a whole-database backup, and its first act was wal_checkpoint(TRUNCATE), which restarts the WAL and destroys the replicator’s resume offset. The page replicator was being reduced to a full upload with extra steps, by the exact path it was replacing.
So I tried the gentle mode, and this is the trap. TRUNCATE leaves the -wal at 0 bytes and forces a snapshot, which is obvious. FULL leaves the -wal unchanged at 168,952 bytes and still forces a snapshot, same reason. FULL looks harmless. It isn’t. SQLite restarts a fully-backfilled WAL on the next write regardless.
The answer is to take no checkpoint at all and stage a consistent copy through SQLite’s Online Backup API, which reads through the WAL and never checkpoints:
static let defaultExporter: Exporter = { store, dest in
guard case .external = store.walCheckpointing else {
return await DataBackup.writeBackup(
checkpoint: { (try? await store.checkpointWAL()) != nil }, to: dest)
}
// Under .external: stage through the Online Backup API instead.First push after the fix shipped 65,789 bytes. That’s 9,739× smaller.
The footgun where nothing throws and nothing logs
external checkpointing is a per-connection PRAGMA wal_autocheckpoint = 0, and my app opens the same database from two independent places with two independent DatabasePools.
Set .external at one call site and not the other and you achieve nothing at all. The other pool auto-checkpoints at ~4 MB, restarts the WAL under the replicator, and every push degrades to a full snapshot. Nothing throws. Nothing logs. The only symptom is that your uploads never get smaller, which reads like the replicator just not being very good.
So the mode isn’t an argument call sites have to remember. It’s the default value of the initializer’s parameters, resolved per call:
public init(path: String,
walCheckpointing: WalCheckpointing = StoreReplication.walCheckpointing,
walBackstop: WalBackstopPolicy = StoreReplication.walBackstop) async throwsEvery opener that exists, and every opener anyone writes later, is correct without knowing this type exists. The defaults are byte-for-byte a build that never configures anything, so upstream compiles the file unchanged.
Retention on a page-replication server is not what the comment says
I wrote this comment in the replicator, and I want to quote it because it’s the wrong assumption that filled a disk:
the bucket is a transport, not an archive… one base snapshot plus ~2 MB per 4-hour sync is ~12 MB/day
Both halves are wrong. The LTX bucket hit 7.8 GB across 27 files and took a 10 GB volume to zero bytes free. Every file spanned a single transaction and every file was 320 to 395 MB, because the phone was still replacing the whole database on each push, so every page was dirty. At two pushes a day that’s about 800 MB a day. You cannot buy your way out of that with a bigger volume.
The deadlock was self-inflicted and did not self-heal. The sink accrued 79,024 errors refusing to apply, and the ingest path refused in parallel. Both write paths died together, so the phone’s uploads were rejected and the app’s “Syncing…” spinner never finished. The spinner was telling the truth.
One thing I want to defend: the sweeper is corpses-only by design, and the assertion pinning that is load-bearing. Committed .ltx files are replication history and are deliberately invisible to it, so sweptTotal: 0 alongside zero temp files is correct, not a failure. “Fixing” the sweeper would have deleted my lineage. Recovery was to keep the newest three segments and prune the rest, after which the sink recovered unaided in about ten minutes with no restart.
Two smaller gotchas from the same week, both of which cost me a wrong timeline:
A bucket .ltx file’s mtime is the LTX header’s creation timestamp, set deliberately for litestream parity. It is when the delta was created on the phone, never when it was uploaded, and files uploaded together in one drain keep their original spread of timestamps. Don’t reconstruct upload timing from an ls -la.
And mirror.sqlite has two writers. The whole-DB ingest path renames a database into place and appends a log row. The Rust sink writes the mirror in place and touches nothing in the server’s own tables. Anything derived from that log row freezes at the last whole-DB upload while the mirror races ahead, which is how freshness reporting told me the data was 30.8 hours stale on a file written seconds earlier. The rule I keep: never derive “how fresh is X” from a marker only one of your writers maintains.
The loop
The nastiest bug was a loop, and every individual piece of it was correct.
Every successful whole-DB ingest resets the server-side liters lineage, and that reset is right, because replacing the whole mirror genuinely orphans the lineage. The phone’s rule was: if the push didn’t work, fall back to ingest. Also reasonable in isolation.
Together: the background window is too short for a forced ~390 MB snapshot, so the push fails, so we fall back to ingest, so the lineage resets, so the next push has to snapshot again. Forever.
And it fired on liters’ healthiest outcome. A push returning uploaded=0 synced=true, meaning “already in sync, nothing to ship,” got answered with a 208 MB whole-DB upload that destroyed the lineage that made the no-op possible.
The fix was to stop treating “uploaded zero bytes” as one state. Four outcomes now: pushed, in-sync, retryable, unavailable. In-sync only counts as verified when the transaction id is greater than zero and both sides agree on it, because a synced=true at txid 0 is vacuous, both sides agreeing on nothing, and stamping the upload token on that makes the next sync skip uploading entirely. Retryable retries the cheap delta next time with the lineage intact. Ingest only opens after three consecutive retryables, counted in a streak that persists across relaunches and is reset only by a liters success, deliberately not by an ingest success.
Proof the delta path works when you leave it alone, three consecutive pushes on one afternoon after a lineage reset: 402,105,931 bytes, then 125,571 bytes, then 198 bytes.
That fix reached my phone on the morning of August 4 and fired for real about an hour later. A ~400 MB re-baseline PUT died at Fly’s proxy with a 408 that never reached the liters route. Old behaviour would have been a silent 1 GB-plus upload plus a lineage reset. What actually happened was one log line, Delta sync didn't complete (attempt 1), will retry next sync, streak 1 of 3, lineage intact. Exactly as designed, on the first real failure it saw.
One more, because I nearly filed it as a bug and I was wrong: a status line reading “Uploaded 1167.6 MB” is not a broken counter. bytesUploaded is per-push, and one push honestly drained a five-file backlog summing to 1,224,353,458 bytes. The real defect is that failed and zero-byte pushes write no telemetry at all, so a backlog builds invisibly and surfaces as one implausibly large sync.
Where it actually stands
Page replication is the primary write path in production, live since July 30, gated behind a deploy secret so the build changes nothing until you turn it on. Freshness reporting says lastWriteSource: "replication" against a mirror a little over a gigabyte.
But I’m not going to tell you this was a clean win. The honest verdict I wrote down at the worst point still mostly stands: liters was the right call on analysis, and the delta primitive works, but integration added failure modes faster than it removed work.
What’s still open, plainly:
Bucket retention is not shipped. The file exists on my disk, uncommitted. Until it lands, the thing that took the server down is still latent.
Background upload is unsolved and it’s structural, not a bug. The bindings surface is entirely synchronous, and iOS background URLSession is delegate-async and file-upload-only. The replicator uses a foreground session by design and can’t hand its work to the app’s background session.
Whole-DB upload has no resume. A 40 MB partial out of a 208 MB body restarts from zero, and I’ve now watched single-PUT fragility three separate ways.
There’s a fix sitting on a branch that stops SQLite’s last-connection-close checkpoint from restarting the WAL under the replicator, which is what makes every graceful termination, most visibly a devicectl install, cost the next push a full snapshot. It is not on main yet, so I’m not going to claim it.
And on the other half of the pipeline, getting banked history off the wearable and onto the phone, I measured about 21× realtime sustained and published a causal story about where the lost time goes. I retracted it the same day, once I counted four confounds including the fact that my measurement method perturbed the app it was measuring. The ceiling stands, the explanation doesn’t.
The code is at vishk23/liters-mobile, MIT. Two of the fixes in there aren’t really mine, they’re bugs in Kurt’s library that I hit because I ran it somewhere he hadn’t, and both are proposed back to mrkurt/liters rather than kept. Upstream is quiet right now. That’s not the same thing as gone, and my repo says so in the first paragraph: if you want liters itself, go to mrkurt/liters. Mine is not a competing project and not a hard fork.
I’d rather run someone else’s page replicator and send the fixes back than own a row protocol for the rest of my life.