The 16-year-old WAL-reset bug in SQLite: what your .NET app should do

There was a post at the top of Hacker News this week that made me stop for a second. Tailscale tracked down a bug in SQLite that had been sitting in the code for sixteen years. No crash, no error message, no stack trace. Just rows that were there one moment and gone the next, and nothing in the system that noticed.

The bug lives in the WAL checkpoint process. Under the right, rare timing, SQLite resets the write-ahead log in a way that loses commits already recorded in the WAL before they reach the main file. The result is data that quietly disappears. And because the rest of the database structure stays intact, your database looks perfectly healthy while rows are missing.

Fixed in SQLite 3.51.3. But the fix is the least interesting part of this story.

What actually went wrong

Write-ahead logging works like this: new writes go to a separate WAL file first, not straight into the database. Readers see the old state, writers append changes to the log, and at intervals a checkpoint moves those changes into the main database file. After that the WAL can be reset and it starts over.

The gap was in that reset. With a specific overlap of a checkpoint and concurrent connections, SQLite could roll the WAL back while it still held frames that had not yet been written to the database. Those frames were then gone. The transaction had committed as far as the application was concerned, because COMMIT returned success cleanly. It had never reached the disk.

The window is tiny. You need concurrent connections, a checkpoint at exactly the wrong moment, and then some luck on top. That is how it survived sixteen years in one of the most heavily tested pieces of software in existence.

Why your test suite never caught it

SQLite runs more than a hundred million test cases on every release. Line-by-line coverage, fuzzing, crash-recovery tests, power-loss simulations. And still this slipped through, for sixteen years.

That is because the bug only exists in the interplay of timing and concurrency. A deterministic test that runs a checkpoint and reads the rows back sees nothing wrong. You have to make two connections collide within microseconds to hit the window, and even then it is a coin toss. Bugs like this live in the space between threads, and that space is almost infinitely large.

The lesson is not that SQLite is unreliable. SQLite is probably more reliable than the code you and I write around it. The lesson is that your bottom layer, the one you never look at because it always works, also has a failure mode you have not met yet.

What this means for your .NET app

If you use Microsoft.Data.Sqlite with WAL mode, and you probably do in production, the first question is simple: which SQLite version is in your container?

using var connection = new SqliteConnection("Data Source=app.db");
connection.Open();

using var cmd = connection.CreateCommand();
cmd.CommandText = "SELECT sqlite_version()";
var version = (string)cmd.ExecuteScalar()!;

logger.LogInformation("SQLite version in use: {Version}", version);

The version Microsoft.Data.Sqlite ships is tied to the SQLitePCLRaw bundle in your dependency tree, not to whatever SQLite happens to sit on your host. So run that query on a real connection in your running app, not in a shell on the server. If you are below 3.51.3, update the NuGet package and redeploy. That is the whole fix.

Not using WAL mode but the default DELETE journal mode instead? Then this bug does not touch you. It lives purely in the WAL checkpoint path. WAL is still the better choice for apps with concurrent readers, so the advice is not to drop it. The advice is: update and keep WAL on.

Detection is the real work

Here it gets uncomfortable. Your first reflex is PRAGMA integrity_check, and that reflex falls just short. Integrity check verifies B-tree consistency and page structure. With this bug the structure is completely fine. Only rows are missing. A structurally sound database with a hole in it passes the check without a complaint.

What does work is verification at the application level. If your critical records carry a checksum or an expected count you can compute independently, you have a detection mechanism separate from SQLite's own bookkeeping. Hang it off your health-check endpoint:

app.MapHealthChecks("/health/db", new HealthCheckOptions
{
    Predicate = check => check.Tags.Contains("db")
});

// Registration
services.AddHealthChecks().AddCheck<SqliteIntegrityCheck>(
    "sqlite-integrity", tags: new[] { "db" });

public sealed class SqliteIntegrityCheck : IHealthCheck
{
    private readonly SqliteConnection _connection;

    public SqliteIntegrityCheck(SqliteConnection connection)
        => _connection = connection;

    public async Task<HealthCheckResult> CheckHealthAsync(
        HealthCheckContext context, CancellationToken ct = default)
    {
        using var cmd = _connection.CreateCommand();
        cmd.CommandText = "PRAGMA integrity_check";
        var result = (string?)await cmd.ExecuteScalarAsync(ct);

        if (result != "ok")
            return HealthCheckResult.Unhealthy($"integrity_check: {result}");

        // The real check: recompute an invariant you own.
        cmd.CommandText = "SELECT COUNT(*) FROM orders WHERE status = 'paid'";
        var paidOrders = (long)(await cmd.ExecuteScalarAsync(ct))!;

        return paidOrders >= await ExpectedMinimumAsync(ct)
            ? HealthCheckResult.Healthy()
            : HealthCheckResult.Degraded("Paid order count lower than expected");
    }
}

I leave the structural check in, because free is free. The line that actually matters is the second one: an invariant from your own domain that you can confirm independently. Paid orders that vanish, balances that no longer add up, a counter that drops when it should only ever climb. That is what makes a missed commit visible.

Watching the WAL file size

By default SQLite triggers an automatic checkpoint once the WAL file reaches 1000 pages. At the default page size of 4096 bytes that is roughly 4 MB. You can read that behaviour and steer it:

using var cmd = connection.CreateCommand();
cmd.CommandText = "PRAGMA wal_autocheckpoint";
var pages = (long)cmd.ExecuteScalar()!;
logger.LogInformation("Auto-checkpoint at {Pages} WAL pages", pages);

A WAL file that keeps growing and never falls back means checkpoints are not succeeding. That is exactly the kind of signal you want before things go wrong. Measure the size of the -wal file on an interval and put an alert on it. It costs you a cron job and gives you eyes on a layer you otherwise never see.

When Tailscale investigated this, they did it with a thorough postmortem, solid logging, and direct contact with the SQLite team. That is not a luxury approach. That is the minimum approach for any system holding your production data.

The wider lesson

Sixteen years. More than a hundred million test cases per release. And the bug just waited for the right timing.

That is no reason to distrust SQLite. It is a reason to never take your infrastructure layer as given. Every system that persists data has a failure mode you have not seen yet. The only question that counts is whether you have the observability to find it before your customers do.

Frequently asked questions about the SQLite WAL-reset bug and .NET

Which version of SQLite contains the fix for the WAL-reset bug? SQLite 3.51.3 contains the fix. Run SELECT sqlite_version() on an open connection to see which version your application actually uses, since that is the version in your SQLitePCLRaw bundle and not the one on the host. If you are below 3.51.3, update the Microsoft.Data.Sqlite NuGet package and redeploy.

Can I hit this bug if I do not use WAL mode? No. The bug lives specifically in the WAL checkpoint process, so applications on the default DELETE journal mode are not vulnerable to this problem. WAL mode is still recommended for production apps with concurrent readers, so the advice stays simple: update to 3.51.3 and keep WAL on.

Is PRAGMA integrity_check reliable for detecting this corruption? Not for this case. The bug removes rows but leaves the structural integrity intact, and integrity_check verifies B-tree consistency and page integrity, not whether rows are missing. A checksum or a recomputable invariant on your critical records is an independent detection mechanism that does fire.

How do I know whether my app runs automatic WAL checkpoints? By default SQLite triggers an automatic PASSIVE checkpoint once the WAL file reaches 1000 pages, which at the default page size of 4096 bytes comes to roughly 4 MB. You read the behaviour with PRAGMA wal_autocheckpoint, and by measuring the size of the -wal file on an interval you can see whether checkpoints are actually succeeding.

Further reading

• SQLite in production: why your .NET app might not need Postgres
• The ripgrep musl crash: why your .NET container on Alpine can betray you
• The valley of webhooks: what to build when delivery is at-least-once

---

Conclusion: Update to SQLite 3.51.3, hang a recomputable invariant off your health-check endpoint, and measure the WAL file size. The bug is patched, but the lesson is bigger: observability and periodic integrity validation are not extras in a system that holds production data.

Sources: We tracked down the 16-year-old WAL-reset SQLite bug · Hacker News discussion.