REQUEST: sqlite3_file_control(...op=SQLITE_FCNTL_TRUNCATE_WAL...)

Previous Topic Next Topic
classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view

REQUEST: sqlite3_file_control(...op=SQLITE_FCNTL_TRUNCATE_WAL...)

Howard Kapustein
I use journal_mode=WAL and have periods of checkpoint starvation (complicated reasons) so I'm looking to prune the -wal file but in less blunt way than SQLITE_CHECKPOINT_TRUNCATE.

Behaviorally I want SQLITE_CHECKPOINT_PASSIVE *and then* if required -wal content < journal_size_limit, to do the truncation a la SQLITE_CHECKPOINT_TRUNCATE

SQLITE_CHECKPOINT_PASSIVE gives me the best-effort checkpointing, but in the best case I don't get -wal shrinkage. SQLITE_CHECKPOINT_TRUNCATE is more of an aggressive do-it-now-wait-if-necessary which gets me the shrinkage behavior, but with *required*-effort rather than best-effort. I'd really like both -- best-effort checkpoint AND best-effort truncate.

sqlite3WalClose does exactly what I want (the key bits)

    sqlite3OsLock(pWal->pDbFd, SQLITE_LOCK_EXCLUSIVE)
    sqlite3WalCheckpoint(pWal, db, SQLITE_CHECKPOINT_PASSIVE,...)
    sqlite3OsFileControlHint(pWal->pDbFd, SQLITE_FCNTL_PERSIST_WAL, &bPersist)
    if (bPersist) { if (pWal->mxWalSize>=0) { walLimitSize(pWal, 0) } }

But AFAICT this is only called when PRAGMA journal_mode changes to not WAL or the pager cache is closed when via sqlite3_close(). I'm a long running process with connection caching and associated prepared statements so torching the connection to trigger this isn't optimal.

Can I indirectly get this behavior if I open then immediately close a connection?

I think I can sorta approximate this it if I disable the busy-handler, do SQLITE_CHECKPOINT_TRUNCATE, ignore Busy/Locked errors and restore the busy-handler before returning, but that's merely 'sorta' -- ugly on several levels.

I don't see any way to directly try to best-effort truncate the -wal file e.g.

    sqlite3_wal_checkpoint_v2(db, NULL, SQLITE_CHECKPOINT_PASSIVE, NULL, NULL))
    sqlite3_file_control(db, NULL, SQLITE_FCNTL_TRUNCATE_WAL, NULL)

That would be ideal. Let's me handle both checkpoint'ing and truncation in low-impact best-effort manners.

    - Howard
sqlite-users mailing list
[hidden email]