Wal checkpoint support

[sonhmai]

Hi folks,

I'd like to create this issue as an umbrella issue for adding WAL checkpoint support. See sqlite doc for wal checkpointing

Rational

1. important feature for write ahead log to support recovery
2. needed in benchmark(s) like mobibench like Mobibench mentioned in Add support for sqlite3_wal_checkpoint_*() #478

todos

• support for pragma wal_checkpoint. Initial support for wal_checkpoint pragma #694
• wal checkpoint infra in core modules: connection, parser, storage.
• improve checkpoint result: add num pages written to wal, num pages moved to db
• integration of checkpoint to sqlite3_wal_checkpoint* apis. Add support for sqlite3_wal_checkpoint_*() #478
• auto checkpoint of database
• integration tests

Context

• checkpoint is applying changes in WAL file to database file in wal journal_mode.
• it is triggered
  • manually by pragma wal_checkpoint and sqlite3_wal_checkpoint*
  • automatically by the database when wal file is over a threshold

Feel free to comment, I will add the missing ones, or add more details.

[sonhmai]

Support for pragma wal_checkpoint; added in #694

[sonhmai]

checkpoint infra already across the modules in wal, pager, connection.

wal module

limbo/core/storage/wal.rs

Line 415 in 4a41736

fn checkpoint(

connection

limbo/core/lib.rs

Line 394 in 4a41736

pub fn checkpoint(&self) -> Result<()> {

It seems that we need to two other things as part of checkpoint result later. Ref this todo.

1. num modified pages written to wal file
2. num pages moved to db after checkpoint