test(stress): concurrent payload-apply stress test across transaction modes

[andinux]

Summary

Adds test/stress/apply_stress.c, a manual stress test that reproduces the SQLITE_BUSY storm seen when a sync server applies uploads from many devices to one database at the same time.

Each device builds payloads that each span several db_versions. One thread per device then applies its payloads concurrently to a shared WAL database file, each on its own connection, in one of four modes:

mode	how the apply runs
autocommit	SELECT cloudsync_payload_apply(?) on its own
deferred	BEGIN; apply; COMMIT — what the SQLite Cloud server does today (implicit BEGIN TRANSACTION)
immediate	BEGIN IMMEDIATE; apply; COMMIT
concurrent	BEGIN CONCURRENT; apply; COMMIT — skipped when the linked SQLite lacks it

Retries follow the server: up to 1 + 6 attempts per request, a busy COMMIT retried 5 times, and a failed request re-sent after a backoff.

Checks: every mode must converge to a reference database that applied all payloads serially, and in immediate mode the apply statement must never return BUSY. Retries, failed requests, BUSY codes, wall time and process CPU time are reported.

Results

32 devices, 640 payloads, built against the SQLite Cloud server's SQLite 3.54, 5000 ms busy timeout:

mode	retries	failed requests	CPU per payload
autocommit	0	0	5.3 ms
deferred	9,000	1,249	6.9 ms
immediate	5	0	6.5 ms
concurrent	902	101	79.9 ms

All modes converged to the reference.

Why deferred fails: the apply reads before its first write. SQLite calls the busy handler only when the connection holds no transaction, so the read→write upgrade returns SQLITE_BUSY immediately and the busy timeout is never used. BEGIN IMMEDIATE takes the write lock before any read, so contention becomes a normal wait.

Running it

Manual only: it needs threads and its numbers depend on the machine. It is not part of make test or CI.

make apply-stress
make apply-stress APPLY_STRESS_SQLITE_DIR=<dir with a sqlite3.c>   # e.g. one with BEGIN CONCURRENT
APPLY_STRESS_DEVICES=32 APPLY_STRESS_ROWS=20 ./dist/apply_stress   # rerun without rebuilding

All APPLY_STRESS_* settings are documented in the file header.

Test plan

• make apply-stress passes with this repo's SQLite 3.45 (concurrent skipped)
• passes with the server's SQLite 3.54, all four modes, 8 and 32 devices
• no changes to the extension or to make test

🤖 Generated with Claude Code