Batch cache hit writes

[montehurd]

Every cached download records its hit with its own UPDATE. The proxy uses one SQLite connection, so each download is a write transaction, and every other download queues behind it. Serving npm tarballs from a warm cache on an 8 vCPU VM, that held the proxy near 1,000 requests a second with the CPU about two thirds idle. Turning off request logging and the access log changed little.

This counts hits in memory and writes them in one transaction per interval.

Measured with this branch on that VM, toggling only hit_flush_interval. The cache was warm, with 5 simultaneous simulated npm installs (15 connections each) from one client VM. Each run was 60 s, with two runs per setting, alternating.

SQLite

hit_flush_interval	Req/s	p50	p99	Proxy CPU (serving the req/s shown)	SQLite's disk, writes/s
"0" (each hit)	933 / 1,005	46 / 46 ms	490 / 450 ms	218% / 231%	119 / 129
"1s"	2,140 / 2,196	11 / 11 ms	489 / 479 ms	393% / 391%	30 / 30

Postgres 18 (default settings, the proxy's pool of 32 connections)

hit_flush_interval	Req/s	p50	p99	Postgres CPU	Postgres commits/s
"0" (each hit)	2,042 / 1,851	17 / 16 ms	476 / 536 ms	269% / 260%	6,150 / 5,601
"1s"	2,182 / 1,568	9 / 11 ms	638 / 1,166 ms	175% / 150%	2,939 / 2,207

On SQLite, batching more than doubles throughput and cuts median latency by about 4x. On Postgres, batching brings no throughput gain; the win is half the commits and about 40% less Postgres CPU. The p99 varied from run to run on Postgres with either setting.

A separate pool of read-only SQLite connections on top of this made no measurable difference to throughput, so the single connection stays.

• database.hit_flush_interval sets the interval. It defaults to "1s", and "0" writes each hit as it happens, as before. It's also available as PROXY_DATABASE_HIT_FLUSH_INTERVAL, and it applies to both databases.
• The buffer holds one entry per distinct artifact hit since the last flush. A failed write puts the hits back for the next flush and is logged, and Close writes whatever is pending. Up to one interval of hits is lost if the process is killed.
• Batching is turned on in server.New. Tests and the mirror command open the database directly and keep writing each hit immediately.
• Only hit_count, last_accessed_at and updated_at are delayed. They feed the stats pages and LRU eviction, which don't need them to the second.
• A hit on an artifact that eviction clears before the flush still updates that row's counters. Today's code has the same window, just narrower.
• Rows are written in a fixed order, so proxies sharing a Postgres database can't deadlock on each other's flushes.
• When proxies share a database, a flush never moves last_accessed_at or updated_at backwards: each keeps the later of the stored and incoming time.

Tested with go test ./..., plus -race and a real Postgres 18 for internal/database.

Closes #322.

[andrew]

One fix needed before merge: writeHits in internal/database/hits.go unconditionally replaces last_accessed_at and updated_at with the batch timestamp. When proxies share a database, an older batch can flush after a newer hit has already been persisted and move both timestamps backwards. This can change LRU eviction order.

I reproduced this through two running proxy processes sharing a temporary SQLite database. One buffered a cache hit with hit_flush_interval: "1h"; the other then recorded a newer hit with batching disabled. Shutting down the first proxy flushed its older timestamp over the newer one. The hit count correctly increased from 1 to 2, but last_accessed_at moved from 12:14:02.360091 back to 12:14:02.238994.

Please keep the count increment while preserving the later of the stored and incoming timestamps, including when the stored access time is NULL. Add a regression test where an older batch flushes after a newer hit has been persisted, covering both SQLite and PostgreSQL.

[montehurd]

Fixed in 55ed6b9. writeHits now keeps the later of the stored and incoming last_accessed_at and updated_at, including when the stored value is NULL, and the count still adds up. TestBatchHitsKeepNewerAccessTime writes a hit, flushes an older batch, then a newer one, on both SQLite and Postgres 18. On SQLite the comparison is on the stored timestamp text, like the code's other time comparisons, so it assumes proxies sharing a database write with the same UTC offset.