Skip to content

feat(better-sqlite3): record queries made through better-sqlite3 - #237

Merged
dividedmind merged 8 commits into
mainfrom
feat/better-sqlite3-hook
Sep 21, 2026
Merged

dividedmind merged 8 commits into
mainfrom
feat/better-sqlite3-hook

Conversation

@evlawler

Copy link
Copy Markdown
Contributor

Applications that use better-sqlite3, directly or through Drizzle, record no SQL events. This adds a hook for it next to the sqlite3, mysql, pg and Prisma hooks.

What is recorded

  • Database.exec and Database.pragma
  • Statement.run, get, all and iterate, with the statement's source as the SQL
  • Statements inside transaction(), including the BEGIN and COMMIT that better-sqlite3 prepares itself
  • A statement that throws is recorded as an exception and rethrown

Where this departs from the sqlite3 hook

  • better-sqlite3 does not export Statement. The Statement prototype is patched the first time prepare() returns one, not at module load.
  • iterate() returns rows lazily. Its return event is emitted when the iterator is exhausted, returned early, or throws. The hook returns a plain iterator that forwards to the native one, because the native iterator's methods cannot be called through a Proxy receiver.
  • Everything is synchronous, so there is no async context capture.

Test

  • test/betterSqlite3: a fixture that runs each method, one statement twice, a transaction, an early break out of iterate(), and a caught unique violation. Snapshot test in test/betterSqlite3.test.ts.
  • The fixture pins better-sqlite3 ^11.10, the last major with prebuilt binaries for Node 18, which CI runs.

Not covered

  • ESM imports. The hook applies when the module is loaded with require. An import Database from "better-sqlite3" in an ESM application is not hooked. The other driver hooks have the same limit.
  • Whether pragma() should be recorded at all is a judgement call. It is included because PRAGMA statements change behavior and are cheap to record.

This was run against one real application, promptfoo, where it recorded the Drizzle inserts and selects that were previously missing.

Written by Claude in a Claude Code session for Elizabeth Lawler. The commit carries Claude as author.

🤖 Generated with Claude Code

https://claude.ai/code/session_01EgK3BVLCovf26kotqTnkau


Generated by Claude Code

@evlawler
evlawler marked this pull request as ready for review September 16, 2026 11:58
Copilot AI lite review requested due to automatic review settings September 16, 2026 11:58

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Four unresolved moderate issues remain in the better-sqlite3 hook.

Get a fresh assessment by requesting another Copilot review.

Pull request overview

Adds CommonJS instrumentation for better-sqlite3, covering queries, pragmas, transactions, iteration, and exceptions.

Changes:

  • Adds the better-sqlite3 require hook.
  • Adds fixture, integration, and snapshot coverage.
  • Registers the fixture workspace and dependency lock entries.
File summaries
File Description
yarn.lock Locks fixture dependencies.
test/betterSqlite3/package.json Adds the fixture dependency.
test/betterSqlite3/index.js Exercises supported database operations.
test/betterSqlite3/appmap.yml Configures the fixture.
test/betterSqlite3.test.ts Adds integration coverage.
test/__snapshots__/betterSqlite3.test.ts.snap Records expected events.
src/requireHook.ts Registers the new hook.
src/hooks/betterSqlite3.ts Implements SQL instrumentation.
package.json Registers the test workspace.
Review details

Suppressed comments (2)

src/hooks/betterSqlite3.ts:172

  • The lazy iterator has a separate exception path here, but the fixture only verifies an exception from run() (test/betterSqlite3/index.js:35-40). Add an integration case that makes iterator.next() throw, verifies the error is rethrown, and snapshots the resulting SQL exception event so this forwarding logic cannot regress unnoticed.
          } catch (exn: unknown) {
            finish(exn ?? new Error("iteration failed"));
            throw exn;

src/hooks/betterSqlite3.ts:56

  • The private transaction controller does not use Database.prototype.prepare; it calls the native database's prepare to create the transaction statements before the returned transaction function runs. On a fresh database, before.run() therefore executes before any user db.prepare() can patch this shared Statement prototype, so BEGIN is not recorded (and other controller statements can likewise be missed when the callback does not prepare a statement). The fixture masks this by preparing insert first; initialize the Statement prototype before transaction setup and add a fresh-database regression test.
function patchStatementPrototype(statement: object) {
  const proto: unknown = Object.getPrototypeOf(statement);
  if (proto === null || typeof proto !== "object" || patchedStatementPrototypes.has(proto)) return;
  patchedStatementPrototypes.add(proto);
  • Files reviewed: 8/9 changed files
  • Comments generated: 2
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread src/hooks/betterSqlite3.ts
Comment thread src/hooks/betterSqlite3.ts Outdated
claude and others added 8 commits September 21, 2026 12:42
Add a hook for the better-sqlite3 driver, alongside the existing hooks for
sqlite3, mysql, pg and Prisma. Applications that use better-sqlite3 (directly
or through Drizzle) recorded no SQL events before this change.

The hook records Database.exec and Database.pragma, and Statement.run, get,
all and iterate, as sql_query events with database_type "sqlite". Statements
inside a transaction() are recorded too, including the BEGIN and COMMIT that
better-sqlite3 prepares itself. A statement that throws is recorded as an
exception and rethrown.

Two places where this hook departs from the sqlite3 hook:

- better-sqlite3 does not export its Statement class, so the Statement
  prototype is patched the first time prepare() returns a statement, rather
  than at module load.
- iterate() returns rows lazily, so its return event is emitted when the
  iterator is exhausted, returned early, or throws. The hook hands back a
  plain iterator object that forwards to the native one, because the native
  iterator's methods cannot be called through a Proxy receiver.

Everything in better-sqlite3 is synchronous, so no async context capture is
needed.

The test fixture pins better-sqlite3 ^11.10, the last major that still ships
prebuilt binaries for Node 18, which the CI matrix runs.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EgK3BVLCovf26kotqTnkau
iterate() has two failure paths that no fixture exercised: the statement
itself can refuse to iterate, and a row can fail to materialize once the
iterator is already running. Only the first is a plain call-time throw; the
second has to be recorded when next() throws, long after iterate() returned.

Add both to the fixture. A user-defined function that throws on the second
row makes next() fail mid-loop, and iterate() on an INSERT throws right
away. Both are recorded as exceptions on the sql_query event.

No behaviour change; the snapshot records what the hook already does.

Assisted-by: Claude:claude-opus-5[1m]
The iterator's return() emitted the return event from a finally block, so a
cleanup that threw was still recorded as a successful query. native.return()
does throw: it goes through the same busy check as next(), so releasing an
iterator from inside a user-defined function running in another query fails
with a TypeError.

Emit the event from the catch instead, with the exception that cleanup threw.

While here, drop the `exn ?? new Error(...)` sentinel from the next() handler
and box the failure passed to finish() instead. The sentinel only fired for a
thrown undefined or null, and when it did it reported an exception the
application never threw. It did nothing for a thrown string, which lands in
the same place. Every other method in this hook already passes whatever was
thrown straight to functionException.

Assisted-by: Claude:claude-opus-5[1m]
The require hook runs for every require of a module, cache hits included, so
a second require re-wrapped exec() and pragma(), and every call then emitted
one event per layer of wrapping. Statement methods were already safe: they go
through a WeakSet of patched prototypes.

Guard the module itself the same way, as the prisma hook does.

Assisted-by: Claude:claude-opus-5[1m]
Two symptoms of the same cause. The Statement prototype was patched when
prepare() first returned a statement, so whether a query was recorded
depended on what the application had done before it.

A transaction started before any prepare() lost its BEGIN: the controller
compiles BEGIN, COMMIT and ROLLBACK on the native database handle, never
through Database.prototype.prepare, and runs BEGIN before the callback gets
a chance to prepare anything. COMMIT then came out recorded and BEGIN did
not.

A pragma run after a prepare() was recorded twice: pragma() compiles a
statement of its own and runs it through all(), so the pragma proxy and the
patched Statement both reported it. The fixture only missed this because it
ran its pragma before its first prepare().

Prime the Statement patch from pragma() and transaction() with a throwaway
statement, and drop the pragma proxy: with the patch in place the inner
statement reports the pragma itself, with the same SQL. The existing
snapshot is unchanged.

Assisted-by: Claude:claude-opus-5[1m]
iterate() handed back a plain object with next, return and Symbol.iterator
on it, which dropped everything else the native iterator carries -- notably
`statement`, the frozen reference to the statement it came from, and the
iterator's own identity.

Forward to the native iterator through a proxy instead, dispatching the
three methods that do the recording and passing everything else through.
Methods reached through it are bound to the native object, which was the
reason for the plain object in the first place: it is a native wrapper and
cannot be unwrapped from a receiver that is not itself. `constructor` is
excluded, being a class rather than a method -- binding renames it.

Symbol.iterator returns the proxy rather than what the native one returns:
handing for..of the native iterator would bypass recording altogether.

The fixture also pins the finished latch: a spent iterator asked for another
row, or settled again, records nothing further.

Assisted-by: Claude:claude-opus-5[1m]
Corrects the premise of b17967a. better-sqlite3's connection-busy check runs
before it looks at the iterator at all and returns without cleaning up, so a
call that fails that way leaves the iteration alive with its rows still to
come -- and it is the only way return() can fail. Recording the query as
failed there both reported a failure that had not happened and latched the
recording shut, so the rest of the query went unrecorded.

Ask the statement instead: better-sqlite3 keeps it locked for as long as an
iterator holds it, so `busy` says whether the query is over. Finish only when
it is; an iterator we cannot ask is treated as over, which keeps the events
balanced.

An iterator that is abandoned outright still leaves its call event open. That
is left alone deliberately, and the reasoning is now next to the code: for..of
always settles the iterator, and better-sqlite3 gives the iterator no
finalizer, so an application that drops a live one has already stranded the
statement and broken db.close() for itself.

Assisted-by: Claude:claude-opus-5[1m]
The fixtures only ever committed. The transaction controller compiles a
statement for every other way a transaction can end too -- ROLLBACK when the
callback throws, and SAVEPOINT/RELEASE when one transaction runs inside
another -- and none of them go through prepare(), so all of them depend on
the Statement patch being primed before the controller is built.

They are recorded correctly today. Pin the whole sequence so the priming
cannot quietly stop covering them.

Assisted-by: Claude:claude-opus-5[1m]
@dividedmind
dividedmind force-pushed the feat/better-sqlite3-hook branch from cde6858 to 54116ee Compare September 21, 2026 13:14
@dividedmind
dividedmind requested a lite review from Copilot September 21, 2026 13:15
@pkg-pr-new

pkg-pr-new Bot commented Sep 21, 2026

Copy link
Copy Markdown

Open in StackBlitz

npm i https://pkg.pr.new/appmap-node@237 -D

commit: 54116ee

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

The new hook and fixtures appear likely to fail the repo’s ESLint/Prettier checks (unbound-method + formatting), which would block CI.

Get a fresh assessment by requesting another Copilot review.

Review effort: Lite
Findings: 1 High severity · 2 Low severity

Open (3)
Resolved since last review (2)

Comment thread src/hooks/betterSqlite3.ts
Comment thread test/betterSqlite3/index.js
Comment thread test/betterSqlite3/transactionFirst.js
@dividedmind
dividedmind merged commit e574ef0 into main Sep 21, 2026
9 checks passed
@dividedmind
dividedmind deleted the feat/better-sqlite3-hook branch September 21, 2026 13:36
appmap-releasebot Bot pushed a commit that referenced this pull request Sep 21, 2026
# [2.27.0](v2.26.4...v2.27.0) (2026-09-21)

### Features

* **better-sqlite3:** record queries made through better-sqlite3 ([#237](#237)) ([e574ef0](e574ef0))
@appmap-releasebot

Copy link
Copy Markdown

🎉 This PR is included in version 2.27.0 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants