Skip to content

perf(courier): sleep until the next job, and deliver as a bot gets ready - #79

Merged
httpsphl merged 1 commit into
mainfrom
perf/courier
Sep 30, 2026
Merged

httpsphl merged 1 commit into
mainfrom
perf/courier

Conversation

@httpsphl

Copy link
Copy Markdown
Owner

Sixth and last slice of the performance round. It is independent of the others. It touches deliveries.rs like #74, in different spots.

What changes

  • The courier sleeps until the next job instead of waking every 500 ms.
    • Before: three queries every 500 ms under the store lock, idle or not.
    • It wakes at the earliest of: the first delivery of a bot's queue coming due, a lease ending, or an open task passing its deadline (courier_next_at).
    • The sleep is at least 250 ms, so a delivery that stays due cannot spin the courier.
    • It is at most poll_interval_ms, whose default goes from 500 ms to 10 s: it is now only the ceiling.
    • New messages and finished sends still wake it at once (notify).
  • Delivers as the bot gets ready. The courier follows bot.state. When a bot can take messages again, its deliveries that never failed (attempts = 0) come due at once.
    • Before: a message to a bot that was starting or paused waited for its 5 s check.
    • Deliveries that failed keep their backoff, so a message that crashes the bot does not come straight back.
  • Spec 6 and 9.1 describe it.

Not done here: the reconcile every 5 s. release_limits depends on it, and the session_id N+1 there is small.

Tests

  • New test a_message_that_waited_for_the_bot_goes_as_soon_as_it_is_ready: a message sent while the bot is paused goes on resume without moving the manual clock. It fails without the change (I checked).
  • Four tests waited for Idle and moved the clock 5 s to release a message. They no longer need to: the bot is Busy before it can be seen Idle. The backoff test (attempts ≥ 1) still waits for the clock, as it should.
  • tests/courier.rs dropped from 8 s to 0.3 s.
  • cargo fmt, clippy -D warnings, cargo test --workspace (three times) pass locally.

Manual check (real Claude)

  • Restart a bot and send it a message while it starts. It goes out as soon as it shows "Available", not 5 s later.
  • With every bot idle, the daemon (botloftd.exe) CPU in Task Manager stays near zero.
  • Pause a bot, send it a message, then resume it: the message goes out right away.

The courier woke every 500 ms, idle or not, and ran three queries under
the store lock each time. A message to a bot that was starting waited
for its 5 s check even when the bot was ready after 1.5 s.

- Between cycles the courier sleeps until the first delivery of a bot's
  queue comes due, a lease runs out or an open task passes its deadline;
  at least 250 ms, at most poll_interval_ms, whose default goes from
  500 ms to 10 s since it is now only the ceiling. New messages and
  finished sends still wake it at once.
- It follows bot.state: when a bot can take messages again, its
  deliveries that never failed come due at once. Those that failed keep
  their backoff, so a message that makes the bot crash does not come
  straight back.
- Tests that waited for Idle and moved the clock 5 s to release a
  message no longer need to; a new test checks a message that waited
  for a paused bot goes on resume without the clock moving.
@httpsphl
httpsphl merged commit 49e60ef into main Sep 30, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant