feat(dictation): Parakeet idle unload/warm-up, tap-or-hold mode, custom dictionary - #279
Conversation
…custom dictionary - Unload the on-device Parakeet worker after a configurable idle period (default 10 minutes, or never) and preload it on hotkey press or mic start. - Add a hybrid "Tap or hold" shortcut mode alongside hold and toggle. - Apply a user dictionary of words and replacements to every transcript. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
![pullfrog[bot]](https://avatars.githubusercontent.com/in/1768019?s=60&v=4){kind=small}
There was a problem hiding this comment.
Important
Hybrid activation and locale-sensitive dictionary matching fail for supported inputs. Please correct these user-visible behavior paths before merging.
Reviewed changes I reviewed PR #279's Parakeet model lifecycle and warm-up, activation-mode state machine and settings, custom dictionary behavior, and composer/onboarding changes.
- Parakeet lifecycle and warm-up. Idle leases cover status, warm-up, and transcription; a new worker request preloads the recognizer and the configurable idle policy terminates the utility process.
- Activation modes. Settings persist toggle, hold, and hybrid behavior, including the Linux portal release path and hybrid tap-versus-hold timing.
- Dictionary and UI. Shared validation and replacement logic apply to global dictation and composer microphone transcripts; Voice settings and onboarding copy expose the feature.
GPT Luna ｜ 𝕏
There was a problem hiding this comment.
✅ No new issues found.
Reviewed changes I reviewed the two new commits since the previous Pullfrog review (6599de34, b43faa61) plus their interaction with the rest of the PR, and re-ran the focused dictation suites locally.
- Hybrid fallback for unmappable shortcuts.
beginHoldWatch()now latches toggle with a pill hint when the shortcut has no key-code mapping or no watch can start, and OS key repeats while held are ignored; regression test covers theCommand+F13class of shortcut. - Leased recognizer release.
releaseRecognizerruns inside the transcription lane under an idle-unloader lease, so an expiring idle timer can no longer dispose the worker mid-request and fail model deletion. - Locale-independent dictionary matching.
matchKeyusestoLowerCase()to agree with the/iumatcher; the TurkishIĞDIRregression test covers it. - CLI speech worker warm handling. The bundled worker now answers
warmrequests explicitly instead of falling through to the PCM16 branch. - Docs and CI registry. Fix summary recorded in
.memory, and the four new test files were added toscripts/ci-test-registry.jsonlanes.
All three open threads from the previous review were verified as addressed against the new code and resolved. Several dictation settings toggles wired to composer/consume paths remain inaccessible in this validated clone on Linux (native helpers are macOS build products, which is a pre-existing constraint noted in the plan's pending real-hardware acceptance).
GLM Flash ｜ 𝕏
There was a problem hiding this comment.
✅ No new issues found.
Reviewed changes A new commit (8c2f386e) landed while the previous incremental review was being finalized, so I re-ran the incremental flow against the new head.
- Dictionary replacement selection reworked. The pattern now keeps each rule's original Unicode spelling inside one capture group per rule (sorted longest-first), and the replace callback picks the rule owning the first defined capture. This removes the earlier locale-map round-trip entirely: dotted Turkish
İ, longſ, and Greek finalςmatches now select the correct replacement via the regex engine's/iucase folding, covered by the new regression test. - I verified the capture-selection edge behavior directly with additional probes: rules sharing a prefix never leak each other's captures, identical-spelling duplicates keep the first entry, and same-key-different-case duplicates behave as before.
Focused dictionary suite re-run at this head: 9 passed.
GLM Flash ｜ 𝕏
|
Pullfrog stalled The agent stopped emitting events for 125s and was killed by the activity-timeout watchdog. The model produced no output at all before the stall — the request was sent but nothing came back. This is usually transient; re-running often succeeds. Recent agent stderr
|
Hermes Review BotConfidence: 5 Engine: SummaryThis change introduces the first slice of Handy P1 dictation capabilities: lease-tracked idle unloading and on-demand model warm-up for the on-device Parakeet engine, a hybrid "Tap or hold" activation mode alongside hold and toggle, and a custom dictionary applied to final transcripts across both global dictation and composer recordings. Model memory is reclaimed by terminating the utility process after a configurable quiet period (defaulting to 10 minutes), while warm-up preloads recognizers in the serialized transcription lane on hotkey press or microphone start. The hybrid activation mode captures timestamps synchronously at call time rather than dequeue time, latching capture on if released within 300 ms and operating as push-to-talk for longer holds. Dictionary parsing and replacement enforce Unicode word boundaries, preserve exact replacement casing without re-scanning, and de-duplicate spoken terms case-insensitively. Maintainers should double-check physical macOS hardware behavior when using global shortcuts during cold window launches to verify that the 300 ms tap threshold feels comfortable under default OS key repeat rates. Confidence Score: 5/55/5 — All changed control flows, async concurrency races, protocol extensions, dictionary regex mechanics, and platform capabilities were traced end-to-end through callers, callees, and tests. 📁 Important Files Changed
FindingsNo findings. 📊 Sequence DiagramsequenceDiagram
autonumber
actor User
participant Coord as DictationCoordinator
participant Lane as TranscriptionLane
participant Unload as ParakeetIdleUnloader
participant Worker as Parakeet Worker
User->>Coord: press() [shortcut down]
Note over Coord: Stamp pressedAt = now()<br/>Start release watch (hybrid)
Coord->>Lane: warmLocalVoice()
Lane->>Unload: begin() [lease++]
Lane->>Worker: postMessage({ kind: "warm", modelId })
Worker-->>Lane: result (model loaded in RAM)
Lane->>Unload: end() [lease-- -> arm(idleDelay)]
alt Quick tap (< 300 ms)
User->>Coord: release() [shortcut up]
Note over Coord: releasedAt - pressedAt < 300ms<br/>Latch recording = true (toggle behavior)
User->>Coord: press() [shortcut down again]
Note over Coord: Stop recording -> transcribing
else Long hold (>= 300 ms)
User->>Coord: release() [shortcut up]
Note over Coord: Grace window expires -> transcribing
end
Coord->>Lane: transcribePcmBase64()
Lane->>Unload: begin() [cancels idle timer, lease++]
Lane->>Worker: postMessage({ kind: "transcribe" })
Worker-->>Lane: text
Lane->>Unload: end() [lease-- -> arm(idleDelay)]
Coord->>Coord: applyDictationDictionary(text)
Coord-->>User: paste(correctedTranscript)
Note over Unload: Idle timer expires after quiet period
Unload->>Worker: dispose() [kill utilityProcess]
Machine-Readable Findings[]
|
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
There was a problem hiding this comment.
✅ No new issues found.
Reviewed changes I reviewed the two commits since the previous Pullfrog review (7abc4a9, fec82e8) and checked them against the full PR context.
- Aligned dictionary row keys. Switched the settings list to locale-independent lowercase keys, matching dictionary deduplication, and added a Turkish-casing regression for distinct
Iand dotless-ıentries. - Removed branch-specific troubleshooting notes. Dropped the two papercut entries introduced by this branch.
GPT Luna ｜ 𝕏
# Conflicts: # docs/plans/README.md # scripts/ci-test-registry.json
# Conflicts: # docs/plans/README.md # package.json
There was a problem hiding this comment.
✅ No new issues found.
Reviewed changes I reviewed the fcca8ab merge from current main and traced its dictation delivery and composer changes against the full PR diff.
- Secure Input delivery integration. The coordinator preserves paste-result reasons and messages, allowing the pill to explain manual paste and keep Secure Input warnings visible longer.
- Composer queue integration. The submission gate now follows
composerInputLocked, matching the editable composer behavior when messages can queue during compaction. - Plan and test-script conflict resolutions. The merge retains current-main entries and tests alongside the dictation plan row and focused suites.
Validation passed: npm run test:voice (152 passed, 3 skipped), composer tests (21 passed), and git diff --check.
GPT Luna ｜ 𝕏
2 participants
Summary
This PR is the first slice of the Handy P1 dictation work. It adds three things:
VAD, history and mute are out of scope and deferred.
Source: the user's request "Parakeet idle unload / warm-up, then dictation activation modes + custom dictionary (Handy P1); VAD / history / mute later."
Plan:
docs/plans/dictation-parakeet-modes-plan.mdChanges by surface
Main process
parakeet-idle-unload.ts(new)Parakeet protocol, worker, process core and engine
warmrequest, serialized through the existing transcription lane.dictation-coordinator.tsdictation.tssettings:setdictationActivationMode, which also setsdictationHoldToTalk, so the Linux portal hold binding keeps working.dictationDictionaryandlocalVoiceIdleUnloadMinutes, and re-arms the idle timer when the period changes.localVoice:warmIPCruntimeSettingsFromRenderer
Settings → Voice
role="alert".Composer mic
Onboarding
Docs
docs/plans/README.md..memory/dictation-parakeet-modes.md..papercuts/troubleshooting.md.Tests run
npm run test:voice: 139 passed, 0 failed. New files are registered in this script:parakeet-idle-unload.test.tsdictation-dictionary.test.tsdictation-preferences.test.tsdictation-dictionary-settings.test.tsxdictation-coordinator.test.ts.main/services/config-store-core.test.tsandmain/services/dictation-coordinator.test.ts: 128 passed. This includes a new test for normalizing hand-edited preferences.npm run test:onboarding: 64 passed.npm run test:settings-design: 60 passed.renderer/components/composer.test.tsxanddictation-hold-settings.test.ts: 25 passed.renderer/components/environment-subagents-contract.test.ts: 28 passed. I replaced its source-grep assertions ondictation-shortcut-settings.tsxwith behavioral checks ofresolveDictationActivationMode.npm run type-check, scoped ESLint andgit diff --checkall pass.Follow-ups
🤖 Generated with Claude Code