diff --git a/CLAUDE.md b/CLAUDE.md index 2acba11..61e9ceb 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -105,6 +105,9 @@ pnpm test # Run tests | `ChatMessage` | Renders individual messages with URL linking | | `ChatSources` | Slide-out sidebar displaying grouped source links | | `SourceIcon` | Icon button with badge count to trigger source sidebar | +| `VoiceInputButton` | Mic button for dictation; toggle and hold-to-talk gestures | +| `VoiceLevel` | Voice-activity indicator shown inside the input while listening | +| `MessageSpeechControls` | Read aloud / pause / resume / stop buttons under assistant replies | | `AttachmentPreview` | Image thumbnail / file chip for pending and sent attachments | ### useChat Hook @@ -173,6 +176,25 @@ renders. Retrieval failures degrade to ungrounded replies. Ingestion: `pnpm rag:ingest ./content --index ` (chunking helpers in `scripts/lib/rag-ingest.ts`). Docs: rag/index.md. +### Voice + +Voice is widget-only and opt-in (`voice` prop / `ClaudiusConfig.voice` / +``), built on the browser's Web Speech API with no dependencies. +`useSpeechRecognition` (owned by `ChatInput`) runs one single-utterance session +at a time and streams interim results into the field; `useSpeechSynthesis` +(owned by `ChatWindow`) reads one settled assistant reply at a time as +sentence-sized utterances. Pure helpers live in `widget/src/utils/voice.ts` +(`resolveVoiceConfig`, `resolveSpeechLang`, `joinDictation`, +`chunkSpeechText`, `voiceOptionsFromAttributes`). Each half is feature +detected separately and renders nothing when unsupported. jsdom has no speech +engines, so tests drive `widget/src/test-utils/fakeSpeech*.ts`. + +Privacy: the widget never touches audio (no `getUserMedia`) and none reaches +the worker, but browsers recognize speech on their vendor's servers by +default. Do not describe it as on-device. Docs: configuration/voice.md; +design: docs/plans/2026-09-18-voice-input-tts-design.md. + ### Chat Request/Response ```typescript diff --git a/clients/_schema.json b/clients/_schema.json index f5c56e2..8e76e01 100644 --- a/clients/_schema.json +++ b/clients/_schema.json @@ -62,6 +62,23 @@ } } ] + }, + "voice": { + "description": "Voice input (mic button) and read-aloud via the browser's Web Speech API. true enables both with the defaults (click-to-toggle mic, no auto-submit).", + "oneOf": [ + { "type": "boolean" }, + { + "type": "object", + "additionalProperties": false, + "properties": { + "input": { "type": "boolean" }, + "output": { "type": "boolean" }, + "mode": { "type": "string", "enum": ["toggle", "hold"] }, + "autoSubmit": { "type": "boolean" }, + "lang": { "type": "string", "minLength": 1 } + } + } + ] } } }, diff --git a/docs/plans/2026-09-18-voice-input-tts-design.md b/docs/plans/2026-09-18-voice-input-tts-design.md new file mode 100644 index 0000000..3f5b320 --- /dev/null +++ b/docs/plans/2026-09-18-voice-input-tts-design.md @@ -0,0 +1,324 @@ +# Voice input and text-to-speech playback: design + +Issue: [#54](https://github.com/PMDevSolutions/Claudius/issues/54) +Date: 2026-09-18 +Status: implemented in the same PR as this document + +## Summary + +Add an opt-in `voice` option to the widget. It puts a mic button in the +composer that dictates into the message field through the browser's +`SpeechRecognition`, and a read-aloud control under each completed assistant +message that plays it through `speechSynthesis`. Both halves are feature +detected independently and render nothing where the browser lacks them. + +The feature is widget-only. No worker, API, or wire-format change. + +## Facts that constrain the design + +These were checked against MDN and the MDN browser-compat data on 2026-09-18, +because several of them contradict common assumptions (including one in the +issue). + +1. **Recognition audio usually leaves the device.** MDN: "By default, using + speech recognition on a web page involves a server-based recognition + engine. Your audio is sent to a web service for recognition processing." + Chrome sends audio to Google, Safari hands it to Apple's dictation service. + The issue's acceptance criterion ("speech is processed on-device by the + browser; no audio leaves the client") is therefore **not true** and must + not be documented as written. What *is* true, and what we document: the + widget never requests microphone access itself, never sees audio, and + never sends audio to the Claudius worker. It receives text from the + browser, exactly as if it had been typed. +2. **On-device recognition exists but is not ready to build on.** + `SpeechRecognition.processLocally` shipped in desktop Chrome 139. It is + marked experimental, is unsupported on Chrome for Android, Safari, and + Firefox, needs a language-pack download flow (`available()` / + `install()`), and has open bugs where `available()` hangs. Out of scope + here; listed under follow-ups. +3. **The API exposes no audio signal.** There is no level, waveform, or + sample access. A true amplitude meter needs a second microphone capture + through `getUserMedia` plus a Web Audio `AnalyserNode`. +4. **Support matrix.** Recognition: Chrome and Edge 33+ (prefixed; unprefixed + from 139), Safari 14.1+ and iOS Safari 14.5+ (prefixed), Firefox only + behind a flag. Synthesis: every current browser except Android WebView. + Some Chromium forks (Brave, some in-app browsers) expose the constructor + but fail at `start()` with a `network` or `service-not-allowed` error, so + detection alone is not enough: start-time errors need a visible message. +5. **`speechSynthesis.pause()` is `cancel()` on Android.** Compat data: "In + Android, `pause()` ends the current utterance." +6. **Chrome cuts long utterances off** at roughly 15 seconds when a network + voice is used, and then fires no `end` event. +7. **Network voices send the text being read to the voice vendor.** + `SpeechSynthesisVoice.localService` distinguishes them. + +## Approaches considered + +**A. Two small hooks over the browser APIs, no dependencies (chosen).** +`useSpeechRecognition` and `useSpeechSynthesis` wrap the platform, pure +helpers hold the logic worth unit testing, and the UI stays thin. Smallest +bundle cost, no global state, works with multiple widget instances. + +**B. A library** (`react-speech-recognition`, `react-speech-kit`). Rejected: +more bytes than writing it directly, singleton state that fights multiple +instances, and nothing for the Android pause or 15-second problems. + +**C. Server-side STT/TTS through the worker.** Would cover Firefox, but sends +audio to our worker, which is the opposite of the privacy goal, and adds +cost, latency, and a large worker surface. Rejected for this issue. + +## Public API + +```ts +type VoiceInputMode = "toggle" | "hold"; + +interface VoiceOptions { + input?: boolean; // mic button. Default true + output?: boolean; // read-aloud control. Default true + mode?: VoiceInputMode; // Default "toggle" + autoSubmit?: boolean; // send when dictation ends. Default false + lang?: string; // BCP-47. Default: derived from the widget locale +} + +// ChatWidgetProps / window.ClaudiusConfig +voice?: boolean | VoiceOptions; // Default false +``` + +- **Off by default**, like `attachments`. A minor release must not add a + microphone button to every existing embed, and turning it on has privacy + implications the site owner should choose deliberately. +- `voice: true` enables both halves. `output: false` makes the speaker + optional, as the issue asks. `input: false` gives a read-aloud-only widget. + Both `false` resolves to disabled. +- An unknown `mode` falls back to `"toggle"` rather than throwing, since it + can arrive from an HTML attribute. +- `` gains `voice`, `voice-mode`, `voice-auto-submit`, + `voice-input`, `voice-output`, and `voice-lang`. Unlike attachment limits + these are plain enums, booleans, and strings, and the web component cannot + read `ClaudiusConfig`, so without them "configurable per widget" would not + hold for that embed style. +- `clients/_schema.json`, CLI validation, and both snippet generators accept + `widget.voice`. + +### Language + +`resolveSpeechLang(locale, override)` returns, in order: + +1. `voice.lang` when set. +2. `` when it carries a region and its primary subtag matches the + widget locale (`en-GB` on an English widget). +3. `navigator.language` under the same rule. +4. A default per built-in locale: `en-US`, `es-ES`, `fr-FR`, `de-DE`. + +Steps 2 and 3 mirror `detectLocale()` and mean a UK site gets British +recognition and a British voice with no configuration. With nothing +detectable the result is `en-US`, as the issue specifies. + +## Architecture + +``` +ChatWidget resolveVoiceConfig(voice, locale ?? detectLocale()) + ChatWindow useSpeechSynthesis(lang) one reader per window + ChatMessage completed assistant messages + ChatInput useSpeechRecognition(lang) + +``` + +| Unit | Responsibility | +|------|----------------| +| `utils/voice.ts` | `resolveVoiceConfig`, `resolveSpeechLang`, `joinDictation`, `chunkSpeechText`, `voiceOptionsFromAttributes`. Pure. | +| `hooks/useSpeechRecognition.ts` | One recognition session at a time: start, stop, abort, transcript, error mapping. Minimal local typings, since TypeScript's DOM lib has none for recognition. | +| `hooks/useSpeechSynthesis.ts` | Reads one message at a time: chunk queue, pause, resume, cancel. | +| `components/VoiceInputButton.tsx` | The mic button and its toggle / hold gesture handling. Presentational. | +| `components/VoiceLevel.tsx` | The listening indicator. Presentational. | +| `components/MessageSpeechControls.tsx` | Read aloud / pause / resume / stop buttons. Presentational. | + +`ChatInput` owns the recognition hook because the transcript has to land in +its controlled field. `ChatWindow` owns the synthesis hook because only one +message may speak at a time and it already holds `streamingMessageId`. + +## Behavior + +### Dictation + +- A **new recognition instance per session**, created in `start()`. Reusing + one after an error is unreliable in Safari. +- `interimResults = true` so words appear in the field as they are spoken. + `continuous = false` in both modes. Continuous mode duplicates results on + Chrome for Android and misreports `isFinal` in Safari; a single utterance + per session is what a chat message needs. If the engine ends the session + during a pause, pressing the mic again appends. +- Dictation **appends** to whatever is already typed. The field value at + `start()` is the base; each result renders `joinDictation(base, transcript)` + clipped to the existing 2000 character limit. +- **Typing takes over.** A manual edit while listening keeps the edited text + and aborts the session, so a late result cannot overwrite it. +- `isListening` turns true at `start()`, not at the `start` event. That gives + immediate feedback and guards against a double start while the permission + prompt is open. +- **Toggle mode:** click starts, click stops. The session also ends by itself + after the utterance. While listening the button shows a stop square, the + same swap the send button makes while a reply streams. A solid mic next to + the solid send button read as two send buttons. +- The field is kept scrolled to its end while dictating. It is not focused, + so the browser does not scroll it, and a long dictation would otherwise show + only its first words. +- **Hold mode:** pointer down starts and pointer up, cancel, or lost capture + stops; Space and Enter work the same way from the keyboard. Releasing + before the engine has actually started calls `abort()` rather than + `stop()`, so a permission prompt answered after release can never leave the + microphone open. A `click` with no preceding pointer or key press, which is + how screen readers and switch devices activate a button, toggles instead, + so hold mode stays usable without a sustained press. +- **Auto-submit** fires only when the session ended normally and produced + final text. Never after an error, an abort, or silence. It goes through the + same submit path as the send button, so pending attachments, the length + limit, and plugins all apply. +- Starting dictation **cancels any read-aloud** so the microphone does not + transcribe the widget's own voice. +- Unmounting (closing the chat) aborts the session. + +### Listening indicator + +Animated bars sit inside the right edge of the message field while listening, +and the placeholder reads "Listening...". The bars have two states, idle and +active, driven by the engine's `speechstart` / `speechend` events and by +arriving results. + +This is a voice-activity indicator, **not an amplitude meter**, and that is a +deliberate deviation from a literal reading of "waveform / level indicator". +A true meter needs the second `getUserMedia` capture described above, which +(a) conflicts with the recognizer's own capture on Android and iOS, the +platforms the issue cares most about, (b) adds a second permission prompt in +Safari, and (c) would give widget code access to raw audio, which would make +the privacy statement in the next section untrue. The indicator still +reflects whether speech is being heard, which is the feedback a user needs. + +Under `prefers-reduced-motion` the bars do not animate; the two states differ +in opacity. + +### Read aloud + +- Offered on assistant messages that have settled (not the streaming one) and + have text. Nothing plays automatically. +- Text passes through the existing `stripAnnouncementFormatting`, so markdown + markers are not read out and URLs collapse to hostnames. +- The text is split at sentence boundaries into chunks of at most 200 + characters and queued as separate utterances. That avoids Chrome's + 15-second cutoff and gives clean pause points. Utterances are held in a ref + because Safari garbage-collects them mid-speech otherwise. +- **The voice is left to the browser**: only `utterance.lang` is set. An + earlier draft picked a local voice explicitly, for privacy and to avoid the + cutoff. That was dropped. Voice lists are alphabetical, so "first local + en-US voice" is the novelty voice "Albert" on a Mac whose system language is + not English; in Edge it would swap the Natural voices for robotic ones; and + Safari and Firefox voices are all local already, so there was nothing to + gain there. Chunking already handles the cutoff, and the docs state plainly + that a network voice sends the reply text to the vendor. +- `speak()` cancels the engine only when something is playing or queued. That + still clears a queue Chrome left stuck, without sending an idle engine a + `cancel()` immediately before `speak()`, which swallows the new utterance + in some browsers. Verified against Chrome 153: with no voices installed + `speak()` fires `error: synthesis-failed`, which returns the control to + idle rather than leaving it stuck. +- Controls: idle shows Read aloud; speaking shows Pause and Stop; paused + shows Resume and Stop. Starting another message stops the current one. +- A session counter makes late events from cancelled utterances harmless. +- **Android pause:** if a chunk ends while a pause was requested and the + engine does not report `paused`, the platform treated pause as cancel. The + rest of the queue is cancelled and the control returns to idle, instead of + the next sentence starting to play after the user pressed pause. + +## Errors + +Recognition errors map to four translated messages shown in the composer's +existing `role="alert"` slot: + +| Engine code | Message | +|-------------|---------| +| `not-allowed`, `service-not-allowed` | Microphone access is blocked | +| `no-speech` | No speech was detected | +| `audio-capture` | No microphone was found | +| `network`, `language-not-supported`, anything else, `start()` throwing | Voice input is unavailable | +| `aborted` | ignored | + +The message clears on the next start or edit. Synthesis errors return the +control to idle silently; there is nothing useful to tell the user. + +## Accessibility + +- The mic is a real `button` with a constant label and `aria-pressed`, rather + than a label that changes with state, to avoid announcing the state twice. +- No live "listening" announcement: a screen reader speaking at that moment + would be transcribed. +- Indicator bars are `aria-hidden`. State is carried by `aria-pressed` and + the placeholder. +- Composer button is 44 px like its neighbours; message controls are 28 px, + above the WCAG 2.5.8 minimum. +- Hold mode has the click fallback described above (WCAG 2.5.1). +- New buttons are picked up by the existing focus trap, which queries + focusable elements on each Tab. + +## Strings + +Eleven keys, translated in en, es, fr, and de (the parity test enforces it): +`voiceInput`, `voiceInputHold`, `voiceListening`, `voicePermissionDenied`, +`voiceNoSpeech`, `voiceNoMicrophone`, `voiceUnavailable`, `readAloud`, +`pauseReading`, `resumeReading`, `stopReading`. + +## Testing + +Test-first, with fake `SpeechRecognition` and `speechSynthesis` +implementations that tests drive event by event. + +- `utils/voice`: config resolution, language resolution, joining, chunking, + attribute parsing. +- `useSpeechRecognition`: unsupported no-op, session setup, interim and final + transcripts, stop versus abort, each error mapping, `start()` throwing, + double start, unmount. +- `useSpeechSynthesis`: chunk queue with language and voice, state from + events, pause and resume, the Android pause path, switching messages, stale + events after cancel, unmount, unsupported no-op. +- Components: visibility rules, both modes by pointer, keyboard, and bare + click, appending, typing takes over, auto-submit and the cases where it + must not fire, error display, read-aloud only on settled assistant + messages, dictation cancels read-aloud. +- Embed attributes, client-config validation, snippets. +- One Playwright spec with injected fakes to prove the wiring in a real + browser, including the one behaviour jsdom cannot show (keeping a long + dictation scrolled into view). Real engines cannot run in CI: no microphone, and Chromium builds ship + without the recognition service key. + +## Documentation + +New `configuration/voice.md`: enabling, options, browser support table, where +voices come from per browser and how to list them, and the privacy posture as +stated under fact 1. Updates to the widget and clients configuration pages, +the FAQ, and `CLAUDE.md`. + +## Bundle budget + +Everything ships in the main bundle: the IIFE build cannot code-split. +Budgets are raised to measured size plus 5%, in their own commit, per +CONTRIBUTING. + +## Acceptance criteria + +| Criterion | Status | +|-----------|--------| +| Mic visible only when recognition is supported | Met | +| Hold and toggle modes, configurable per widget | Met, across all three embed styles | +| Inline waveform / level indicator | Met as a voice-activity indicator, not an amplitude meter. See Listening indicator | +| Auto-submit, off by default | Met, as a configuration option | +| Speaker with cancel and pause | Met. Pause degrades to stop on Android, a platform limit | +| Locale follows the widget, `en-US` default; voices documented | Met | +| Documented privacy posture | Met, **with corrected wording**. The issue's claim is inaccurate. See fact 1 | +| Graceful no-op when unsupported | Met | + +## Out of scope, possible follow-ups + +- `onDevice` option built on `processLocally`, including the language-pack + install flow, once the API is stable beyond desktop Chrome. +- Opt-in amplitude meter for desktop browsers. +- Auto-reading new replies, voice / rate / pitch selection, read-aloud for + the welcome message. diff --git a/docs/src/content/docs/configuration/clients.md b/docs/src/content/docs/configuration/clients.md index 31bff65..2b3d27b 100644 --- a/docs/src/content/docs/configuration/clients.md +++ b/docs/src/content/docs/configuration/clients.md @@ -47,7 +47,7 @@ pnpm claudius snippet acme # generate the embed snippet(s) | `slug` | Yes | URL-safe identifier; must match the filename | | `apiUrl` | Yes | The client's worker chat endpoint | | `allowedDomains` | Yes | Domains where the widget may be embedded | -| `widget` | No | Appearance: `title`, `subtitle`, `welcomeMessage`, `placeholder`, `theme`, `position`, `accentColor`; `attachments` (`true` or `{ maxSizeBytes, maxCount, allowedTypes }`) | +| `widget` | No | Appearance: `title`, `subtitle`, `welcomeMessage`, `placeholder`, `theme`, `position`, `accentColor`; `attachments` (`true` or `{ maxSizeBytes, maxCount, allowedTypes }`); `voice` (`true` or `{ input, output, mode, autoSubmit, lang }`, see [Voice](/configuration/voice/)) | | `worker` | No | `model`, `maxTokens` (1–8192), `rateLimitMinute`, `rateLimitHour`, `systemPrompt` (path to a markdown file), `attachments` (`enabled`, `maxBytes`, `maxCount`, `maxRequestBytes`, `allowedTypes`, `storage`, `retentionHours`, `quotaIpBytesPerDay`, `quotaTenantBytesPerDay`) | `widget.attachments` flows into the generated embed snippet. `worker.attachments` diff --git a/docs/src/content/docs/configuration/localization.md b/docs/src/content/docs/configuration/localization.md index e625a15..5c078e3 100644 --- a/docs/src/content/docs/configuration/localization.md +++ b/docs/src/content/docs/configuration/localization.md @@ -49,6 +49,7 @@ window.ClaudiusConfig = { | Input | `placeholder`, `sendMessage`, `typeYourMessage` | | Toggle / bubble | `openChat`, `dismissGreeting` | | Errors | `errorGeneric`, `errorConnection`, `errorTimeout`, `errorRateLimitMinute`, `errorRateLimitHour`, `errorRetry` | +| [Voice](/configuration/voice/) | `voiceInput`, `voiceInputHold`, `voiceListening`, `voicePermissionDenied`, `voiceNoSpeech`, `voiceNoMicrophone`, `voiceUnavailable`, `readAloud`, `pauseReading`, `resumeReading`, `stopReading` | Note these localize the widget UI only. The AI's reply language follows the conversation and your [system prompt](/configuration/worker/#system-prompt) — diff --git a/docs/src/content/docs/configuration/voice.md b/docs/src/content/docs/configuration/voice.md new file mode 100644 index 0000000..1dadf8c --- /dev/null +++ b/docs/src/content/docs/configuration/voice.md @@ -0,0 +1,247 @@ +--- +title: Voice +description: Let visitors dictate messages and have replies read aloud, and understand where the audio goes. +sidebar: + order: 8 +--- + +Voice adds two controls, both built on the browser's +[Web Speech API](https://developer.mozilla.org/en-US/docs/Web/API/Web_Speech_API): + +- A **mic button** in the composer. Visitors dictate instead of typing, and + the words appear in the message field as they speak. +- A **read-aloud button** under each completed assistant reply, with pause and + stop. + +Voice is **off by default**. Each half appears only in browsers that support +it, so enabling voice is safe everywhere: an unsupported browser simply shows +the normal text chat. Nothing changes on the worker. + +Read [Privacy posture](#privacy-posture) before you switch it on. The widget +never touches audio, but most browsers send it to their vendor for +recognition, and your privacy policy may need to say so. + +## Enable voice + +```tsx + +``` + +```html + +``` + +```html + +``` + +`true` turns on both the mic and read-aloud. Pass an object to tune them: + +| Option | Default | Description | +|--------|---------|-------------| +| `input` | `true` | Show the mic button | +| `output` | `true` | Show the read-aloud button on assistant replies | +| `mode` | `"toggle"` | `"toggle"`: click to start, click again to stop. `"hold"`: record only while the button is held | +| `autoSubmit` | `false` | Send the message as soon as dictation ends | +| `lang` | from the widget locale | BCP-47 tag for recognition and playback, e.g. `"en-GB"`. See [Language](#language) | + +```tsx + +``` + +The web component takes the same options as attributes: + +| Attribute | Example | +|-----------|---------| +| `voice` | `voice` or `voice="true"` to enable, `voice="false"` to disable | +| `voice-mode` | `voice-mode="hold"` | +| `voice-auto-submit` | `voice-auto-submit` or `voice-auto-submit="false"` | +| `voice-input` | `voice-input="false"` for read-aloud only | +| `voice-output` | `voice-output="false"` for dictation only | +| `voice-lang` | `voice-lang="en-GB"` | + +In a [client config](/configuration/clients/), set `widget.voice` to `true` or +to the same object; `pnpm claudius snippet` carries it into both embed styles. + +## Dictation + +Dictated text is **appended** to whatever is already in the field, so a +visitor can type part of a message and speak the rest. Recognition stops by +itself at the end of a sentence; pressing the mic again adds more. If the +visitor types while the mic is listening, typing wins: their edit is kept and +the session ends, so a late result can never overwrite it. + +Without `autoSubmit`, the text stays in the field to be checked and sent by +hand. Speech recognition makes mistakes, which is why this is the default. +With `autoSubmit`, the message goes out when dictation ends, but only if the +session finished normally and actually heard something. It never fires after +an error or a silent session. It uses the same path as the send button, so +pending [attachments](/configuration/attachments/), the length limit, and +[plugins](/plugins/) all apply. + +Starting dictation stops any reply that is being read aloud, so the +microphone does not transcribe the widget's own voice. + +### Toggle or hold + +`"toggle"` is the default and the more accessible choice, because it needs no +sustained press. While listening, the button shows a stop icon. + +`"hold"` is push-to-talk: recording runs from press to release, with a mouse, +a finger, or Space / Enter on the keyboard. If the visitor lets go before the +browser's permission prompt has been answered, nothing starts, so granting +permission afterwards can never open the microphone with nobody at the +button. Screen readers and switch devices activate buttons with a click they +cannot hold, so in hold mode a click of that kind toggles instead. + +### Listening indicator + +While the mic is open, animated bars appear inside the message field and the +placeholder reads "Listening...". The bars idle while the mic waits and liven +up while speech is being heard. + +They show **voice activity, not loudness**. The Web Speech API gives a page no +access to the audio signal. A true level meter would mean opening the +microphone a second time through `getUserMedia`, which conflicts with the +recognizer on Android and iOS, prompts for permission twice in Safari, and +would hand raw audio to the widget. Claudius deliberately does not do that: +see the first point under [Privacy posture](#privacy-posture). + +With `prefers-reduced-motion`, the bars hold still and the two states differ +in opacity only. + +## Read aloud + +The button appears once a reply has finished streaming. Formatting marks are +not spoken and links are shortened to their hostname, the same cleanup used +for screen-reader announcements. One reply plays at a time; starting another +stops the first, and closing the chat stops playback. + +Long replies are queued sentence by sentence. Chrome cuts a single long +utterance off after about 15 seconds when it is using a network voice, and +short utterances avoid that. + +On **Android**, browsers implement "pause" as "stop". Claudius detects this +and returns the control to its idle state, rather than carrying on with the +next sentence after the visitor pressed pause. + +## Language + +Recognition and playback use one BCP-47 language tag, chosen in this order: + +1. `voice.lang`, when you set it. +2. The page's ``, if it names a region of the widget's language. + An English widget on an `en-GB` page listens for, and speaks, British + English. +3. The browser's language, under the same rule. +4. A default for the widget [locale](/configuration/localization/): + +| Locale | Default speech language | +|--------|-------------------------| +| `en` | `en-US` | +| `es` | `es-ES` | +| `fr` | `fr-FR` | +| `de` | `de-DE` | + +Tags in a different language from the widget are ignored, so a French browser +visiting a Spanish widget still gets `es-ES`. Set `voice.lang` for anything +more specific, such as `es-MX` or `pt-BR`. + +## Browser support + +| Browser | Dictation | Read aloud | Notes | +|---------|-----------|------------|-------| +| Chrome, desktop and Android | Yes | Yes | Needs a network connection | +| Edge | Yes | Yes | | +| Safari 14.1+ on macOS, 14.5+ on iOS and iPadOS | Yes | Yes | The visitor must have Siri enabled in system settings | +| Firefox | No | Yes | Recognition exists only behind a flag. The mic is hidden; read-aloud works | +| Brave, and many in-app browsers | No | Varies | They expose the API but cannot reach a speech service. The mic appears, and using it shows "Voice input is not available right now." | +| Android WebView | Varies | No | | + +Dictation also requires a secure page (HTTPS, or `localhost` in development). + +### Voices + +Claudius does not pick a voice. It sets the language and lets the browser +choose, because the available voices depend on the visitor's browser *and* +operating system, and the browser's own choice is the most reliable one: + +| Browser | Where voices come from | +|---------|------------------------| +| Chrome (desktop) | The operating system's voices, plus Google network voices for about 20 languages | +| Edge | The operating system's voices, plus Microsoft's online "Natural" voices | +| Safari, and every browser on iOS | Apple's system voices | +| Firefox | The operating system's voices | +| Chrome (Android) | The device's text-to-speech engine, usually Google's | + +To see exactly what a given browser offers, run this in its console: + +```js +speechSynthesis.getVoices().map((v) => `${v.name} (${v.lang})${v.localService ? "" : " - network"}`); +``` + +Chrome loads the list asynchronously, so run it twice if the first result is +empty. A desktop with no voices installed at all, which is common on Linux, +produces no sound: the button returns to idle straight away. + +## Privacy posture + +What Claudius does, and does not do: + +- **The widget never handles audio.** It does not call `getUserMedia`, never + sees a sample of sound, and cannot record. The browser listens, and gives + the widget text, exactly as if it had been typed. +- **No audio reaches your worker or Anthropic.** Only the resulting text is + sent, and only when the message is sent. +- **Nothing is on unless you enable it**, and the microphone opens only while + the visitor is pressing or has toggled the mic button. Closing the chat or + leaving the page ends the session. + +What the browser does is outside Claudius's control, and it is **not +on-device in most browsers**: + +| Browser | Where speech is recognized | +|---------|----------------------------| +| Chrome | Google's servers. [MDN](https://developer.mozilla.org/en-US/docs/Web/API/Web_Speech_API/Using_the_Web_Speech_API): "Your audio is sent to a web service for recognition processing" | +| Edge | The browser vendor's speech service | +| Safari | The same engine as Siri. Apple does not say whether a given request is handled on the device or on its servers, so treat it as leaving the device | + +Read-aloud is similar in one case: when Chrome or Edge choose a **network +voice**, the text of the reply is sent to Google or Microsoft to be +synthesized. System voices are generated on the device. The console snippet +under [Voices](#voices) marks which is which. + +If you enable voice, say so in your privacy policy: that dictation is +processed by the visitor's browser vendor, and name the vendors above. The +browser asks the visitor for microphone permission itself, the first time +they use the mic. + +On-device recognition exists (`SpeechRecognition.processLocally`, in desktop +Chrome 139 and later) but is experimental, is unavailable on Android, Safari, +and Firefox, and requires downloading a language pack first. Claudius does not +use it yet. + +### Site policies that block the microphone + +- A `Permissions-Policy: microphone=()` header disables dictation. Allow your + own origin: `Permissions-Policy: microphone=(self)`. +- If the page hosting the widget is itself inside an `