Skip to content

feat(snippets): add universal keyboard snippets with Dynamic Island & Duo integration - #1170

Open
Balajitechlabs wants to merge 34 commits into
developfrom
feature/ime-snippets
Open

Balajitechlabs wants to merge 34 commits into
developfrom
feature/ime-snippets

Conversation

@Balajitechlabs

@Balajitechlabs Balajitechlabs commented Sep 25, 2026 •

Copy link
Copy Markdown
Collaborator

Description

Adds universal text snippet expansion for external keyboards (Gboard, SwiftKey, physical keyboards) along with display integration across Dynamic Island, Duo camera ring, and a floating pill.

  • Dynamic Island Integration:
    • Displays snippet suggestions as a line peek when trigger keywords are typed.
    • Tap: Inserts the snippet into the active field and copies it to the system clipboard.
    • Long-Press: Expands into a detailed card showing snippet title, keyword badge (!keyword), multi-line text preview, and quick action buttons (Insert, Copy, Dismiss).
    • Isolated touch handling so existing Island gestures and plugins are unaffected.
  • Duo Camera Ring Integration:
    • Displays an animated drop-down pill below the camera cutout with tap-to-insert support.
  • Display Modes & Smart Recommendation:
    • Adds setting to choose where suggestions appear: Floating Pill, Dynamic Island, Duo Ring, or Both.
    • Intelligently recommends the best style based on which display overlays are currently active.
  • Universal External Keyboard Support:
    • Space-based auto-expansion and floating pill positioned dynamically above the keyboard.

Type of change

  • Bug fix
  • New feature
  • UI / visual update
  • Performance improvement
  • Refactoring / cleanup
  • Translations
  • Build / CI tooling

UI changes

Demo of the new snippet UI

Checklist

Before submitting, please make sure:

  • The base branch is set to develop (not main).
  • The app builds cleanly locally with ./gradlew assembleDebug.
  • Unit tests pass with ./gradlew testDebugUnitTest.
  • You tested the changes on a device or emulator (including Shizuku or Root if applicable).
  • New preferences use SettingsRepository for keys and getter/setter methods.
  • New user-facing settings are registered in FeatureRegistry.kt for search indexing.
  • New UI follows the design system in ui/core/ (RoundedCardContainer, IconToggleItem, etc.).
  • Strings are added to res/values/strings.xml and drawables use rounded variants (R.drawable.rounded_*).

- Implement animated drop-down capsule beneath the camera ring with OvershootInterpolator showing keyword and title preview
- Expand Duo touch anchor window to 280dp x (diameter + 72dp) with screen-width boundary clamping
- Prioritize snippet expansion tap with forgiving human touch slop (up to 650ms) and suppress ring gestures during active snippet interaction
…rgonomics

- Dynamically measure active input field bounds directly on text change to position pill comfortably above keyboard
- Add vertical drag gesture support to floating pill frame so users can move it out of the way
- Route SNIPPETS_DISPLAY_BOTH concurrently to Dynamic Island and Duo camera ring while keeping keyboard typing area clear
…tegration

- Implement getRecommendedSnippetsSuggestionDisplayMode() to dynamically evaluate active features (Island, Duo, or both) and default accordingly
- Display (Recommended) badge on matching setup option in SnippetDisplayStyleSheet
- Add Text snippets toggle with gear button and live mode descriptions to Dynamic Island and Duo settings screens
- Streamline SnippetsBottomSheet settings hierarchy
…ard settings

- Add SnippetIslandPlugin to display snippet keyword peeks on Dynamic Island
- Register SnippetIslandPlugin in IslandCoordinator and expose showSnippet/hideSnippet
- Update IslandController to execute custom interactions.onTap on Line stage items
- Wire SnippetAccessibilityHandler lifecycle into ScreenOffAccessibilityService
- Expose Text snippets item in KeyboardSettingsUI to open management bottom sheet
…-collapse

- Allow InteractionOverrides.onLongPress to return Any? to support both Unit and Boolean lambdas
- Prevent premature collapse in IslandController when onLongPress returns true
- Route targeted item long-press events in IslandRoot before falling back to compact gestures
…n tap

- Add localized action strings for insert and copy actions
- Implement SnippetExpanded composable with header badge, preview, and button row
- Wire clipboard copying on tap and long-press expansion in SnippetIslandPlugin
@Balajitechlabs Balajitechlabs added the enhancement New feature or request label Sep 25, 2026
@Balajitechlabs Balajitechlabs self-assigned this Sep 25, 2026
@Balajitechlabs
Balajitechlabs marked this pull request as ready for review September 25, 2026 15:18
@sameerasw
sameerasw force-pushed the develop branch 3 times, most recently from 9fb6cff to a453ce1 Compare October 5, 2026 19:02
@sameerasw
sameerasw marked this pull request as draft October 6, 2026 10:45
@Balajitechlabs
Balajitechlabs marked this pull request as ready for review October 11, 2026 12:51

@sameerasw sameerasw left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Request changes.

Blockers

  1. The feature defaults to on, and the PR adds typeViewTextChanged to the shared accessibility service. Every user's text input in every app starts feeding it with no opt-in. Default it to off.
  2. {clipboard} expands into whatever field has focus. The default !quote snippet uses it, so it can leak clipboard contents into the wrong field.
  3. replaceKeywordInNode rewrites the whole field with ACTION_SET_TEXT.
    • It wipes formatting and spans.
    • It uses lastIndexOf, so it can replace the wrong occurrence when the cursor isn't at the end.
    • It never checks whether performAction succeeded.
    • expandSnippet relies on the stale lastTypedWord.
  4. There is no guard against re-triggering. ACTION_SET_TEXT fires another text-changed event, which can show a suggestion right after expansion.
  5. The accessibility handler also receives events from Essentials' own IME, which already expands on space. The same keystroke can expand twice.
  6. Keyword matching is inconsistent. The accessibility path is case-sensitive and the IME path is case-insensitive.

Serious
7. SnippetRepository can lose data.

  • Writes run concurrently on Dispatchers.IO, so an older save can overwrite a newer one.
  • The file is written in place, so a crash can truncate it.
  • A corrupt file loads as empty, and the next edit overwrites it.
  • Updates should use MutableStateFlow.update.
  1. findMatching matches any prefix and any title substring of two or more characters. Ordinary typing triggers suggestions in every app, and snippets also rank first in the IME suggestions.
  2. SuggestionEngine uses distinctBy { it.text }, which can drop a snippet whose title equals a dictionary word. The spell-checker callback can also keep stale snippet suggestions from the previous word.
  3. showFloatingPill and hideFloatingPill run in separate post blocks on shared state. Rapid events can leave an orphan overlay. activeNode is never released when replaced.
  4. The IslandController change makes Line stage tap and long-press call onTap and onLongPress first. This touches the shared gesture path, so existing plugins need a regression check.
  5. While a Duo snippet suggestion is active, all ring gestures are swallowed. If nothing hides it, the ring stays dead.

Minor

  • Narrating comments and multi-line KDoc on expand and findMatching, against the repo rules.
  • The unrelated intent?. to intent. change in ScreenOffAccessibilityService.
  • isKeyboardPackage uses contains("keyboard"), which can match non-IME apps.
  • Magic numbers for keyboard height and offsets.
  • No tests for the expander, matching or replace logic.

@sameerasw
sameerasw marked this pull request as draft October 11, 2026 14:51
@Balajitechlabs
Balajitechlabs marked this pull request as ready for review October 11, 2026 15:24
@Balajitechlabs

Copy link
Copy Markdown
Collaborator Author

done 👍🏻 check now :}

@sameerasw
sameerasw marked this pull request as draft October 11, 2026 16:01
@Balajitechlabs
Balajitechlabs marked this pull request as ready for review October 18, 2026 06:00
@Balajitechlabs

Copy link
Copy Markdown
Collaborator Author

minor change: should add permission checker and some minor improvements; everything else is fine

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

Status: In review

Development

Successfully merging this pull request may close these issues.

2 participants