Close the deferred minors from the initial review

README.md

@@ -8,12 +8,11 @@ A [Toga](https://toga.beeware.org) widget for editing code, with line numbers an
 pip install toga-code-editor
 ```
 
-The package depends on `toga-core` and `pygments`. Your app installs the Toga backend for its platform as usual; `toga-code-editor` contributes its implementation for that backend through entry points, the same way Toga's own widgets are found.
+The package depends on `pygments` and on `toga-core` 0.5.7 or later in the 0.5 series, because it subclasses Toga backend internals that can change between minor versions. Your app installs the Toga backend for its platform as usual; `toga-code-editor` contributes its implementation for that backend through entry points, the same way Toga's own widgets are found.
 
 ## Usage
 
 ```python
-import toga
 from toga_code_editor import CodeEditor, language_for_filename
 
 editor = CodeEditor(
@@ -34,11 +33,11 @@ editor.language = language_for_filename(path)
 - `show_line_numbers`: shows or hides the gutter. Default `True`.
 - `theme`: a mapping from `TokenKind` to `Style`. `None` selects the built-in `DEFAULT_THEME`.
 
-Unless you give the widget a font family, it uses a monospace font. Autocorrect, smart quotes, smart dashes, auto-capitalization, and spell checking are always off.
+Unless you give the widget a font family, it uses a monospace font. Autocorrect, smart quotes, smart dashes, smart insert and delete, auto-capitalization, and spell checking are always off.
 
 ### Themes
 
-A theme is a plain mapping. Kinds you leave out are drawn in the widget's normal text color.
+A theme is a plain mapping. Kinds you leave out are drawn in the widget's normal text color. `DEFAULT_THEME` is read-only, so unpack it into a new dict to change it, as below.
 
 ```python
 from toga_code_editor import DEFAULT_THEME, Style, TokenKind
@@ -51,15 +50,16 @@ theme = {
 editor.theme = theme
 ```
 
-`Style.color` accepts anything Toga's color properties accept. The token kinds are `KEYWORD`, `BUILTIN`, `DEFINITION`, `DECORATOR`, `STRING`, `NUMBER`, `COMMENT`, `OPERATOR`, `PUNCTUATION`, `TAG`, `ATTRIBUTE`, and `VARIABLE`.
+`Style.color` accepts anything Toga's color properties accept. The token kinds are `KEYWORD`, `BUILTIN`, `DEFINITION`, `DECORATOR`, `STRING`, `NUMBER`, `COMMENT`, `OPERATOR`, `PUNCTUATION`, `TAG`, `ATTRIBUTE`, and `VARIABLE`. Everything else, including plain names, is `TEXT`, which is never styled.
 
 ## Platform notes
 
 - **macOS** uses an `NSRulerView` for the gutter and the system's adaptive color mapping, so theme colors follow dark mode.
+- **Android** 12 and later are supported. The example app is checked on Android 12 and Android 15 emulators.
 - **iOS** and **Android** draw the gutter themselves. Theme colors do not change with the system appearance; the default theme is chosen to be legible on both light and dark backgrounds.
 - Highlighting re-lexes the whole buffer after a short pause in typing. Files of a few thousand lines are fine on a desktop; very large files are slower on phones.
 - Some third-party Android keyboards ignore the flag that disables suggestions.
-- On Android, the widget asks the window to shrink its content when the soft keyboard appears, so the editor scrolls internally and the rest of the layout stays put. It only does this when the app has not chosen a soft-input mode itself.
+- On Android, the widget asks the window to shrink its content when the soft keyboard appears, so the editor scrolls internally and the rest of the layout stays put. It only does this when the app has not chosen a soft-input mode itself. Android deprecated that request in API 30 in favour of handling window insets, so a future release may stop honouring it. With Briefcase's current template, which targets SDK 36 and enables edge-to-edge drawing, the layout still shrinks for the keyboard on Android 12 and Android 15 emulators.
 
 ## Developing
 
@@ -78,7 +78,7 @@ briefcase run iOS
 briefcase run android
 ```
 
-On macOS and iOS the example installs the widget straight from this repository; add `-r` after changing the widget source so Briefcase reinstalls it. On Android the example installs the widget from a wheel in `dist/`, because Gradle rejects a path requirement whose directory contains the Android build tree. Build the wheel before each Android run that should pick up widget changes:
+On macOS and iOS the example installs the widget straight from this repository; add `-r` after changing the widget source so Briefcase reinstalls it. On Android the example installs the widget from a wheel in `dist/`, because Gradle rejects a path requirement whose directory contains the Android build tree. The example's Android requirement names that wheel file, version and all, so a version bump must update `examples/editor/pyproject.toml` too, and a stale wheel in `dist/` ships the old code without complaint. Build the wheel from the repository root before each Android run that should pick up widget changes:
 
 ```console
 uv build --wheel

changes/5.bugfix.md

@@ -0,0 +1 @@
+Closed the minor findings from the initial review: the editor paints once at construction, re-highlight failures are reported immediately, pasted rich text is stripped on macOS and Android, and the default theme is read-only.

examples/editor/pyproject.toml

@@ -19,8 +19,9 @@ author = "Kattni"
 author_email = "hello@kattni.com"
 description = "A code editor demonstrating the toga-code-editor widget."
 # macOS and iOS install the widget from this repository; Android installs it from a
-# built wheel (run `uv build` first), because Gradle rejects a path requirement whose
-# directory contains the Android build tree.
+# built wheel (run `uv build --wheel` from the repository root first), because Gradle
+# rejects a path requirement whose directory contains the Android build tree. The
+# wheel filename below carries the package version, so a version bump must update it.
 requires = []
 
 [tool.briefcase.app.editor]

examples/editor/src/editor/app.py

@@ -2,10 +2,12 @@
 
 import toga
 from toga.constants import COLUMN, ROW
+from toga.platform import current_platform
 
 from toga_code_editor import CodeEditor, language_for_filename
 
 SAMPLES = Path(__file__).parent / "resources" / "samples"
+MOBILE = current_platform in {"iOS", "android"}
 NO_LANGUAGE = "none"
 LANGUAGES = ["python", "json", "html", NO_LANGUAGE]
 
@@ -14,7 +16,11 @@ class Editor(toga.App):
     def startup(self):
         self.editor = CodeEditor(flex=1, on_change=self.on_edit)
         self.sample = toga.Selection(
-            items=sorted(path.name for path in SAMPLES.iterdir()),
+            items=sorted(
+                path.name
+                for path in SAMPLES.iterdir()
+                if path.is_file() and not path.name.startswith(".")
+            ),
             on_change=self.load_sample,
         )
         self.language = toga.Selection(items=LANGUAGES, on_change=self.set_language)
@@ -23,21 +29,15 @@ def startup(self):
         )
         self.status = toga.Label("", flex=1)
 
-        # Two rows so the controls fit a phone's width.
+        # Two rows on mobile, so the controls fit a narrow screen.
+        if MOBILE:
+            rows = [[self.sample, self.language], [self.line_numbers, self.status]]
+        else:
+            rows = [[self.sample, self.language, self.line_numbers, self.status]]
         toolbar = toga.Box(
             children=[
-                toga.Box(
-                    children=[self.sample, self.language],
-                    direction=ROW,
-                    align_items="center",
-                    gap=5,
-                ),
-                toga.Box(
-                    children=[self.line_numbers, self.status],
-                    direction=ROW,
-                    align_items="center",
-                    gap=5,
-                ),
+                toga.Box(children=row, direction=ROW, align_items="center", gap=5)
+                for row in rows
             ],
             direction=COLUMN,
             margin=5,

pyproject.toml

@@ -30,7 +30,7 @@ classifiers = [
     "Topic :: Software Development :: Widget Sets",
 ]
 dependencies = [
-    "toga-core >= 0.5.7",
+    "toga-core >= 0.5.7, < 0.6",
     "pygments >= 2.19",
 ]
 

src/toga_code_editor/code_editor.py

@@ -23,6 +23,27 @@
 """Seconds to wait after the last native edit before re-highlighting."""
 
 
+def _highlighter_for(language: str | None) -> Highlighter:
+    return NullHighlighter() if language is None else PygmentsHighlighter(language)
+
+
+def _theme_for(theme: Theme | None) -> Theme:
+    return DEFAULT_THEME if theme is None else theme
+
+
+def _report_rehighlight_failure(task: asyncio.Task) -> None:
+    # Hand a failure to the loop's exception handler as soon as the task finishes.
+    # Otherwise asyncio only reports it when the task object is garbage collected.
+    if not task.cancelled() and (exc := task.exception()) is not None:
+        task.get_loop().call_exception_handler(
+            {
+                "message": "Re-highlighting the code editor failed",
+                "exception": exc,
+                "task": task,
+            }
+        )
+
+
 class CodeEditor(toga.MultilineTextInput):
     def __init__(
         self,
@@ -61,12 +82,14 @@ def __init__(
         ):
             kwargs["font_family"] = MONOSPACE
 
-        # State the inherited value setter needs before super().__init__() runs.
-        self._highlighter: Highlighter = NullHighlighter()
-        self._language: str | None = None
-        self._theme: Theme = DEFAULT_THEME
+        # Resolve the highlighting state before the widget exists: an unknown
+        # language raises before anything is created, and the inherited constructor
+        # paints the initial value once, with the highlighter and theme in hand.
+        self._highlighter: Highlighter = _highlighter_for(language)
+        self._language = language
+        self._theme: Theme = _theme_for(theme)
         self._show_line_numbers = True
-        self._pending_rehighlight = None
+        self._pending_rehighlight: asyncio.Task | None = None
 
         super().__init__(
             id=id,
@@ -78,8 +101,6 @@ def __init__(
             **kwargs,
         )
 
-        self.theme = theme
-        self.language = language
         self.show_line_numbers = show_line_numbers
 
     @cached_property
@@ -97,7 +118,11 @@ def factory(self):
             return get_factory("toga_code_editor")
 
     def _create(self) -> Any:
-        return self.factory.CodeEditor(interface=self)
+        impl = self.factory.CodeEditor(interface=self)
+        # The backend needs the theme before the inherited constructor sets the
+        # initial value, which paints the first highlights.
+        impl.set_theme(self._theme)
+        return impl
 
     @toga.MultilineTextInput.value.setter
     def value(self, value: object) -> None:
@@ -115,7 +140,7 @@ def language(self) -> str | None:
 
     @language.setter
     def language(self, value: str | None) -> None:
-        highlighter = NullHighlighter() if value is None else PygmentsHighlighter(value)
+        highlighter = _highlighter_for(value)
         self._language = value
         self._highlighter = highlighter
         self._rehighlight()
@@ -127,7 +152,7 @@ def theme(self) -> Theme:
 
     @theme.setter
     def theme(self, value: Theme | None) -> None:
-        self._theme = DEFAULT_THEME if value is None else value
+        self._theme = _theme_for(value)
         self._impl.set_theme(self._theme)
         self._rehighlight()
 
@@ -154,9 +179,9 @@ def _schedule_rehighlight(self) -> None:
         timer added from a native callback would otherwise never fire.
         """
         self._cancel_pending_rehighlight()
-        self._pending_rehighlight = toga.App.app.loop.create_task(
-            self._rehighlight_after_delay()
-        )
+        task = toga.App.app.loop.create_task(self._rehighlight_after_delay())
+        task.add_done_callback(_report_rehighlight_failure)
+        self._pending_rehighlight = task
 
     async def _rehighlight_after_delay(self) -> None:
         await asyncio.sleep(REHIGHLIGHT_DELAY)

src/toga_code_editor/highlighting.py

@@ -3,7 +3,8 @@
 from collections.abc import Iterable, Mapping
 from dataclasses import dataclass
 from enum import StrEnum
-from os import PathLike
+from os import PathLike, fsdecode
+from types import MappingProxyType
 from typing import Protocol
 
 from pygments.lexers import find_lexer_class_for_filename, get_lexer_by_name
@@ -52,35 +53,47 @@ class Span:
 
 @dataclass(frozen=True)
 class Style:
-    """How a token kind is drawn. ``color`` accepts any Toga color value."""
+    """How a token kind is drawn.
 
-    color: Color | str
+    ``color`` accepts any Toga color value, including a string, and is stored as a
+    :class:`~toga.colors.Color`.
+    """
+
+    color: Color
     bold: bool = False
     italic: bool = False
 
-    def __post_init__(self):
-        object.__setattr__(self, "color", Color.parse(self.color))
+    # Written out so the constructor's type is Color | str while the field is Color;
+    # keep its parameters in step with the fields above.
+    def __init__(self, color: Color | str, bold: bool = False, italic: bool = False):
+        object.__setattr__(self, "color", Color.parse(color))
+        object.__setattr__(self, "bold", bold)
+        object.__setattr__(self, "italic", italic)
 
 
 Theme = Mapping[TokenKind, Style]
 """A mapping from token kind to style. Kinds absent from the theme are left unstyled."""
 
-DEFAULT_THEME: Theme = {
-    TokenKind.KEYWORD: Style("#a626a4"),
-    TokenKind.BUILTIN: Style("#0184bc"),
-    TokenKind.DEFINITION: Style("#4078f2", bold=True),
-    TokenKind.DECORATOR: Style("#986801"),
-    TokenKind.STRING: Style("#50a14f"),
-    TokenKind.NUMBER: Style("#986801"),
-    TokenKind.COMMENT: Style("#8a8a8a", italic=True),
-    TokenKind.TAG: Style("#e45649"),
-    TokenKind.ATTRIBUTE: Style("#986801"),
-    TokenKind.VARIABLE: Style("#e45649"),
-}
-
-# Ordered most-specific first. The first row whose Pygments type contains the token
-# wins, so Operator.Word must precede Operator, and the Name.* rows precede nothing
-# more general because bare Name falls through to TEXT.
+# Read-only, so no editor can change the default under another; unpack it with
+# ``{**DEFAULT_THEME, ...}`` to build a custom theme.
+DEFAULT_THEME: Theme = MappingProxyType(
+    {
+        TokenKind.KEYWORD: Style("#a626a4"),
+        TokenKind.BUILTIN: Style("#0184bc"),
+        TokenKind.DEFINITION: Style("#4078f2", bold=True),
+        TokenKind.DECORATOR: Style("#986801"),
+        TokenKind.STRING: Style("#50a14f"),
+        TokenKind.NUMBER: Style("#986801"),
+        TokenKind.COMMENT: Style("#8a8a8a", italic=True),
+        TokenKind.TAG: Style("#e45649"),
+        TokenKind.ATTRIBUTE: Style("#986801"),
+        TokenKind.VARIABLE: Style("#e45649"),
+    }
+)
+
+# Ordered most-specific first: the first row whose Pygments type contains the token
+# wins, so Operator.Word must precede Operator. Anything unmatched, including bare
+# Name, is TEXT.
 _TOKEN_TABLE = [
     (Comment, TokenKind.COMMENT),
     (String, TokenKind.STRING),
@@ -108,10 +121,10 @@ def token_kind(token_type) -> TokenKind:
 
 
 def merge_spans(spans: Iterable[Span]) -> list[Span]:
-    """Drop TEXT spans and merge adjacent spans of the same kind."""
+    """Drop TEXT and empty spans, and merge adjacent spans of the same kind."""
     merged: list[Span] = []
     for span in spans:
-        if span.kind is TokenKind.TEXT:
+        if span.kind is TokenKind.TEXT or span.end <= span.start:
             continue
         if merged and merged[-1].kind is span.kind and merged[-1].end == span.start:
             merged[-1] = Span(merged[-1].start, span.end, span.kind)
@@ -155,12 +168,12 @@ def highlight(self, text: str) -> list[Span]:
         )
 
 
-def language_for_filename(path: str | PathLike) -> str | None:
+def language_for_filename(path: str | bytes | PathLike) -> str | None:
     """Return the Pygments lexer alias for a filename, or ``None`` if there is none.
 
     The whole filename is matched, so names such as ``Makefile`` resolve.
     """
-    lexer_class = find_lexer_class_for_filename(str(path))
+    lexer_class = find_lexer_class_for_filename(fsdecode(path))
     return None if lexer_class is None else lexer_class.aliases[0]
 
 
@@ -192,8 +205,9 @@ def utf16_line_starts(text: str) -> list[int]:
     """Return the UTF-16 offset of the first character of each logical line."""
     starts = [0]
     offset = 0
-    for ch in text:
-        offset += 2 if ord(ch) > 0xFFFF else 1
-        if ch == "\n":
-            starts.append(offset)
+    for line in text.split("\n")[:-1]:
+        # An astral character is two UTF-16 units. "surrogatepass" keeps a lone
+        # surrogate at one unit rather than raising.
+        offset += len(line.encode("utf-16-le", "surrogatepass")) // 2 + 1
+        starts.append(offset)
     return starts