Skip to content

Rebase FastLED infrastructure onto kernal-api - #242

Merged
zackees merged 95 commits into
mainfrom
feat/kernal-api-migration
Sep 15, 2026
Merged

zackees merged 95 commits into
mainfrom
feat/kernal-api-migration

Conversation

@zackees

@zackees zackees commented Sep 13, 2026 •

Copy link
Copy Markdown
Member

Closes #229.
Closes #230, closes #231, closes #232, closes #233, closes #234, closes #235, closes #236, closes #237, closes #238, closes #239, closes #243, closes #248.
Supersedes #244 (its SHA-256 encoding tests are ported onto kernel JSON in eb5a51d).
Refs #247, #250.

Summary

Moves FastLED's shared infrastructure into kernal-api. FastLED product policy and protocols stay in this repo.

fastled-cli now has exactly one direct Rust dependency: kernal-api, from the v0.1.6 release tag, with no [patch], path or revision override. Direct backends are removed from the manifests and sources; they remain only as private transitive kernel implementations. The full per-slice record is in docs/kernal-api-migration.md.

Removed direct dependencies: fs2, globset, notify, zccache-fingerprint, running-process, windows-sys, dirs, tokio, tokio-stream, reqwest, axum, tower-http, zip, tar, zstd, flate2, sha2, tauri, tauri-build, gtk, webkit2gtk, crossterm, ctcb-core, tree-sitter, tree-sitter-cpp, clap, serde, serde_json, toml, anyhow, strsim, shell-words, tempfile, getrandom, indexmap, thiserror. Python runtime: typeguard, zcmds_win32.

Release

kernal-api v0.1.3 was cut on release/0.1.3 from 762a1d2, the revision this branch had already validated. It was also the last kernal-api main commit with green CI; main was failing CI at the time, so later main commits are excluded. The release adds only a version bump over 762a1d2.

v0.1.4 (zackees/kernal-api#268) forced WebKitGTK's worker WebGL feature and froze the viewer on NVIDIA Wayland. v0.1.5 reverts it (zackees/kernal-api#272), so its code is v0.1.3 again. zackees/kernal-api#269, the main-branch port of that change, is closed unmerged. v0.1.6 adds the opt-in windows-app-resources build-script capability (zackees/kernal-api#276). crates.io publishing isn't enabled for kernal-api and fastled-cli isn't on crates.io, so the dependency is a git tag.

Linux viewer rendering (#247)

WebKitGTK has WebGL2 on a regular canvas but, by default, not on OffscreenCanvas. The bundled frontend required OffscreenCanvas WebGL2 and rejected the Linux viewer.

It now carries FastLED#4405, which is the path the pre-migration viewer used:

  • The sketch stays in the worker, which posts each frame to the page.
  • The page draws each frame with the regular-canvas WebGL2 renderer.
  • The --test harness reads the page canvas in this mode.

Evidence (NVIDIA 595.99.02, WebKitGTK 2.52.5, KDE): focused-window OS captures, with core dumps matched by timestamp.

Renderer Runs Animated throughout Froze
Worker WebGL forced on (kernal-api v0.1.4) 8 Wayland 3 5, each with a Skia GrResourceCache SIGILL
Main-thread rendering (this branch + v0.1.5) 8 Wayland, 3 X11 11 0

A separate libnvidia-eglcore SEGV shows up with both renderers and with the pre-migration binary; it doesn't stop rendering.

Correction: earlier versions of this description called the failure an upstream WebKitGTK/NVIDIA crash. That was wrong. The MiniBrowser "reproduction" lacked __NV_DISABLE_EXPLICIT_SYNC=1. The Blink captures failed because Blink sets no screen map in setup(), now tracked in #250.

Boundary

tests/unit/test_kernal_boundary.py enforces:

  • kernal-api is the only dependency in every package, workspace and target table.
  • No dev dependencies, [patch] or generated Tauri output.
  • One build.rs, whose only build dependency is kernal-api from the same release with only windows-app-resources, and which names no other crate. It embeds the Windows executable's icon, version information and manifest (see below).
  • The hash-sha256 feature is required.
  • Per-backend source bans.

Obsolete generated Tauri ACL schemas are deleted.

Validation

  • Linux x86-64: Rust workspace tests pass (303 library, 3 binary, 1 integration, 1 doc). bash lint passes (rustfmt, strict Clippy, dylint, Ruff, Black, isort, Pyright). Python: 45 passed, 1 skipped.

  • fastled <sketch> --test on Linux:

    • X11: rc=0; the red/blue sketch alternates across 12 in-page captures.
    • Wayland with the viewer window visible: rc=0, same alternation (2 of 2 runs).
    • With the DMA-BUF renderer, WebKit produces no frames while the compositor isn't drawing the window. A Wayland run whose window never becomes visible times out waiting for the first frame.
  • Build measurements (same host, warm soldr cache, fresh target dir, debug fastled bin), comparable round:

    • main: 12.1 s clean / 1.5 s incremental, 455 crates compiled.
    • This branch: 15.5 s clean (8 cache misses remaining) / 1.5 s incremental, 386 crates.

    No build-speed improvement is claimed: the graph is 69 crates smaller, but clean and incremental times aren't faster. soldr can't disable its cache, so uncached clean builds weren't measured. The full table is in the doc.

macOS live test in PR CI

New macos-arm-live-test.yml:

  • Linux job: cross-compiles for aarch64-apple-darwin with Soldr's toolchain. It produces the release fastled (checked to be an arm64 Mach-O), the macOS wheel, and a nextest archive of the workspace tests.
  • macos-15 job: compiles nothing. It runs the test archive, then installs the wheel and runs the shipped viewer and real Safari.

The 8 native macOS build/lint/unit/integration workflows no longer run on pull requests; they still run on main pushes and manual dispatch. Intel macOS therefore has no PR coverage; #251 tracks moving execution into a docker-mac-x64 guest on a Linux runner.

Result on e1c9580 (run 35003749031):

  • Tests: 307 of 307 cross-built Rust tests pass on the Mac.
  • Shipped WKWebView viewer: the wasm_vfs fixture's asset is sha256-verified and loaded before setup(). --test captures the expected solid-green grid.
  • Safari 26.6.1: 38 frames rendered, cross-origin isolated, no page errors. The WebDriver screenshot shows the same green grid.
  • Timing: cross-build 3.5 min on Linux; the Mac runner is held 3 min, of which the Rust tests take 9 s and the viewer and Safari smoke take 2 min 13 s.

Not yet done (merge gates)

  • Real Safari smoke test on macOS: ci/safari_smoke.py in the macOS lane (above).
  • Windows executable resources: kernal-api v0.1.6 adds windows-app-resources (feat: embed Windows application resources from a build script zackees/kernal-api#276), and crates/fastled-cli/build.rs calls it. A locally cross-built fastled.exe embeds the same RT_ICON, RT_GROUP_ICON (32512), RT_VERSION ("FastLED Viewer" / "fastled" / 2.0.20) and Common-Controls v6 RT_MANIFEST as the Tauri-era release; this PR's earlier builds embedded none.

🤖 Generated with Claude Code

https://claude.ai/code/session_01VfXV1EpgAvMZzXJfbGZPRG

This was referenced Sep 13, 2026
zackees and others added 15 commits September 13, 2026 11:13
- Replace the migration-only [patch.crates-io] git revision with the
  kernal-api v0.1.3 release tag (cut from the validated 762a1d2 pin).
- Boundary test now requires kernal-api to be the only Rust dependency,
  forbids [patch], build.rs and generated Tauri output, and requires the
  hash-sha256 feature.
- Remove obsolete generated Tauri ACL schemas.
- Port the SHA-256 encoding lock-down tests from #244 onto kernel JSON.

Refs #229, #235

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VfXV1EpgAvMZzXJfbGZPRG
Adds a current-status section and replaces the final audit checklist with
results: resolved dependency inventory, the kernal-api v0.1.3 release cut
from the validated 762a1d2 pin, CI-enforced sole-dependency boundary, Linux
validation, build timing measurements (69 fewer crates, no speedup claimed),
and the #247 viewer comparison showing identical WebGL2 failure on
pre-migration main and on NixOS generation 118 (no hardware GL until the
NVIDIA driver mismatch is cleared by reboot).

Refs #229, #247

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VfXV1EpgAvMZzXJfbGZPRG
kernal-api v0.1.4 enables WebKitGTK's AllowWebGLInWorkers runtime feature in
its Linux webview adapter (zackees/kernal-api#268). WebKitGTK ships it
disabled, and it gates WebGL on every OffscreenCanvas, so the viewer's WebGL2
compatibility gate always failed on Linux. The fix lives in kernal-api; no
local workaround is added here.

- Pin the workspace dependency to the v0.1.4 tag and require it in the
  boundary test.
- Log the frontend's OffscreenCanvas WebGL2 probe as unavailable when
  getContext returns null, instead of reporting SUCCESS unconditionally.
- Correct the migration audit: the shared #247 failure was the WebKitGTK
  feature flag, not host GL. A WebKitGTK 2.52 + NVIDIA Wayland crash with
  document plus worker WebGL still blocks real rendering and is tracked
  upstream in #247.

Refs #229, #247

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VfXV1EpgAvMZzXJfbGZPRG
…bGL2

Port FastLED#4405 to the bundled frontend. WebKitGTK (the Linux viewer)
has WebGL2 on a regular canvas but, by default, not on OffscreenCanvas.
The sketch stays in the worker (the pthread build blocks in
extern_setup()/extern_loop()), which now posts each frame's pixel
buffers to the main thread; the page draws them with the regular-canvas
WebGL2 graphics manager. The compatibility check requires WebGL2 on a
regular canvas and only warns about OffscreenCanvas.

This is the path the pre-kernal-api viewer used. Forcing worker WebGL
(kernal-api v0.1.4) instead crashes the WebKit web process on NVIDIA
Wayland (Skia GrResourceCache SIGILL) and freezes the canvas: 5 of 8
runs froze, versus 0 of 10 with main-thread rendering (7 Wayland, 3 X11).

The --test harness reads the page canvas in this mode (drawing buffers
are already preserved) instead of asking the worker for a WebGL readback.

Refs #247

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VfXV1EpgAvMZzXJfbGZPRG
kernal-api v0.1.5 reverts zackees/kernal-api#268, so WebKitGTK views keep
their default of no WebGL on OffscreenCanvas. The viewer now renders on
the main thread in that case (previous commit), which is the path the
pre-migration viewer used. Forcing worker WebGL froze the canvas on NVIDIA
Wayland.

The migration record now corrects the earlier claim that #247 was an
upstream WebKitGTK/driver crash. It records the renderer comparison and
the Linux --test results.

Refs #247

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VfXV1EpgAvMZzXJfbGZPRG
zackees and others added 5 commits September 15, 2026 10:29
macos-arm-live-test.yml builds everything for aarch64-apple-darwin on
Linux through Soldr's cross toolchain:
- the release fastled binary (checked to be an arm64 Mach-O),
- the macOS wheel,
- a nextest archive of the workspace tests.

A macos-15 runner then only executes:
- the test archive, remapped onto its checkout;
- the installed wheel's live render in the shipped WKWebView viewer;
- a real Safari smoke test of the same compiled sketch. safaridriver
  loads it from the shipped server, and the check requires cross-origin
  isolation, rendered frames and no page errors.

The native macOS build/lint/unit/integration workflows no longer run on
pull requests; they still run on main pushes and manual dispatch. The
installed-wheel smoke moves from _build.yml into
ci/smoke_installed_wheel.sh so both lanes run the same steps.

Refs #242, #251

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VfXV1EpgAvMZzXJfbGZPRG
setup-soldr v0.9.62 pins Soldr 0.7.51. That version has no `build`
subcommand, so `soldr build --target aarch64-apple-darwin` was looked up
as a downloadable tool ("webbrandon/build", HTTP 404), and it has no
managed macOS SDK. Use the setup-soldr revision kernal-api's Apple cross
lanes use, which installs a current Soldr (0.9.x locally cross-builds
fastled and its nextest archive for aarch64-apple-darwin).

Refs #242

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VfXV1EpgAvMZzXJfbGZPRG
setup.py always rebuilds the bundled binary and passes --target only when
FASTLED_RUST_TARGET is set. Without it, the wheel step rebuilt fastled
for the Linux host while Soldr's aarch64-apple-darwin cross environment
was active. A host build script (serde_core) was then linked with
ld64.lld, which rejects `-z`. Set FASTLED_RUST_TARGET, pre-build with the
exact command setup.py runs, and check the binary that ends up in the
wheel is an arm64 Mach-O.

Refs #242

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VfXV1EpgAvMZzXJfbGZPRG
cache_upgrade_cli started the CLI from env!("CARGO_BIN_EXE_fastled"),
a path fixed when the test was compiled. The macOS lane cross-builds the
tests on Linux and runs them on a Mac, where that path does not exist
("No such file or directory"). Prefer nextest's runtime
NEXTEST_BIN_EXE_fastled and keep the compile-time path for plain
`cargo test`. The other 303 archived tests already passed on macOS.

The macOS lane also runs the viewer render and Safari smoke when a
Rust test fails, so one cycle reports every result.

Refs #242

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VfXV1EpgAvMZzXJfbGZPRG
The Tauri-era fastled.exe carried RT_ICON, RT_GROUP_ICON (32512),
RT_VERSION ("FastLED Viewer", "fastled", package version) and a
Common-Controls v6 RT_MANIFEST from tauri-build; the kernal-api build
carried no resources.

kernal-api v0.1.6 adds the windows-app-resources build-script capability
(zackees/kernal-api#276). crates/fastled-cli/build.rs declares the
product and icon and calls it; it does nothing on non-Windows targets.
The boundary test now allows exactly this one build script: kernal-api
is its only build dependency, from the same release, with only that
feature, and the script names no other crate.

A fastled.exe cross-built from Linux for x86_64-pc-windows-msvc embeds
the same four resources as the Tauri-era release. Linux: Rust workspace
tests, Python unit tests and bash lint pass.

Refs #242

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VfXV1EpgAvMZzXJfbGZPRG
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment