Repository navigation
Latch 5: the streaming engine in Rust - #30
Merged
Merged
Conversation
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
docs/PROTOCOL.md and docs/ENGINE.md define the Rust engine that replaces Sunshine, moonlight-common-c, OpenH264 and Opus. The three new crates hold the public APIs the implementations fill in. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…script - Switch latch-core from rustls/ring to rustls-graviola (pure-Rust TLS with s2n-bignum verified inline assembly, eliminating cc, C, and external assembly compilers from latch-core). - Add network verification test fetching the latest release from the real GitHub API using latch-core's update logic. - Add scripts/audit-native.sh and scripts/audit-native.allow to audit transitive dependencies of latch-host for native build tools (cc, cmake, nasm-rs, bindgen), shipped native sources, and links directives across all four target platforms. Co-Authored-By: gemini-3.8-flash-high <noreply@google.com>
…re, and keymap - Color conversion (crates/codec/src/color.rs): BT.709 fixed-point integer matrix with chroma 2x2 block averaging, odd-dimension handling, and NV12 interleave / deinterleave functions. Verified within +/- 1 LSB against floating-point reference. - Audio codec (crates/codec/src/audio.rs): lpcm-rice lossless 16-bit stereo PCM codec with FLAC-style fixed linear predictors (orders 0..=4), partitioned Rice residuals, 4 stereo channel decorrelation modes, bounds checks, and zero panic. - Synthetic desktop (crates/engine/src/pattern.rs): CpuCapture implementation with SMPTE 75% colour bars, 1-pixel checkerboard, embedded 5x7 font, big digital frame counter, Still/Sweep/Noise motion modes, and 64x64 key-press latency feedback. - Keymap (crates/engine/src/input.rs): Windows VK mapping to macOS Carbon kVK_* and X11 keysyms with collision testing and documentation. - Tests & benchmarks: comprehensive integration test suites and release benchmarks for color (1080p in 1.38 ms) and audio (< 12 µs/frame encode+decode). Co-Authored-By: Gemini 3.8 Flash High <noreply@google.com>
latch-stream: the session speaks docs/PROTOCOL.md over an upgraded control connection and UDP; decoding is VideoToolbox, Media Foundation or latch-codec; audio is lpcm-rice. Pairing, nvhttp, the RSA identity and the vendored moonlight-common-c build are gone from the crate. latch-engine: the session server (one viewer at a time, takeover, keyframe requests, reliable input, keepalives). latch-core: HTTP upgrade on both sides. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
LATCH_TEST_PATTERN streams the synthetic desktop and feeds it the viewer's input, so a whole session runs on any machine. The loopback test checks the decoded picture against the drawn one and times key presses to decoded pictures on the viewer's clock. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: GPT-6.1 Sol <noreply@openai.com>
…a still desktop Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Keep capture, video transforms, and loopback audio on MTA workers. Add DXGI cursor composition, NV12 conversion, hardware/software H.264 MFTs, WASAPI capture, SendInput, virtual display modes, Windows smoke and round-trip tests, and a capture timing example. Co-Authored-By: gpt-6.1-sol <noreply@openai.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ions The build of latch-host compiles no C, C++ or assembler on any of the four targets. The remaining entries ship such files but never compile them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ped at its usage limit)
# Conflicts: # crates/stream/Cargo.toml
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
serve runs the engine behind the stream upgrade without the service; view connects, prints per-second stats, times key presses to the decoded picture and reports its own CPU. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…aphics) - Implement open_video, open_audio, open_input, and encodable in crates/engine/src/mac - Video capture: ScreenCaptureKit on main display with NV12 IOSurface buffers - Video encode: VideoToolbox compression session with low-latency rate control, real-time CBR/VBR, Annex B framing, and in-band parameter sets - Audio capture: ScreenCaptureKit audio stream, float-to-int16 resampling, bounded 10 ms ring buffer - Input injection: CoreGraphics CGEvent generation, coordinate clamping, drag events, wheel scrolling, modifier tracking, unicode chunks - TCC permission guards: check and return exact Screen Recording and Accessibility error messages - Verification: unit tests for synthetic VideoToolbox encode/decode roundtrip, input event generation, and permission failure reporting Co-Authored-By: Gemini <gemini@google.com>
# Conflicts: # Cargo.lock
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Re-introduce Live.codec populated from Stats.codec once known. - Enforce Windows check in can_upgrade so non-Windows hosts receive clear instructions. - Update lobby test assertion to expect the new manual install instructions. - Restore ui::NOTICES in open source notices view and remove redundant crates/client/NOTICE. Co-Authored-By: gemini-3.8-flash-high <noreply@google.com>
Co-Authored-By: gemini-3.8-flash-high <antigravity@google.com>
- id 14: use cryptographically random nonce and create_new(true) for elevated setup script in %SystemRoot%\Temp - id 15: reject plain Caller::Local on POST /v1/stream; only allow verified Tailscale peers - id 16: do not fail migration or startup on malformed host.toml; load fallback and log error - id 17: retire legacy Latch 4 sunshine config by backing up to .bak-<stamp> and leave other sunshine configs untouched - id 18: treat tun == None like Some(false) in loopback_caller to avoid failing open to Caller::Local - id 19: enforce overall head deadline (IDLE_TIMEOUT) and authenticate caller before reading large /v1/update bodies - id 20: pass raw_arg for cmd.exe /D /S /C on Windows to preserve script quote semantics - id 23: remove legacy Latch SunshineService via CIM/WMI check while leaving user-installed Sunshine services intact - id 24: pipe mcookie directly into xauth via stdin in latch-display.service to prevent auth cookie leakage in process listings Co-Authored-By: gemini-3.8-flash-high
… builds Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Enable low-latency Media Foundation decoding; preserve capture aspect and physical dimensions; tolerate missing display modes and back off duplication recovery. Restore macOS click counts and scroll direction, retry dropped frames, and fall back to software VideoToolbox encoding when hardware fails at encode time. Downmix WASAPI speaker layouts and preserve resampler continuity across chunks. Add platform regression coverage and software VideoToolbox round trips. Co-Authored-By: GPT-6.1-Sol <noreply@openai.com>
Upgrading from Latch 4 only removes the engine unit and the drop-in's lines that required and waited for it; a custom Xorg display or session (as on the machine this was tested against) is no longer replaced with the repository's Xvfb default. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…rt, prep/undo) Run by hand against a copy of the data folder, then 4.1.1 restored. Note for anyone repeating it: --background rewrites the user's latch.service to point at the running binary, as 4.x does; the test restored the unit. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The security fix asked whois about loopback callers whenever Tailscale's state was unknown, and a whois that failed refused them: with Tailscale not installed or not running (CI, a stopped daemon) the machine's own window was refused. No daemon, no relayed peer: that answer is "nobody". Other whois failures still admit no one. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sample scroll detection once per band and frame, reject unrelated offsets, and reuse stack prediction/search storage. Add scalar-checked NEON transforms, quantization, half-pel generation and chroma interpolation; bypass exact-zero residuals and stop losing intra trials early. Batch bit writes and reduce unnecessary deblock loads. Score all luma rows, refine fractional motion adaptively, and start rate control from bits per pixel. Use the full QP range before coefficient suppression and allow bounded frame variation to avoid repeated whole-frame encodes. Always verify reconstruction with the Rust decoder as well as FFmpeg. Extend natural-motion and SIMD oracles, and support saved benchmark binaries and matched actual bitrate in the x264 comparison script. Co-Authored-By: OpenAI GPT-6.1 Sol <noreply@openai.com>
Also the session logs of the live runs, which *.log had kept out. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e idle interval The relay polled two sockets with 1 ms timeouts; Windows rounds those to about 15 ms, so the relay itself dropped most of what it carried and the picture stopped. One blocked thread per direction now. The prompt-keyframe test's bound is two idle intervals rather than a number written for 25 ms. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The macOS runner slept 202 ms for a 50 ms sleep; timing bounds there measure the runner. The keyframe property is checked by the mock's own assertion on its second call, the pacing test only that it is never early. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…kflow that ran them Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…sion asserts flow, not speed The host migration had the CI seam write the real retire script without the setup preamble: Step was undefined and the dummy folder and service CI created were not what it targeted. It targets them again, refuses the real engine paths, and carries the preamble. The bad-network session asserted 30 fps; a debug build on the macOS runner makes about 10. It now asserts that the picture keeps coming to the end; speed is measured in release builds on real machines. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
MrBeldum
marked this pull request as ready for review
October 29, 2026 09:39
install-host.ps1 named the signed-in user as USERDOMAIN\USERNAME, which an SSH session does not set, so starting Latch in the desktop session failed when installing remotely. The retire step also removes Latch 4's cursor-guard task, which would otherwise point at a deleted executable. Both found while installing on the Windows PC. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Installing over SSH started Latch before setup had opened the firewall; Windows turned its unanswered allow-access prompt into inbound Block rules for the program, which beat setup's Allow rules: the control port was unreachable on the PC this was installed on. Setup now removes inbound Block rules for its own executable, and only those, before adding its own. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Setup removed latch-engine.service but left the drop-in lines that required it and waited for its port, so a service upgraded through the window's Set up (rather than the migration script) would never start. Setup now strips those lines and keeps the owner's others. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
On an upgrade the old latch-host is still running, and 'enable --now' leaves it running: the new binary would not run until the next reboot. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Latch 5 replaces Sunshine (host), and moonlight-common-c, OpenH264 and Opus (viewer), with its own streaming engine in Rust. Design:
docs/ENGINE.md, wire format:docs/PROTOCOL.md, evidence and what is not yet verified:docs/RUST_REWRITE_STATUS.md(raw data indocs/measurements/2026-10-29/).What changes
latch-proto: the stream protocol. HTTP-upgraded control on 47850, UDP media with Reed–Solomon FEC, keyframe on loss, input made reliable by redundancy.latch-codec: an H.264 Constrained Baseline encoder built for desktops and a decoder (reconstruction kernels adapted from MIT-licensed pith-h264), BT.709 conversion,lpcm-ricelossless audio.latch-engine: capture, encode, audio and input per OS (ScreenCaptureKit + VideoToolbox; DXGI + D3D11 + Media Foundation; X11 MIT-SHM + the Rust encoder) and the session server, inside the host's own process.latch-stream: the viewer. VideoToolbox, Media Foundation (with the Rust decoder behind it) or the Rust decoder; cpal audio.third_party/and every GameStream path are removed; TLS uses a pure-Rust provider (graviola).scripts/audit-native.sh(CI): nothing in the build compiles C, C++ or assembler on any target.Evidence
Not yet verified (needs the machines or permissions)
Rollout: nothing is released or installed (stuzha, the Mac and Hermes still run 4.1.1). A 4.x viewer cannot watch a 5.0 host, so the Mac app goes first, then each host; Hermes by hand the first time, keeping its 4.1.1 executable.
🤖 Generated with Claude Code