background blur changes + examples - #606
Open
MiloszFilimowski wants to merge 10 commits into
Open
Conversation
… the raw one A middleware was applied once, at the moment `setTrackMiddleware` was called, and never again. The device track is replaced far more often than that — camera off and on, a camera switch, a track that ended — and on each of those the effect silently disappeared. Setting a middleware before starting the camera, which is the natural order for a hook, meant it was stored and never applied at all. Three fixes, all in the shared react-client path so web and React Native get them: - Re-apply `currentMiddleware` when the raw device track changes. An `appliedTo` ref, claimed before the middleware is awaited, keeps the effect from racing a caller that is already applying it to the same track. - Process a freshly acquired track before publishing it. `selectDevice` and `toggleDevice` handed the raw track straight to `replaceTrack`/`addTrack`, so unprocessed camera video went on the wire for as long as the middleware took to start — seconds, for an effect that loads a model. `applyMiddlewareToTrack` takes the track explicitly, because `rawTrack` still belongs to the previous device until React re-renders. - Make release idempotent. `cleanupRef` was called but never cleared, so a stopped device followed by `setTrackMiddleware(null)` ran a consumer's `onClear` twice, freeing resources it no longer owned. Tests cover cold start, off/on, double-clear and the publish path; four of the five fail against the previous implementation. A middleware that finishes setting up after it was replaced or cleared now releases itself: A middleware whose setup was still pending when the next apply or a clear arrived had no cleanup registered yet, so it was never released; once it resolved it overwrote the newer middleware's track and cleanup. Each apply now carries a generation: a superseded result releases its own track and yields whatever replaced it.
…d track is swapped On React Native a middleware's onClear may dispose its track natively. Running it before replaceTrack left the published stream holding a dead track it could no longer remove, so the swap failed and the local tile froze. applyMiddleware / applyMiddlewareToTrack now detach the previous middleware and hand its release back to the caller as `releasePrevious`; the track manager calls it only after replaceTrack (or right away when nothing is published). The superseded-generation and stopped-device paths still release immediately. Claude-Session: https://claude.ai/code/session_01MUzB8Nef5pJqJMBoPHgJNJ
Expo SDK 57 pods require iOS 16.4, so `expo prebuild` with the 15.1 default failed with "Specs satisfying the Expo dependency were found, but they required a higher minimum deployment target". `ios.iphoneDeploymentTarget` still overrides it. Claude-Session: https://claude.ai/code/session_01MUzB8Nef5pJqJMBoPHgJNJ
A blur button on the preview and room screens hands a video-effects middleware to useCamera().setCameraTrackMiddleware, so the effect runs on the track the app already publishes and stays on across screens. Expo moves to SDK 57 / React Native 0.86.3 because react-native-worklets 0.12 needs React Native 0.83 or newer.
This was referenced Sep 11, 2026
…the plugin The plugin default applies to every app that uses the SDK, so it stays at 15.1. The example needs 16.4 for Expo SDK 57 pods and asks for it through the plugin's iphoneDeploymentTarget option.
applyMiddleware read the device track from React state and applyMiddlewareToTrack took it as an argument. One function that always takes the track covers both: callers with a freshly acquired track pass it, setTrackMiddleware passes the device's raw track, now exposed as rawDeviceTrack.
applyMiddleware now builds the new track, publishes it through a callback the caller passes, and only then releases the previous one. The latest request wins: one replaced while it sets up releases itself. That replaces the generation counter, the applied-track and latest-apply refs, and the releasePrevious hand-off through the track manager.
Named helpers to start and stop a middleware, descriptive names for the running middleware and the latest request, explicit early returns, and separate effects for a stopped device and a new device track.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Background blur in the fishjam-chat example, plus the react-client fixes it needs:
applyMiddlewarenow builds the new track, publishes it through a callback from the caller, and releases the old one last. The latest request wins: one that is replaced while it sets up releases itself.@fishjam-cloud/video-effectstouseCamera().setCameraTrackMiddleware. No provider, no custom source:The model is read through
expo-assetandexpo-file-system, because an Android release build cannotfetcha bundled asset. The example moves to Expo SDK 57 / React Native 0.86.3, which react-native-worklets 0.12 requires, and setsiphoneDeploymentTarget: "16.4"in its Fishjam plugin options because Expo SDK 57 pods need it. The plugin default stays 15.1.Motivation and Context
Camera effects on Fishjam's own camera track, shown in the example with the smallest possible app code. The react-client fixes came out of testing it on devices.
Uses
@fishjam-cloud/video-effects0.1.3 (fishjam-cloud/video-effects#4), which addscreateCameraEffectMiddleware.Related fork fix, released in
@fishjam-cloud/react-native-webrtc0.30.4 and not required by this PR: fishjam-cloud/fishjam-react-native-webrtc#91 (a released track leaves every local stream).Checked
cameraMiddleware.spec.tscovers cold start, camera off and on, replacement and clearing while pending, onClear running once, and the previous middleware being released afterreplaceTrack.Documentation impact
Types of changes
not work as expected)
https://claude.ai/code/session_01MUzB8Nef5pJqJMBoPHgJNJ