Skip to content

docs: add repeatable device acceptance record #45

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
YashasVM wants to merge 3 commits into from
Closed

docs: add repeatable device acceptance record #45

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
3c7113d
docs: add repeatable device acceptance record
YashasVM Sep 23, 2026
ee2ba88
docs: record local emulator and OBS checks
YashasVM Sep 24, 2026
6afbe6e
docs: note supplemental V8 scene load limits
YashasVM Sep 24, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
86 changes: 86 additions & 0 deletions docs/evidence/2026-09-24-local-emulator-and-obs.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
# Local Android emulator and OBS checks

This is a partial development check, not a physical-device acceptance run. It
does not change the phone/host matrix in `docs/testing.md` from `INCONCLUSIVE`.

## Android emulator

| Field | Value |
| --- | --- |
| Date | 2026-09-24, Asia/Kolkata |
| Device | Pixel 7 AVD, x86_64 emulator |
| Android | Android 15, API 35 |
| Candidate source commit | `2f2b22f33c534a5798c01a30feedfe747484b9f5` |
| Debug APK SHA-256 | `7dc355a80d4859751446f20242c6f91bba3035da7e770c8c79545a9562d86957` |
| OBS host used in this run | None; lifecycle instrumentation only |

Commands run from `android/`:

```sh
./gradlew :app:connectedDebugAndroidTest
./gradlew :app:testDebugUnitTest :app:assembleDebug
```

The emulator instrumentation suite completed with **4 tests passed**. It covered
Activity recreation with a service-owned runtime, returning from background,
explicit Stop and relaunch, camera-permission loss handling, and Start while the
permission result is denied. The permission-edge tests supplied a denied result
to the service handler; they did not revoke Android's actual runtime permission
because API 35 terminates the target process when its permission is revoked.

The lifecycle test log showed the native SRT listener starting on ports 9000 and
9100. This verifies the JNI bridge and listener startup on the emulator. It does
not verify media delivery to OBS.

Repository checks on this source tree:

```sh
python3 -m pytest -q
```

Result: **118 passed**. Android debug unit tests and APK assembly also passed.

## Linux OBS module load

| Field | Value |
| --- | --- |
| Host | Linux x86_64 |
| OBS | 32.2.2 |
| Source commit | `33280463e42d4a0d6a4c052a1b524e62a28ecf6b` (O1 branch) |
| Module SHA-256 | `2029002dd8adf50e9c2ce4f2e766110bdd7800399f69ea53e4aa7ffd15880a87` |
| Result | Module loaded in an isolated OBS profile |

This checks module loading only. No published old scene collection with known
provenance was available, and no Windows OBS host was available. Old-scene
settings retention, source media, and Windows UI behavior remain
`INCONCLUSIVE`.

### Supplemental local V8 scene check

I also opened the local historical `openstream_phone_v8_source` scene snapshot
from an isolated OBS profile with the same module. OBS 32.2.2 logged the source
creation and loaded-scene entry without a missing-source error. The persistent
identity fields `slot_id`, `slot_label`, `source_instance_id`, and
`phone_target_hint` were unchanged after OBS shut down. The settings hash did
change: OBS added `enabled` and `listener_enabled`, and the plugin refreshed
runtime fields (`pairing_hint`, `pairing_url`, `slot_status`, and `srt_url`).
The local scene file has no provenance connecting it to the published v1.0.1
release, so this does not satisfy the published-scene check. No phone was
connected, so media receipt was not tested. OBS shutdown emitted source/dock
cleanup warnings, which also need review before treating this as a clean
compatibility pass.

The saved scene itself is not included because it contains local user settings.
The isolated OBS log records the module load and V8 source creation; the
original and post-run settings hashes were `cea1e2969e56075de428fe5ccc4057280f7009f0401965c8b1ff643fd76e63fe`
and `d13ff519b3127d6133b83ed308accf6618d4ecac22f6b8549624e5a86c513a9e`.

## Still required

- Physical Android checks for camera behavior under Home, screen lock, task
removal, and process recreation.
- Actual Android runtime-permission revoke and restore on a device.
- A published legacy scene opened in real OBS on Windows and Linux, with its
settings retained and media received.
- End-to-end Android-to-OBS video/audio, controls, reconnect, Virtual Camera,
A/V sync, and thermal runs from the device acceptance template.
60 changes: 40 additions & 20 deletions docs/testing.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -67,21 +67,40 @@ executing Kotlin or C++ production code. New parsers and protocol state
machines should receive native unit tests rather than Python copies of their
implementation.

## Device acceptance tests

- One Android phone streams 1080p30 for 30 minutes without receiver crash.
- Android discovers a `shin` OBS source without manual IP entry.
- Tapping a discovered OBS device starts the stream directly.
- Stopping the OBS listener removes the device from Android discovery within about 5 seconds.
- SRT reconnect completes within 2 seconds after a short Wi-Fi interruption.
- OBS receives video through the selected `shin` source.
- OBS receives mono AAC audio at 48 kHz on the source's mixer channel.
- The OBS source shows only the phone camera feed, never the Android screen.
- Starting OBS Virtual Camera while the source is live does not create an
ever-growing delay; video remains within the configured low-latency window.
- A clap or tone remains aligned between the OBS video and mono audio paths
after Virtual Camera starts and after a reconnect.
- Telemetry updates at least once per second.
## Device acceptance runs

Use the [device acceptance record template](../tools/device-acceptance-record.md)
for every phone and OBS host pair. Run the same checks on at least three
distinct Android devices and both supported OBS hosts before G1. Record exact
APK and plugin SHA-256 values for each run. A device or host you did not run is
unverified for that release.

Track each phone and host pair in this matrix. Replace each status only after
you complete the run and link its record. An unrun pair stays `INCONCLUSIVE`.

| Android device | Android version | Windows OBS | Linux OBS | Run records |
| --- | --- | --- | --- | --- |
| Device 1 | | INCONCLUSIVE | INCONCLUSIVE | |
| Device 2 | | INCONCLUSIVE | INCONCLUSIVE | |
| Device 3 | | INCONCLUSIVE | INCONCLUSIVE | |

The partial local development checks on an API 35 emulator and Linux OBS are
recorded in [the 2026-09-24 evidence note](evidence/2026-09-24-local-emulator-and-obs.md).
They do not count as physical-device or old-scene acceptance, so the matrix
above remains `INCONCLUSIVE`.

The template starts every result as `INCONCLUSIVE`. Change a result to
`VERIFIED` only after you run the check and save evidence. Use `NOT VERIFIED`
when a run fails or does not meet its expected result. Do not infer physical
behavior from CI, source inspection, or a binary string.

The run checks install and update, clean OBS startup and plugin load, phone
discovery and reservation, first video and AAC audio, camera controls, Stop,
short and long network loss, OBS restart, source removal, Virtual Camera, a
30-minute stream, A/V sync, thermal readings, OBS shutdown with the phone
unreachable, and isolation of unrelated OBS sources during an OpenStream
failure. The record captures latency, dropped media, recovery time, battery,
temperature, and logs.

## Android/OBS compatibility matrix

Expand Down Expand Up @@ -141,8 +160,9 @@ send-path capacity, and stale-frame threshold were configuration comparisons,
not measurements of phone temperature, latency, A/V offset, reconnect time, or
drops. Do not use them as evidence that the candidate performs better.

For each future paired device run, record the exact APK and plugin SHA-256,
phone model and Android version, OBS version and host OS, Wi-Fi conditions,
profile, Virtual Camera state, duration, temperature delta, measured
end-to-end latency, maximum audio/video offset, reconnect time, and dropped
frame count for both builds.
For each paired device run, record the exact APK and plugin SHA-256, phone
model and Android version, OBS version and host OS, Wi-Fi conditions, profile,
Virtual Camera state, duration, temperature delta, measured end-to-end latency,
maximum audio/video offset, reconnect time, and dropped frame count for both
builds. The template separates recorded values from expected behavior so an
unmeasured target stays `INCONCLUSIVE`.
159 changes: 159 additions & 0 deletions tools/device-acceptance-record.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,159 @@
# OpenStream device acceptance record

Copy this file for each Android device and OBS host pair. Run the checks in
order. Keep raw logs, photos, and measurements with the record. Do not change an
unrun result from `INCONCLUSIVE`.

## Run identity

| Field | Value |
| --- | --- |
| Record ID | |
| Date and time, including time zone | |
| Tester | |
| Git commit | |
| Android device model and hardware revision | |
| Android version and security patch | |
| OBS host OS, version, and architecture | |
| OBS version and build | |
| Phone install type: clean install or update | |
| App version name and version code | |
| APK path | |
| APK SHA-256 | |
| APK signer certificate SHA-256 | |
| Plugin package path and format | |
| Plugin package SHA-256 | |
| Loaded plugin module path and SHA-256 | |
| Scene collection used | |
| Result folder for logs and captures | |

Compute artifact hashes before installing. On Linux or macOS, run:

```sh
sha256sum /path/to/openstream-android.apk /path/to/plugin-package
```

On Windows PowerShell, run:

```powershell
Get-FileHash -Algorithm SHA256 .\openstream-android.apk
Get-FileHash -Algorithm SHA256 .\plugin-package.zip
```

Inspect Android package metadata and signer before installation:

```sh
aapt dump badging /path/to/openstream-android.apk
apksigner verify --verbose --print-certs /path/to/openstream-android.apk
```

Install with `adb install /path/to/openstream-android.apk` for a clean app
install. For an update run, use `adb install -r /path/to/openstream-android.apk`
and record the result. Do not uninstall the published app before an update
check.

Record the package hash and the hash of the module OBS actually loads. If more
than one OpenStream module copy exists, stop and record `NOT VERIFIED` for
plugin startup until the duplicate is removed and the run is repeated.

## Network and stream settings

| Field | Value |
| --- | --- |
| Router or access point model and firmware | |
| Wi-Fi band and channel | |
| Phone SSID and subnet | |
| OBS host interface, SSID or wired, and subnet | |
| VPN enabled on phone or host | |
| Guest network or client isolation enabled | |
| Signal strength at phone | |
| Video profile, resolution, and frame rate | |
| Bitrate | |
| Configured SRT latency | |
| Virtual Camera enabled during run | |
| Other network traffic or impairment | |

## Check results

Use `VERIFIED`, `NOT VERIFIED`, or `INCONCLUSIVE`. For every result, record the
time, observed behavior, and evidence path. Keep `INCONCLUSIVE` if a required
device, host, permission, artifact, or measurement is unavailable.

| # | Check | Result | Observation and evidence path |
| --- | --- | --- | --- |
| 1 | Install the exact APK. For an update run, install over the previous published APK signed with the same identity. Confirm the package ID, version code, and signer match the candidate manifest. | INCONCLUSIVE | |
| 2 | Install OBS and the plugin on a clean host setup. Confirm the intended plugin module loads once, record the module path, and record the source name and available controls. | INCONCLUSIVE | |
| 3 | Add one OpenStream source. Start the phone app and confirm first discovery without manual IP entry. Record discovery time. | INCONCLUSIVE | |
| 4 | Select the phone and confirm reservation. Record the phone state and OBS state before media arrives. | INCONCLUSIVE | |
| 5 | Confirm live video from the selected camera and mono AAC audio in OBS. Confirm the source does not show the phone screen. | INCONCLUSIVE | |
| 6 | Try front/rear lens selection, torch when supported, and zoom. Record the selected camera and observed control response. | INCONCLUSIVE | |
| 7 | Use Stop in the expected user flow. Confirm media stops, reservation releases, and the UI reports a stopped or disconnected state. Record whether another Start works. | INCONCLUSIVE | |
| 8 | Restore the stream, interrupt Wi-Fi for less than 5 seconds, and record whether media recovers without restarting either app. Measure recovery time. | INCONCLUSIVE | |
| 9 | Restore the stream, make the phone unreachable for at least 60 seconds, then restore network access. Record recovery time and whether the user must act. | INCONCLUSIVE | |
| 10 | Keep the phone streaming, restart OBS, reopen the same scene, and record whether the plugin and source return and whether the phone reconnects. | INCONCLUSIVE | |
| 11 | Remove the OpenStream source while streaming. Confirm the phone reservation releases and record any OBS or app error. | INCONCLUSIVE | |
| 12 | Start the OBS Virtual Camera while the source is live. Record startup result, delay behavior, and any media loss. | INCONCLUSIVE | |
| 13 | With Virtual Camera active, clap once in view of the phone camera and microphone. Record video and audio, then measure the A/V offset. Repeat after reconnect. | INCONCLUSIVE | |
| 14 | Run video and audio continuously for 30 minutes. Record start/end times, dropped frames, audio interruptions, and any app or OBS failure. | INCONCLUSIVE | |
| 15 | During the 30-minute run, record battery level and phone temperature at start, every 5 minutes, and at end. Record encoder state and bitrate when available. | INCONCLUSIVE | |
| 16 | Add and keep an unrelated OBS source active. Make the phone unreachable during an OpenStream failure. Confirm the unrelated source stays active and OBS exits without a hang when you stop it. | INCONCLUSIVE | |
| 17 | Restore the network and repeat discovery, reservation, media, controls, Stop, and Start after the failures above. | INCONCLUSIVE | |

## Measurements

| Measurement | Value and method |
| --- | --- |
| Time from app launch to discovery | |
| Time from selection to reservation | |
| Time from reservation to first video | |
| Time from reservation to first audio | |
| End-to-end video latency and method | |
| Maximum A/V offset before reconnect | |
| Maximum A/V offset after reconnect | |
| Short-loss recovery time | |
| Long-loss recovery time | |
| Dropped video frames during 30 minutes | |
| Audio gaps or dropouts | |
| Battery level at start and end | |
| Phone temperature at start and end, including units | |
| Thermal warning or throttling observed | |
| Other media gaps or failure reasons | |

## Logs and evidence

Save the Android and OBS logs from before app launch through the final reconnect
check. Record commands and file paths. For Android, an example capture command
is:

```sh
adb logcat -c
adb logcat -v threadtime > android-logcat.txt
```

Stop the capture with Ctrl+C after the run. Save the OBS log from the OBS Help
menu or the host's OBS log directory. Include screenshots or short clips for
discovery, reservation, controls, sync, and any failure. Redact Wi-Fi secrets,
tokens, and personal data before sharing the record.

| Evidence | Path or description |
| --- | --- |
| Android log | |
| OBS log | |
| OBS scene collection | |
| Screen capture or photos | |
| Measurements or analysis files | |
| Failure details and reproduction steps | |

## Run conclusion

| Field | Value |
| --- | --- |
| Overall result | INCONCLUSIVE |
| Checks not run | |
| Failed checks | |
| Follow-up issue or PR | |
| Tester notes | |

Keep the overall result `INCONCLUSIVE` while any required check remains
inconclusive. A failed required check makes the run `NOT VERIFIED`. Mark the run
`VERIFIED` only when every required check passes and the evidence is saved.
Loading