[tests] Add infrastructure to run monotouch-test inside app extensions - #26265
Draft
rolfbjarne wants to merge 6 commits into
Draft
[tests] Add infrastructure to run monotouch-test inside app extensions#26265rolfbjarne wants to merge 6 commits into
rolfbjarne wants to merge 6 commits into
Conversation
Add a test harness that runs the monotouch-test NUnit suite inside an Audio Unit (AUv3) app extension process, verifying that .NET code works correctly in the extension sandbox. Key changes: - New test projects: AudioUnitExtension and SpotlightImportExtension (tests/dotnet/) with ContainerApp + AppExtension layout for iOS and macOS. - Convert ExtensionProject from Share/Thumbnail to QuickLook Preview (iOS) and Audio Unit (macOS) for more reliable triggering. - Runner script (scripts/run-audio-unit-extension-tests/) that builds, registers, launches the host, instantiates the AU out-of-process, runs NUnit tests inside the extension, and collects results. - Makefiles (tests/monotouch-test/dotnet/extensions/audio-unit/) with run-monovm / run-coreclr targets and TEST_FILTER support. - TouchRunner modifications to support extension-hosted execution. - Test for issue #24869: extensions with managed-static registrar. Fixes #24869
…le ctors Move ExtensionTestRunner from the monotouch-test project into the shared Touch.Client runner (namespace MonoTouch.NUnit.UI), so it can be reused by any Touch.Unit-based test app. Since the shared runner can't reference the per-app generated TestLoader, CreateHeadlessRunner now takes the test assemblies as an argument, and the caller passes TestLoader.GetTestAssemblies (). Also change the app extension `(IntPtr handle)` base constructors to `(NativeHandle handle)` across the extension test projects. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 3f83ba49-2d01-4cee-b4ca-8fbb6912c597
…nts into the csproj Set AssemblyName=monotouchtest so the extension's native executable and bundle name match the container app conventions, which lets BundleTest use the same expected executable name on all platforms again. Replace the six per-platform Info.plist/Entitlements.plist files with: * a single shared AppExtension-Info.plist (referenced via PartialAppManifest) containing only the keys with no MSBuild property equivalent (NSExtension, CFBundleDisplayName/CFBundleName). * CustomEntitlements items in AppExtension-shared.csproj (the network client/server entitlements come from the shared monotouch-test project). The remaining Info.plist keys (bundle identifier, version, package type, etc.) are derived automatically by CompileAppManifest from the csproj properties. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 3f83ba49-2d01-4cee-b4ca-8fbb6912c597
…ARIATION Remove the CoreCLR/MonoVM split (the XAMARIN_RUNTIME variable and the build/run-monovm/coreclr convenience targets) from the audio-unit extension makefiles, and select the runtime, registrar and configuration using TEST_VARIATION instead (the same mechanism the other .NET test projects use, see tests/common/test-variations.csproj). The per-platform makefiles now just include the shared makefile. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 3f83ba49-2d01-4cee-b4ca-8fbb6912c597
Rework the audio-unit extension test harness to report results the same way the regular Touch.Unit test apps do: over a localhost TCP connection. * The run script (run-audio-unit-extension-tests) now opens a TcpListener on a free 127.0.0.1 port, configures the extension via NSUserDefaults (network.enabled/host.name/host.port/transport=TCP + xml.enabled), and reads the NUnit XML result stream back over the socket (until the '<!-- the end -->' marker). Pass/fail is parsed from the <test-results> element and the raw NUnit XML is persisted for CI consumption. The system log is still captured, but for diagnostics only. * Subprocess launches use Xamarin.Utils.Execution (tools/common/Execution.cs) instead of hand-rolled Process helpers. * The extension's test filter now comes solely from NSUserDefaults (test.name); the environment-variable, bundle-file and Info.plist fallbacks are removed, along with the build-time MonotouchExtensionTestName property. This relies on the empirically-verified fact that 'defaults write <extension-bundle-id> ...' reaches the sandboxed extension's NSUserDefaults.StandardUserDefaults, whereas environment variables set on the container host do not propagate to the OS-spawned extension process. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 3f83ba49-2d01-4cee-b4ca-8fbb6912c597
This comment has been minimized.
This comment has been minimized.
Collaborator
✅ API diff for current PR / commitNET (empty diffs)✅ API diff vs stableNET (empty diffs)ℹ️ Generator diffGenerator Diff: vsdrops (html) vsdrops (raw diff) gist (raw diff) - Please review changes) Pipeline on Agent |
This comment has been minimized.
This comment has been minimized.
Collaborator
🔥 [CI Build #cf28e17] Test results 🔥Test results❌ Tests failed on VSTS: test results 1 tests crashed, 160 tests failed, 42 tests passed. Failures❌ dotnettests tests (iOS)1 tests failed, 0 tests passed.Failed tests
Html Report (VSDrops) Download ❌ dotnettests tests (MacCatalyst)1 tests failed, 0 tests passed.Failed tests
Html Report (VSDrops) Download ❌ dotnettests tests (macOS)1 tests failed, 0 tests passed.Failed tests
Html Report (VSDrops) Download ❌ dotnettests tests (tvOS)1 tests failed, 0 tests passed.Failed tests
Html Report (VSDrops) Download ❌ interdependent-binding-projects tests4 tests failed, 0 tests passed.Failed tests
Html Report (VSDrops) Download ❌ introspection tests4 tests failed, 0 tests passed.Failed tests
Html Report (VSDrops) Download ❌ linker tests (iOS)15 tests failed, 0 tests passed.Failed tests
Html Report (VSDrops) Download ❌ linker tests (MacCatalyst)12 tests failed, 3 tests passed.Failed tests
Html Report (VSDrops) Download ❌ linker tests (macOS)8 tests failed, 13 tests passed.Failed tests
Html Report (VSDrops) Download ❌ linker tests (tvOS)12 tests failed, 3 tests passed.Failed tests
Html Report (VSDrops) Download ❌ monotouch tests (iOS)19 tests failed, 0 tests passed.Failed tests
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
TODO:
Early prototype that runs the monotouch-test suite from inside an app
extension (an AUv3 audio-unit extension), so we can exercise the test
suite in the more constrained app-extension environment (e.g. to validate
the trimmable static registrar there).
Highlights:
A new audio-unit AUv3 extension test harness under
tests/monotouch-test/dotnet/extensions/audio-unit/, with a containerapp that instantiates the extension out-of-process and per-platform
projects (macOS/iOS/tvOS/MacCatalyst) that share almost all logic via
AppExtension-shared.csproj.Test results are streamed back over a localhost TCP connection, reusing
the existing Touch.Client reporting infrastructure. The run script
(
scripts/run-audio-unit-extension-tests) opens a TcpListener, configuresthe sandboxed extension through NSUserDefaults (
defaults write), andreads the NUnit XML result back over the socket.
The extension is configured entirely through NSUserDefaults, which — unlike
environment variables set on the container host — reliably reach the
OS-spawned, sandboxed extension process.
Runtime/registrar/configuration variations are selected with
TEST_VARIATION(consistent with the rest of the test infrastructure).Still an early prototype; xharness/CI wiring is proposed but not yet
included here.
🤖 Pull request created by Copilot