Skip to content

Remove deprecated multi-agent workflow samples and update evaluation … - #48795

Open
Howie Leung (howieleung) wants to merge 1 commit into
feature/azure-ai-projects/vnextfrom
howie/reemit
Open

Remove deprecated multi-agent workflow samples and update evaluation …#48795
Howie Leung (howieleung) wants to merge 1 commit into
feature/azure-ai-projects/vnextfrom
howie/reemit

Conversation

@howieleung

Copy link
Copy Markdown
Member

…scripts with new role assignment parameters. Update YAML configuration for additional directories and commit reference.

Description

Please add an informative description that covers that changes made by the pull request and link all relevant issues.

If an SDK is being regenerated based on a new API spec, a link to the pull request containing these API spec changes should be included above.

All SDK Contribution checklist:

  • The pull request does not introduce [breaking changes]
  • CHANGELOG is updated for new features, bug fixes or other significant changes.
  • I have read the contribution guidelines.

General Guidelines and Best Practices

  • Title of the pull request is clear and informative.
  • There are a small number of commits, each of which have an informative message. This means that previously merged commits do not appear in the history of the PR. For more information on cleaning up the commits in your PR, see this page.

Testing Guidelines

  • Pull request includes test coverage for the included changes.

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).
9 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

@github-actions

This comment has been minimized.

@rhurey

Copy link
Copy Markdown
Member

class RealtimeConversationItem(_Model): # pylint: disable=docstring-keyword-should-match-keyword-only

The Realtime classes can come from the OpenAI SDK, right?


Refers to: sdk/ai/azure-ai-projects/azure/ai/projects/models/_models.py:14274 in ab6b8d0. [](commit_id = ab6b8d0, deletion_comment = False)

@howieleung

Copy link
Copy Markdown
Member Author

RealtimeConversationItem

Yes
From
node_modules@azure-tools\openai-typespec\src\realtime\models.tsp

@github-actions

This comment has been minimized.

@github-actions

github-actions Bot commented Sep 1, 2026

Copy link
Copy Markdown
Contributor
[Pilot] PR Pipeline Failure Analysis

What failed

Build: 6769789 (public) — failed

Tests failing on all platforms (macOS 3.11, Ubuntu 24.04 3.10/3.10-coverage/3.13/3.14, Windows 2022 3.12 — both whl and sdist artifacts):

  • sdk.ai.azure-ai-projects.tests.foundry_features_header.test_foundry_features_header_on_beta_operations
  • sdk.ai.azure-ai-projects.tests.foundry_features_header.test_foundry_features_header_on_beta_operations_async
Relevant pipeline output
tests/foundry_features_header/test_foundry_features_header_on_beta_operations.py:104: in _discover_test_cases
    assert sc_name in EXPECTED_FOUNDRY_FEATURES, (
AssertionError: New .beta sub-client 'agent_endpoint_conversations' discovered but not found in
EXPECTED_FOUNDRY_FEATURES. Please add an entry for '.beta.agent_endpoint_conversations' to the
EXPECTED_FOUNDRY_FEATURES mapping in base_test.py.
assert 'agent_endpoint_conversations' in {'agents': 'WorkflowAgents=V1Preview,...', ...}

Root cause

This PR introduced a new .beta sub-client agent_endpoint_conversations on the AIProjectsClient, but the guard assertion in tests/foundry_features_header/base_test.py (EXPECTED_FOUNDRY_FEATURES) was not updated. The test module fails at collection time (before any test runs) on every platform and install type.

Recommended next steps

  • In sdk/ai/azure-ai-projects/tests/foundry_features_header/base_test.py, add an entry to the EXPECTED_FOUNDRY_FEATURES dict for key 'agent_endpoint_conversations' with the appropriate Foundry feature string value (e.g., 'AgentEndpointConversations=V1Preview' or whatever header value the new sub-client should send — check the sub-client implementation for the expected value).
  • Re-run the pipeline after the fix.
  • See https://aka.ms/ci-fix

Automated fix: Fix found, view and apply fix

Generated by Pipeline Analysis Next Steps · 42 AIC · ⌖ 6.37 AIC · ⊞ 6K ·

Glenn Harper (glharper) added a commit to Azure/azure-sdk-for-js that referenced this pull request Sep 1, 2026
### Packages impacted by this PR

`@azure/ai-projects`

### Issues associated with this PR

Follow-up to the Agent Insights API regeneration in #39797 and the
cross-language customization in Azure/azure-sdk-for-python#48795.

### Describe the problem that is addressed by this PR

Agent Insights `createRun` is emitted as an LRO that resolves to
`AgentInsightRunResult`. That terminal result omits the initial
`AgentInsightRun.id`, even though callers need the ID for the paired
`getRun` and `cancelRun` operations.

The feature branch already routes this operation through `JobPoller`,
following the job LRO work in #39237 and #39455, but that exposes an
Agent Insights run as `operationState.jobId`. This PR adds the
run-specific public shape while keeping the shared implementation
generic:

- Adds `RunOperationState<TResult>` and `RunPoller<TResult>`, exposing
`operationState.runId` after `submitted()` resolves.
- Changes `project.beta.agentInsightMonitors.createRun` to return
`RunPoller<AgentInsightRunResult>`.
- Refactors `JobPoller` and `RunPoller` through one identity-preserving
wrapper covering `operationState`, `poll()`, `onProgress()`,
serialization, and resume.
- Adds focused unit coverage for run IDs, serialized poller restoration,
resource-location fallback, and existing `JobPoller` behavior.
- Records the customization in the package's post-emitter workaround
documentation and skill so regeneration does not remove it.

The Agent Insights API is new in the unreleased `2.6.0` feature branch,
so replacing its temporary `JobPoller` return type does not break a
published API.

### Are there test cases added in this PR? _(If not, why?)_

Yes. `test/public/pollingHelpers.spec.ts` covers the new run poller and
regression coverage for job pollers.

### Provide a list of related PRs _(if any)_

- Azure/azure-sdk-for-python#48795
- #39237
- #39455
- #39797

### Validation

- `npm run check-format`
- ESLint on all changed TypeScript files
- `tsc -p tsconfig.src.json --noEmit`
- `tsc -p tsconfig.test.json --noEmit`
- `TEST_MODE=playback npm run test:node`
- `TEST_MODE=playback npm run test:browser`
- `turbo build --filter=@azure/ai-projects... --token 1`
- `check-package-version ai -packages azure-ai-projects`

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: da906d50-380f-43e0-bfe1-a5d82afd7ac6
- Introduced `BetaAgentInsightMonitorsOperations` for managing agent insight monitors.
- Updated `BetaOperations` to include the new agent insight monitors operations.
- Removed outdated sample scripts for multi-agent workflows and MCP approval.
- Updated evaluation samples to reflect changes in Azure SDK and improve functionality.
- Adjusted role assignment parameters in evaluation samples for better clarity and structure.
- Updated `tsp-location.yaml` to include new directories for agent insights and voice agents.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants