Skip to content

Fix/pipeline component asset default runtime - #48513

Open
Chakradhar886 wants to merge 12 commits into
mainfrom
fix/pipeline-component-asset-default-runtime
Open

Fix/pipeline component asset default runtime#48513
Chakradhar886 wants to merge 12 commits into
mainfrom
fix/pipeline-component-asset-default-runtime

Conversation

@Chakradhar886

@Chakradhar886 Chakradhar886 commented Aug 10, 2026

Copy link
Copy Markdown
Member

Description

Fixes support for default values on asset-type inputs in Azure ML pipeline components.

Previously, loading a pipeline component with an asset input default failed with:
Default value of Input cannot be set: Non-primitive type Input has no default value.

Changes:

1.Allow string defaults for uri_file, uri_folder, mltable, mlflow_model, and custom_model inputs.
2.Apply asset defaults when a pipeline component is invoked without explicitly providing those inputs.
3.Preserve defaults when registered components are returned or retrieved from the service.
4.Prevent default [Input] objects from being shared between component invocations.
5.Add unit and end-to-end test coverage.
6.Update the changelog.

Validation
1.Verified that load_component() accepts a pipeline component with a uri_file default.
2.Verified that defaults survive serialization, registration, and retrieval.
3.Verified that an omitted uri_file input is populated when the pipeline runs.
4.All Azure SDK CI checks pass.

Testing

Samples validations: Azure/azureml-examples#4102

All SDK Contribution checklist:

  • The pull request does not introduce [breaking changes]
  • CHANGELOG is updated for new features, bug fixes or other significant changes.
  • I have read the contribution guidelines.

General Guidelines and Best Practices

  • Title of the pull request is clear and informative.
  • There are a small number of commits, each of which have an informative message. This means that previously merged commits do not appear in the history of the PR. For more information on cleaning up the commits in your PR, see this page.

Testing Guidelines

  • Pull request includes test coverage for the included changes.

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).
10 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds asset-type defaults to Azure ML pipeline components and carries them into runtime job inputs.

Changes:

  • Accepts and serializes defaults for data/model asset inputs.
  • Materializes defaults when pipeline nodes are instantiated.
  • Adds unit and end-to-end coverage plus a changelog entry.

Reviewed changes

Copilot reviewed 12 out of 12 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
sdk/ml/azure-ai-ml/CHANGELOG.md Documents asset-default support.
azure/ai/ml/_schema/job/input_output_entry.py Adds default fields to asset schemas.
azure/ai/ml/constants/_component.py Permits defaults in asset input validation.
azure/ai/ml/dsl/_component_func.py Supplies asset defaults to generated callables.
azure/ai/ml/entities/_inputs_outputs/input.py Accepts string defaults for asset types.
azure/ai/ml/entities/_job/pipeline/_io/mixin.py Materializes defaults as runtime inputs.
azure/ai/ml/operations/_component_operations.py Restores defaults omitted from create responses.
tests/component/unittests/test_component_operations.py Tests create-response restoration.
tests/component/unittests/test_pipeline_component_entity.py Tests loading and serialization.
tests/pipeline_job/e2etests/test_pipeline_job.py Exercises an omitted default end to end.
tests/test_configs/components/pipeline_component_with_asset_defaults.yml Defines serialization test inputs.
tests/test_configs/components/pipeline_component_with_uri_file_default.yml Defines the runtime test component.

💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.

Comment thread sdk/ml/azure-ai-ml/azure/ai/ml/dsl/_component_func.py
Comment thread sdk/ml/azure-ai-ml/azure/ai/ml/operations/_component_operations.py
@github-actions

This comment has been minimized.

Copilot AI review requested due to automatic review settings August 10, 2026 12:50

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 15 out of 15 changed files in this pull request and generated 1 comment.

Suppressed comments (3)

sdk/ml/azure-ai-ml/azure/ai/ml/_schema/job/input_output_entry.py:118

  • DataInputSchema is used for actual job inputs, where default is never promoted to path; _resolve_job_input consequently raises Input path can't be empty for jobs. A pipeline component's input definition is parsed by the separate InputPortSchema, which already supports defaults, so this addition broadens job YAML without enabling valid runtime behavior.
    default = generate_path_property(azureml_type=AzureMLResourceType.DATA)

sdk/ml/azure-ai-ml/azure/ai/ml/_schema/job/input_output_entry.py:136

  • This also makes an MLTable job input with only default pass schema loading, but job submission requires path and does not consume default. Component interface defaults are already handled by _schema/component/input_output.py::InputPortSchema; remove this job-schema field to preserve early validation.
    default = generate_path_property(azureml_type=AzureMLResourceType.DATA)

sdk/ml/azure-ai-ml/azure/ai/ml/entities/_inputs_outputs/input.py:377

  • Local-path defaults are now accepted, but component registration does not resolve or upload top-level component input defaults: _resolve_dependencies_for_component only resolves component code/environment and inputs of nested jobs. The registered component therefore persists a machine-local path, and invoking it elsewhere later fails when job submission tries to resolve that path relative to the caller. Either upload/replace local defaults during component registration or restrict defaults to portable AzureML/remote references.
            # Asset-type inputs accept a string default value (e.g. "azureml:", "https://",
            # local path) matching the public CLI v2 YAML schema.

Comment thread sdk/ml/azure-ai-ml/azure/ai/ml/_schema/job/input_output_entry.py Outdated
…t-asset-default-runtime

# Conflicts:
#	sdk/ml/azure-ai-ml/api.metadata.yml
Copilot AI review requested due to automatic review settings August 10, 2026 12:57

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 13 out of 13 changed files in this pull request and generated no new comments.

Suppressed comments (1)

sdk/ml/azure-ai-ml/azure/ai/ml/entities/_inputs_outputs/input.py:381

  • Accepting local paths here makes registered defaults non-portable. Component registration resolves dependencies in pipeline jobs but does not upload or rewrite top-level component.inputs[*].default; the fallback metadata consequently stores the raw local path. Fetching and running that component from another working directory or machine then attempts to use a path that does not exist. Either restrict defaults to portable remote references or resolve/upload local defaults before registration and persist the resulting URI.
        if not self._is_primitive_type and default_value is not None:
            # Asset-type inputs accept a string default value (e.g. "azureml:", "https://",
            # local path) matching the public CLI v2 YAML schema.
            if not self._multiple_types and self.type in IOConstants.ASSET_INPUT_TYPES:
                if isinstance(default_value, str):
                    self.default: Any = default_value
                    return

Copilot AI review requested due to automatic review settings August 10, 2026 13:03

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 13 out of 13 changed files in this pull request and generated no new comments.

Suppressed comments (1)

sdk/ml/azure-ai-ml/azure/ai/ml/entities/_inputs_outputs/input.py:381

  • This changes the shared public Input type, not just pipeline-component ports. As a result, a top-level job can now be constructed with Input(type="uri_file", default="azureml:data:1"), but it has no path; job submission still rejects that object with Input path can't be empty for jobs in _job_operations.py:1517-1525. The job YAML schemas were kept from accepting default for this reason. Please scope asset defaults to component interfaces (or add context-specific job validation) so programmatic job inputs continue to reject this unsupported shape instead of accepting it and failing later during submission.
            if not self._multiple_types and self.type in IOConstants.ASSET_INPUT_TYPES:
                if isinstance(default_value, str):
                    self.default: Any = default_value
                    return

Copilot AI review requested due to automatic review settings August 10, 2026 13:42

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 13 out of 13 changed files in this pull request and generated no new comments.

Suppressed comments (2)

sdk/ml/azure-ai-ml/azure/ai/ml/entities/_component/pipeline_component.py:532

  • Local paths are explicitly accepted as asset defaults, but this persists the raw path without resolving/uploading it. _resolve_dependencies_for_component only resolves code, environment, and inputs of pipeline jobs (_component_operations.py:871-906,1151-1169), not component.inputs. After the component is registered and fetched, the original YAML base directory is unavailable, so an omitted local-path default is resolved relative to the caller (or fails) instead of referencing an uploaded asset. Resolve local interface defaults during registration and persist the resulting cloud URI.
        asset_input_defaults = {
            input_name: component_input.default
            for input_name, component_input in self.inputs.items()
            if component_input.type in IOConstants.ASSET_INPUT_TYPES and component_input.default is not None
        }

sdk/ml/azure-ai-ml/azure/ai/ml/entities/_inputs_outputs/input.py:381

  • Input is also the public type for top-level job inputs, so this now lets a programmatic job accept Input(type="uri_file", default="..."). That job has no path; submission reaches _job_operations.py:1517-1525, which ignores default and raises “Input path can't be empty for jobs.” The job YAML schemas reject default, but direct construction bypasses those schemas. Please scope asset defaults to component-interface inputs, or add context-aware job validation so this unsupported configuration is not accepted by the shared API.
            if not self._multiple_types and self.type in IOConstants.ASSET_INPUT_TYPES:
                if isinstance(default_value, str):
                    self.default: Any = default_value
                    return

@github-actions

Copy link
Copy Markdown
Contributor
[Pilot] PR Pipeline Failure Analysis

A CI pipeline failed on this pull request. Here is an automated analysis of what went wrong and how to get the build green.

What failed

The python - pullrequest pipeline (build 6680616) failed in the Build Analyze stage (direct link) due to a black formatting check violation on azure-ai-ml. The check reports that exactly one file needs reformatting:

would reformat sdk/ml/azure-ai-ml/azure/ai/ml/entities/_component/pipeline_component.py

Black wants to reformat a dictionary assignment (collapsing a multi-line subscript assignment into a parenthesized single-line form). This is a validation failure — no code logic is broken, only formatting.

Recommended next steps

  • Run black sdk/ml/azure-ai-ml/azure/ai/ml/entities/_component/pipeline_component.py locally (or black sdk/ml/azure-ai-ml/) and commit the result.
  • Alternatively, run azpysdk black . from sdk/ml/azure-ai-ml/ to apply and verify all black fixes for the package.
  • See the CI troubleshooting guide: https://aka.ms/ci-fix
  • Push the reformatted file to re-trigger the pipeline.
Raw pipeline analysis (azsdk ci analyze)
Build: 6680616 Project: public
PipelineUrl: https://dev.azure.com/azure-sdk/public/_build/results?buildId=6680616

Failed Tasks
------------
[azure-ai-ml :: black] would reformat /mnt/vss/_work/1/s/sdk/ml/azure-ai-ml/azure/ai/ml/entities/_component/pipeline_component.py

Oh no! 💥 💔 💥
1 file would be reformatted, 1469 files would be left unchanged.

black check completed with exit code 1
[END 1/1] black :: /mnt/vss/_work/1/s/sdk/ml/azure-ai-ml -> FAIL(1) in 25.65s

Failing checks:
  python - pullrequest [FAILURE]
    Url: https://dev.azure.com/azure-sdk/29ec6040-b234-4e31-b139-33dc4287b756/_build/results?buildId=6680616
  python - pullrequest (Build Analyze) [FAILURE]
    Url: https://dev.azure.com/azure-sdk/29ec6040-b234-4e31-b139-33dc4287b756/_build/results?buildId=6680616&view=logs&jobId=b70e5e73-bbb6-5567-0939-8415943fadb9

Copilot detected the failing pipeline and generated the analysis above. To have it attempt a fix automatically, reply with `@copilot please fix the failing pipeline on this PR`.

Generated by Pipeline Analysis - Next Steps · 25 AIC · ⌖ 6.17 AIC · ⊞ 6.6K ·

Copilot AI review requested due to automatic review settings August 10, 2026 14:26

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 13 out of 13 changed files in this pull request and generated no new comments.

Suppressed comments (2)

sdk/ml/azure-ai-ml/azure/ai/ml/entities/_inputs_outputs/input.py:385

  • The string-only check also accepts an empty asset reference. That value is serialized and later converted to Input(path=""), where job submission always fails the non-empty path check. Reject empty defaults while loading the component so the error is reported at the invalid declaration.
                if isinstance(default_value, str):
                    self.default: Any = default_value
                    return
                msg = (
                    f"{msg_prefix}cannot be set: default for type '{self.type}' must be a "
                    f"string asset reference, got '{type(default_value)}'."
                )

sdk/ml/azure-ai-ml/azure/ai/ml/entities/_inputs_outputs/input.py:381

  • Input is also the public value type for top-level job inputs, so this now accepts Input(type="uri_file", default="azureml:data:1") outside component interfaces even though its path remains None. Job submission then deterministically fails in _job_operations.py:1517-1525 with “Input path can't be empty for jobs.” Scope asset defaults to component-port construction/deserialization (or add context-aware validation) so the shared job-input API does not accept a configuration it cannot submit.

This issue also appears on line 379 of the same file.

            if not self._multiple_types and self.type in IOConstants.ASSET_INPUT_TYPES:
                if isinstance(default_value, str):
                    self.default: Any = default_value
                    return

Copilot AI review requested due to automatic review settings August 21, 2026 05:01

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 13 out of 13 changed files in this pull request and generated no new comments.

Suppressed comments (1)

sdk/ml/azure-ai-ml/azure/ai/ml/entities/_job/pipeline/_io/mixin.py:122

  • Returning only the default path drops the input's declared mode on the non-dynamic node-loading path: NodeInput._build_data wraps this string as Input(type=..., path=...), so defaults configured with download or direct are submitted without that mode. Return an Input here, as the pipeline-job override below already does, so omitted asset defaults preserve their delivery mode.
        if isinstance(val, Input) and val.type in IOConstants.ASSET_INPUT_TYPES and val.default is not None:
            return val.default

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants