LCORE-2080: Added E2E Steps for Agent Skills

[jrobertboos]

Description

Added the missing E2E steps for testing agent skills.

Type of change

• Refactor
• New feature
• Bug fix
• CVE fix
• Optimization
• Documentation Update
• Configuration Update
• Bump-up service version
• Bump-up dependent library
• Bump-up library or tool used for development (does not change the final image)
• CI configuration change
• Konflux configuration change
• Unit tests improvement
• Integration tests improvement
• End to end tests improvement
• Benchmarks improvement

Tools used to create PR

Identify any AI code assistants used in this PR (for transparency and review context)

• Assisted-by: Cursor (Composer 2.5)
• Generated by: Cursor (Composer 2.5)

Related Tickets & Documents

• Related Issue LCORE-2080
• Closes LCORE-2080

Checklist before requesting a review

• I have performed a self-review of my code.
• PR has passed all pre-merge test jobs.
• If it is a core feature, I have added thorough tests.

Testing

• Please provide detailed steps to perform tests related to this code change.
• How were the fix/results from this change verified? Please provide relevant screenshots or results.

[coderabbitai]

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: ded61893-0c6a-4f40-b77c-ae3457d32b76

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

✨ Simplify code

• Create PR with simplified code

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[jrobertboos]

TODO: Need to reflect skill tools (list_skills, load_skill, read_skill_resource) in /tools.

[jrobertboos]

The "Error Paths" will have to be skipped for now as the skill tools do fail and produce a result, but it's a different type that the response-building code silently discards.

Below I have helpful part of conversation with Claude about the issue.

---

Pydantic-ai catches ModelRetry and wraps the error in a RetryPromptPart (not a ToolReturnPart). The FunctionToolResultEvent.part is typed as ToolReturnPart | RetryPromptPart — it can be either.

Where LCS drops it:

In the non-streaming path, build_turn_summary_from_agent_run only processes ToolReturnPart:

query.py
Lines 266-269

        elif isinstance(message, ModelRequest):
            for request_part in message.parts:
                if isinstance(request_part, ToolReturnPart):
                    process_function_tool_result(state, request_part)

In the streaming path, the same filter exists:

streaming.py
Lines 522-524

    part = event.part
    if not isinstance(part, ToolReturnPart):
        return None

Both paths explicitly ignore RetryPromptPart, so the retry/error message for load_skill is never surfaced as a tool_result in the API response.

The result:

• Both tool calls appear (because both ToolCallPart instances from the ModelResponse are processed)
• Only the list_skills result appears (because it succeeded and produced a ToolReturnPart)
• The load_skill result is missing (because it raised ModelRetry → became a RetryPromptPart → silently dropped)

[jrobertboos]

This will likely be quite flaky as the LLM (through appendage of system prompt, I think) is given only the "names" of skills so sometimes will result in just load_skill and read_skill_resource being used completely skipping list_skills.