Repository navigation
Add yylo-benchmark registry card - #13
Open
InsightFactoryAPP wants to merge 2 commits into
Open
InsightFactoryAPP wants to merge 2 commits into
InsightFactoryAPP wants to merge 2 commits into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
YYLO Benchmark is a thin trusted-host experiment runner for agent tasks: independent attempts over historical Ledger tasks, supplied coding prompts, and workflows. Retained outputs can be evaluated later with different checks or judges; no combined score, winner, or capability inference is made. Categorized
test+agentfollowing the inspect-ai / swe-agent / deepeval entries. Sibling of theyyloCLI card (#9, merged); one tool per PR.Contribution Rules & Checklist
data/tools/yylo-benchmark.yamladded; README regenerated locally for validation only and not included in the diffdata/tools/yylo-benchmark.yaml,name: yylo-benchmarkexecute_code,network_access,read_files,write_files,secret_exposure— all from the whitelistrisk.level: highwith two concrete guardrailsDisclosure
Self-submission by the YYLO team (affiliated), prepared with AI-agent assistance. Every summary / use_when / avoid_when phrase is quotable from the live README. Happy to adjust category, risk framing, or wording to fit the registry's conventions.