Skip to content

YNM 1.3.0: add empirical self-evaluation gate - #2

Merged
RobLe3 merged 10 commits into
mainfrom
codex/ynm-1.3.0-self-evaluation
Aug 20, 2026
Merged

RobLe3 merged 10 commits into
mainfrom
codex/ynm-1.3.0-self-evaluation

Conversation

@RobLe3

@RobLe3 RobLe3 commented Aug 20, 2026 •

Copy link
Copy Markdown
Owner

Summary

  • bootstraps YNM against its own repository using the existing project-native release state
  • freezes empirical protocol revision 2 before execution and separates primary effectiveness, trigger selectivity, and cross-model replication
  • executes 200 trigger requests and 40 paired CONTROL/YNM benchmark requests
  • preserves anonymized scoring packets and maintainer-operated blinded adjudication
  • validates current empirical records and evidence references
  • preserves the generated runtime package unchanged

Validation

  • repository/schema validation: PASS
  • release-integrity validation: PASS
  • deterministic tests: 80 PASS
  • package build: PASS
  • package signature unchanged: 00ee94ff9af56c2e13b39463ef7d6f567d232823e3020784bb3634795b04f976
  • Agent Skills reference validation: PASS
  • security-boundary validation: PASS
  • Linux, macOS, and Windows CI: PASS

Empirical evaluation

Primary executor: gpt-5.6-sol
Replication executor: gpt-5.6-terra, the first available frozen candidate

  • trigger executions: 200/200
  • paired benchmark executions: 40/40
  • blinded benchmark scores: 40/40
  • primary material recall: 1.000 control / 1.000 YNM
  • primary material precision: 0.8167 control / 0.8500 YNM
  • primary unsupported-claim rate: 0.1833 control / 0.1500 YNM
  • primary input-token ratio: 2.52x
  • primary output-token ratio: 2.82x
  • primary elapsed-time ratio: 2.34x

The aggregate primary metrics improved modestly, but YNM failed a frozen hard-safety criterion on the clean-project fixture by asserting correctness while failing to preserve explicitly unreviewed production evidence. Trigger behavior also produced material false negatives under the precommitted behavioral-inference rule. Replication showed several improvements but introduced a paired false-finding regression.

Release state

Version: 1.3.0
Status: release candidate
YNM-EMP-001: NO
YNM-EMP-002: NO
YNM-EMP-003: NO
Publication authorization: REQUIRES_HUMAN

The evidence does not support releasing this 1.3.0 candidate. Any correction belongs to a later candidate and requires fresh fixtures; the consumed benchmark cannot be reused as independent evidence. YNM-VAL-001 remains MAYBE.

This PR does not merge, tag, publish, or claim independent interoperability.

@RobLe3
RobLe3 merged commit 5f6c009 into main Aug 20, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant