Skip to content

[case] domux-4057-full-eval - 97.58% accuracy on full 4057-sample official eval - #21

Closed
gebibd00-jpg wants to merge 2 commits into
iflytek:mainfrom
gebibd00-jpg:case/domux-4057-full-eval
Closed

[case] domux-4057-full-eval - 97.58% accuracy on full 4057-sample official eval#21
gebibd00-jpg wants to merge 2 commits into
iflytek:mainfrom
gebibd00-jpg:case/domux-4057-full-eval

Conversation

@gebibd00-jpg

Copy link
Copy Markdown

Ref #20

结果(官方 4057 条测试集全量,指标逐字沿用 eval/run_eval.py)

  • 格式合规 100.00% / 结果准确率 97.58% / Slot F1 99.60% / Intent F1 98.17%
  • 独立复现与模型卡 98.37% 仅差 0.79pp;失败模式分析见案例

环境

  • RTX 4090D 24GB(AutoDL 按量付费),transformers 5.15.1 + torch 2.13.0,BF16,greedy,batch 8
  • 模型 revision 6c71a32f4d624cadfd9fce9d10240d8068e53456(Hugging Face 下载,非 ModelScope)

公开 Discussion(唯一有效渠道)

注:依赖 #15(案例框架)合并后 validator 生效;本 PR 仅新增 cases/ 下案例文件,与 #15 无冲突。

@CLAassistant

CLAassistant commented Aug 23, 2026

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@gebibd00-jpg

Copy link
Copy Markdown
Author

Superseded by #22 (branch was rebuilt with DCO sign-off after the case framework #15 merged).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants