Issues

• The benchmark finished 94% by GPT-6 Astra

  CritPt-Benchmark/CritPt#14 · Filter by author adybag14-cyber opened on Sep 3, 2026
  1

• One-step vs two-step

  CritPt-Benchmark/CritPt#7 · Filter by author zaccharieramzi opened on Mar 24, 2026
  1

• System Prompt with web search has 3 reminders

  CritPt-Benchmark/CritPt#6 · Filter by author zaccharieramzi opened on Mar 24, 2026

• No subproblems?

  CritPt-Benchmark/CritPt#5 · Filter by author zaccharieramzi opened on Mar 24, 2026

• Subproblems in quantum_error_correction.json have the wrong "problem_description" field (they are all off by 1)

  CritPt-Benchmark/CritPt#3 · Filter by author scienceetonnante opened on Mar 14, 2026

• evaluation result return 0

  CritPt-Benchmark/CritPt#2 · Filter by author rhyang2021 opened on Jan 7, 2026