All pull requests

Pull requests

• fix: prevent code execution in FollowBench post-processing

  #2669·JimmyWang0417 opened Oct 11, 2026·

  ·
  1

  1

  

• [Compassistant] EarthSE: A Benchmark for Evaluating Earth Scientific Exploration Capability of LLMs adapter for opencompass

  #2667·mzr1996 opened Oct 10, 2026Collaborator·

  ·
  

• Fix GSM8K parsing of thousands-separated answers

  #2664·Hahaa909 opened Oct 9, 2026·

  ·
  1

  

• [Fix] Support CascadeEvaluator custom scoring without a rule evaluator

  #2663·YueshenZ opened Oct 9, 2026·3 tasks done·

  ·

• [Fix] Correct MATH JudgeLLM config imports in example and guides

  #2662·YueshenZ opened Oct 2, 2026·2 of 3 tasks·

  ·
  

• [Fix] score an EESE word judgment of correct as 10

  #2661·SashaMIT opened Sep 29, 2026·

  ·
  

• [Fix] Compare TydiQA answers case-insensitively

  #2660·MohammadHijjawi97 opened Sep 28, 2026·3 of 6 tasks·

  ·

• [Fix] Escape think tokens in extract_non_reasoning_content

  #2659·MohammadHijjawi97 opened Sep 28, 2026·3 of 6 tasks·

  ·
  

• fix(followbench): use ast.literal_eval instead of eval on model output

  #2656·linhongyu510 opened Sep 25, 2026·

  ·
  1

• Fix gsm8k_postprocess truncating numbers at thousands separators

  #2648·FOWEPJF255 opened Sep 18, 2026·4 of 6 tasks·

  ·
  1

  3

  

• [Feature] Add OpenAISDK model using completion API

  #2645·qq1069730908 opened Sep 16, 2026·

  ·
  1

• [Datasets] Add Berkeley Function Calling Leaderboard (BFCL)

  #2644·DingNova opened Sep 15, 2026·

  ·
  1

• fix(evaluator): make ToxicEvaluator max score NaN-aware

  #2643·qq1069730908 opened Sep 15, 2026·

  ·
  1

  

• fix lcb stringIO executor

  #2638·KL35kkk opened Sep 9, 2026Collaborator·

  ·

• Replace removed tokenizer.encode_plus in DatasetEncoder

  #2636·rrrxxx0510 opened Sep 8, 2026·

  ·
  1

  1

  

• Replace removed tokenizer.batch_encode_plus with direct call

  #2634·rrrxxx0510 opened Sep 7, 2026·

  ·
  1

  1

  

• [Fix] Follow redirects when a custom httpx client is supplied to the OpenAI SDK

  #2633·CastilloLuis opened Sep 5, 2026·

  ·
  

• [Submission] WeddooBench: A Wedding Industry Benchmark for Large Language Models (Weddoo-Doo-Q3.8.W9)

  #2632·Weddoo99 opened Sep 5, 2026·5 tasks done·

  ·
  

• feat(dataset): add generative CivilComments config

  #2625·motodriver opened Aug 31, 2026·

  ·
  1

• docs: clarify prompt version in summaries

  #2623·motodriver opened Aug 31, 2026·

  ·
  

• test: cover HumanEval evaluator details

  #2622·motodriver opened Aug 31, 2026·

  ·
  

• Fix MBPP answer extraction

  #2621·Marchematics opened Aug 30, 2026·

  ·
  1

  1

  

• Fix gsm8k modelscope config

  #2620·Marchematics opened Aug 30, 2026·

  ·
  1

• Skip token type ids in Hugging Face backend

  #2619·Marchematics opened Aug 30, 2026·3 tasks done·

  ·
  1

  4

  

• docs: correct three docstring defaults that disagree with the signatures

  #2618·avalyset opened Aug 30, 2026·3 of 6 tasks·

  ·