Flagging something that's adjacent to this repo rather than squarely in it, so apologies if this isn't the right place — happy to be redirected.
EvalPort is an open, Apache-2.0 JSON format (with real JSON Schemas + Python/TS SDKs) for moving LLM eval test cases/suites/results between tools that otherwise all use incompatible shapes. I built a braintrust-openeval-adapter that converts a Braintrust Eval() result to/from an EvalPort suite:
from braintrust import Eval
from autoevals import Factuality
from braintrust_openeval_adapter import to_openeval, from_openeval
result = Eval(
"my-project",
data=lambda: [{"input": "What is 2+2?", "expected": "4"}],
task=my_task,
scores=[Factuality],
)
suite = to_openeval(result) # each scorer (Factuality, ExactMatch, ...) -> its own EvalPort graderSource + tests: https://github.com/adhabnr-ux/evalport/tree/main/adapters/braintrust-openeval-adapter (6 tests, run against the real EvalPort validator)
The mismatch: that adapter is Python, and this repo is Braintrust's JS/TS SDK (I saw the note in your README that Python split into its own repo). The Eval() signature it targets — data/task/scores, with input/expected/output/scores per case — looks identical between the Python and TS SDKs from your own README's tutorial.eval.ts example, so a toOpenEval()/fromOpenEval() pair on the JS side seems like it would look almost the same shape-wise:
import { Eval } from "braintrust";
// hypothetical:
// import { toOpenEval } from "some-js-adapter";
// const suite = toOpenEval(evalResult);Two questions rather than an ask: is a JS/TS equivalent of this something that would be useful to have, and is this repo the right venue for that conversation, or does it belong somewhere else given the recent Python split? No pressure — just didn't want to build a JS port speculatively without checking first whether it'd land anywhere useful.
Flagging something that's adjacent to this repo rather than squarely in it, so apologies if this isn't the right place — happy to be redirected.
EvalPort is an open, Apache-2.0 JSON format (with real JSON Schemas + Python/TS SDKs) for moving LLM eval test cases/suites/results between tools that otherwise all use incompatible shapes. I built a
braintrust-openeval-adapterthat converts a BraintrustEval()result to/from an EvalPort suite:Source + tests: https://github.com/adhabnr-ux/evalport/tree/main/adapters/braintrust-openeval-adapter (6 tests, run against the real EvalPort validator)
The mismatch: that adapter is Python, and this repo is Braintrust's JS/TS SDK (I saw the note in your README that Python split into its own repo). The
Eval()signature it targets —data/task/scores, withinput/expected/output/scoresper case — looks identical between the Python and TS SDKs from your own README'stutorial.eval.tsexample, so atoOpenEval()/fromOpenEval()pair on the JS side seems like it would look almost the same shape-wise:Two questions rather than an ask: is a JS/TS equivalent of this something that would be useful to have, and is this repo the right venue for that conversation, or does it belong somewhere else given the recent Python split? No pressure — just didn't want to build a JS port speculatively without checking first whether it'd land anywhere useful.