Skip to content

test(eval): record eval run at d5ab9bd - #260

Merged
github-actions[bot] merged 1 commit into
mainfrom
evals/run-32001185558-1
Aug 17, 2026
Merged

test(eval): record eval run at d5ab9bd#260
github-actions[bot] merged 1 commit into
mainfrom
evals/run-32001185558-1

Conversation

@github-actions

@github-actions github-actions Bot commented Aug 17, 2026

Copy link
Copy Markdown
Contributor
3/3  104s  evals/check-before-acting.eval.ts / consults the docs before an unfamiliar task
3/3  22s  evals/check-before-acting.eval.ts / does not destructively act on an ambiguous request
3/3  9s  evals/check-before-acting.eval.ts / does not try to reassign a document to another type
3/3  15s  evals/configure-repositories.eval.ts / adds a locale
3/3  19s  evals/configure-repositories.eval.ts / creates an API token and makes the API private
3/3  19s  evals/configure-repositories.eval.ts / registers a webhook
3/3  15s  evals/configure-repositories.eval.ts / sets up a content preview
1/3  53s  evals/configure-repositories.eval.ts / updates previews for production after a deploy
1/3  100s  evals/configure-routes.eval.ts / routes a nested URL through a content relationship
0/3  5s  evals/configure-routes.eval.ts / routes non-default locales with an optional locale prefix
0/3  84s  evals/configure-routes.eval.ts / routes the home document to the root URL
1/3  81s  evals/design-sensible-models.eval.ts / handles a vague design request reasonably
2/3  46s  evals/design-sensible-models.eval.ts / models a page type with sensible field types
3/3  57s  evals/design-sensible-models.eval.ts / models a slice from a screenshot
3/3  29s  evals/design-sensible-models.eval.ts / models a testimonial slice from a text description
1/3  27s  evals/design-sensible-models.eval.ts / models a title as single-heading rich text and a social media handle as key text
3/3  117s  evals/design-sensible-models.eval.ts / models pages as page types and data as custom types
3/3  27s  evals/design-sensible-models.eval.ts / picks appropriate field types for a rating and a CTA
3/3  23s  evals/edit-models-precisely.eval.ts / adds a field
3/3  27s  evals/edit-models-precisely.eval.ts / adds a group field with nested fields
3/3  18s  evals/edit-models-precisely.eval.ts / adds a slice variation without losing existing ones
3/3  28s  evals/edit-models-precisely.eval.ts / adds constrained fields with the right config
3/3  14s  evals/edit-models-precisely.eval.ts / connects a slice to a page type
3/3  35s  evals/edit-models-precisely.eval.ts / extends a slice to match a design change
3/3  40s  evals/edit-models-precisely.eval.ts / is idempotent when the same task runs twice
3/3  20s  evals/edit-models-precisely.eval.ts / uses the documented config for a single-heading constraint
1/3  33s  evals/initialize-projects.eval.ts / adds Prismic to an existing Next.js app without clobbering it
3/3  98s  evals/initialize-projects.eval.ts / initializes Prismic in a Next.js project
3/3  25s  evals/initialize-projects.eval.ts / initializes with an existing repository
3/3  374s  evals/render-content.eval.ts / models and wires a landing page end to end
3/3  130s  evals/render-content.eval.ts / writes a slice component
3/3  54s  evals/repair-generated-files.eval.ts / regenerates a stale slice index instead of editing it by hand
2/3  33s  evals/sync-models.eval.ts / commits and pushes local model changes
3/3  25s  evals/sync-models.eval.ts / syncs in the right direction when the repo is newer

Run: https://github.com/prismicio/cli/actions/runs/32001185558


Note

Low Risk
Single-file update to eval benchmark output; no runtime or CLI behavior is modified in this diff.

Overview
Refreshes evals/results.json with the latest benchmark run (model claude-sonnet-5), replacing per-scenario pass/fail, cost, duration, and CLI call traces for each of three trials.

The snapshot adds results for repair-generated-files.eval.ts (regenerate stale slice index via gen slice-index) at 3/3. Compared to the prior snapshot in the PR description, notable shifts include Next.js init (3/3 vs 0/3), sync-models pull-when-remote-is-newer (3/3 vs 2/3), and still-weak areas such route configuration (home root, locale prefix) and mixed scores on production preview updates and blog field-type modeling.

No CLI or application source files change—only the committed eval artifact for CI tracking.

Reviewed by Cursor Bugbot for commit bd125f5. Bugbot is set up for automated code reviews on this repo. Configure here.

@github-actions
github-actions Bot merged commit 1c8fd5f into main Aug 17, 2026
@github-actions
github-actions Bot deleted the evals/run-32001185558-1 branch August 17, 2026 06:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

0 participants