Skip to content

Show technical output when learner hints do not match - #187

Merged
kbuffardi merged 3 commits into
mainfrom
fix/issue-186-cxxtest-fallback
Sep 17, 2026
Merged

kbuffardi merged 3 commits into
mainfrom
fix/issue-186-cxxtest-fallback

Conversation

@kbuffardi

Copy link
Copy Markdown
Contributor

Summary

  • stop synthesizing generic learner hints for unrecognized CxxTest, JUnit, and pytest failures
  • show verbatim technical output directly in Outcome when no tailored hint matches
  • preserve recognized hints, structured Expected/Actual comparisons, multi-failure navigation, and full Output details
  • render empty expected or actual output explicitly

Depends on codewit-us/codeval#33 for structured CxxTest comparison extraction. This PR remains backward compatible with the current evaluator and can deploy first.

Fixes #186

Testing

  • npx nx test api --runInBand --testPathPattern=learnerHints.spec.ts
  • npx nx test client -- src/components/codeblock/CodeSubmission.spec.tsx
  • npx nx test api --runInBand
  • npx nx build api --configuration=production
  • npx nx build client --configuration=production
  • changed implementation and client test files pass ESLint

Existing repository failures

  • Full client tests stop in src/app/app.spec.tsx because @uiw/react-markdown-preview requires an ES module from CommonJS (ERR_REQUIRE_ESM); the changed component suite passes.
  • Repository-wide lint reports existing errors outside this change. The only report in a touched API test file is the pre-existing no-useless-escape violation at line 772.

Let unrecognized evaluator formats fall through to their technical output while preserving tailored guidance for known failures.\n\nRefs #186\n\nCo-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Render verbatim technical results in Outcome when no tailored hint exists while preserving structured diffs, navigation, and full Output details.\n\nFixes #186\n\nCo-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Render both sides of structured output mismatches and label empty values explicitly so valid CxxTest comparisons remain complete.\n\nRefs #186\n\nCo-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@kbuffardi
kbuffardi merged commit c0ac015 into main Sep 17, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Fall back to CxxTest output when beginner-friendly hints do not match

1 participant