Repository navigation
fix(ci): install Julia + QuartoNotebookRunner in ci-benchmark - #328
Merged
Merged
Conversation
Cold benchmark run 38109155362 (force_full_rebuild=true) failed at "Render website with Babelquarto" on [221/270] Julia/Heatmap.qmd: with _freeze deleted, Julia qmds re-execute and the julia-engine tried to bootstrap its QuartoNotebookRunner server at render time, dying with "ERROR: Ensuring an updated julia server environment failed" (after a zstd library-version mismatch inside Pkg's registry setup). The benchmark workflow had no Julia setup at all — its only Julia reference was the worker pre-render skip. Mirror the proven setup from quarto-publish-update-cache.yml: julia-actions/setup-julia, the pinned QNR env (QUARTO_JULIA_PROJECT skips Quarto's network-dependent ensureQuartoNotebookRunnerEnvironment at render time), and the Julia/ tutorial project instantiate. Two deliberate deviations from the update-cache workflow: - cache/restore instead of cache: the benchmark must never SAVE caches (its header says so — saving would pollute the shared pool); restore still lets warm benchmark runs reuse the shared ~/.julia cache. - Julia setup is unconditional (no need_julia gate): the benchmark has no changed-file detection step to feed such a gate, and a cold run always re-executes Julia qmds. Also clean up the phase marker: Julia setup time lands between python_setup and pre_render marks, so "Pre-render non-frozen QMDs" in the report absorbs it — acceptable for a manual-diagnostic workflow, not worth extra clock entries. Co-Authored-By: Claude Code <noreply@anthropic.com>
Contributor
✅ QMD Format CheckNo new QMD tutorial files to check.
|
Second validation run 38114723635 proved the Julia setup works (setup/ cache/install all green, zstd bootstrap error gone, QUARTO_JULIA_PROJECT set) but babelquarto still failed at [221/270] Julia/Heatmap.qmd with "ERROR: SyntaxError: Unexpected end of JSON input" from julia-engine's JSON handshake with the QuartoNotebookRunner server (writeJuliaCommand, julia-engine.js:1202). Structural difference vs. the working quarto-publish-update-cache.yml: the benchmark had no dedicated "Pre-render Julia notebooks" step, so Julia qmds executed inline inside babelquarto's temp-dir render. Port the proven pre-render step from update-cache: render Julia/*.qmd before the parallel pre-render phase, with QUARTO_JULIA_PROJECT + TMPDIR set and a julia_server_log tail on failure. This freezes Julia outputs into the workspace _freeze so babelquarto only pandoc-renders them. Co-Authored-By: Claude Code <noreply@anthropic.com>
Contributor
✅ QMD Format CheckNo new QMD tutorial files to check.
|
Third validation run 38120426338 failed in the new pre-render step itself: every Julia/Heatmap.qmd render died in under a second with "NotFound: No such file or directory (os error 2): tmpdir" from quarto's initSessionTempDir (makeTempDirSync). The step sets TMPDIR/TMP/TEMP=/mnt/TMP, but in ci-benchmark that directory was only created later (in the R pre-render step's own mkdir); in quarto-publish-update-cache.yml a dedicated "Create temporary directory" step runs before the render pipeline. Move that mkdir/chown step ahead of "Pre-render Julia notebooks". Co-Authored-By: Claude Code <noreply@anthropic.com>
Contributor
✅ QMD Format CheckNo new QMD tutorial files to check.
|
Member
Author
Validation — all green ✅Cold benchmark run 38121976629: success (1 h 42 min total, force_full_rebuild=true). Progression across the three validation runs:
Final state: Also verified: the regular PR checks ( |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Cold benchmark run 38109155362 (
force_full_rebuild=true) failed at "Render website with Babelquarto" on[221/270] Julia/Heatmap.qmd:Root cause:
ci-benchmark.ymlhad no Julia setup at all (its only Julia reference was the worker pre-render skip[[ "$rel" == Julia/* ]] && continue). With_freezedeleted by a cold run,Julia/*.qmdre-execute during the babelquarto render, and Quarto's julia-engine tried to bootstrap QuartoNotebookRunner at render time — which needs network +Pkg.update()and died.Fix
Mirror the proven Julia setup from
quarto-publish-update-cache.yml(in place on main since the Julia support PR):julia-actions/setup-julia@v3~/.juliakeyed onJulia/Project.tomlQUARTO_JULIA_PROJECTexport (skips Quarto's network-dependentensureQuartoNotebookRunnerEnvironment)julia --project=Juliainstantiate of the tutorial environment (CairoMakie, DataFrames, Statistics)Two deliberate deviations:
actions/cache/restoreinstead ofactions/cache— the benchmark must never save caches (workflow header: saving would pollute the shared freeze/julia pool with benchmark entries). Restore-only still lets warm benchmark runs reuse the shared~/.juliacache.need_juliagate like quarto-publish.yml) — the benchmark has no changed-file detection step to feed such a gate, and a cold run always re-executes Julia qmds. On warm runs Julia setup costs ~1–2 min with the package cache restored; cold runs already cost 60+ min, so this is negligible.Also re-ran a cold benchmark after the fix lands to validate the 677-package renv.lock cold path end-to-end (B1).
Test plan
workflow_dispatchcold benchmark (force_full_rebuild=true) on this branch:Julia/Heatmap.qmdrenders via the pre-provisioned QNR env, noensureQuartoNotebookRunnerEnvironmentfailure — run 38121976629, all steps green🤖 Generated with Claude Code