Exercise 04: add Part 2 with augmented-state cases - #160
Open
nicolejrkim wants to merge 37 commits into
Open
Conversation
added 30 commits
August 16, 2026 16:28
Three cases extending ex04, each breaking the Markov property of the bare grid; students choose the state that restores it and reuse their ex04 VI/PI unchanged: - momentum (time lag): state (i, j, heading) - forecast (information): state (i, j, fog) - glitch (correlated noise): state (i, j, gear) Design decisions (from supervisor review): fixed augmented state so all students share one (M, N, Z) ground truth; no signature changes (one GridMdp subclass per case); maps are the ex04 family with the wonderland removed (same seeds, n_wonderland=0); an automated VI/PI coincidence check on the converged matrices emits a review label when the two submissions agree to bit level. Ground truth generated by a validated reference implementation (reproduces the shipped ex04 expected results when each case's modulation is switched off; VI==PI on every case and map; policy ground truth stores all optimal actions per state, fair to any tie-breaking). End-to-end smoke test: 15/15 transition probes, 18/18 algo tests at policy_accuracy 1.0 / R2 1.0, coincidence label clean on honest pairs and raised on duplicated solvers.
12 extra wonderland-free maps (6x6 to 12x12, sanity-filtered so the goal is reachable and worth reaching), solved by the validated reference for all three cases, with the maps stored inside the npz so nothing is regenerated at load time. exercises_def/ex04b/practice.py lets students run check_solution(MySolver) and see, per map, the exact policy-accuracy and value-R2 formulas the graded evaluation uses. Handout gains a Practice maps section. Also: npz meta is now literal_eval-safe. Smoke test: reference solver scores 1.0 on all 36 (case, map) runs; a deliberately broken solver is flagged.
Exercise 04 is now one exercise in two parts. Part 1 is the base MDP with the WONDERLAND removed completely: maps use the same generator and seeds with n_wonderland=0, the 5x5 example replaces its wonderland cells with grass, and the handout loses the teleport spec, hints, rewards and worked rows (replaced with rows recomputed on the new map). Part 2 folds in the three augmented cases (momentum, forecast, glitch) on the same maps: same method signatures, one GridMdp subclass per case, solvers gain an optional max_iters parameter. Ground truth regenerated wonderland-free by the validated reference: expected_results.npz (base, OptimalActions policies), expected_transition_results.npz (dense 2646-entry dict on the 5x5), expected_results_aug.npz (the nine augmented case/map pairs), and practice_results.npz (12 practice maps now covering base + all cases, with the student self-check moved to exercises_def/ex04/practice.py). The VI/PI coincidence label now covers base and augmented pairs alike. The separate ex04b exercise is removed. End-to-end smoke test: 54 tests assembled; 30/30 transition probes; 24/24 algo tests at policy_accuracy 1.0 / R2 1.0; coincidence check clean on 12 honest pairs and raised on 12 duplicated ones; 48/48 practice runs perfect.
New exercises_def/ex04/selfcheck.py: random_map() generates unlimited wonderland-free maps (BFS-checked goal reachability), and self_check() verifies a solution with no answer key, using only the student's own model: probability sums (full-grid scan on small maps), ABANDON mass on START, V <= 500 with V(goal) = 500, per-cell dominance for forecast and glitch, VI-vs-PI agreement, and a Monte-Carlo rollout reproducing V(start) within statistical error. Deliberately contains no solver logic; the exact certificate (the Bellman residual) is described in the handout in math only, for students to implement themselves. Handout gains a "Make your own maps, and verify without an answer key" section with the checklist, the certificate, and the honest caveats (checks are necessary not sufficient; a wrong model can self-certify, which the worked examples and practice maps anchor against). Smoke test: reference solutions pass all checks on a fresh random map (base and forecast); corrupted values are caught by three independent checks.
The transition-probability file pickled Action enums under a bare "structures" module path, so it could not be unpickled inside the student container (and it contained stray STAY entries for non-goal states). All four data files are now produced by the exercises-private generator with plain-int keys and lists, saved with savez_compressed. Values and optimal action sets are verified equivalent to the previous files.
Drops the Cell.WONDERLAND enum member (CLIFF keeps its value 5, so all stored maps stay valid), the unused placement machinery and n_wonderland parameter in generate_map, the wonderland plot branches and evaluation-mask special case, the purple color entry, and every remaining mention in the handout and comments. The generator's RNG call sequence is unchanged: all practice maps regenerate byte-identically and the full validation suite passes; only expected_results_aug.npz changes (a metadata string).
…e example figure Brings back the separate Value Iteration, Policy iteration, and Expected outcome subsections with full class skeletons (updated to the current solve signature with max_iters), the output-format and CLIFF-exclusion paragraph, and Figure 1 - regenerated from the new 5x5 map with the reference solution so it matches what students see in their reports.
… no max_iters - Cell.CLIFF renumbered from 5 to 4, closing the gap; data files regenerated (verified: only the cliff byte changes in the stored maps, every value, policy, and transition probability is identical). - The Part-2 model constants are no longer pre-defined in structures.py; students take the numbers from the handout themselves. - solve() loses the max_iters parameter everywhere (templates and handout); the automated VI/PI coincidence label remains the anti-copying check.
Three practice maps (6x6, 9x9, 12x12) now run inside get_exercise4 as ungraded maps 3-5, reported and scored exactly like the public test maps, for Part 1 and every Part-2 case. The maps regenerate deterministically from PRACTICE_MAP_SPECS, so only their solutions ship (expected_results_practice.npz); the separate practice.py module, its check_solution API, and the 12-map practice_results.npz are removed. Verified: the three maps and their solutions are identical to entries 0, 6 and 10 of the previous practice set.
The no-answer-key consistency checks are dropped; random_map (random maps with the goal guaranteed reachable) moves verbatim into map.py, where the practice-map regeneration and students can keep using it. Verified: the practice maps regenerate byte-identically through the relocated function.
get_test_grids() now returns six maps (5x5, 10x10, 40x40, 6x6, 9x9, 12x12); the smaller three regenerate deterministically from SMALL_TEST_MAP_SPECS. Their solutions merge into expected_results(.aug).npz (keys 3-5, loaders size themselves from the file), the separate practice plumbing and expected_results_practice.npz are removed, and the handout lists the six maps in the test-cases section. Still 78 built-in test cases; the server evaluation now covers the small maps too.
The default "--exercise 04" is back to the three public maps (5x5, 10x10, 40x40). A second exercise id "04quick" runs the identical pipeline on three smaller maps (6x6, 9x9, 12x12, report ids 3-5) for fast iteration while developing - a full naive-solution run finishes in a fraction of the time. The expected-results files already carry solutions for map ids 0-5; the loaders now take the id subset. The handout explains both run modes.
…er ones "--exercise 04" runs the three public maps (54 test cases); "--exercise 04all" runs all six maps (78 test cases), adding the 6x6, 9x9 and 12x12 maps as ids 3-5 for broader coverage.
available_exercises is back to its original single "04" entry. Students opt into the three extra maps (6x6, 9x9, 12x12, report ids 3-5) with PDM4AR_EX04_ALL_MAPS=1; the handout shows the command.
The env variable is replaced by ALL_MAPS = False at the top of exercises_def/ex04/ex04.py; students flip it to True to also run the 6x6, 9x9 and 12x12 maps (report ids 3-5). Handout updated.
The student's and ground-truth panels each auto-scale their colormap; a student array holding 0.0 at CLIFF cells (which are not states) stretched its scale to [0, max] while the ground truth (NaN at cliffs) spanned only the real values, so identical values rendered in different colors. Both the base and per-slice plots now mask cliffs to NaN before imshow.
added 7 commits
August 31, 2026 13:53
Value slices now paint black rectangles over CLIFF cells (instead of leaving them transparent), and the policy slices skip cliff cells so no arrow can be drawn on top of one.
The Rectangle overlays (inherited from the original code) could leave
hairline seams at small cell sizes in the PDF panels. Cliff cells are
already masked to NaN for the color scale; cmap.set_bad("k") now paints
them black inside the image raster itself, pixel-perfect, and the patch
overlays are removed from both the base and per-slice value plots.
- ex4_evaluation_algo and ex4_transition_prob_evaluation initialize their scores before the ground-truth guard, so a test case without an expected result scores 0 instead of raising UnboundLocalError. - get_test_grids' evaluation_tests annotation matches the actual 3-field tuples ((m, n), n_cliff, seed). - Removed the dead pkl-based transition-matrix helper cluster in data.py (no callers, reads a file no generator produces) and the unused is_too_close_to_rift_or_border leftover in map.py. - get_exercise4 reuses the already-built grids for the transition probes instead of regenerating all maps.
- Replace the in-module VI/PI coincidence check, which could never fire (each test case runs in its own run_with_timer process, so the VI cache was never visible to the PI case), with case_key/value_checksum stamps on every value-test result; the comparison itself moves to the evaluation side. Drop the docs sentence describing the mechanism. - Fix the misspelled __post__init__ dataclass hooks so the performance sanity asserts actually run, and make the empty-aggregation path return zeros explicitly (np.mean([]) is NaN, not ZeroDivisionError). - Accept any integer dtype for plotted policies (int32 previously crashed the whole test case) and score object-dtype student policies by set membership (list-in-list could never match). - Make the solver templates shape-aware so a fresh checkout returns (M, N, Z) zeros for Part-2 mdps and scores 0 instead of crashing every augmented test with an IndexError.
expected_transition_results_aug.npz holds full (state, action, next_state) coverage per case on the 5x5 map; AUG_PROBES is now scenario-only and get_expected_results_transition_aug looks expectations up in the file, mirroring the Part-1 loader. As in Part 1, students can add their own probe triples and the file already covers them. Verified to load inside the student container (numpy 1.26).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Changes
(M, N, Z)arrays. Handout rewritten with per-case transition tables and worked examples.WONDERLANDremoved entirely;Cell.CLIFFrenumbered to 4.case_key/value_checksumstamps so the server can screen VI/PI submissions for verbatim reuse.ALL_MAPS;random_map(...)also generates student practice maps with the goal guaranteed reachable.Validation
ALL_MAPS) executed end to end in the devcontainer at accuracy 1.0; all data files load in the student container.