Slim plan contract and browser proof checks#124
Conversation
There was a problem hiding this comment.
Code Review
This pull request replaces the retired machine-validated JSON UI Proof Contract with a simpler, markdown-based Browser Proof Contract. It updates the planner, executor, and verifier workflows, as well as the CLI verification logic in bin/lib/phase.mjs to parse and validate browser proof plans and observation records. Feedback on the changes highlights potential parsing failures in sectionFieldValue due to markdown formatting and inline comments, as well as a potential CLI crash in readLinkedObservationRecords when attempting to read unreadable paths or directories without error handling.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
| function sectionFieldValue(section, label) { | ||
| const text = String(section || ''); | ||
| const escapedLabel = String(label).replace(/[.*+?^${}()|[\]\\]/g, '\\$&'); | ||
| const match = new RegExp(`^\\s*(?:[-*]\\s*)?${escapedLabel}:\\s*(.*)$`, 'im').exec(text); | ||
| if (!match) return ''; | ||
|
|
||
| const inlineValue = normalizeScalarValue(match[1]); | ||
| if (inlineValue) return inlineValue; | ||
|
|
||
| const nestedValues = []; | ||
| const followingLines = text.slice(match.index + match[0].length).split(/\r?\n/); | ||
| for (const line of followingLines) { | ||
| if (!line.trim()) continue; | ||
| if (/^\S/.test(line)) break; | ||
| const nestedBullet = line.match(/^\s+[-*]\s*(.+)$/); | ||
| if (nestedBullet) { | ||
| nestedValues.push(nestedBullet[1].trim()); | ||
| continue; | ||
| } | ||
| const nestedText = line.match(/^\s{2,}(.+)$/); | ||
| if (nestedText) { | ||
| nestedValues.push(nestedText[1].trim()); | ||
| continue; | ||
| } | ||
| break; | ||
| } | ||
| return normalizeScalarValue(nestedValues.join(' ')); | ||
| } |
There was a problem hiding this comment.
The current implementation of sectionFieldValue has two issues:
- The regular expression used to match the label (
^\s*(?:[-*]\s*)?${escapedLabel}:\s*(.*)$) does not account for bold or italic markdown formatting (e.g.,**Result**:or*Result*:), which is commonly used in markdown lists and explicitly shown inobservation-record.md. This will cause parsing to fail for valid observation records. - When parsing multi-line nested values,
normalizeScalarValueis called on the joined string. If any of the nested lines contain an inline comment (e.g.,path1 # comment),stripInlineCommentwill truncate the entire joined string from the first#, discarding all subsequent lines.
Normalizing each nested line individually and updating the regex to support optional markdown bold/italic syntax around the label resolves both issues.
function sectionFieldValue(section, label) {
const text = String(section || '');
const escapedLabel = String(label).replace(/[.*+?^${}()|[\]\\]/g, '\\$&');
const match = new RegExp(`^\\s*(?:[-*]\\s*)?[*_]*${escapedLabel}[*_]*:\\s*(.*)$`, 'im').exec(text);
if (!match) return '';
const inlineValue = normalizeScalarValue(match[1]);
if (inlineValue) return inlineValue;
const nestedValues = [];
const followingLines = text.slice(match.index + match[0].length).split(/\r?\n/);
for (const line of followingLines) {
if (!line.trim()) continue;
if (/^\S/.test(line)) break;
const nestedBullet = line.match(/^\s+[-*]\s*(.+)$/);
if (nestedBullet) {
nestedValues.push(normalizeScalarValue(nestedBullet[1]));
continue;
}
const nestedText = line.match(/^\s{2,}(.+)$/);
if (nestedText) {
nestedValues.push(normalizeScalarValue(nestedText[1]));
continue;
}
break;
}
return nestedValues.join(' ');
}| function readLinkedObservationRecords(summaryContent, summaryFullPath, workspaceRoot) { | ||
| const records = []; | ||
| const matcher = /^\s*(?:[-*]\s*)?(?:Browser Proof Observation|Browser proof observation|Observation record|Browser proof record):\s*(.+)$/gim; | ||
| let match; | ||
| while ((match = matcher.exec(String(summaryContent || '')))) { | ||
| const linkedPath = sanitizeLinkedObservationPath(match[1]); | ||
| if (!linkedPath) continue; | ||
| const candidates = isAbsolute(linkedPath) | ||
| ? [linkedPath] | ||
| : [join(dirname(summaryFullPath), linkedPath), join(workspaceRoot, linkedPath)]; | ||
| const existingPath = candidates.find((candidate) => existsSync(candidate)); | ||
| if (existingPath) records.push(readFileSync(existingPath, 'utf-8')); | ||
| } | ||
| return records; | ||
| } |
There was a problem hiding this comment.
The readFileSync call is performed on existingPath without a try-catch block. Since existingPath is derived from user-provided paths in markdown files, it is untrusted and could potentially point to a directory (which existsSync returns true for) or be unreadable due to permission issues. If this happens, readFileSync will throw an unhandled exception (such as EISDIR or EACCES) and crash the CLI. Wrapping the read operation in a try-catch block prevents these crashes.
function readLinkedObservationRecords(summaryContent, summaryFullPath, workspaceRoot) {
const records = [];
const matcher = /^\s*(?:[-*]\s*)?(?:Browser Proof Observation|Browser proof observation|Observation record|Browser proof record):\s*(.+)$/gim;
let match;
while ((match = matcher.exec(String(summaryContent || '')))) {
const linkedPath = sanitizeLinkedObservationPath(match[1]);
if (!linkedPath) continue;
const candidates = isAbsolute(linkedPath)
? [linkedPath]
: [join(dirname(summaryFullPath), linkedPath), join(workspaceRoot, linkedPath)];
const existingPath = candidates.find((candidate) => existsSync(candidate));
if (existingPath) {
try {
records.push(readFileSync(existingPath, 'utf-8'));
} catch (err) {
// Skip unreadable files or directories safely
}
}
}
return records;
}There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: bc819b0bb2
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| }; | ||
| } | ||
|
|
||
| if (requiredPlans.length === 1) { |
There was a problem hiding this comment.
Require the single required plan to match its observation
When a phase has multiple plan artifacts but exactly one plan sets browser_proof_required: true, this shortcut accepts any complete Browser Proof Observation found in any matching summary, even if that observation names a different plan. Since matchingSummaries is collected phase-wide, a stale or unrelated observation from another slice can make gsdd verify report the UI-sensitive plan as verified without proof for that plan.
Useful? React with 👍 / 👎.
Summary:
Verification: