Skip to content

Fix runcluster.pl - #856

Open
mernst wants to merge 314 commits into
codespecs:masterfrom
mernst:issue365
Open

mernst wants to merge 314 commits into
codespecs:masterfrom
mernst:issue365

Conversation

@mernst

@mernst mernst commented Oct 5, 2026 •

Copy link
Copy Markdown
Member

Merge #885, #886, #887, #888, #880, #876, #875, #877, #867, #868, #870, and #869 first. (#880 is required because #875 depends on it, and #870 because #869 depends on it.) This pull request includes their changes, because its tests depend on them. After they are merged, this PR changes only the runcluster scripts, their tests, and the manual.

Fixes #365.
Fixes #537.

runcluster.pl failed on every input, and on a malformed trace it gave no useful error message.

Scripts:

  • runcluster.pl:
    • Run DtraceNonceFixer once, into a temporary copy of each trace that both extract_vars.pl and dtrace-add-cluster.pl read, instead of overwriting the user's dtrace file.
    • Write a spinfo section for each clustered program point, rather than one that matched only :::OBJECT program points.
    • Spell --var-omit-pattern correctly.
    • Decls files are optional: declarations may be in the dtrace files, as Chicory writes them.
    • When extract_vars.pl rejects a trace, say which input file the temporary copy came from.
  • extract_vars.pl:
    • Support the version 2 declaration format and gzipped decls files. Daikon no longer reads version 1 files (Drop support for the version 1 file format #880).
    • Check that the variables in each dtrace record match the declaration, and fail with a clear message if they do not. The trace attached to Trouble terminating when running 'runcluster.pl' and help with inputs for daikon #365 omits the first variable from every record; it now produces a message such as outfile_2a_runcluster_temp_nonces.dtrace line 4: expected variable Time, got 179.736000 for program point 0x305programpoint:::POINT.
    • Allow a program point to be declared in more than one decls file.
    • Fix $value =~ -5, which should be an assignment.
  • dtrace-add-cluster.pl: escape . in filename regexes. s/.cluster// matched "ncluster" in "runcluster_temp", so every record was discarded and Daikon reported "No samples found". Skip declarations in dtrace files.
  • decls-add-cluster.pl: support the version 2 declaration format, gzipped files, dtrace files that contain declarations, and a decls file in the current directory.

Tests are in tests/script-tests/runcluster/, run by scripts/test-misc.sh. They cover declarations in a decls file and in the dtrace file, a malformed trace, and a check that Daikon loads every splitter in the resulting .spinfo file.

@coderabbitai

coderabbitai Bot commented Oct 5, 2026 •

Copy link
Copy Markdown

Warning

Review limit reached

  • Run on-demand review

This review includes 73 billable files and costs up to $18.25.

Or wait 38 minutes for your next included review.

Check out review usage here.

View limit details

Limit details: You’ve used all 2 included reviews currently available.

Learn how review limits work.

Review configuration:

⚙️ Run configuration
  • Configuration used: Repository: codespecs/daikon/.coderabbit.yaml
  • Review profile: ASSERTIVE
  • Plan: Advanced
  • Run ID: 78d65f71-66a8-4ec5-b3ce-2760f1e76026
📥 Commits

Reviewing files that changed from the base of the PR and between b7136ac and 40c22d5.

⛔ Files ignored due to path filters (2)
  • java/daikon/tools/runtimechecker/InstrumentHandler.doc is excluded by !**/*.doc
  • tests/daikon-tests/SplitPoint/SplitPoint.dtrace.gz is excluded by !**/*.gz
📒 Files selected for processing (73)
  • doc/CHANGELOG.md
  • doc/daikon.texinfo
  • doc/developer.texinfo
  • java/Makefile
  • java/daikon/AnnotateNullable.java
  • java/daikon/Chicory.java
  • java/daikon/Daikon.java
  • java/daikon/DaikonGetopt.java
  • java/daikon/DynComp.java
  • java/daikon/FileIO.java
  • java/daikon/MergeInvariants.java
  • java/daikon/PptConditional.java
  • java/daikon/PptRelation.java
  • java/daikon/PptTopLevel.java
  • java/daikon/PrintInvariants.java
  • java/daikon/SplitDtrace.java
  • java/daikon/UnionInvariants.java
  • java/daikon/VarInfo.java
  • java/daikon/dcomp/BuildJDK.java
  • java/daikon/dcomp/BuildJDK24.java
  • java/daikon/diff/Diff.java
  • java/daikon/split/PptSplitter.java
  • java/daikon/split/SplitterFactory.java
  • java/daikon/split/SplitterJavaSource.java
  • java/daikon/split/SplitterJavaSourceTest.java
  • java/daikon/split/SplitterList.java
  • java/daikon/test/AllTestsSuite.java
  • java/daikon/test/BadOptionTest.java
  • java/daikon/test/DtraceNonceFixerTest.java
  • java/daikon/test/SampleTester.java
  • java/daikon/test/split/SplitterFactoryTestUpdater.java
  • java/daikon/test/split/SplitterListTest.java
  • java/daikon/tools/DtraceDiff.java
  • java/daikon/tools/DtraceNonceFixer.java
  • java/daikon/tools/ExtractConsequent.java
  • java/daikon/tools/ExtractConsequentTest.java
  • java/daikon/tools/InvariantChecker.java
  • java/daikon/tools/ReadTrace.java
  • java/daikon/tools/TraceSelect.java
  • java/daikon/tools/compare/LogicalCompare.java
  • java/daikon/tools/jtb/Annotate.java
  • java/daikon/tools/jtb/CreateSpinfo.java
  • java/daikon/tools/jtb/CreateSpinfoC.java
  • java/daikon/tools/runtimechecker/CommandHandler.java
  • java/daikon/tools/runtimechecker/InstrumentHandler.java
  • java/daikon/tools/runtimechecker/Main.java
  • java/daikon/tools/runtimechecker/WriteViolationFile.java
  • scripts/decls-add-cluster.pl
  • scripts/dtrace-add-cluster.pl
  • scripts/extract_vars.pl
  • scripts/runcluster.pl
  • scripts/test-misc.sh
  • scripts/trace-purge-vars.pl
  • tests/daikon-tests/KnownValues/KnownValues.txt-daikon.goal
  • tests/daikon-tests/KnownValuesNewDecls/KnownValuesNewDecls.txt-daikon.goal
  • tests/daikon-tests/Makefile
  • tests/daikon-tests/NonZeroLargeRange/NonZeroLargeRange.txt-daikon.goal
  • tests/daikon-tests/SplitPoint/Makefile
  • tests/daikon-tests/SplitPoint/SplitPoint.decls
  • tests/daikon-tests/SplitPoint/SplitPoint.spinfo
  • tests/daikon-tests/SplitPoint/SplitPoint.txt-daikon.goal
  • tests/diff-tests/Makefile.common
  • tests/diff-tests/StackAr/StackAr-u.txt.goal
  • tests/diff-tests/schedule2/schedule2-u.txt.goal
  • tests/script-tests/runcluster/.gitignore
  • tests/script-tests/runcluster/Makefile
  • tests/script-tests/runcluster/good-km-2.spinfo.goal
  • tests/script-tests/runcluster/good-newdecls.dtrace
  • tests/script-tests/runcluster/good.decls
  • tests/script-tests/runcluster/good.dtrace
  • tests/script-tests/runcluster/malformed-newdecls.dtrace
  • tests/script-tests/runcluster/malformed.decls
  • tests/script-tests/runcluster/malformed.dtrace
  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.

mernst and others added 16 commits October 6, 2026 10:55
…are-hierarchy

# Conflicts:
#	java/daikon/Daikon.java
#	java/daikon/PptRelation.java
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
….pl unescaping, remove version 1 leftovers

* developer.texinfo: release snippets no longer test for the deleted
  front-end/ directory; example .decls is now in version 2 format;
  clarify that a file with no decl-version record is read as version 2.
* CHANGELOG.md: accurately describe what identifies a version 1 file.
* convertcsv.pl: unescapeDecl now handles \n and \r like
  FileIO.unescape_decl.
* Remove lwpp-init.pl (only used with the removed lwpp.pl) and unused
  version 1 Kvasir goal files for SimpleDisambigStructTest.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…trace, cluster scripts

- ppt_perc: never exclude GLOBAL/OBJECT/CLASS ppts (they are parents of
  included ppts); read ppt names via FileIO.declared_ppt_name, which
  applies user_mod_ppt_name.
- MergeInvariants with a .decls file: create combined exits before
  init_hierarchy, and clean the ppt map for merging on both paths.
- SplitDtrace: handle traces without declarations; pass header records
  through.
- decls-add-cluster.pl: link the cluster variable to its parents.
- dtrace-count.pl, dtrace-indent.pl: match declaration records exactly.
- Remove simplify_expression/simplify_variable_names and the unused
  return value of read_decl_version.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…tDtrace comments

- FileIO.is_parent_only_ppt_name: use PptName rather than a substring
  match on "GLOBAL".
- read_ppt_decl: reuse declared_ppt_name to parse the ppt name.
- SplitDtrace: use new FileIO.is_header_record, which also recognizes
  comment records.
- MergeInvariants: clean ppts for merge only when reading .inv files.
- developer manual: there is no longer a Perl front end.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ion 2

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Add DaikonGetopt, a Getopt subclass that throws a UserError describing an
unrecognized, ambiguous, or malformed option and handles -h and --help.
Use it in Daikon and in the other tools that use Getopt.

Remove Diff's -u and -z options, which had no effect.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Tools that take no options other than -h and --help now use DaikonGetopt, so
they print a usage message for -h and --help and reject other options.  Tools
that use plume-lib Options exit with status 0 after printing help.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…per call

Report a UserError for missing or non-integer num_reps and sample_size and for
a malformed -SEED.  Pass every argument that is not a TraceSelect option to
Daikon rather than silently dropping some.  mainHelper resets its static state
on each call.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…nd --help

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The instrumenter and WriteViolationFile use DaikonGetopt.  The instrument
command reports a bad option and its usage message rather than ignoring the
option.  A missing .doc file is reported as a bug.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
mernst and others added 5 commits October 6, 2026 17:30
…onditions

* Write "pkg.Foo.bar(int):::" for an entry or exit program point, so that
  SplitterList.get applies the splitters to every program point of the
  method (not, say, "Foo.bar():::EXIT1" to "Foo.bar():::EXIT12"), and so
  that conditions found at different program points of one method are
  output once.
* Group conditions over orig(cluster) with those over cluster.
* Omit "b == i" where exactly one operand is a boolean.
* Parenthesize conjuncts that contain "?:", "==>", or other operators of
  lower precedence than conjunction.
* Do not treat names such as "borig(" as pre-state values.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Treat a lexical error (TokenMgrError) in a splitter condition like a parse error.
- Include compiler standard output in the error output only when compilation fails.
- Normalize paths when matching javac error output against source files.
- Compute splitter source and class paths in one place, in SplitterObject.
- Remove the redundant nonexistent-file filter from FileCompiler.
- Test unwritable splitter files and lexical errors; clean up SplitterList after tests.
QuantFixer now runs after the StatementReplacer and the name fixers, and it
uses the VarInfos to produce type-correct Java code that matches daikon.Quant
semantics: a null array yields Quant's default value, the index may be a long,
boolean elements are stored as longs, and char arrays are Strings.
PrefixFixer leaves alone any name that does not start with a variable name,
such as java.lang.Math.abs.  The QuantCalls splitter test covers the new cases.
Test the .spinfo program point name, parenthesization of conjuncts, the
pre-state pattern, the filter for numeric invariants over booleans, and
the grouping of orig(cluster) with cluster.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
mernst added 30 commits October 11, 2026 22:18
# Conflicts:
#	doc/CHANGELOG.md
#	java/daikon/DaikonGetopt.java
#	java/daikon/SplitDtrace.java
#	java/daikon/test/BadOptionTest.java
#	java/daikon/tools/DtraceNonceFixer.java
#	java/daikon/tools/ReadTrace.java
…rmat

# Conflicts:
#	java/daikon/SplitDtrace.java
# Conflicts:
#	doc/CHANGELOG.md
#	java/daikon/SplitDtrace.java
# Conflicts:
#	doc/CHANGELOG.md
#	java/daikon/SplitDtrace.java
# Conflicts:
#	java/daikon/DaikonGetopt.java
#	java/daikon/test/BadOptionTest.java
# Conflicts:
#	java/daikon/DaikonGetopt.java
…n-files

# Conflicts:
#	java/daikon/test/AllTestsSuite.java
# Conflicts:
#	java/daikon/test/AllTestsSuite.java
#	java/daikon/tools/DtraceNonceFixer.java
…litter-quant-calls

# Conflicts:
#	java/daikon/test/split/SplitterFactoryTestUpdater.java
…are-hierarchy

# Conflicts:
#	doc/CHANGELOG.md
# Conflicts:
#	doc/CHANGELOG.md
#	java/daikon/PptTopLevel.java
…6-integration

# Conflicts:
#	java/daikon/test/split/targets/DataStructures_QueueAr_dequeue_25.java.goal
#	java/daikon/test/split/targets/DataStructures_QueueAr_dequeue_27.java.goal
#	java/daikon/test/split/targets/DataStructures_QueueAr_enqueue_29.java.goal
#	java/daikon/test/split/targets/DataStructures_QueueAr_enqueue_30.java.goal
#	java/daikon/test/split/targets/DataStructures_QueueAr_enqueue_31.java.goal
#	java/daikon/test/split/targets/DataStructures_QueueAr_enqueue_32.java.goal
#	java/daikon/test/split/targets/DataStructures_QueueAr_isFull_21.java.goal
#	java/daikon/test/split/targets/DataStructures_QueueAr_isFull_22.java.goal
#	java/daikon/test/split/targets/Math__BigFloat_bdiv__39.java.goal
…tion

# Conflicts:
#	java/daikon/test/AllTestsSuite.java
…integration

# Conflicts:
#	java/daikon/VarInfo.java
#	java/daikon/split/SplitterList.java
…tegration

# Conflicts:
#	java/daikon/test/AllTestsSuite.java
# Conflicts:
#	doc/CHANGELOG.md
#	java/daikon/DaikonGetopt.java
#	java/daikon/test/AllTestsSuite.java
#	java/daikon/test/BadOptionTest.java
#	java/daikon/tools/jtb/CreateSpinfoC.java
#	java/daikon/tools/runtimechecker/CommandHandler.java
#	java/daikon/tools/runtimechecker/InstrumentHandler.java
#	java/daikon/tools/runtimechecker/Main.java
#	java/daikon/tools/runtimechecker/WriteViolationFile.java
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Daikon Failed to Execute when Using runcluster.pl with kmeans Option Trouble terminating when running 'runcluster.pl' and help with inputs for daikon

1 participant