Skip to content

Drop support for the version 1 file format - #880

Open
mernst wants to merge 49 commits into
codespecs:masterfrom
mernst:dont-support-old-format
Open

mernst wants to merge 49 commits into
codespecs:masterfrom
mernst:dont-support-old-format

Conversation

@mernst

@mernst mernst commented Oct 6, 2026 •

Copy link
Copy Markdown
Member

This PR has been split into smaller PRs that can be reviewed and merged independently. This PR includes their changes, so it will shrink as they are merged:

What remains here is the change to Daikon itself (plus the related documentation). This PR overlaps #875, much of which it makes unnecessary.

The changelog entry for version 5.8.6 says "Removed support for the obsolete version 1 file format", but that change only stopped Chicory and DynComp from writing version 1 files. Daikon still read them. Now it doesn't: a version 1 file (one that starts with a DECLARE or VarComparability record, or that says decl-version 1.0) produces an error that says to re-generate the file with a current front end.

Daikon:

  • Remove FileIO.new_decl_format and the code that ran only for version 1 files: the version 1 parser, the name-based program point hierarchy, and many VarInfo methods' VarInfoName branches.
  • init_hierarchy_new is renamed init_hierarchy. MergeInvariants used the version 1 hierarchy code even for version 2 decls; now it uses the declaration-based code.
  • Remove configuration option remove_post_vars. It had an effect only for version 1 files (with version 2 files it failed an assertion).
  • Change the serialVersionUID of FileIO.SerialFormat and InvMap, whose serialized form changed, so that old .inv files produce the existing "regenerate your .inv file" message.

Front ends and scripts:

Documentation: remove the "Version 1 Declarations" chapter of the developer manual and the dfepl sections of the user manual, and rewrite the troubleshooting entry for this error.

mernst added 3 commits October 6, 2026 10:55
Convert every version 1 .decls and .dtrace test input to the version 2
format, in preparation for dropping support for version 1 files.  The
converted inputs produce the same output, with two exceptions:
* pine_mdiep loses one invariant over a two-dimensional variable, which
  version 2 cannot represent.
* Nine more splitters compile, because the version 2 declarations let
  the splitter generator find array variables.

Also delete test inputs that no test uses, and delete
tests/diff-tests/StackAr, which has been excluded from testing (and
broken) since 2010.
Daikon no longer reads version 1 .decls and .dtrace files, and reports
an error when it encounters one.  Remove FileIO.new_decl_format and all
code that was used only for version 1 files, including the name-based
program point hierarchy (MergeInvariants now uses the declaration-based
one) and the remove_post_vars configuration option.

Remove the Perl front end (dfepl), which produced version 1 files.
Port convertcsv.pl and the cluster analysis scripts to version 2, and
remove scripts that processed only version 1 files.
mernst and others added 7 commits October 6, 2026 12:59
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
….pl unescaping, remove version 1 leftovers

* developer.texinfo: release snippets no longer test for the deleted
  front-end/ directory; example .decls is now in version 2 format;
  clarify that a file with no decl-version record is read as version 2.
* CHANGELOG.md: accurately describe what identifies a version 1 file.
* convertcsv.pl: unescapeDecl now handles \n and \r like
  FileIO.unescape_decl.
* Remove lwpp-init.pl (only used with the removed lwpp.pl) and unused
  version 1 Kvasir goal files for SimpleDisambigStructTest.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…trace, cluster scripts

- ppt_perc: never exclude GLOBAL/OBJECT/CLASS ppts (they are parents of
  included ppts); read ppt names via FileIO.declared_ppt_name, which
  applies user_mod_ppt_name.
- MergeInvariants with a .decls file: create combined exits before
  init_hierarchy, and clean the ppt map for merging on both paths.
- SplitDtrace: handle traces without declarations; pass header records
  through.
- decls-add-cluster.pl: link the cluster variable to its parents.
- dtrace-count.pl, dtrace-indent.pl: match declaration records exactly.
- Remove simplify_expression/simplify_variable_names and the unused
  return value of read_decl_version.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…tDtrace comments

- FileIO.is_parent_only_ppt_name: use PptName rather than a substring
  match on "GLOBAL".
- read_ppt_decl: reuse declared_ppt_name to parse the ppt name.
- SplitDtrace: use new FileIO.is_header_record, which also recognizes
  comment records.
- MergeInvariants: clean ppts for merge only when reading .inv files.
- developer manual: there is no longer a Perl front end.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ion 2

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
mernst added 14 commits October 6, 2026 17:52
…tants

- extract_vars.pl: exclude variables by rep type rather than by name; drop redundant check
- dtrace-count.pl, dtrace-indent.pl, sort-dtrace.pl, dtrace-diff.pl, convertcsv.pl: reject version 1 declarations explicitly
- convertcsv.pl: skip constant variables; escape newlines and carriage returns
- dtrace-add-cluster.pl: handle EOF in skip_till_next
- decls-add-cluster.pl: link cluster variable for all parent relation types
- CHANGELOG: note that version 1 input is no longer accepted
… update comments, delete version 1 goal file
Graphviz and netpbm were used only to build the dfepl-flow image.
Also restore an attributed historical note, format the changelog entry
as a bullet, and clarify that the input-language string is arbitrary.
…w: faster ppt_included, share is_declaration_header, sort-dtrace.pl header paragraphs
…iff test

* In pine_mdiep, every "[]" variable has a scalar rep-type, so it is a
  pointer dereference, not an array.  Declare it as "p[0]" with
  "var-kind field [0]", as Kvasir does, rather than as an array.  This
  keeps the 613 doubly-dereferenced variables, and the output matches the
  version 1 output except for variable names.
* Convert and restore tests/diff-tests/StackAr, with ENTER declarations
  moved before their EXIT declarations, and regenerate its goal files.
* tests/diff-tests/Makefile: run every subdirectory, and stop if a
  subdirectory's make fails.
* SplitterFactoryTestUpdater: read each decls file once, with
  read_data_trace_file, which throws IOException.
…zip output and dropped line; dtrace-add-cluster.pl: do not overwrite input
- util_daikon.pm: add die_if_version_1_decl (checks only a paragraph's
  first line), is_declaration_paragraph, and skip_till_next
- dtrace-count.pl, dtrace-indent.pl, sort-dtrace.pl, dtrace-diff.pl,
  convertcsv.pl: use the shared helpers
- dtrace-diff.pl: skip declaration paragraphs in combined dtrace files;
  skip constant variables
- sort-dtrace.pl: pass through comment paragraphs
- decls-add-cluster.pl: link the cluster variable only for "parent"
  relations, not "user" relations
- extract_vars.pl: check for constants before hashcodes; fix null-object
  assignment; handle end of file without a trailing blank line
- convertcsv.pl, extract_vars.pl: replace closures with simpler code
- decls2comp.py: implement the no-hashcodes option; remove Lackwit note
… checks

- Reject version 1 input in the cluster scripts.
- Accept # comments, and comments not followed by a blank line.
- Detect version 1 files with CRLF line endings or trailing whitespace.
- Detect an existing variable named "cluster".
- In dtrace-diff.pl, explain mismatches involving constants.
- Move escape_decl, unescape_decl, and a version 2 decls parser to util_daikon.pm.
- Classify each record with a single regular expression.
- Remove decl-input.
…ords

* util_daikon.pm: new read_record reads one record from a file the way
  Daikon does: decl-version, var-comparability, and input-language are a
  single line, and leading comments are returned with the record.  All
  the scripts use it instead of paragraph mode or skip_till_next.
* record_kind matches Daikon's tests ("ppt " and an exact
  "ListImplementors" line).
* read_ppt_decls rejects data trace records in a declaration file.
* decls-add-cluster.pl: name the output after the file's basename, and
  die if two inputs would map to the same output.
* sort-dtrace.pl: comments stay with the declaration or variable that
  follows them.
* extract_vars.pl: exclude variables by name as well as by rep type.
* dtrace-diff.pl: compare variable lists element by element.
* Share the "cluster" variable check as check_no_cluster_var.
Pull request codespecs#856 rewrites runcluster.pl and the scripts it calls
(extract_vars.pl, decls-add-cluster.pl, and dtrace-add-cluster.pl),
including their support for version 2 files.
…pull requests

Pull request codespecs#867 fixes DtraceNonceFixer, including its handling of
header records, declarations, compressed files, and samples without
nonces.  Pull request codespecs#856 rewrites runcluster.pl and the scripts it
calls, including their support for version 2 files.
mernst added a commit to mernst/daikon that referenced this pull request Oct 7, 2026
Daikon reads version 2 declarations both now and after codespecs#880 removes
support for version 1.  In version 2, array variables are named
"this.a[..]" rather than "this.a[]", so update the goal files and the
test's expected values accordingly.
mernst and others added 12 commits October 7, 2026 15:01
The tests run `perl -Si sort-dtrace.pl FILE`, but -i has no effect on
files that a script opens itself, so the sorted output went to standard
output and the file was left unsorted.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…te declarations

Read records a paragraph at a time, which is faster; end a record only at an
empty line, as Daikon does.  Share the record loop via for_each_record, which
reads through ARGV so that perl -i works.  Give ListImplementors its own record
kind.  Keep indented comments with the following variable in sort-dtrace.pl.
Reject unknown keys in program point declarations.  Remove the unused
check_no_cluster_var.
…arations and escaping

* util_daikon.pm: accept \r\n line endings; process paragraphs by offset
  (linear time); reject a variable declared twice or lacking var-kind,
  dec-type, or rep-type; build error messages only when needed.
* decls2comp.py: list variables with negative comparability on a "-1:" line.
* dtrace-diff.pl: read declarations from the traces (decls file optional);
  escape names in all messages; strip comments with one regex; remove the
  no-op --ignore_exitno option.
* convertcsv.pl: match -decl variables to columns by either form of the
  name, die if none matches, and write the declared names.
…t-support-old-format

# Conflicts:
#	java/daikon/test/split/SplitterFactoryTestUpdater.java
#	tests/daikon-tests/pine_mdiep/pine_mdiep.decls.gz
#	tests/daikon-tests/pine_mdiep/pine_mdiep.dtrace.gz
#	tests/daikon-tests/pine_mdiep/pine_mdiep.txt-daikon.goal
#	tests/diff-tests/Makefile
#	tests/diff-tests/StackAr/Makefile
#	tests/diff-tests/StackAr/StackAr-1.dtrace.gz
#	tests/diff-tests/StackAr/StackAr-1m2.txt.goal
#	tests/diff-tests/StackAr/StackAr-2.dtrace.gz
#	tests/diff-tests/StackAr/StackAr-2m1.txt.goal
#	tests/diff-tests/StackAr/StackAr-e.txt.goal
#	tests/diff-tests/StackAr/StackAr-jt.txt.goal
#	tests/diff-tests/StackAr/StackAr-t.txt.goal
#	tests/diff-tests/StackAr/StackAr-u.txt.goal
#	tests/diff-tests/StackAr/StackAr-union.txt.goal
#	tests/diff-tests/StackAr/StackAr-xor.txt.goal
#	tests/diff-tests/StackAr/StackAr.decls
#	tests/diff-tests/StackAr/StackAr.txt.goal
…-support-old-format

# Conflicts:
#	Makefile
#	doc/CHANGELOG.md
#	doc/Makefile
…ont-support-old-format

# Conflicts:
#	doc/CHANGELOG.md
#	scripts/convertcsv.pl
#	scripts/dtrace-count.pl
#	scripts/dtrace-diff.pl
#	scripts/dtrace-indent.pl
#	scripts/sort-dtrace.pl
…arability, truncated records

* dtrace-diff.pl reports a difference when two declarations of a program
  point have different variables, and reuses read_ppt_decls.
* sort-dtrace.pl writes the input file's line endings.
* decls2comp.py recognizes arrays by their rep-type.
* dtrace-count.pl honors a "// EOF" line among a record's leading comments.
* util_daikon.pm and dtrace-diff.pl use English without match variables.
@mernst

mernst commented Oct 11, 2026

Copy link
Copy Markdown
Member Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Oct 11, 2026 •

Copy link
Copy Markdown
⚠️ Action not completed

Review skipped: 140 files exceed the limit of 100.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant