Skip to content

New versions of scripts for HyperPipe - #202

Open
mebiri wants to merge 9 commits into
oshaughn:rift_O4dfrom
mebiri:rift_O4d
Open

mebiri wants to merge 9 commits into
oshaughn:rift_O4dfrom
mebiri:rift_O4d

Conversation

@mebiri

@mebiri mebiri commented Sep 29, 2026

Copy link
Copy Markdown

Updated files:
-util_ConstructIntrinsicPosterior_GenericCoordinates.py -> CIP_faster_v2.py
-allows n_events_to_analyze > 1
-conditional imports speed up initialization
-util_ConstructEOSPosterior.py -> util_ConstructHyperPosterior.py
-implements external rotation differently than rift_O4d AI code
-util_HyperparameterPuffball.py -> util_HyperPuffball_new.py
-implements external rotation, etc. differently than rift_O4d AI code
-Also 1 change to create_eos_posterior_pipeline to enable new argument structure for PUFF (arguments for parameter bounds now match CEP/CHP format param:[lower,upper])

HyperPuffball: new version with flexible rotation/reflection/bounds system (better than RoboCode)
ConstructHyperPosterior: ConstructEOSPosterior with flexible rotation system (better than RoboCode), but missing newer features added to CEP > April 2026
CIP_faster: a faster version of util_ConstructIntrinsicPosterior_GenericCoordinates, via optional imports and n_events > 1 functionality
it should work now
now behaves like puff for rotation and param bounds
@mebiri

mebiri commented Sep 29, 2026

Copy link
Copy Markdown
Author

Actually that last change to create_eos_posterior_pipeline has not been done in this PR; line 191:
puff_args = puff_args.replace('[', ' \'[') -> puff_args = puff_args.replace('[', '\'[') (removed space inside quotes)

@oshaughnessy-junior

Copy link
Copy Markdown
Contributor

Thanks for this. We audited the three files against current oshaughnessy-junior:rift_O4d (76ead4c) and ran small known-answer tests. Summary below. We plan to port the useful ideas into the existing tools rather than merge new parallel files; that port PR will be linked here.

Where you are right

  • --n-events-to-analyze > 1. Our CIP accepts the flag but evaluates only one row. Under create_eos_posterior_pipeline chunking, (n-1)/n of the grid is skipped with no error. That is a real bug in our line.
  • Startup cost. Conditional imports save about 6 s per job (--help ~3 s vs ~9 s on ldas-grid). The cost comes from corner (via arviz, 5.4 s) and senni (torch, 3.7 s), both of which were already unconditional in March.
  • Coordinate passing. The hyperpipe never passes the coordinate module to the puff stage (RIFT/hyperpipe/coords.py, to_puff_args), so a rotated puff cannot be configured from a hyperpipe config. This is the main thing we will fix.
  • param:[lo,hi] ranges for the puffball, and the one-character quoting fix in create_eos_posterior_pipeline (line 191; lines 138 and 216 have the same pattern).
  • Comment bulk in CEP: the measurement table at L866-915 and long narrative comments belong in a design note.

Problems in the PR files

File Problem Evidence
CIP_faster_v2.py Forked from CIP at d87a815 (March); drops 53 later commits, 29 of them already in this PR's base git log d87a815f4..rift_O4d
Every non-EOS run crashes: str(opts.using_eos_index+i) with index None (L2611) measured, rc=1
Per-file output names built from an already-mutated name: idx=0, n=4 writes MARG-0-{0,1,3,6}; neighbouring chunks overwrite rows measured
Lacks options that util_RIFT_pseudo_pipe, cip_pipeline.py, helper_LISA_Events and NMB (nmbseq:) emit
util_ConstructHyperPosterior.py Gaussian at (3,3), 45 deg rotation, true lnZ = 19.33: CEP gives 19.34 in both plugin modes. CHP with physical-frame ranges gives mean (0.63, 2.78), lnZ = -26.7, with no warning measured
Re-introduces fixed bugs: double log of the likelihood (lnZ 2.96 vs 19.33); --internal-use-lnL with the adaptive sampler (lnZ 3.52); tempering_exp ignored measured
Loads the plugin only when --supplementary-coordinate-function is set, so under hyperpipe (which passes only -code) the plugin is dropped silently; .py paths and rift_default fail
util_HyperPuffball_new.py CRLF line endings (python\r: No such file) measured
With no inverse available, turns rotation off but still applies rotated-frame bounds to physical coordinates
--downselect-parameter x ... --reflect-parameter x (the old idiom, used in your hyperpipetests/Analysis/Makefile) now raises measured
The error-handling fix in 46231ee landed on the root-level copy that 1fb4aa2 deleted

For linear maps the two puffball rotation designs agree (covariance and reflection match). The CEP refusals that look like bloat are correctness fixes; the runs above show the wrong answers they prevent.

What the port will contain (all additive; no flag removed or renamed)

  • CIP: lazy imports; working n>1 with the same output names as n separate jobs; --chunk-save; --save-hyperfile-only.
  • CEP: optional get_bounds hook via --get-range-from-external and --external-range-args k=v, which raises on failure.
  • Puffball: name:[lo,hi] ranges; reflection from a range without a downselect.
  • Hyperpipe: the coordinate module reaches the puff stage.
  • create_eos_posterior_pipeline quoting fix.

If your rotation plugin keeps physical column names, a small adapter that renames the outputs will let it run under CEP's plugin interface. We can help write it.

@oshaughnessy-junior

Copy link
Copy Markdown
Contributor

Port PR: oshaughnessy-junior#383. It adds hyperpipe coordinate passing to the puff stage (post.coord-input-parameters, puff.coord-basis), name:[lo,hi] puff ranges, the get_bounds hook in CEP and the puffball, and CIP multi-row support with --chunk-save and lazy imports. Evidence is in its first comment.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants