Repository navigation
fix(convert_output): repair the Rceattle branch and correct its units - #401
grantdadams wants to merge 3 commits into
Conversation
f1c6f84 to
6516e75
Compare
|
I reviewed this before anyone had to and force-pushed a corrected version. What changed, in case you read the first one:
|
|
@grantdadams thank you so much Grant! I appreciate you tackling a first review. I plan to do a review tomorrow, but my main checks will be to verify that this works in convert_output and that all of our figures and tables look reasonable as well. This is a huge help with the changes to Rceattle! I will lyk if I have any questions, but I think I can take it from here. |
ad02425 to
a8038fc
Compare
Fixes nmfs-ost#400. `convert_output(model = "rceattle")` errored on every Rceattle fit, in both input modes: - `index_ln_q` is `index_log_q` and `catch_h` is `catch_hat`, so neither was found; `Q_block` is deprecated, now selected with `any_of()`. - Module selection was positional, but a fit's element list depends on how it was fitted, so a fixed offset lands on different modules for different fits. Selected by name instead. `sdrep` holds every derived time series, so a `getsd = FALSE` fit now warns rather than returning only the input data. - `file[["data_list"]]` read the argument, not the loaded object, so the path mode could never work; `dat[["data_list"]]` in 16 places, and the `.rds` check is anchored and case-insensitive. - `comp_data` was overwritten with the catch data, and predicted composition used `cross_join(age_hat)`, repeating the 2,478 correct predictions 59 times for 146,202 against 2,928 observed. - The catch and composition frames set no `module_name`, leaving 5,586 of 6,648 rows without one. - A multispecies fit has no species dimension in this schema; refused on `nspp`, read with `[[` so a renamed element cannot partial-match past it. Output is 35 columns, 6,648 rows, and biomass equal to the fit to 2.33e-10. Adds a fixture built from the bundled `GOAatf` with its rebuild script, and tests pinning the composition and index alignment by value.
Groundwork for nmfs-ost#388-nmfs-ost#392, which will read units from the converter: once a figure takes its unit from this sheet, a wrong one becomes a wrong axis label. All 35 unit-carrying rows in rceattle_var_names.csv were wrong -- 31 said kg and 4 said numbers. CEATTLE carries numbers-at-age in thousands of fish and weight-at-age in kg, so their product is metric tons. Corrected against `Rceattle::quantity_dictionary()`: biomass and the B0/SB0 family to mt, numbers-at-age to thousands of fish, the depletions and compositions to proportion, log_ssb to log mt, pop_scalar to multiplier, nbyagespr to fish per recruit, consumption_at_age to kg per fish per yr. catch_hat, max_catch_hat, comp_weights and diet_comp_weights are blanked rather than corrected: a fleet's catch follows its own Observation_units column, so no single value is right for every fleet, and a composition weight is a likelihood multiplier (log Dirichlet-multinomial theta under that likelihood) rather than a mass. Flimit is the FOFL proxy (F35% at Tier 3) and Ftarget the maximum FABC proxy (F40%), but the two alt_labels were crossed, so each reference point would publish under the other's name. The ln_flimit / ln_ftarget rows in the same sheet are already the right way round. Only the four sdrep rows reach the output today, since the converter emits sdrep and data_list and the sheet has no data_list block; the rest are corrections for when quantities is processed. See nmfs-ost#405.
a8038fc to
df8915a
Compare
| i = "Refit with {.code fit_control(getsd = TRUE)} to standardize the derived time series." | ||
| )) | ||
| } | ||
| for (module_name in intersect(c("sdrep", "data_list"), names(dat))) { |
There was a problem hiding this comment.
Is there a condition where sdrep or data_list would not be in names(dat)?
There was a problem hiding this comment.
sdrep, yes. fit_mod() leaves it out when getsd = FALSE, when estimateMode = 3 (no estimation), for a projection-only fit with a constant-F control rule, and when the Hessian is not positive definite. The output then has no biomass, SSB, recruitment or parameter estimates, so the converter warns. The input data, selectivity and M still come through.
data_list is always on a fit returned by fit_mod(). The check is for a fit that was trimmed or built by hand. Without it the converter can't attach years or read the data, so it stops with a clear message.
Schiano-NOAA
left a comment
There was a problem hiding this comment.
@grantdadams This generally looks good to me. A couple questions before I merge:
(1) is there selectivity somewhere in the output file?
(2) is there error for recruitment deviations?
(3) is natural mortality reported anywhere? is it a single point or a time series?
Please lmk where they are located if these values are in the output file. Thanks!
…r errors * Read `sel_at_age` and `M_at_age` from `quantities` as `selectivity_age` and `natural_mortality`, by fleet, sex, age and year. Ages are offset by `minage`, since the array's age dimension is the bin index. * Give the fixed effects a standard error from `sdrep$cov.fixed`, so recruitment deviations now carry one. * Read `sdrep$par.random`, so recruitment deviations are kept when fitted as random effects (`random_rec = TRUE`). * The fixture now keeps `minage`, the 2 arrays and the fixed-effect variances.
|
@Schiano-NOAA thanks! All 3 are in the Rceattle fit, and my commit adds them to the converted output.
Ages on these rows are the actual ages: the array's age dimension is offset by |
What is the feature?
convert_output#400 —convert_output(model = "rceattle")errored on every Rceattle fit I could try, in both input modes.rceattle_var_names.csvwerekgornumbers, where CEATTLE reports metric tons and thousands of fish. Groundwork for [feat] Call units from converter, not arguments: B, discard, F #388–[feat] Call units from converter, not arguments: index, landings, projections, total catch #392, since once a figure reads its unit from the sheet a wrong one becomes a wrong axis label. It also unswaps theFlimit/Ftargetlabels, which had each other's name.How have you implemented the solution?
index_log_q,catch_hat, deprecatedQ_block), positional module selection,file[[instead ofdat[[,comp_dataoverwritten with catch data, and across_join()on the predictions. Per-defect detail with the measured before/after is in the commit messages.GOAatf, bundled with Rceattle: 35 columns, 21,768 rows (was 150,372), including 11,340 selectivity and 3,780 natural mortality rowsdevtools::test()has 0 failures, with the same 67 warnings asmain.GOAatf, withprepare_rceattle_output_file.Ralongside it so the recipe survives the merge.Does the PR impact any other area of the project, maybe another repo?
model == "rceattle"branch.data/example_data.rdaregenerates bit-identically, so theupdate-example-datatrigger has nothing to rebuild.data_listblock); the rest are dormant untilquantitiesis processed. Also in [fix]: Rceattle converter publishes wrong composition, age and uncertainty values #405.action_required— first PR from a fork, so they need approval to run.