Skip to content

add base.context.dump to save experiment config as metadata - #131

Open
vlap wants to merge 1 commit into
uwefladrich:masterfrom
vlap:add-toyaml-jinja-filter
Open

add base.context.dump to save experiment config as metadata #131
vlap wants to merge 1 commit into
uwefladrich:masterfrom
vlap:add-toyaml-jinja-filter

Conversation

@vlap

@vlap vlap commented Aug 13, 2026

Copy link
Copy Markdown

Adds toyaml (and to_yaml alias) Jinja filter to dump SE context as a yaml file (via yaml.dump).

@vlap
vlap force-pushed the add-toyaml-jinja-filter branch 3 times, most recently from e4ae491 to 0b4186f Compare August 13, 2026 23:27
@vlap

vlap commented Aug 13, 2026

Copy link
Copy Markdown
Author

hi @uwefladrich .

I am implementing "saveic" in ECE4. To make saved initial conditions fully self-documenting and traceable, I'd like saveic to also save a snapshot of the experiment config that defines the model setup (component list, resolutions/grids, forcing settings, and dates) that generated those initial conditions.

I did not find an easy way to dump from ScriptEngine in-memory context. With the new toyaml filter I can create
experiment-config.yml.j2 template

       {{ {'base.context': {'experiment': experiment, 'model_config': model_config}} | toyaml }}

and render it

            - base.template:
                src: "experiment-config.yml.j2"
                dst: "saveic/experiment-config.yml"

I hope this PR makes sense to you. Let me know if there is a better way to achieve the same.

@vlap
vlap force-pushed the add-toyaml-jinja-filter branch 8 times, most recently from b41fb84 to 6c097db Compare August 14, 2026 00:14
@vlap

vlap commented Aug 14, 2026

Copy link
Copy Markdown
Author

when testing, I realized implementing a toyaml filter is complicated by ! tags, which are not rendered well.

So, I decided to add base.context.dump, a counterpart of base.context.load

@vlap
vlap force-pushed the add-toyaml-jinja-filter branch 3 times, most recently from feaecca to ec59b4d Compare August 14, 2026 00:34
@vlap vlap changed the title feat(jinja): add toyaml and to_yaml Jinja filters add base.context.dump to save experiment config as metadata Aug 14, 2026
@vlap
vlap force-pushed the add-toyaml-jinja-filter branch 2 times, most recently from 164c46f to b0499c9 Compare August 14, 2026 08:53
@vlap

vlap commented Aug 14, 2026

Copy link
Copy Markdown
Author

I thought about copying the input file but it's fragile (cli args position uncertain, can be more than 1). So, base.context.dump is needed

   ```bash
   se my-user-settings.yml ../platforms/marenostrum5.yml my-experiment.yml scriptlib/main.yml

Also, some meaningful context, added via base.include can be missed (such as model tuning presets).

It can also filter out platform and job context to the model physics and experiment parameters.

base.context.dump is not meant to create a SE yml files with - base.context, just pure YAML, which can be loaded back into the context directly in downstream scripts using base.context.load.

@vlap
vlap force-pushed the add-toyaml-jinja-filter branch from b0499c9 to da33f7a Compare August 14, 2026 09:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant