ft: added preliminar changes to gallery

docs/guide/experiment_config.rst

@@ -14,9 +14,10 @@ Configuration files are written in ``YAML`` format and are divided into differen
 2. **Temporal Configuration** (``time_config``): Temporal characteristics of the experiment, such as the start and end dates, experiment class (time-independent or time-dependent), the testing intervals, etc.
 3. **Spatial and Magnitude Configuration** (``region_config``): Describes the testing region, such as its geographic bounds, magnitude ranges, and depth ranges.
 4. **Seismic Catalog** (``catalog``): Defines the seismicity data source to test the models. It can either link to a seismic network API, or an existing file in the system.
-5. **Models** (``models``): Configuration of forecasting models. It can direct to an additional configuration file with ``model_config`` for readability. See :ref:`model_config`.
-6. **Evaluation Tests** (``tests``): Configuration of the statistical tests to evaluate the models. It can direct to an additional configuration file with ``test_config`` for readability. See :ref:`evaluation_config`.
-7. **Postprocessing** (``postprocess``): Instructions on how to process and visualize the experiment's results, such as plotting forecasts or generating reports. See :ref:`postprocess`.
+5. **Input Data** (``input_config``): Describes the data handed to the models (e.g., the input catalog of time-dependent models), when it should differ from the testing settings.
+6. **Models** (``models``): Configuration of forecasting models. It can direct to an additional configuration file with ``model_config`` for readability. See :ref:`model_config`.
+7. **Evaluation Tests** (``tests``): Configuration of the statistical tests to evaluate the models. It can direct to an additional configuration file with ``test_config`` for readability. See :ref:`evaluation_config`.
+8. **Postprocessing** (``postprocess``): Instructions on how to process and visualize the experiment's results, such as plotting forecasts or generating reports. See :ref:`postprocess`.
 
 .. note::
 
@@ -228,7 +229,7 @@ Configuring the spatial and magnitude definitions is done through the ``region_c
    resulting in the ``[4.0, 4.1)``, ``[4.1, 4.2)`` and ``[4.2, 4.3)``.
 
 
-3. Depths: The minimum and maximum depths are just required to filter out seismicity outside those ranges.
+3. Depths: The minimum and maximum depths are just required to filter out seismicity outside those ranges. The testing catalogs contain the events with ``depth_min <= depth < depth_max``.
 
 
 Some example of region configurations would be:
@@ -282,6 +283,47 @@ The seismicity catalog can be defined with the ``catalog`` parameter. It represe
   The main catalog will be stored, and consecutively filtered to the extent of each testing time-window, as well as to the experiment's spatial domain, and magnitude- and depth- ranges.
 
 
+Input Data
+----------
+
+Time-dependent models receive, at each time window, an **input catalog** with the seismicity that occurred *before* the forecast start (see :ref:`model_config`). By default, this catalog is filtered from the main catalog using the testing magnitude range only (``mag_min``, ``mag_max``), without spatial, depth or lower time bounds. The ``input_config`` option allows to set the input data independently from the testing settings, as is usual in forecasting experiments (e.g., models are calibrated with small events in a wider region than the one they are tested in).
+
+.. code-block:: yaml
+
+   input_config:
+     catalog:
+       region: italy_csep_collection_region
+       mag_min: 2.5
+       mag_max: 8.0
+       depth_min: 0
+       depth_max: 30
+       start_date: 2005-01-01T00:00:00
+
+The ``catalog`` block admits the following parameters, all optional:
+
+.. list-table::
+   :widths: 20 80
+
+   * - **region**
+     - Spatial domain of the input catalog, with the same options as the testing ``region`` (a **CSEP** region or a file). Default: no spatial filter.
+   * - **mag_min**
+     - Minimum magnitude of the input catalog. Default: the testing ``mag_min``.
+   * - **mag_max**
+     - Maximum magnitude of the input catalog (exclusive). Default: the testing ``mag_max``.
+   * - **depth_min**
+     - Minimum depth (in `km`) of the input catalog. Default: no filter.
+   * - **depth_max**
+     - Maximum depth (in `km`, exclusive) of the input catalog. Default: no filter.
+   * - **start_date**
+     - Earliest event (``datetime``) of the input catalog. Default: no lower bound. The upper bound is always the start of the forecast time window.
+
+.. note::
+
+    When the main catalog is queried from a network API, the query bounds are extended to cover both the testing and the input settings (i.e., the lowest magnitude, the largest depth, the earliest date and the union of the regions' bounding boxes).
+
+The input data configuration is written in the reproducibility configuration file, so an experiment reproduced with ``floatcsep reproduce`` hands the same input catalogs to the models. The ``input_config`` block is meant to be extended with other types of input data in the future.
+
+
 Run Configuration
 -----------------
 

docs/guide/model_config.rst

@@ -344,17 +344,18 @@ We distinguish **input data** versus **input arguments**. The input data is give
 
    Therefore, the model source-code should be at least able to dynamically read the obligatory arguments (simply the time window of the issued forecast)
 
-2. **Input Data**: Correspond to any data source outside the control of the modeler (e.g., authoritative input catalog, testing region). For now, **floatCSEP** just handles an input **catalog**, which are all the events within the **main catalog**  until the forecast **start_date**. The catalog is written by default in ``model_path/input/catalog.csv`` in the CSEP ascii format (see :doc:`pycsep:concepts/catalogs`) as:
+2. **Input Data**: Correspond to any data source outside the control of the modeler (e.g., authoritative input catalog, testing region). For now, **floatCSEP** just handles an input **catalog**, which are the events of the **main catalog** that occurred before the forecast **start_date**. By default, the input catalog contains the events within the testing magnitude range, without spatial or depth filters. The ``input_config`` option of the experiment configuration (see :ref:`experiment_config`) sets the region, magnitude range, depth range and earliest date of the input catalog, independently from the testing settings. The catalog is written by default in ``model_path/input/catalog.csv`` in the CSEP ascii format (see :doc:`pycsep:concepts/catalogs`), with a header line, as:
 
   .. code-block:: none
 
-      longitude, latitude, magnitude, time_string, depth, event_id
+      lon,lat,mag,time_string,depth,catalog_id,event_id
 
-  - **longitude**: Decimal degrees of the forecasted event location.
-  - **latitude**: Decimal degrees of the forecasted event location.
-  - **magnitude**: Magnitude of the forecasted event.
-  - **time_string**: Timestamp in UTC following the ISO8601 format (`%Y-%m-%dT%H:%M:%S`).
+  - **lon**: Decimal degrees of the event longitude.
+  - **lat**: Decimal degrees of the event latitude.
+  - **mag**: Magnitude of the event.
+  - **time_string**: Timestamp in UTC following the ISO8601 format (`%Y-%m-%dT%H:%M:%S`, with fractional seconds `.%f` when the event has them).
   - **depth**: Depth of the event in kilometers.
+  - **catalog_id**: Always ``-1`` for an observed catalog (as opposed to the index of a synthetic catalog in a forecast).
   - **event_id**: The event ID in case is necessary to map the event to an additional table.
 
 

docs/reference/api_reference.rst

@@ -125,6 +125,8 @@ These are the helper functions of ``floatCSEP``
     parse_timedelta_string
     read_time_cfg
     read_region_cfg
+    read_input_cfg
+    parse_region
     time_windows_ti
     time_windows_td
 
@@ -241,8 +243,11 @@ catalogs, forecasts), abstracting the experiment logic from the pyCSEP io functi
    CatalogRepository
    CatalogRepository.set_main_catalog
    CatalogRepository.catalog
+   CatalogRepository.input_cat_config
+   CatalogRepository.query_bounds
    CatalogRepository.get_test_cat
    CatalogRepository.set_test_cats
+   CatalogRepository.get_input_cat
    CatalogRepository.set_input_cats
    CatalogRepository.filter_catalog
 

docs/tutorials/case_h.rst

@@ -81,7 +81,7 @@ As in :ref:`Tutorial G<case_g>`, each **Model** requires to build and execute a
     .. literalinclude:: ../../tutorials/case_h/models.yml
         :caption: tutorials/case_h/models.yml
         :language: yaml
-        :lines: 1-3, 12-13, 22-23
+        :lines: 1-3, 12-13, 23-24
 
 2. A ``path`` needs to be indicated for each model, to both download the repository contents therein and from where the source code will be executed.
 
@@ -132,7 +132,7 @@ As in :ref:`Tutorial G<case_g>`, each **Model** requires to build and execute a
     .. literalinclude:: ../../tutorials/case_h/models.yml
         :caption: tutorials/case_h/models.yml
         :language: yaml
-        :lines: 1,7,12,15,22,25
+        :lines: 1,7,12,15,23,26
 
     .. important::
         Please refer to :ref:`Tutorial G<case_g>` for example of how to set up ``func`` for the model and interface it to **floatCSEP**.
@@ -142,7 +142,7 @@ As in :ref:`Tutorial G<case_g>`, each **Model** requires to build and execute a
     .. literalinclude:: ../../tutorials/case_h/models.yml
         :caption: tutorials/case_h/models.yml
         :language: yaml
-        :lines: 12,16,22, 26
+        :lines: 12,16,23,27
 
     The experiment will read the forecasts as:
 
@@ -157,7 +157,7 @@ As in :ref:`Tutorial G<case_g>`, each **Model** requires to build and execute a
     .. literalinclude:: ../../tutorials/case_h/models.yml
         :caption: tutorials/case_h/models.yml
         :language: yaml
-        :lines: 12,18-21,22,28-33
+        :lines: 12,18-22,23,29-35
 
 .. note::
 
@@ -178,6 +178,18 @@ Catalog
 
     The catalog was obtained *prior* to the experiment using :func:`query bsi <csep.query_bsi>`, but it was filtered from 2006 onwards, so it has enough data for the model calibration.
 
+Input Catalog
+~~~~~~~~~~~~~
+
+    The models are time-dependent, so they receive an **input catalog** at each time window, with the seismicity before the forecast start. The ``input_config`` block sets how this catalog is filtered, independently from the testing settings of ``region_config``: here the models are calibrated with events down to magnitude 2.5 within the Italy *collection* region, whereas the forecasts are tested for magnitudes above 3.5 within the Italy *testing* region.
+
+    .. literalinclude:: ../../tutorials/case_h/config.yml
+        :caption: tutorials/case_h/config.yml
+        :language: yaml
+        :lines: 17-22
+
+    Without ``input_config``, the input catalog would contain the events of the testing magnitude range only, with no spatial or depth filter. See :ref:`experiment_config` for the admissible parameters.
+
 
 Tests
 ~~~~~
@@ -201,7 +213,7 @@ Custom Post-Process
     .. literalinclude:: ../../tutorials/case_h/config.yml
         :caption: tutorials/case_h/config.yml
         :language: yaml
-        :lines: 22-23
+        :lines: 29-30
 
     This option provides `hook` for a Python script and a function within as:
 

floatcsep/experiment.py

@@ -22,6 +22,7 @@
     NoAliasLoader,
     read_time_cfg,
     read_region_cfg,
+    read_input_cfg,
     timewindow2str,
     parse_nested_dicts,
 )
@@ -75,6 +76,21 @@ class Experiment:
             - depth_min (:class:`float`): Minimum depth. Defaults to -2
             - depth_max (:class:`float`): Maximum depth. Defaults to 6000
 
+        input_config (dict): Settings of the data handed to the models, as opposed to the
+            testing settings above. Contains a ``catalog`` block with the filters applied to
+            the main catalog before it is written as a model's input catalog:
+
+            - region: Spatial domain of the input catalog (same options as the testing
+              ``region``). Defaults to no spatial filter.
+            - mag_min, mag_max (:class:`float`): Magnitude range of the input catalog.
+              Default to the testing magnitude range.
+            - depth_min, depth_max (:class:`float`): Depth range of the input catalog.
+              Default to no depth filter.
+            - start_date (:class:`datetime.datetime`): Earliest event of the input catalog.
+              Defaults to no lower bound. The upper bound is always the forecast start.
+
+            See :func:`~floatcsep.utils.helpers.read_input_cfg`.
+
         model_config (str): Path to the models' configuration file
         test_config (str): Path to the evaluations' configuration file
         run_mode (str): 'sequential' or 'parallel'
@@ -106,6 +122,7 @@ def __init__(
         LICENSE: str = None,
         time_config: dict = None,
         region_config: dict = None,
+        input_config: dict = None,
         catalog: str = None,
         models: str = None,
         tests: str = None,
@@ -156,6 +173,7 @@ def __init__(
         self.seed = kwargs.get("seed", None)
         self.time_config = read_time_cfg(time_config, **kwargs)
         self.region_config = read_region_cfg(region_config, **kwargs)
+        self.input_config = read_input_cfg(input_config, self.region_config, **kwargs)
         self.model_config = models if isinstance(models, str) else None
         self.test_config = tests if isinstance(tests, str) else None
 
@@ -182,14 +200,25 @@ def __init__(
             else "Time-Independent"
         )
         log.info(f"\tExperiment class: {exp_class_str}")
+        if self.input_config:
+            inp = self.input_config["catalog"]
+            inp_region = inp["region"].name if inp["region"] is not None else None
+            log.info(
+                f"\tInput catalog: region {inp_region}, "
+                f"magnitudes [{inp['mag_min']}, {inp['mag_max']}), "
+                f"depths [{inp.get('depth_min')}, {inp.get('depth_max')}), "
+                f"from {inp.get('start_date')}"
+            )
 
         self.catalog = None
         self.models = []
         self.tests = []
 
         self.postprocess = postprocess if postprocess else {}
         self.default_test_kwargs = default_test_kwargs
-        self.catalog_repo.set_main_catalog(catalog, self.time_config, self.region_config)
+        self.catalog_repo.set_main_catalog(
+            catalog, self.time_config, self.region_config, self.input_config
+        )
 
         self.models = self.set_models(
             models or kwargs.get("model_config"), kwargs.get("order", None)
@@ -582,13 +611,17 @@ def make_repr(self) -> None:
         repr_config = self.registry.get_attr("repr_config")
 
         # Dropping region to results folder if it is a file
-        region_path = self.region_config.get("path", False)
-        if isinstance(region_path, str):
-            if isfile(region_path) and region_path:
-                new_path = join(self.registry.run_dir, self.region_config["path"])
-                shutil.copy2(region_path, new_path)
-                self.region_config.pop("path")
-                self.region_config["region"] = self.registry.rel(new_path)
+        configs = [self.region_config]
+        if self.input_config:
+            configs.append(self.input_config["catalog"])
+        for cfg in configs:
+            region_path = cfg.get("path", False)
+            if isinstance(region_path, str):
+                if isfile(region_path) and region_path:
+                    new_path = join(self.registry.run_dir, cfg["path"])
+                    shutil.copy2(region_path, new_path)
+                    cfg.pop("path")
+                    cfg["region"] = self.registry.rel(new_path)
 
         # Dropping catalog to results folder
         target_cat = join(
@@ -632,10 +665,20 @@ def as_dict(self, extra: Sequence = (), extended=False) -> dict:
                 for i, j in self.region_config.items()
                 if (i not in ("magnitudes", "depths") or extended)
             },
-            "catalog": self.registry.rel(self.catalog_repo.cat_path).as_posix(),
-            "models": [i.as_dict() for i in self.models],
-            "tests": [i.as_dict() for i in self.tests],
         }
+        if self.input_config:
+            dict_walk["input_config"] = {
+                "catalog": {
+                    i: j for i, j in self.input_config["catalog"].items() if j is not None
+                }
+            }
+        dict_walk.update(
+            {
+                "catalog": self.registry.rel(self.catalog_repo.cat_path).as_posix(),
+                "models": [i.as_dict() for i in self.models],
+                "tests": [i.as_dict() for i in self.tests],
+            }
+        )
         dict_walk.update(extra)
         return parse_nested_dicts(dict_walk)