Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
56 commits
Select commit Hold shift + click to select a range
fa26087
BUG: fixed TODO calls
aburrell Sep 25, 2026
6169884
BUG: fixed meta data assignment
aburrell Sep 25, 2026
1ce2394
BUG: fixed yaml indentation
aburrell Sep 25, 2026
f587599
STY: fixed PEP8 violations
aburrell Sep 25, 2026
09ab5f7
STY: changed import order
aburrell Sep 25, 2026
670a62d
DOC: updated documentation
aburrell Sep 25, 2026
e5032b7
BUG: changed import order
aburrell Sep 25, 2026
545d8bc
MAINT: updated dependencies
aburrell Sep 25, 2026
3994aa3
BUG: remove numpy float type
aburrell Sep 25, 2026
55bb412
DEP: change dims to sizes
aburrell Sep 25, 2026
7af0f4f
BUG: fixed assignment with index
aburrell Sep 25, 2026
431d9f3
BUG: added a debug line
aburrell Sep 25, 2026
585fec1
BUG: updated debug statement
aburrell Sep 25, 2026
168a78f
BUG: moved debug statement
aburrell Sep 25, 2026
e1b882d
DOC: updated quickstart
aburrell Sep 25, 2026
33a027c
ENH: added space weather instruments
aburrell Sep 28, 2026
07388fb
ENH: added space weather instrument support
aburrell Sep 30, 2026
f1a0d9d
BUG: fixed instrument meta data assignment
aburrell Sep 30, 2026
4aa2348
DOC: updated docstring
aburrell Sep 30, 2026
304df02
DOC: updated changelog
aburrell Sep 30, 2026
1735f05
STY: updated import order
aburrell Sep 30, 2026
6143628
STY: changed import order
aburrell Sep 30, 2026
7bfb9e3
TST: added tests for space weather
aburrell Sep 30, 2026
7e6788b
BUG: fixed instrument assignment for Python 3.14
aburrell Sep 30, 2026
80a2839
TST: removed pysatSpaceWeather
aburrell Sep 30, 2026
2f055a2
BUG: attempt to fix an I/O bug
aburrell Sep 30, 2026
35317f0
BUG: removed tab from yaml
aburrell Sep 30, 2026
e78d186
BUG: add options to special cases
aburrell Sep 30, 2026
5fa379b
BUG: fixed test to assign a different value
aburrell Oct 1, 2026
1906947
BUG: updated data assignment
aburrell Oct 1, 2026
65ef129
BUG: removed tabs from yaml
aburrell Oct 1, 2026
8832987
BUG: xarray needs netcdf4
aburrell Oct 2, 2026
b35b222
BUG: fixed test
aburrell Oct 2, 2026
1322610
BUG: added error type to except
aburrell Oct 2, 2026
df73758
BUG: added test mark
aburrell Oct 2, 2026
694071d
MAINT: removed unused lines
aburrell Oct 2, 2026
ab37b69
BUG: added debug statements
aburrell Oct 2, 2026
7593c0a
BUG: use explicit multiplication
aburrell Oct 2, 2026
7ce533c
BUG: remove untestable test
aburrell Oct 2, 2026
6f8bf2b
STY: remove bad lines
aburrell Oct 2, 2026
0097db4
BUG: fixed manifest
aburrell Oct 2, 2026
5b64802
BUG: fixed links
aburrell Oct 2, 2026
f6b6f0b
BUG: fixed links
aburrell Oct 2, 2026
6f3c303
Merge pull request #5 from USNavalResearchLaboratory/space_weather
aburrell Oct 2, 2026
37a303a
DOC: add examples
aburrell Oct 2, 2026
b8212fb
DOC: add examples to index
aburrell Oct 2, 2026
e827f28
DOC: add the space weather instruments
aburrell Oct 2, 2026
746a3e4
DOC: added test badges
aburrell Oct 2, 2026
7b896af
TST: remove debug statements
aburrell Oct 2, 2026
0fe5538
DOC: fixed references and directory name
aburrell Oct 2, 2026
33784d6
DOC: updated changelog
aburrell Oct 2, 2026
7edfd4a
DOC: fixed typos
aburrell Oct 2, 2026
7243a32
Merge pull request #7 from USNavalResearchLaboratory/space_weather_docs
aburrell Oct 2, 2026
8c24d0b
REL: updated version number
aburrell Oct 2, 2026
8663d23
DOC: updated changelog
aburrell Oct 2, 2026
43e433d
MAINT: Update requirements.txt
aburrell Oct 2, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions .github/workflows/docs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ jobs:
- name: Check documentation links
run: sphinx-build -b linkcheck docs dist/docs

- name: Validate the CITATION.cff
uses: citation-file-format/cffconvert-github-action@2.0.0
with:
args: "--infile ./CITATION.cff --validate"
- name: Validate the CITATION.cff
uses: citation-file-format/cffconvert-github-action@2.0.0
with:
args: "--infile ./CITATION.cff --validate"
12 changes: 10 additions & 2 deletions .github/workflows/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ jobs:
matrix:
os: ["ubuntu-latest", "windows-latest", "macos-latest"]
python-version: ["3.10", "3.11", "3.13", "3.14"]
options: ["standard", "space_weather"]
numpy_ver: ["latest"]
test_config: ["latest"]
include:
Expand All @@ -27,11 +28,13 @@ jobs:
xarray_ver: "2024.5.0"
os: ubuntu-latest
test_config: "SPEC0000"
options: "standard"
# Operational compliance settings
- python-version: "3.12"
numpy_ver: "2.0.0"
os: "ubuntu-22.04"
test_config: "Ops"
options: "space_weather"
# Minimum version settings
- python-version: "3.10"
dask_ver: "2022.12.1"
Expand All @@ -45,7 +48,8 @@ jobs:
xarray_ver: "2022.12.0"
os: "ubuntu-22.04"
test_config: "Min"
name: Python ${{ matrix.python-version }} on ${{ matrix.os }} with numpy ${{ matrix.numpy_ver }}
options: "standard"
name: Python ${{ matrix.python-version }} on ${{ matrix.os }} with numpy ${{ matrix.numpy_ver }} for ${{ matrix.options }}
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v6
Expand Down Expand Up @@ -87,10 +91,14 @@ jobs:
if: ${{ matrix.test_config == 'latest'}}
run: pip install .[test]

- name: Install extra dependencies
if: ${{ !startsWith(matrix.options, 'standard') }}
run: pip install .[${{ matrix.options }}]

- name: Set up the data directory
run: |
mkdir gdmData
python -c "import GeospaceDataManagement; GeospaceDataManagement.params['data_dirs'] = 'gdmData'"
python -c "import GeospaceDataManagement; GeospaceDataManagement.params['data_dirs'] = 'gdmData'; print(GeospaceDataManagement.params)"

- name: Test PEP8 compliance
run: flake8 . --count --select=D,E,F,H,W --show-source --statistics
Expand Down
16 changes: 15 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,20 @@ Change Log
All notable changes to this project will be documented in this file.
This project adheres to [Semantic Versioning](https://semver.org/).

[0.0.1] - 2026-10-XX
[0.0.2] - 2026-10-02
--------------------
* ENH:
- Added Space Weather instruments as an optional installation
- Allow `generate_instrument_list` to include only a subset of the instruments
* BUG:
- Fixed assignment issue in Instrument, where meta data could not be added if
a variable did not exist
- Fixed assignment issue in Instrument, where xarray behaviour changed in
more recent versions of Python
* DOC:
- Added missing sections to docstrings
- Added information about space weather instruments and methods

[0.0.1] - 2026-09-25
--------------------
Initial Release
2 changes: 1 addition & 1 deletion CITATION.cff
Original file line number Diff line number Diff line change
Expand Up @@ -21,4 +21,4 @@ keywords:
- "Observations"
- "Models"
license: MIT
version: 0.0.1
version: 0.0.2
59 changes: 25 additions & 34 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -1,46 +1,40 @@
Contributing
============
# Contributing

Bug reports, feature suggestions, and other contributions are greatly
appreciated! GeospaceDataManager welcomes both feedback and contributions.
appreciated! GeospaceDataManagement welcomes both feedback and contributions.

Short version
-------------
## Short version

* Submit bug reports, feature requests, and questions at
[GitHub](https://github.com/USNavalResearchLaboratory/GeospaceDataManager/issues)
[GitHub](https://github.com/USNavalResearchLaboratory/GeospaceDataManagement/issues)

* Make all pull requests to the ``develop`` branch

Issues
------
## Issues

Bug reports, questions, and feature requests should all be made as GitHub
Issues. Templates are provided for each type of issue, to help you include
all the necessary information.

Questions
^^^^^^^^^
### Questions

Not sure how something works? Ask away! The more information you provide, the
easier the question will be to answer.

Bug reports
^^^^^^^^^^^
### Bug reports

When [reporting a bug](https://github.com/GeospaceDataManager/GeospaceDataManager/issues) please include:
When [reporting a bug](https://github.com/USNavalResearchLaboratory/GeospaceDataManagement/issues) please include:

* Your operating system name and version

* Any details about your local setup that might be helpful in troubleshooting

* Detailed steps to reproduce the bug

Feature requests and feedback
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
### Feature requests and feedback

The best way to send feedback is to file an
[issue](https://github.com/GeospaceDataManager/GeospaceDataManager/issues).
[issue](https://github.com/USNavalResearchLaboratory/GeospaceDataManagement/issues).

If you are proposing a new feature or a change in something that already exists:

Expand All @@ -49,17 +43,16 @@ If you are proposing a new feature or a change in something that already exists:
* Keep the scope as narrow as possible, to make it easier to implement.


Development
-----------
## Development

To set up `GeospaceDataManager` for local development:
To set up `GeospaceDataManagement` for local development:

1. [Fork GeospaceDataManager on GitHub](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/fork-a-repo).
1. [Fork GeospaceDataManagement on GitHub](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/fork-a-repo).

2. Clone your fork locally:

```
git clone git@github.com:your_name_here/GeospaceDataManager.git
git clone git@github.com:your_name_here/GeospaceDataManagement.git
```

3. Create a branch for local development:
Expand All @@ -75,8 +68,8 @@ To set up `GeospaceDataManager` for local development:
standard tests.

Tests for custom functions should be added to the appropriately named file
in ``GeospaceDataManager/tests``. For example, custom functions for the time
utilities are tested in ``GeospaceDataManager/tests/test_utils_time.py``. If
in ``GeospaceDataManagement/tests``. For example, custom functions for the time
utilities are tested in ``GeospaceDataManagement/tests/test_utils_time.py``. If
no test file exists, then you should create one. This testing uses pytest,
which will run tests on any Python file in the test directory that starts
with ``test``. Classes must begin with ``Test``, and methods must begin with
Expand All @@ -95,7 +88,7 @@ To set up `GeospaceDataManager` for local development:
flake8 . --count --select=D,E,F,H,W --show-source --statistics
```

Note that GeospaceDataManager uses the `flake-docstrings` and `hacking`
Note that GeospaceDataManagement uses the `flake-docstrings` and `hacking`
packages to ensure standards in docstring formatting.

6. Update/add documentation (in ``docs``). Even if you don't think it's
Expand All @@ -109,7 +102,7 @@ To set up `GeospaceDataManager` for local development:
git commit -m "AAA: Brief description of your changes"
```
Where AAA is a standard shorthand for the type of change (e.g., BUG or DOC).
`GeospaceDataManager` follows the [numpy development workflow](https://numpy.org/doc/stable/dev/development_workflow.html),
`GeospaceDataManagement` follows the [numpy development workflow](https://numpy.org/doc/stable/dev/development_workflow.html),
see the discussion there for a full list of this shorthand notation.

9. Once you are happy with the local changes, push to GitHub:
Expand All @@ -121,11 +114,10 @@ To set up `GeospaceDataManager` for local development:
10. Submit a pull request through the GitHub website. Pull requests should be
made to the ``develop`` branch. Note that automated tests will be run on
GitHub Actions, but these must be initialized by a member of the
GeospaceDataManager team for outside contributors.
GeospaceDataManagement team for outside contributors.


Pull Request Guidelines
-----------------------
## Pull Request Guidelines

If you need some code review or feedback while you're developing the code, just
make a pull request. Pull requests should be made to the ``develop`` branch.
Expand All @@ -145,10 +137,9 @@ will delay tests by other developers, please ensure that the code passes all
tests on your local system first.


Project Style Guidelines
------------------------
## Project Style Guidelines

In general, GeospaceDataManager follows PEP8 and numpydoc guidelines. Pytest
In general, GeospaceDataManagement follows PEP8 and numpydoc guidelines. Pytest
runs the unit and integration tests, flake8 checks for style, and sphinx-build
performs documentation tests. However, there are certain additional style
elements that have been adopted to ensure the project maintains a consistent
Expand All @@ -160,8 +151,8 @@ coding style. These include:
* Use no more than 80 characters per line
* Avoid using Instrument class key attribute names as unrelated variable names:
`platform`, `name`, `tag`, and `inst_id`
* The GeospaceDataManager logger is imported into each sub-module and provides
status updates at the info and warning levels (as appropriate)
* The GeospaceDataManagement logger is imported into each sub-module and
provides status updates at the info and warning levels (as appropriate)
* Several dependent packages have common nicknames, including:
* `import datetime as dt`
* `import numpy as np`
Expand All @@ -175,7 +166,7 @@ coding style. These include:
* Use setup_method (or setup_class) and teardown_method (or teardown_class) in
test classes
* Use pytest parametrize in test classes when appropriate
* Use GeospaceDataManager testing utilities when appropriate
* Use GeospaceDataManagement testing utilities when appropriate
* Provide testing class methods with informative failure statements and
descriptive, one-line docstrings
* Block and inline comments should use proper English grammar and punctuation
Expand Down
14 changes: 4 additions & 10 deletions GeospaceDataManagement/__init__.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
"""GeospaceDataManagement
"""GeospaceDataManagement (a.k.a. gdm).

A package providing a simple and flexible interface for
downloading, loading, cleaning, and managing scientific measurements.
Expand Down Expand Up @@ -43,15 +43,9 @@
gdm_dir = os.path.join(home_dir, '.gdm')

# Set directory for test data
if resources is None:
test_data_path = os.path.join(os.path.realpath(os.path.dirname(__file__)),
'tests', 'test_data')
citation = os.path.join(os.path.realpath(os.path.dirname(__file__)),
'citation.txt')
else:
test_data_path = str(resources.files(__package__).joinpath('tests',
'test_data'))
citation = str(resources.files(__package__).joinpath('citation.txt'))
test_data_path = str(resources.files(__package__).joinpath('tests',
'test_data'))
citation = str(resources.files(__package__).joinpath('citation.txt'))

# Create a .gdm directory or parameters file if one doesn't exist.
settings_file = os.path.join(gdm_dir, 'gdm_settings.json')
Expand Down
44 changes: 35 additions & 9 deletions GeospaceDataManagement/_instrument.py
Original file line number Diff line number Diff line change
Expand Up @@ -834,6 +834,11 @@ def __setitem__(self, key, new_data):
# Simple assignment
inst['name'] = newData

# Assignment with Metadata
inst['name'] = {'data':new_data,
'long_name':long_name,
'units':units}

Raises
------
ValueError
Expand All @@ -846,7 +851,6 @@ def __setitem__(self, key, new_data):
If a single new value is set, the value will be broadcast over time.

"""

new = copy.deepcopy(new_data)

# xarray format chosen for Instrument object
Expand Down Expand Up @@ -885,9 +889,27 @@ def __setitem__(self, key, new_data):
# Try loading as values
self.data[var_key].loc[indict] = in_data
except (TypeError, KeyError, IndexError):
# Original code
# Try loading indexed as integers
self.data[key[-1]][indict] = in_data
# Attempt to use loc, which will currently fail for Python 3.14
sel_dict = {
self.data[var_key].dims[i]: self.data[var_key][
self.data[var_key].dims[i]].values[kind]
for i, kind in enumerate(ind_keys)}

try:
self.data[var_key].loc[sel_dict] = in_data
except (ValueError, KeyError, IndexError, TypeError):
# Input is probably an integer, get the desired data
# through array assignment. Because the original behaviour
# used indices for each dimension instead of paired indices,
# we invoke meshgrid to get the full range of index
# combinations
sel_dat = self.data[var_key].values.copy()
sel_ind = np.meshgrid(ind_keys)
sel_dat[sel_ind] = in_data
self.data[var_key].values = sel_dat

# Finish updating by adding meta data
self.data[var_key].attrs.update(new)

return
elif isinstance(key, str):
Expand Down Expand Up @@ -957,6 +979,10 @@ def __setitem__(self, key, new_data):
for keyname in key:
self.data[keyname] = in_data[keyname]

# Attach meta data, if it exists
if len(new) > 0:
self.data[key].attrs.update(new)

return

def __delitem__(self, key):
Expand Down Expand Up @@ -1191,13 +1217,13 @@ def _assign_attrs(self, by_name=False):

Notes
-----
methods
methods include:
init, preprocess, and clean
functions
functions include:
load, list_files, download, and list_remote_files, concat_data
attributes
attributes include:
directory_format, file_format, and multi_file_day
test attributes
test attributes include:
_test_download, _test_download_ci, _new_tests, and _password_req

"""
Expand Down Expand Up @@ -1608,7 +1634,7 @@ def _get_var_type_code(self, coltype):
np.dtypes.StringDType, np.dtypes.BytesDType,
pds.StringDtype]
except AttributeError:
# TODO(#1227) np.dtypes not introduced until somewhere around
# TODO(#1) np.dtypes not introduced until somewhere around
# numpy version 1.25
str_types = [str, np.str_, np.bytes_, pds.StringDtype]

Expand Down
2 changes: 1 addition & 1 deletion GeospaceDataManagement/citation.txt
Original file line number Diff line number Diff line change
@@ -1 +1 @@
Burrell, A. G. (2026). USNavalResearchLaboratory/GeospaceDataManagement v0.0.1 (Version v0.0.1). https://github.com/USNavalResearchLaboratory/GeospaceDataManagement
Burrell, A. G. (2026). USNavalResearchLaboratory/GeospaceDataManagement v0.0.2 (Version v0.0.2). https://github.com/USNavalResearchLaboratory/GeospaceDataManagement
18 changes: 16 additions & 2 deletions GeospaceDataManagement/instruments/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,22 @@
Each instrument is contained within a subpackage of this set.
"""

__all__ = ['gdm_ndtesting', 'gdm_netcdf', 'gdm_testing',
'gdm_testmodel']
__test__ = ['gdm_ndtesting', 'gdm_netcdf', 'gdm_testing',
'gdm_testmodel']
__all__ = list(__test__)

from GeospaceDataManagement.instruments import methods # noqa 401
try:
from GeospaceDataManagement.instruments import sw_methods # noqa 401
__sw__ = ['ace_epam', 'ace_mag', 'ace_sis', 'ace_swepam', 'norp_rf',
'sw_ae', 'sw_al', 'sw_ap', 'sw_apo', 'sw_au', 'sw_cp',
'sw_dst', 'sw_f107', 'sw_flare', 'sw_hpo', 'sw_kp',
'sw_mgii', 'sw_polarcap', 'sw_sbfield', 'sw_ssn',
'sw_stormprob']
__all__.extend(__sw__)
except ImportError as ierr:
import GeospaceDataManagement
GeospaceDataManagement.logger.info(ierr)

for inst in __all__:
exec("from GeospaceDataManagement.instruments import {x}".format(x=inst))
Expand Down
Loading
Loading