Add Ma2022 (SHU) cross-session motor imagery dataset - #1178
Conversation
Add adapter for the SHU dataset (Ma et al. 2022, Scientific Data): 25 BCI-naive subjects, cued left/right-hand grasping MI, 5 sessions on 5 different days, 32 EEG channels at 250 Hz, 11988 retained 4 s trials (concatenated 1000-sample MI windows in .mat files). The loader targets the open version-1 figshare archive cited by the data paper; record versions 2/3 re-uploaded the same recordings inside AES-encrypted zips. Closes NeuroTechX#893 Co-Authored-By: Claude <noreply@anthropic.com>
d950cf7 to
2675260
Compare
|
This dataset is interesting! But the main problem is that the "public" version doesn't include all subjects, if I remember correctly. If the license allows, we can repost the public version without the password, since the password system violates the Scientific Data policy. Can you confirm the license, @qinxwew? |
|
Thanks for taking a look! Confirming the license: the figshare record (19228725) is CC BY 4.0, on all published versions (v1-v3). One small clarification about the public version: I verified the openly downloadable v1 The v2/v3 archives are indeed password-protected -- the record asks users to email the corresponding author for the zip password -- which I agree conflicts with the Scientific Data open-access policy. Under CC BY 4.0, re-posting an unencrypted copy would be allowed. If the authors publish one, updating this adapter should only require re-pointing the pinned version, with no structural changes. |
Closes #893.
What
Adds an adapter for the SHU cross-session motor imagery dataset (Ma et al. 2022, Scientific Data 9:531), requested in #893 and originally suggested by @PierreGtch in #1.
Data access: why version 1 of the figshare record
The figshare record 19228725 exists in three versions. Only version 1 exposes the data archives without password protection — versions 2 and 3 re-uploaded the same recordings inside AES-encrypted zips (the record description asks users to email the author for the password). The data paper itself cites version 1 as "open access for free download" (reference 17 in the paper), and the record is CC-BY-4.0. The loader therefore downloads the open version-1
mat.ziparchive (figshare file id36324114) and documents this in the module and class docstrings.Implementation notes
.matsession file stores epoched trials(n_trials, 32, 1000)in µV plus alabelsvector (1 = left, 2 = right). Trials are concatenated along time into a continuousRawArray; every trial onset is marked with an MNE annotation (not a stim channel) so that the first trial, which starts at sample 0, is not dropped bymne.find_events.interval=[0, 4 - 1/250]:mne.Epochsincludes the tmax sample, so[0, 4]would borrow one sample from the next concatenated trial and drop the last trial of each session. The chosen interval extracts exactly the 1000 stored samples per trial.task-motorimagery_channels.tsvsidecar, with case normalized. The dataset uses the older 10-20 nomenclature (T3/T4/T5/T6) plus mastoids A1/A2;standard_1005is applied withon_missing="ignore".participants.tsvis attached toraw.info["subject_info"]... important::note that the released data is preprocessed by the authors (bad-segment rejection, baseline removal, 0.5–40 Hz FIR band-pass);data_processed=Trueand aPreprocessingMetadatasection record this in the metadata catalog.DatasetMetadataprovided (acquisition / participants / experiment / documentation / preprocessing / paradigm-specific / data-structure / signal-processing / cross-validation / BCI-application).This implementation was informed by the exploratory draft on the
codex/mi-dataset-expansionbranch (which was not merged as part of #1038). Relative to that draft, this PR downloads only the MAT archive (dropping the local-EDF fallback path that required the password-protected release), fills in the complete metadata sections, adds per-subject demographics, and removes the manual docstring summary table in favor of the auto-generated one.Files
moabb/datasets/ma2022.py— the adaptermoabb/datasets/__init__.py,docs/source/api.rst— registrationmoabb/datasets/summary_imagery.csv— summary-table row (auto-generates the docstring summary card)moabb/tests/test_metadata.py— bump metadata-catalog count 160 → 161moabb/tests/test_datasets.py— addMa2022toNEMAR_ID_EXEMPT(no NEMAR deposit exists for this dataset)docs/source/whats_new.rst— changelog entryTesting
pytest moabb/tests/test_datasets.py— 1278 passed (2 pre-existing network-dependent failures,test_dataset_acceptandtest_epochs, fail identically ondevelop)pytest moabb/tests/test_metadata.py— all passpre-commit run— all hooks pass (ruff, ruff-format, codespell, …)f577fbd4…):.matfiles parsed: total trial count is exactly 11,988 (as stated in the docstring), per-session counts range 74–100, every file is shaped(n_trials, 32, 1000)with labels ⊆ {1, 2}; overall label balance is 5,983 left / 6,005 right.dataset.get_data()for subjects 1–2 loads 5 sessions each (492 and 482 trials);MotorImagery(n_classes=2).get_data()returns492 × 32 × 1000epochs with correct session metadata.channels.tsvordering were scrambled.Review focus
on_missing="ignore").