Skip to content

Add Ma2022 (SHU) cross-session motor imagery dataset - #1178

Open
qinxwew wants to merge 1 commit into
NeuroTechX:developfrom
qinxwew:add-ma2022-dataset
Open

Add Ma2022 (SHU) cross-session motor imagery dataset#1178
qinxwew wants to merge 1 commit into
NeuroTechX:developfrom
qinxwew:add-ma2022-dataset

Conversation

@qinxwew

@qinxwew qinxwew commented Sep 5, 2026

Copy link
Copy Markdown

Closes #893.

What

Adds an adapter for the SHU cross-session motor imagery dataset (Ma et al. 2022, Scientific Data 9:531), requested in #893 and originally suggested by @PierreGtch in #1.

  • 25 healthy, BCI-naive subjects (13 M / 12 F, aged 20–24)
  • Cued left- vs right-hand grasping MI, 2 classes
  • 5 sessions per subject recorded on 5 different days (2–3 days apart) — the dataset is specifically designed for cross-session MI variability studies, complementing the existing same-limb (Ma2020) and multi-day (Yang2025) datasets
  • 32 EEG channels (10-20 system, reference M1, ground AFz) at 250 Hz
  • 11,988 retained trials in total (74–100 per session after source-side bad-segment rejection), each stored as a 1000-sample (4 s) motor-imagery window

Data access: why version 1 of the figshare record

The figshare record 19228725 exists in three versions. Only version 1 exposes the data archives without password protection — versions 2 and 3 re-uploaded the same recordings inside AES-encrypted zips (the record description asks users to email the author for the password). The data paper itself cites version 1 as "open access for free download" (reference 17 in the paper), and the record is CC-BY-4.0. The loader therefore downloads the open version-1 mat.zip archive (figshare file id 36324114) and documents this in the module and class docstrings.

Implementation notes

  • Each .mat session file stores epoched trials (n_trials, 32, 1000) in µV plus a labels vector (1 = left, 2 = right). Trials are concatenated along time into a continuous RawArray; every trial onset is marked with an MNE annotation (not a stim channel) so that the first trial, which starts at sample 0, is not dropped by mne.find_events.
  • interval=[0, 4 - 1/250]: mne.Epochs includes the tmax sample, so [0, 4] would borrow one sample from the next concatenated trial and drop the last trial of each session. The chosen interval extracts exactly the 1000 stored samples per trial.
  • Channel names follow the acquisition order from the BIDS task-motorimagery_channels.tsv sidecar, with case normalized. The dataset uses the older 10-20 nomenclature (T3/T4/T5/T6) plus mastoids A1/A2; standard_1005 is applied with on_missing="ignore".
  • Per-subject sex/age from participants.tsv is attached to raw.info["subject_info"].
  • The docstring carries an .. important:: note that the released data is preprocessed by the authors (bad-segment rejection, baseline removal, 0.5–40 Hz FIR band-pass); data_processed=True and a PreprocessingMetadata section record this in the metadata catalog.
  • Full DatasetMetadata provided (acquisition / participants / experiment / documentation / preprocessing / paradigm-specific / data-structure / signal-processing / cross-validation / BCI-application).

This implementation was informed by the exploratory draft on the codex/mi-dataset-expansion branch (which was not merged as part of #1038). Relative to that draft, this PR downloads only the MAT archive (dropping the local-EDF fallback path that required the password-protected release), fills in the complete metadata sections, adds per-subject demographics, and removes the manual docstring summary table in favor of the auto-generated one.

Files

  • moabb/datasets/ma2022.py — the adapter
  • moabb/datasets/__init__.py, docs/source/api.rst — registration
  • moabb/datasets/summary_imagery.csv — summary-table row (auto-generates the docstring summary card)
  • moabb/tests/test_metadata.py — bump metadata-catalog count 160 → 161
  • moabb/tests/test_datasets.py — add Ma2022 to NEMAR_ID_EXEMPT (no NEMAR deposit exists for this dataset)
  • docs/source/whats_new.rst — changelog entry

Testing

  • pytest moabb/tests/test_datasets.py — 1278 passed (2 pre-existing network-dependent failures, test_dataset_accept and test_epochs, fail identically on develop)
  • pytest moabb/tests/test_metadata.py — all pass
  • pre-commit run — all hooks pass (ruff, ruff-format, codespell, …)
  • Real-data validation against the downloaded version-1 archive (MD5 verified, f577fbd4…):
    • All 125 .mat files parsed: total trial count is exactly 11,988 (as stated in the docstring), per-session counts range 74–100, every file is shaped (n_trials, 32, 1000) with labels ⊆ {1, 2}; overall label balance is 5,983 left / 6,005 right.
    • dataset.get_data() for subjects 1–2 loads 5 sessions each (492 and 482 trials); MotorImagery(n_classes=2).get_data() returns 492 × 32 × 1000 epochs with correct session metadata.
    • Signal magnitudes are consistent with the disclosed preprocessed EEG (per-trial std 1.5–4.4 µV, 0.5–40 Hz band-passed).
    • Channel-order sanity: the left-vs-right 8–30 Hz band-power contrast at C3/C4 follows the lateralization direction reported in the data paper (left-hand trials show higher C3 power, lower C4 power at group level), which would not hold if the assumed channels.tsv ordering were scrambled.

Review focus

  1. The version-1 download choice (see "Data access" above) — is pinning to the open v1 archive acceptable policy-wise?
  2. A1/A2 mastoid channels are kept among the 32 EEG channels, matching the BIDS sidecar; T3/T4/T5/T6 keep their old nomenclature (montage applied with on_missing="ignore").
  3. The released data being pre-epoched/pre-filtered is unusual for MOABB; it is flagged prominently in the docstring and metadata.

Add adapter for the SHU dataset (Ma et al. 2022, Scientific Data):
25 BCI-naive subjects, cued left/right-hand grasping MI, 5 sessions
on 5 different days, 32 EEG channels at 250 Hz, 11988 retained 4 s
trials (concatenated 1000-sample MI windows in .mat files).

The loader targets the open version-1 figshare archive cited by the
data paper; record versions 2/3 re-uploaded the same recordings
inside AES-encrypted zips.

Closes NeuroTechX#893

Co-Authored-By: Claude <noreply@anthropic.com>
@bruAristimunha

Copy link
Copy Markdown
Collaborator

This dataset is interesting!

But the main problem is that the "public" version doesn't include all subjects, if I remember correctly.

If the license allows, we can repost the public version without the password, since the password system violates the Scientific Data policy.

Can you confirm the license, @qinxwew?

@qinxwew

qinxwew commented Sep 5, 2026

Copy link
Copy Markdown
Author

Thanks for taking a look!

Confirming the license: the figshare record (19228725) is CC BY 4.0, on all published versions (v1-v3).

One small clarification about the public version: I verified the openly downloadable v1 mat_files.zip file by file, and it actually contains all 25 subjects with all 5 sessions each. What is reduced is the per-session trial count: 74-100 trials are retained per session, for 11,988 trials in total versus the 12,500 designed in the paper. So this adapter runs entirely on the unencrypted v1 archive, and users never need a password.

The v2/v3 archives are indeed password-protected -- the record asks users to email the corresponding author for the zip password -- which I agree conflicts with the Scientific Data open-access policy. Under CC BY 4.0, re-posting an unencrypted copy would be allowed. If the authors publish one, updating this adapter should only require re-pointing the pinned version, with no structural changes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Dataset] Add Ma2022 Motor Imagery dataset (25 subjects, 5 sessions)

2 participants