Migrate to datajoint v2

[thomasZen]

This branch already has changes from #72 integrated.

When testing the tutorial schema, make sure to create a new config based on https://github.com/eulerlab/djimaging/blob/migration/datajointv2/djimaging/djconfig/djconf_template.json, adapt the path for processed files, and run the make_user.py script to have access to the updated example notebook, and the mentioned folders for the processed files have to exists already.

The example notebook then partially works already, I have to rerun it and will check it again, though.

Changes (AI Summary)

1. Move to DataJoint 2

• The dependency is now pinned to datajoint>=2.3.1,<2.4 (it was >=0.13).
• The core tables have been rewritten for the new API: roi_mask, stimulus, averages, snippets, traces, presentation and preprocesstraces.
• A new module, dj_storage.py, handles DataJoint 2 object references: it can open a remote object like a local file or give you a local copy.
• Larger data is kept as files, with paths relative to a single reference folder. Smaller data uses blobs.
  load_array is now used everywhere to read arrays.
• The pytest settings now turn DataJoint's warnings about the deprecated fetch() and ERD into errors, so any leftover old-style calls will fail the tests.

2. Removing deprecated and duplicate code

• The whole old tables/classifier/ package is gone (celltype_assignment, rgc_classifier, baden16_traces). classifier_v2 replaces it, and the celltype-assignment tables were updated for the new DataJoint interface.
• Also deleted: receptivefield/sta.py (FastSta takes over), chirp_surround.py, orientation_utils_v1.py and orientation_utils_v2.py (merged into orientation_utils.py), and utils/import_helpers.py.
  fit_rf_utils.py shrank by about 360 lines.

3. Features and fixes

• Stimulus logs: a new PresentationLogTemplate, time-based matching of stimulus logs, and a fixed QDSpy log reader.
• ROI masks: masks are now saved as numpy files, can be scanned from different sources, and the correlation masks and autorois were fixed.
• Receptive fields: improvements to FastSta (including a bug that broke it when no restriction was set), the GLM utilities, and the cascade spike-estimation table. There's also an AIC/BIC typo fix.
• Noise stimulus: better colour-noise processing, updated trigger upsampling, and the default frame rate corrected to 60.

4. Dependencies and CI

• NumPy 2 is now required, so other compiled packages have minimum versions that support it. Scikit-learn must be at least 1.7.2 to load the saved classifiers.
• Some indirect dependencies (pillow, aiohttp, idna) are held at versions with security fixes.
  uv.lock was regenerated, and CI runs on Python 3.12 to 3.14.

5. Tests

• New tests cover DataJoint 2 table definitions and runtime behaviour, the storage helpers (unit and integration), and the tutorial pipeline, which runs against MySQL 8.

Other notes

• Rewrote classifier to make it compatible with scikit-learn-1.7: /gpfs01/euler/data/Resources/Classifier_v2/rgc_classifier_v2_sklearn_1_7.pkl
• Most files will be written as blobs into the database, only big files will be written as files (for the tutorial notebook the only file was stim_trace_vAsiIasY.npy with 3.4 MB)

To check

Moving Bar Stimulus warning

Stimulus().add_movingbar(
    ntrigger_rep=1,
    trial_info=[0, 180, 45, 225, 90, 270, 135, 315],
    bardx=300,
    bardy=1000,
    velumsec=1000,
    tmovedurs=4.0,
    skip_duplicates=True,
)
[djimaging/tables/core/stimulus.py:66](http://localhost:4444/lab/workspaces/auto-R/tree/GitRepos/djimaging/djimaging/user/dj_v2_try3_tz/notebooks/GitRepos/djimaging/djimaging/tables/core/stimulus.py#line=65): UserWarning: Number of triggers in trial_info=8 must match ntrigger_rep=1.
  warnings.warn(msg)

To test

• CellAssignment
• AutoRoi (the example data already had roi masks for all cells)