Repository navigation
Support negative slice steps on fields stored in HDF5 - #277
Open
aaronkanzer wants to merge 1 commit into
Open
aaronkanzer wants to merge 1 commit into
aaronkanzer wants to merge 1 commit into
Conversation
HDF5 hyperslabs require positive steps, so field[::-1] raised 'Step must be >= 1' whenever the field data lived in a file or in the core memory file used for large fields, although the same index works on small fields held in memory. Read and write the equivalent slice with a positive step and reverse the affected axes in NumPy.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Slicing an NXfield with a negative step works when the data are in memory. It fails once the data are in a file, or in the HDF5 core memory file used for large fields. So whether
field[::-1]works depends on the size of the array.Observed:
ValueError: Step must be >= 1 (got -1). Writes such asx[::-2] = ...on a file opened with"rw"fail the same way. So doesNXfield(shape=(200, 100), dtype=float)[::-1], because fields larger thanmaxsizeare kept in an HDF5 core file.Expected: the same result as
a[::-1], which is what a small array returns.Cause:
NXFile.readvalue,NXFile.writevalue,_get_memdataand_put_memdatapass the index straight to h5py. HDF5 hyperslabs only accept positive steps.Fix: a small helper turns each slice with a negative step into the matching slice with a positive step. It reads or writes that slice, then reverses the affected axes with NumPy. Indices without a negative step, and fancy indices, still go to h5py unchanged. Tests cover reads and writes on file-backed and memory-file fields. The helpers were also checked against NumPy on 5000 random int, slice and Ellipsis indices, with no mismatches.