Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
46 commits
Select commit Hold shift + click to select a range
a3f247b
Expect eumGetNextEqvUnit to report units while rc is non-zero
ryan-kipawa Oct 12, 2026
4d5153c
Read eumGetNextEqvUnit's return code like the other eum wrappers
ryan-kipawa Oct 12, 2026
0d49b2d
Expect DfsuBuilder to write the element and node ids it is given
ryan-kipawa Oct 12, 2026
b169d7a
Store DfsuBuilder element ids, and node and element ids as int32
ryan-kipawa Oct 12, 2026
5a63c43
Expect MeshBuilder to keep z as given
ryan-kipawa Oct 12, 2026
fdbc893
Keep z as float64 in MeshBuilder.SetNodes
ryan-kipawa Oct 12, 2026
0cbcc7d
Expect Dfs0ToAscii to stamp rows in the time axis unit
ryan-kipawa Oct 12, 2026
1b82f89
Convert Dfs0ToAscii item times with the time axis's ToSeconds
ryan-kipawa Oct 12, 2026
3a52a74
Expect ReadStaticItem to reject item numbers below 1
ryan-kipawa Oct 12, 2026
6340f38
Reject static item numbers below 1 in ReadStaticItem
ryan-kipawa Oct 12, 2026
602237b
Test reshape on items read through a generic DfsFile
ryan-kipawa Oct 12, 2026
7b59a9e
Reshape 2-D and 3-D items in DfsFile.ReadItemTimeStepNext
ryan-kipawa Oct 12, 2026
bee7614
Expect reused item builders to keep settings and leave items alone
ryan-kipawa Oct 12, 2026
37bf124
Keep a private copy of the item in DfsBuilder's item builders
ryan-kipawa Oct 12, 2026
825d654
Expect DfsuBuilder to write and validate Spectral0D files
ryan-kipawa Oct 12, 2026
ccf25e3
Write DfsuSpectral0D files in DfsuBuilder
ryan-kipawa Oct 12, 2026
dd5e32d
Expect SetElementIds(None) to restore defaults, and int32 range checks
ryan-kipawa Oct 12, 2026
6f82e0a
Restore default element ids on None, and reject ids outside int32
ryan-kipawa Oct 12, 2026
72db4eb
Expect MeshBuilder.SetElementIds to take None and reject ids outside …
ryan-kipawa Oct 13, 2026
f201509
Restore default mesh element ids on None, and reject ids outside int32
ryan-kipawa Oct 13, 2026
d5afdac
Expect DfsuBuilder to write the ids of a mesh built by MeshBuilder
ryan-kipawa Oct 13, 2026
04707fa
Store the ids SetFromMeshFile takes from a mesh as int32
ryan-kipawa Oct 13, 2026
1c48e76
Lower the pyrefly warning ceiling to 325
ryan-kipawa Oct 13, 2026
cc9bc10
Share one int32 id helper between MeshBuilder and DfsuBuilder
ryan-kipawa Oct 20, 2026
50c9a3b
Expect id setters to reject 2-D, scalar and boolean ids
ryan-kipawa Oct 20, 2026
43f3287
Reject 2-D, scalar and boolean ids in the id setters
ryan-kipawa Oct 20, 2026
4a4a81e
Expect MeshBuilder's default ids to be int32, as MeshFile.ReadMesh re…
ryan-kipawa Oct 20, 2026
c8133e0
Make MeshBuilder's default ids int32
ryan-kipawa Oct 20, 2026
d628cc8
Expect Spectral0D files to be written from frequency and direction lists
ryan-kipawa Oct 20, 2026
ae26532
Store DfsuBuilder frequencies and directions as float64 arrays
ryan-kipawa Oct 20, 2026
be550ec
Drop the reshape test that set an item's axis to None by hand
ryan-kipawa Oct 20, 2026
7485ae7
Expect ReadStaticItem to return None for numbers past int32
ryan-kipawa Oct 20, 2026
10f5585
Return None from ReadStaticItem for numbers past int32
ryan-kipawa Oct 20, 2026
d366890
Expect a static item to keep its data when the caller changes the array
ryan-kipawa Oct 20, 2026
3273233
Copy the data into the static item GetStaticItem returns
ryan-kipawa Oct 20, 2026
b1e34e2
Expect item builders to need setting again after returning an item
ryan-kipawa Nov 10, 2026
7103cf4
Reset item builders after they return an item
ryan-kipawa Nov 10, 2026
3aba0f4
Merge remote-tracking branch 'origin/master' into pr-76
ryan-kipawa Nov 10, 2026
c7e480a
Raise the pyrefly warning ceiling to 326 for master's new warning
ryan-kipawa Nov 10, 2026
b80a4bf
Expect ReadStaticItem to raise for a number with no static item
ryan-kipawa Nov 10, 2026
e2e2d69
Raise ValueError from ReadStaticItem past the last static item
ryan-kipawa Nov 10, 2026
b3c4c87
Remove reshape from DfsFile reads; Dfs123File owns it
ryan-kipawa Nov 10, 2026
96b59de
Expect id setters to reject float ids, even whole ones
ryan-kipawa Nov 10, 2026
75ece93
Check id dtype and int32 range directly instead of round-tripping
ryan-kipawa Nov 10, 2026
f2aff3d
Rename _Int32Ids to _ToInt32Ids and pass None through it
ryan-kipawa Nov 10, 2026
f191559
Say in the CHANGELOG that id setters reject floats
ryan-kipawa Nov 10, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/type_check.yml
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ jobs:
# (the step above checks it); if it crashes, the JSON parse fails.
shell: bash
env:
MAX_WARNINGS: 330
MAX_WARNINGS: 326
run: |
warnings=$( (uv run pyrefly check --output-format=json --min-severity=warn --summary=none || true) | uv run python -c "import json, sys; print(sum(e['severity'] == 'warn' for e in json.load(sys.stdin)['errors']))")
echo "pyrefly warnings: $warnings (ceiling $MAX_WARNINGS)"
Expand Down
57 changes: 57 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,14 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
with `subprocess`, no longer inherit it.
- The architecture check at import requires an x86-64 CPU as well as a 64-bit
Python.
- **Breaking:** `DfsFile.ReadStaticItem` raises `ValueError` for a number past
the last static item instead of returning `None`, as reading a dynamic item
past the last one does. To read every static item, call `ReadStaticItemNext`
until it returns `None`.
- **Breaking:** `DfsFile.ReadItemTimeStepNext` and `ReadItemTimeStep` no longer
take `reshape`, which a file opened with `DfsFileFactory.DfsGenericOpen`
ignored. Open dfs1, dfs2 and dfs3 files with `Dfs1FileOpen`, `Dfs2FileOpen`
or `Dfs3FileOpen` to reshape their data.

### Fixed

Expand Down Expand Up @@ -107,5 +115,54 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
([#47](https://github.com/DHI/mikecore-python/issues/47)).
- Reading an item's value type raises the DFS error when the native library
reports one, instead of returning `DataValueType.Instantaneous`.
- `eumWrapper.eumGetNextEqvUnit` returns the units equivalent to the base unit.
It read the native return code the wrong way round, so it reported no unit
where there was one, and raised `ValueError` after the last one, where the
native description is null.
- `DfsuBuilder.SetElementIds` stores the ids, so the file has them instead of
the default 1, 2, .... `SetElementIds` and `SetNodeIds` accept int64 arrays
and lists and store int32 ids, raising `ValueError` for ids that do not fit in
int32. Int64 node ids raised "Data type not supported: int64" in
`CreateFile`. `SetElementIds(None)` goes back to the default ids, as
`SetNodeIds(None)` does. `SetFromMeshFile` stores a mesh's ids as int32 too,
so a mesh from `MeshFile.Create` with int64 ids can be written.
- `MeshBuilder.SetElementIds(None)` goes back to the default ids instead of
raising `TypeError` after `SetElements`, and ids that do not fit in int32
raise `ValueError`. They were written to a mesh file that `MeshFile.ReadMesh`
could not read.
- `MeshBuilder.CreateMesh()` makes its default node and element ids int32, as
`MeshFile.ReadMesh` reads them. They were int64 on Linux, so the id dtype of
a built mesh depended on whether ids were set.
- The id setters of `DfsuBuilder` and `MeshBuilder` raise `ValueError` for ids
that are not a 1-D array of integers. A 2-D array whose size matched the
number of nodes or elements, booleans as 0 and 1, and floats were accepted;
whole-number floats such as 1.0 are now rejected too.
- `MeshBuilder.SetNodes` keeps z as doubles, as mesh files store it, instead of
rounding it to float32. Float32 z passed to `SetNodes`, such as
`DfsuFile.Z`, is now written with the double text of those float32 values
(`-1.7859922647476196` rather than `-1.7859923`). `MeshBuilder.Create(dfsuFile)`
is unchanged.
- `miketools.Dfs0ToAscii` converts item times from the time axis unit to
seconds, so a file with a time unit other than seconds gets the right
timestamps. A 10 minute step was written as 10 seconds.
- `DfsFile.ReadStaticItem` raises `ValueError` for an item number below 1. It
returned the first static item, numbered 0 or below. A number too large for
int32, which returned another item or raised `ctypes.ArgumentError`, now
raises `ValueError` as any number past the last item does (see Changed).
- `DfsDynamicItemBuilder` and `DfsStaticItemBuilder` reset after
`GetDynamicItemInfo` and `GetStaticItem`. A reused dynamic builder used to
return a blank item that passed validation and failed only when added to a
file. Each builder now starts empty for the next item, and validation reports
any setting that is not set again. Changing a builder no longer changes the
item it returned.
- `DfsuBuilder` creates `DfsuSpectral0D` files. It raised an `Exception` with
no message. The caller supplies the frequency-direction mesh, one element per
frequency and direction, and `Validate` reports a Spectral0D builder with no
frequencies or directions, or an element count that does not match them, so
the file reads back as `DfsuSpectral0D`.
- `DfsuBuilder.SetFrequencies` and `SetDirections` accept lists and store the
values as doubles, as dfsu files store frequencies and directions. A list
passed `Validate` and then failed in `CreateFile` with `AttributeError`,
leaving a half-written file, and float32 arrays were written as floats.

[Unreleased]: https://github.com/DHI/mikecore-python/compare/v0.3.a1...HEAD
15 changes: 14 additions & 1 deletion mikecore/Dfs123File.py
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,20 @@ def Reshape(self, reshape):
def ReadItemTimeStepNext(
self, itemData: DfsItemData | None = None, reshape: bool = False
) -> DfsItemData | None:
res = super().ReadItemTimeStepNext(itemData)
return self.__Reshaped(super().ReadItemTimeStepNext(itemData), reshape)

def ReadItemTimeStep(
self,
itemNumber: int | DfsItemData,
timestepIndex: int,
reshape: bool = False,
) -> DfsItemData | None:
return self.__Reshaped(
super().ReadItemTimeStep(itemNumber, timestepIndex), reshape
)

def __Reshaped(self, res: DfsItemData | None, reshape: bool) -> DfsItemData | None:
# Reshaping an array that already has the axis shape leaves it unchanged
if res is not None and (self.reshape or reshape):
res.Data = res.Data.reshape(self.SpatialAxis.Shape, order="F")
return res
Expand Down
26 changes: 17 additions & 9 deletions mikecore/DfsBuilder.py
Original file line number Diff line number Diff line change
Expand Up @@ -610,6 +610,10 @@ class DfsAbstractItemBuilder(Generic[ItemInfoT]):
ItemInfo: ItemInfoT

def __init__(self):
self._Reset()

def _Reset(self):
"""Start a new, empty item. Each subclass also sets a new ItemInfo."""
self.isSetNameQuantityDataType = False
self.isSetSpatialAxis = False

Expand Down Expand Up @@ -663,8 +667,8 @@ class DfsDynamicItemBuilder(DfsAbstractItemBuilder[DfsDynamicItemInfo]):
written to file, i.e., length of strings can be changed.
"""

def __init__(self):
super().__init__()
def _Reset(self):
super()._Reset()
self.isSetDataValueType = False
self.ItemInfo = DfsDynamicItemInfo()

Expand All @@ -690,10 +694,10 @@ def GetDynamicItemInfo(self):
msgs = DfsBuilder.ErrorMessage(errors)
raise Exception(msgs)

# Create and store a clone, such that item-builder can be reused when creating more than one item
# Otherwise updates to the new item-builder would just update the same item.
# Hand over the item and start empty, so reusing the builder
# cannot change the item returned here.
res = self.ItemInfo
self.ItemInfo = DfsDynamicItemInfo()
self._Reset()
return res


Expand All @@ -712,14 +716,14 @@ class DfsStaticItemBuilder(DfsAbstractItemBuilder[DfsStaticItem]):
can be added to the <see cref="DfsBuilder.
"""

def __init__(self):
super().__init__()
def _Reset(self):
super()._Reset()
self.isSetData = False
self.ItemInfo = DfsStaticItem()
self.SetDefaults()

def SetData(self, data):

"""Keeps a reference to data; changes to the array before CreateFile change the item."""
self.ItemInfo.Data = data
self.isSetData = True

Expand Down Expand Up @@ -747,4 +751,8 @@ def GetStaticItem(self):
msgs = DfsBuilder.ErrorMessage(errors)
raise Exception(msgs)

return self.ItemInfo
# Hand over the item and start empty, so reusing the builder
# cannot change the item returned here.
res = self.ItemInfo
self._Reset()
return res
33 changes: 25 additions & 8 deletions mikecore/DfsFile.py
Original file line number Diff line number Diff line change
Expand Up @@ -943,24 +943,44 @@ def ReadStaticItem(self, staticItemNo):
"""
Read the number <paramref name="staticItemNo"/> static item from the file.

To read every static item, use `ReadStaticItemNext`, which returns None
after the last one.

:param staticItemNo: Number of static item in the file to read. First static item has number 1
:returns: The static item number <paramref name="staticItemNo"/>, null if there are not that many static items.
:returns: The static item number <paramref name="staticItemNo"/>.
:raises ValueError: If <paramref name="staticItemNo"/> is below 1 or past the last static item.
"""
self.__CheckIfOpen()
if self.fpState == DfsFilePointerState.CreatingItems:
raise Exception("Can not read static items when file is being created.")
if staticItemNo < 1:
raise ValueError(
f"Static item number must be 1 or more, got {staticItemNo}"
)
pastTheEnd = ValueError(
f"Static item number {staticItemNo} is past the last static item"
)
# dfsFindItemStatic takes an int32, and a larger number is past the last item
if staticItemNo > np.iinfo(np.int32).max:
raise pastTheEnd

if (
self.fpState != DfsFilePointerState.StaticItem
or self.fpItemNumber != staticItemNo
):
DfsDLL.Wrapper.dfsFindItemStatic(
rc = DfsDLL.Wrapper.dfsFindItemStatic(
self.headPointer, self.filePointer, staticItemNo
)
self.fpState = DfsFilePointerState.StaticItem
self.fpItemNumber = staticItemNo
self.fpTimeStepIndex = -1
return self.ReadStaticItemNext()
if rc != 0:
raise pastTheEnd
# dfsFindItemStatic also succeeds one past the last item
staticItem = self.ReadStaticItemNext()
if staticItem is None:
raise pastTheEnd
return staticItem

def WriteStaticItemData(self, staticItem, data: np.ndarray):
"""
Expand Down Expand Up @@ -1006,7 +1026,7 @@ def WriteStaticItemData(self, staticItem, data: np.ndarray):
self.fpItemNumber += 1

def ReadItemTimeStepNext(
self, itemData: DfsItemData | None = None, reshape: bool = False
self, itemData: DfsItemData | None = None
) -> DfsItemData | None:
"""
Reads the next dynamic item-timestep. First time called it returns the first
Expand All @@ -1026,7 +1046,6 @@ def ReadItemTimeStepNext(
since it iterates exactly as the data is stored on the disk.

:param itemData DfsItemData: DfsItemData for item to store timestep values in, for reuse of memory.
:param reshape bool: Reshape data array to dimension of data, 2D or 3D depending on spatial axis.
:returns DfsIemData: The next dynamic item-timestep, None if no more items are present.
"""
self.__CheckIfOpen()
Expand Down Expand Up @@ -1071,7 +1090,6 @@ def ReadItemTimeStep(
self,
itemNumber: int | DfsItemData,
timestepIndex: int,
reshape: bool = False,
) -> DfsItemData | None:
"""
Reads the dynamic item-timestep as specified from the file. It throws an
Expand All @@ -1080,7 +1098,6 @@ def ReadItemTimeStep(

:param itemNumber Union[int,DfsItemData]: Item number (1-based), or DfsItemData for item
:param timestepIndex int: Time step index (0-based)
:param reshape bool: Reshape data array to dimension of data, 2D or 3D depending on spatial axis.
:return DfsItemData: The dynamic item-timestep as specified
"""
self.__CheckIfOpen()
Expand Down Expand Up @@ -1115,7 +1132,7 @@ def ReadItemTimeStep(

# Position file pointer
self.__FpFindItemTimeStep(itemNumber, timestepIndex)
return self.ReadItemTimeStepNext(itemData, reshape)
return self.ReadItemTimeStepNext(itemData)

def __GetTime(self, time, timestepIndex):
# TODO: This assumes time in seconds?
Expand Down
Loading
Loading