Skip to content

Upgrade FFTPACK 4.1 -> 5.1, add 2D FFT - #3412

Open
andrew-platt wants to merge 12 commits into
OpenFAST:devfrom
andrew-platt:f/fftpack5.1
Open

Upgrade FFTPACK 4.1 -> 5.1, add 2D FFT#3412
andrew-platt wants to merge 12 commits into
OpenFAST:devfrom
andrew-platt:f/fftpack5.1

Conversation

@andrew-platt

@andrew-platt andrew-platt commented Jul 30, 2026

Copy link
Copy Markdown
Collaborator

Baselines may need updating before merging.

Feature or improvement description*

Upgrade FFTPACK from version 4.1 to 5.1 in nwtc-library. This brings the FFT library up to the latest FFTPACK release and adds new 2D FFT capabilities.

Key changes:

  • Replace fftpack4.1.f with fftpack5.1.f (aggregated single-file source)
  • Update NWTC_FFTPACK.f90 wrapper to call FFTPACK 5.1 API (RFFT1I/RFFT1B/RFFT1F, CFFT1I/CFFT1B/CFFT1F, COST1I/COST1F, SINT1I/SINT1B) with normalization compensation to maintain full backward compatibility with existing callers
  • Add new 2D FFT wrapper routines exposing FFTPACK 5.1's RFFT2 and CFFT2 families:
    • FFT2D_DataType — handle type for 2D transforms
    • InitFFT2D / ApplyFFT2D / ApplyFFT2D_f / ExitFFT2D — real 2D forward/backward FFT
    • InitCFFT2D / ApplyCFFT2D / ApplyCFFT2D_f / ExitCFFT2D — complex 2D forward/backward FFT
    • Note: The 2D routines are internally normalized (forward * backward = identity), unlike the 1D routines which follow the legacy convention (forward * backward = x·N)
  • Update CMakeLists.txt with per-file compile flags for GNU (-fallow-argument-mismatch -fno-default-real-8 -fno-default-double-8) to keep FFTPACK internal REAL at 4 bytes matching SiKi
  • Update VS build (NWTC-Library.vfproj) with RealKIND=realKIND4 for fftpack5.1.f across all configurations
  • Add unit tests in modules/nwtc-library/tests/test_NWTC_FFTPACK.F90 covering all public routines (9 tests total)

Related issue, if one exists

None

Impacted areas of the software

  • modules/nwtc-library — FFTPACK source and wrapper module
  • All modules consuming NWTC_FFTPACK (unchanged, backward compatible):
    • AeroDyn (AeroAcoustics)
    • HydroDyn (Conv_Radiation, WAMIT, WAMIT2)
    • MoorDyn
    • SeaState (Waves, Waves2, UserWaves)
    • TurbSim

Additional supporting information

FFTPACK 5.1 introduced internal normalization in RFFT1B/RFFT1F and different scaling in COST/SINT transforms that 4.1 did not have. The wrapper compensates for these differences with pre/post-scaling so that all 9 existing consumer modules produce identical results without modification.

The 2D FFT routines (RFFT2B/RFFT2F, CFFT2B/CFFT2F) are newly exposed — they were not available in FFTPACK 4.1. Unit tests for these are in modules/nwtc-library/tests/test_NWTC_FFTPACK.F90 (tests FFT2D_roundtrip and CFFT2D_roundtrip).

Generative AI usage

Co-authored-by: Anthropic Claude claude@anthropic.com

Test results, if applicable

All unit tests pass (9/9 in nwtc_library_utest). Full regression test suite passes with one borderline case:

  • IEA22MW_ModalDamping: 2 of 60 channels (TwrBsFyt, YawBrFyp) at 1.52% max relative difference vs 1% threshold. These are lateral force channels in a floating offshore turbine; the difference is within engineering precision and likely due to minor numerical differences in trig factor computation propagating through the wave excitation chain. Baseline regeneration planned.

  • r-test branch merging required


FFTPACK 5.1 Upgrade - Regression Test Detailed Results

These notes were generated during the ai driven upgrade process.

Summary

Upgrade from FFTPACK 4.1 to 5.1 passes all Fortran-based regression tests
within the standard tolerance (rtol=2, atol=1.9, corresponding to 1% relative /
1.26% absolute threshold), with one borderline exception documented below.

Test Environment

  • Build: build-docker-double (gfortran, double precision)
  • Platform: aarch64 Linux (Docker)
  • Total tests: 205
  • Passing: 165
  • Failing: 40 (35 pre-existing environment issues + 5 parallel-execution OOM)

Pre-existing Failures (not caused by this change)

Count Category Root Cause
19 *_Linear tests Missing pandas Python package
16 *_py / py_* tests Missing pyOpenFAST Python package

These failures reproduce identically on the dev branch without FFTPACK changes.

OOM Kills During Parallel Execution

When running with -j$(nproc) (4 cores), 3-4 large OpenFAST simulations are
occasionally killed by the OOM killer (exit code 137). These pass individually:

  • 5MW_OC4Jckt_DLL_WTurb_WavesIrr_MGrowth - Killed (OOM)
  • 5MW_MRSemi_DLL_WSt_WavesIrr - Killed (OOM)
  • md_waterkin3 - Killed (OOM)
  • hd_ExctnMod1_ExctnDisp2_PtfmYMod1 - Killed (OOM during comparison step)

All pass when run individually with ctest -R "^<testname>$".

Borderline Test Case: IEA22MW_ModalDamping

This test has 2 of 60 channels that exceed the 1% tolerance threshold:

Channel Max Relative Diff Notes
TwrBsFyt 1.52% Tower base side-side force
YawBrFyp 1.52% Yaw bearing side-side force

Assessment: These are lateral (side-side) force channels in a floating
offshore turbine with wave loading. The 1.52% difference is:

  • Just barely above the 1% tolerance (passes at rtol=1.8)
  • Consistent across both channels (same physical DOF at different locations)
  • Likely caused by slightly different numerical characteristics of FFTPACK 5.1
    trig factor computation vs 4.1, propagated through the HydroDyn wave
    excitation → structural response chain
  • All other 58 channels (including the dominant fore-aft loads) pass

Recommendation: This is acceptable for a library upgrade. The difference is
within engineering precision and does not indicate algorithmic error. The
baseline could be regenerated to close this gap.

Executed change: since these two channels are ~7 orders of magnitude smaller than the comparable Fx channels, they were removed from the regression test (they are close to zero and essentially noise). Since these channels differed in results with gcc 15.2 locally but passed on GitHub actions, it is preferable to simply remove them to avoid confusion in local testing.

Key Tests That Now Pass

All major wave/hydro test cases pass within tolerance:

  • 5MW_OC3Trpd_DLL_WSt_WavesReg (regular waves)
  • 5MW_OC3Spar_DLL_WTurb_WavesIrr (irregular waves, spar)
  • 5MW_OC4Semi_WSt_WavesWN (white noise waves, semi-sub)
  • 5MW_TLP_DLL_WTurb_WavesIrr_WavesMulti (multi-directional waves, TLP)
  • MHK_RM1_Floating (marine hydrokinetic)
  • All hd_* HydroDyn driver tests (13 tests)
  • All seastate_* SeaState tests (5 tests)
  • StC_test_OC4Semi (structural control with waves)

Normalization Differences Between FFTPACK 4.1 and 5.1

The key technical challenge was that FFTPACK 5.1 introduced internal
normalization that 4.1 did not have:

Transform FFTPACK 4.1 FFTPACK 5.1 Compensation Applied
RFFTB (backward) Un-normalized RFFT1B pre-scales by ±0.5 Pre-multiply interior by ±2
RFFTF (forward) Un-normalized RFFT1F post-scales by 1/N, 2/N Post-multiply to restore N, N/2
COST (cosine) Un-normalized, self-inverse COST1F normalized forward Use COST1F + non-uniform post-scale
SINT (sine) Un-normalized SINT1B = old/2 Post-multiply by 2
CFFT (complex) Un-normalized CFFT1B/1F un-normalized None needed

Concatenate all 111 individual FFTPACK 5.1 source files into a single
fftpack5.1.f (7740 lines), matching the aggregated format of fftpack4.1.f.

Source: FFTPACK 5.1 by Paul N. Swarztrauber and Richard A. Valent (NCAR)
- Replace FFTPACK 4.1 API calls with FFTPACK 5.1 equivalents
- Add normalization compensation to maintain backward compatibility:
  * ApplyFFT/ApplyFFT_cx: pre-scale x2/-x2 before RFFT1B
  * ApplyFFT_f: post-scale after RFFT1F to undo SN/TSN/TSNM
  * ApplyCOST: use COST1F with non-uniform post-scaling
  * ApplySINT: post-scale by 2 after SINT1B
  * ApplyCFFT/ApplyCFFT_f: no change needed
- Replace heap-allocated wWork with stack-local automatic arrays
- Use explicit SiKi kind in LOG expressions for wsave sizing
- Replace fftpack4.1.f with fftpack5.1.f in source list
- Add per-file compile flags for fftpack5.1.f (GNU only):
  -fallow-argument-mismatch: legacy COMPLEX<->REAL type punning
  -fno-default-real-8 -fno-default-double-8: keep FFTPACK internal
  REAL as 4 bytes to match SiKi arrays passed by callers
All Fortran regression tests pass within tolerance except one
borderline case (IEA22MW_ModalDamping: 2 of 60 channels at 1.52%
vs 1% threshold). See FFTPACK_UPGRADE_TESTS.md for full analysis.
Tests verify all public NWTC_FFTPACK routines:
- FFT forward/backward roundtrip (x*N scaling)
- FFT forward of constant signal (DC coefficient)
- ApplyFFT_cx produces same result as ApplyFFT
- Complex FFT forward spectral bin placement
- Cosine transform vs analytical DCT-I formula
- Sine transform vs analytical DST-I formula
- FFT normalization flag behavior
Set RealKIND=realKIND4 on fftpack5.1.f for all 8 configurations,
matching the CMake -fno-default-real-8 flag to keep FFTPACK's
internal REAL as 4 bytes (SiKi).
New public API in NWTC_FFTPACK:
- FFT2D_DataType: handle for 2D transforms
- InitFFT2D / ApplyFFT2D / ApplyFFT2D_f / ExitFFT2D: real 2D FFT
- InitCFFT2D / ApplyCFFT2D / ApplyCFFT2D_f / ExitCFFT2D: complex 2D FFT

FFTPACK 5.1's 2D routines are internally normalized (forward * backward
= identity), unlike the 1D routines which give x*N.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR upgrades the embedded NetLib FFTPACK dependency in modules/nwtc-library from 4.1 to 5.1, updates the NWTC_FFTPACK Fortran wrapper to the FFTPACK 5.1 API (while aiming to preserve legacy 1D scaling behavior), and exposes new 2D FFT wrapper routines. It also wires in new unit tests and adjusts build system settings to keep FFTPACK’s internal REAL precision consistent with existing SiKi expectations.

Changes:

  • Replace FFTPACK 4.1 source usage with FFTPACK 5.1 and update wrapper calls to 5.1 entry points (RFFT1*, CFFT1*, COST1*, SINT1*), plus add new 2D FFT wrapper APIs.
  • Add FFTPACK-focused unit tests and register them in the unit test build and runner.
  • Update CMake and Visual Fortran project settings to compile FFTPACK 5.1 with required precision/compatibility flags.

Reviewed changes

Copilot reviewed 6 out of 7 changed files in this pull request and generated 6 comments.

Show a summary per file
File Description
vs-build/modules/NWTC-Library.vfproj Switches VS build to fftpack5.1.f and forces RealKIND=4 for that file across configurations.
unit_tests/CMakeLists.txt Adds the new FFTPACK unit test source to nwtc_library_utest.
modules/nwtc-library/tests/test_NWTC_FFTPACK.F90 Introduces a new test module covering 1D/2D FFT and cosine/sine transforms.
modules/nwtc-library/tests/nwtc_library_utest.F90 Registers the new FFTPACK test suite in the unit test runner.
modules/nwtc-library/src/NetLib/fftpack/NWTC_FFTPACK.f90 Updates FFT wrapper to FFTPACK 5.1 APIs, compensates scaling for legacy 1D behavior, and adds 2D FFT wrappers.
modules/nwtc-library/CMakeLists.txt Switches to fftpack5.1.f and adds GNU per-file compile flags to avoid real/double promotion and allow argument mismatch.
Suppressed comments (1)

modules/nwtc-library/src/NetLib/fftpack/NWTC_FFTPACK.f90:1212

  • ExitCFFT2D has the same handle-reset issue as ExitFFT2D: it deallocates wSave but leaves TransformType/L/M/LenWork unchanged, so a stale handle can pass ApplyCFFT2D checks and crash when FFTPACK is called with a deallocated wSave.
   SUBROUTINE ExitCFFT2D(FFT_Data, ErrStat)

      TYPE(FFT2D_DataType), INTENT(INOUT) :: FFT_Data
      INTEGER, INTENT(OUT), OPTIONAL :: ErrStat

      IF ( PRESENT(ErrStat) ) ErrStat = ErrID_None

      IF ( ALLOCATED(FFT_Data%wSave) ) DEALLOCATE( FFT_Data%wSave )


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread modules/nwtc-library/src/NetLib/fftpack/NWTC_FFTPACK.f90
Comment thread modules/nwtc-library/src/NetLib/fftpack/NWTC_FFTPACK.f90
Comment thread modules/nwtc-library/src/NetLib/fftpack/NWTC_FFTPACK.f90
Comment thread modules/nwtc-library/tests/test_NWTC_FFTPACK.F90
Comment thread modules/nwtc-library/tests/test_NWTC_FFTPACK.F90
Comment thread modules/nwtc-library/src/NetLib/fftpack/NWTC_FFTPACK.f90
Check IER return code from all FFTPACK 5.1 Init and Apply calls,
reporting fatal error via ProgAbort if nonzero. Also add missing
ErrStat checks between sequential FFT calls in unit tests.

Co-authored-by: Anthropic Claude <claude@anthropic.com>
Co-authored-by: Microsoft Copilot <copilot@microsoft.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 6 out of 7 changed files in this pull request and generated no new comments.

Suppressed comments (1)

modules/nwtc-library/src/NetLib/fftpack/NWTC_FFTPACK.f90:727

  • InitCOST returns the raw ALLOCATE status code (Sttus) via ErrStat on allocation failure. Everywhere else in this module ErrStat is treated as an NWTC error code (e.g., ErrID_Fatal), so returning an allocation status breaks the implied contract and makes error handling inconsistent for callers.
      IF ( Sttus /= 0 )  THEN
         CALL ProgAbort ( 'Error allocating memory for the cosine transform working array.', PRESENT(ErrStat) )
         ErrStat = Sttus
         RETURN

@andrew-platt

andrew-platt commented Aug 6, 2026

Copy link
Copy Markdown
Collaborator Author

For the IEA22MW_ModalDamping test case, the two channels that sometimes fail are

Screenshot 2026-08-06 at 3 56 55 PM Figure: `TwrBsFyt` is nearly zero with a very slight change near the zero crossing. This is negligible Screenshot 2026-08-06 at 3 58 07 PM Figure: `YawBrFyp` is similarly close to zero. This is a negligible.

Since these two channels may cause erroneous failures depending on the computation platform, they channels will be removed from the test (they are ~7 orders of magnitude smaller than the comparable Fx terms).

andrew-platt and others added 3 commits August 6, 2026 16:22
Was returning raw ALLOCATE status code instead of ErrID_Fatal,
inconsistent with all other Init routines in this module.

Co-authored-by: Anthropic Claude <claude@anthropic.com>
Co-authored-by: Microsoft Copilot <copilot@microsoft.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants