POT3D is a Fortran code that computes potential field solutions to approximate the solar coronal magnetic field using observed photospheric magnetic fields as a boundary condition. It can be used to generate potential field source surface (PFSS), potential field current sheet (PFCS), and open field (OF) models. It has been (and continues to be) used for numerous studies of coronal structure and dynamics. The code is highly parallelized using MPI and is GPU-accelerated using Fortran standard parallelism (do concurrent) and OpenMP Target for data movement and device selection, along with an option to use the NVIDIA cuSparse library. The HDF5 file format is used for input/output.
POT3D is the potential field solver for the WSA/DCHB model in the CORHEL software suite publicly
hosted at the Community Coordinated Modeling Center (CCMC).
A version of POT3D that includes GPU-acceleration with both MPI+OpenACC and MPI+OpenMP
was released as part of the Standard Performance Evaluation Corporation's (SPEC) SPEChpc(TM) 2021 benchmark suite.
POT3D is also included in the SPEC CPU(TM) 2026 benchmark suite.
Details of the code can be found in these publications:
- Variations in Finite Difference Potential Fields.
Caplan, R.M., Downs, C., Linker, J.A., and Mikic, Z. Ap.J. 915,1 44 (2021) - From MPI to MPI+OpenACC: Conversion of a legacy FORTRAN PCG solver for the spherical Laplace equation.
Caplan, R.M., Mikic, Z., and Linker, J.L. arXiv:1709.01126 (2017)
The included build.sh script will use a configuration file to generate a Makefile and build the code.
The folder conf contains example configuration files for various compilers and systems.
We recommend copying the configuration file closest to your setup and then
modifying it to conform to your compiler and system.
Given a configure script conf/my_custom_build.conf, the build script is invoked as:
> ./build.sh ./conf/my_custom_build.conf
After building the code, it can be tested by running the testsuite.
Enter the testsuite directory and run:
./run_test_suite.sh -np=<N>
where "" is the number of MPI ranks to use.
To see available options, run run_test_suite.sh -h
POT3D uses a namelist in an input text file called pot3d.dat to set all parameters of a run. See the provided pot3d_input_documentation.txt file for details on the various parameter options. For any run, an input 2D data set in HDF5 format is required for the lower radial magnetic field (Br) boundary condition. Examples of this file are contained in the benchmarks and testsuite folders.
To run POT3D, set the desired run parameters in a pot3d.dat text file, then copy or link the pot3d executable into the same directory as pot3d.dat and run the command:
<MPI_LAUNCHER> -np <N> ./pot3d
where <N> is the total number of MPI ranks to use (typically equal to the number of CPU cores or number of GPUs) and <MPI_LAUNCHER> is your MPI run command (e.g. mpiexec,mpirun, ibrun, srun, etc).
For example: mpiexec -np 1024 ./pot3d
POT3D uses a preconditoned Conjugate Gradient solver with two preconditioner options:
ifprec=1: Diagonal scaling.ifprec=2: Non-overlapping ILU0
Typically, usingifprec=2will run POT3D faster thanifprec=1, but use more memory. For NVIDIA GPUs,ifprec=2requires building the code with thecuSparselibrary.
For Intel and AMD GPUs, onlyifprec=1is currently available.
POT3D will auto-detect how it is being built, and may overrideifprecto the option best suited for the current build.
For standard cases, one should launch the code such that the number of MPI ranks per node is equal to the number of GPUs per node
e.g.
mpiexec -np <N> --ntasks-per-node 4 ./pot3d
For Intel GPUs, one must set the following ENV variable before running:
export I_MPI_OFFLOAD=1
To estimate how much memory (RAM) is needed for a run (using ifprec=1), compute:
memory-needed = nr*nt*np*8*13.6/1000/1000/1000 GB
where nr, nt, and np are the chosen problem sizes in the r, theta, and phi dimension.
If using ifprec=2, the required memory can be two to four times higher.
Depending on the input parameters, POT3D can have various outputs. Typically, the three components of the potential magnetic field is output as HDF5 files. In every run, the following two text files are output:
pot3d.outAn output log showing grid information and magnetic energy diagnostics.timing.outTime profile information of the run.
Some useful python scripts for reading and plotting the POT3D input data, and reading the output data can be found in the bin folder.
In the benchmarks folder, we provide two run cases that can be used to benchmark the performance of POT3D.
The following is a list of the included benchmark runs, their problem size, and their memory requirements:
bench_tiny(Equivalent to SPEChpc(TM)2021's "tiny" run)
Grid size: 173x361x1171 = 73.2 million cells
Memory (RAM) needed (usingifprec=1): ~7.96 GBisc2023(Equivalent to SPEChpc(TM)2021's "small" run) Grid size: 325x450x2050 = 299.8 million cells
Memory (RAM) needed (usingifprec=1): ~32.62 GB
In the testsuite folder, we provide the following example runs of POT3D:
/potential_field_source_surface
A standard PFSS run with a source surface radii of 2.5 Rs./potential_field_current_sheet
A standard PFCS run using the outer boundary of the PFSS example as its inner boundary condition, with a domain that extends to 30 Rs. The magnetic field solution produced is unsigned./open_field
An example of computing the "open field" model from the solar surface out to 20 Rs using the same input surface Br as the PFSS example. The magnetic field solution produced is unsigned.
Each test case contains an input folder with the run input files, a run folder used to run the test, and a reference folder containing the output diagnostics used to validate the test.
The validation is done with the magnetic energy diagnostics in the pot3d.out file.
All tests run with ifprec=1 by default - use the -pc=2 option to run with ifprec=2.
To run the testsuite, use the included script run_test_suite.sh
To see available options, run run_test_suite.sh -h