Skip to content

Developer Guide

Jessica Meixner edited this page Jul 17, 2023 · 65 revisions

This page details instructions for developers of WAVEWATCH III® (henceforth WW3) to add features to the existing framework, including new science and technological advancements, improve performance, fix bugs etc. WW3 is a community model and we follow the GitFlow model to facilitate collaboration and contributions to WW3.

This guide will take you through the main steps and practical actions for contributing to WW3. If you are interested in further details, the WW3 Gitflow page provides more information on GitFlow. For information to clone, build and run WW3, see the Quick Start guide.

WW3 GitFlow Rationale

GitFlow is a branching style where there are two main branches, main and develop. The main branch contains stable, tested code that is ready for production type use, while develop is a faster moving branch where recent developments are reintegrated too. There are two other types of branches, feature and bugfix. These two types of branches are created from the develop branch for adding features or bug fixes respectively before being reintegrated to the develop branch.

WW3 has one authoritative repository part of the NOAA-EMC organization. This repository is maintained by NCEP/EMC code managers and will be used to produce public releases and is the central reference to all copies of the code. In addition to the authoritative repository, there are trusted institutional repositories/forks. A developer may choose to create a fork from the authoritative repo or from one of the trusted institutional repos.

All development will be made by forks out of one of these trusted institutional repositories. The develop branch is the primary source of code that is currently being updated. All new feature branches are created from there, and completed development threads target updates to that branch via pull request.

Updates to the main branch will be done by the authoritative fork. Updates to the develop branch should be done via a pull request to the authoritative or trusted institutional forks.

To contribute to WW3, developers will fork the authoritative repo or one of the trusted institutional repos using their own GitHub accounts. In the developer's fork,
new developments should be made in a feature or bug fix branch created from the develop branch. Developers should make an effort to communicate via GitHub issues with the code manager from the repository associated with the developer's fork to indicate what developments are being made.

Once a feature branch has been tested and is mature to become a candidate for reintegration to the main repo, the developer sends a pull request to the authoritative or institutional forks to start the process of reintegration.

Development Workflow

The "fork and branch" workflow we are adopting is explained in more detail in this Scott's Weblog post. The main steps are:

  1. Create an issue associated to the new development.
  2. Fork the WW3 GitHub repository.
  3. Clone the forked repository to your local system.
  4. Add a Git remote for the original repository.
  5. Create a feature branch, make changes, commit and push changes to your GitHub fork.
  6. Communicate via GitHub issues with the source repository code manager to indicate what is being developed.
  7. Open a pull request from the new branch to the original repo.
  8. Clean up your fork after your pull request is merged.

1.Create an issue associated to the new development.

Create a new issue on the GitHub page, choosing between the options for features or bugs. In the issue, complete the template as much as possible detailing the work that is to be done. If you are going to be doing the work, either assign yourself to the issue or indicate in the issue that you will follow up with a Pull Request.

2. Fork the WW3 GitHub Repo

  • Create and account on github.com,
  • On github.com choose one of the WW3 trusted repositories
  • Locate and click the fork icon on the upper right-hand corner of the WW3 GitHub page.

This creates a full copy of the WW3 repository under your user account. More information can be found on GitHub documentation here.

3. Clone the Forked Repo Locally

You can now navigate to your GitHub dashboard and choose the newly-created fork. To start your development work, you need a copy of your WW3 fork on your local machine. To do that, you must git clone your fork locally. First, find the right address for the fork on your GitHub fork page clicking the "Clone or Download" dropdown menu. Then, on your local box:

git clone https://github.com/<username>/WW3.git

If you are using ssh:

git clone ssh://[email protected]/<username>/WW3.git

4. Add Remotes and update you fork/branch with authoritative/trusted repos

Remotes are GitHub repositories that are linked to your local copy. A first remote origin is automatically created when you clone a repo. In this case, that links your local copy to the fork you created in your GitHub account. You can keep your local copy synced with your fork and vice-versa by simply using commands such as git pull to sync the local copy with the fork, or git push the way around. We will also want to keep the local copy and your fork synced with changes made in the repository your forked from.

That is possible by creating a remote link within your local copy that points to the upstream repo:

git remote add upstream https://github.com/NOAA-EMC/WW3

or

git remote add upstream https://github.com/<TrustedInstitution>/WW3

Sync with the main repo

Syncing your local copy to the upstream remote is done within your local copy by using the commands:

  • Change to the active development branch
    git checkout develop

  • Copy over changes to upstream repo
    git fetch upstream

  • Merge changes to your local repo
    git merge upstream/develop

  • Push changes to your GitHub fork
    git push

Likewise, when the main changes, you will want to sync your fork accordingly.

Using git push to upstream repos will fail, and should not be attempted.

5. Creating a feature or bugfix Branch for Development Work

Adding new or modifying existing features of WW3 should be done in a feature branch. Fixing a bug can be done in a bugfix branch.
This involves the development cycle steps below.

Before you get started, please note some rules of engagement here:

  • Create an issue with the planned feature update or reporting the bug fix being worked on.
  • Create one feature or bugfix branch for every single feature or single bugfix you will be working on. Do not bundle all features or bugfixes in a single branch,
    • This creates a situation where undeveloped features will get in the way of reintegrating mature work,
    • Pull requests with more than one feature per branch will be asked to split their development into multiple pull requests.

The development cycle steps:

  • Create a new feature branch from the develop branch git checkout -b <branch name> develop
  • Make changes to the files,
  • Test your changes using the regtests
    • To ensure regression tests work, you need to download large files that are not part of the WW3 GitHub repo,
    • In your local copy from the top level directory, run the script:
      sh model/bin/ww3_from_ftp.sh
    • Make note of the added files and ensure they remain untracked (eg, do not use git add for these files),
  • Frequently commit your changes to the branch.
    • Check changes made
      git status
    • Add files indicated by status
      git add <some files>
    • Commit changes locally,
      • Use a commit message in the form "branchname>: [description of changes]":
        git commit -m "<branch name>: [description of changes and files affected]"
  • Push changes to your GitHub fork
    git push origin <branch name>

7. Reintegrate to Main Repo: Open a Pull Request

Pull requests are proposed changes to a repository submitted by a user and accepted or rejected by the main repo code managers. This should happen when you made changes in one of your feature branches that are mature and could become part of the WW3 package. Before you submit a pull request, make sure to verify if your changes satisfy our checklist for a developer submitting pull requests. When you get to that point, follow the guidelines well-established in GitHub:

  • Using your browser, navigate to the original repository you created your fork from.

    • To the right of the Branch menu, click New pull request.
  • On the Compare page, click the highlighted link compare across forks.

  • Confirm that the base fork is the repository you'd like to merge changes into.

    • Use the base branch drop-down menu to select the branch of the upstream repository you'd like to merge changes into.
  • Use the head fork drop-down menu to select your fork,

    • then use the compare branch drop-down menu to select the branch you made your changes in.
  • Type a title and description for your pull request. Follow the PR template instructions to provide the necessary details.

  • If you do not want to allow anyone with push access to the upstream repository to make changes to your PR, unselect Allow edits from maintainers.

  • To create a pull request that is ready for review, click Create Pull Request.

    • To create a draft pull request, use the drop-down and select Create Draft Pull Request, then click Draft Pull Request.

For more information about draft pull requests, see About pull requests.

8. Clean up after your pull request is merged

After your changes have been merged into the develop branch, it can be helpful to delete the branch with the feature and do any other clean-up in your fork. Now, you can restart the development cycle with new branches created from an up-to-date develop branch of the authoritative repository.

Regression Testing in WAVEWATCH III®

A series of regression tests are prepared to check the WW3 with different options and physics. Developers are encouraged to check their implementations with the regression tests to ensure the code compilation, consistency of model output and data reproducibility. Regression tests are enforced when new main or develop branches are created, and this typically involves running a complete set including all available regression tests with different configurations (see “The Matrix” section below). If you add new features, it is strongly encouraged that you add a regtest for the new feature, as outlined in the checklist for adding or modifying a regtest.

The following regression tests are available under retests directory:

Test Label Description
ww3_tp1.1 1D propagation around the world along the equator (no land).
ww3_tp1.2 1D propagation, along meridian (no land).
ww3_tp1.3 1D propagation, shoaling test.
ww3_tp1.4 1D propagation, spectral refraction (x).
ww3_tp1.5 1D propagation, spectral refraction (y).
ww3_tp1.6 1D propagation, wave blocking by current.
ww3_tp1.7 1D propagation, IG wave generation.
ww3_tp1.8 1D propagation, wave breaking on a beach.
ww3_tp1.9 1D propagation, Beji and Battjes (1993) barred flume case.
ww3_tp1.10 1D propagation, Bottom scattering beach.
ww3_tp1.11 1D propagation, Test for spectral interpolation in ww3_bounc.
ww3_tp2.1 2D propagation under angle with grid.
ww3_tp2.2 2D propagation over half the globe without land (with directional spread).
ww3_tp2.3 2D propagation, GSE test.
ww3_tp2.4 2D propagation, East Pacific curvilinear grid test.
ww3_tp2.5 2D propagation, Arctic Grid, curvilinear grid test.
ww3_tp2.6 2D propagation, Limon Harbor unstructured grid test.
ww3_tp2.7 Reflection on a 2D unstructured grid.
ww3_tp2.8 Tidal constituents on a 2D regular grid.
ww3_tp2.9 Tests for obstruction grids.
ww3_tp2.10 Tests for SMC grid.
ww3_tp2.11 Tests for rotated grid.
ww3_tp2.12 Test for system tracking.
ww3_tp2.13 Test for propagation under angle with grid (tripole).
ww3_tp2.14 Test for toy-model using OASIS coupler.
ww3_tp2.15 Test for space-time extremes parameters.
ww3_tp2.16 Test for two-dimensional SMC propagation with Arctic pole handling.
ww3_tp2.17 Unstructured grid with Card Deck vs Domain Decomposition for Explicit vs Implicit schemes; grid integration (ww3_gint); grib2 outputs (ww3_grib).
ww3_tp2.18 Test for tidal constituents in ww3_prnc/ww3_prtide.
ww3_tp2.19 Unstructured grid with Domain Decomposition/Implicit schemes with Neumann Boundary Condition for Depth Breaking and Triad source terms (Boers 1996).
ww3_tp2.20 Vegetation source term (Anderson and Smith 2012).
ww3_tp2.21 Global Unstrucured Grid (periodicity and obstruction).
ww3_ts1 Source term test, time limited growth.
ww3_ts2 Source term test, fetch limited growth.
ww3_ts3 Source term test, hurricane with single moving grid.
ww3_ts4 Source term test, unresolved obstacles.
ww3_tic1.1 Wave-ice interaction, 1D test of S_{ice}.
ww3_tic1.2 Wave-ice interaction, 1D test of “shoaling” effect.
ww3_tic1.3 Wave-ice interaction, 1D test of refraction effect.
ww3_tic1.4 Wave-ice interaction, 1D test with ice floes and ice thickness.
ww3_tic2.1 Wave-ice interaction, 2D test of S_ice.
ww3_tic2.2 Wave-ice interaction, 2D test with non-uniform ice.
ww3_tic2.3 Wave-ice interaction, 2D test with uniform ice with increasing thickness.
ww3_tbt1.1 Wave-mud interaction, 1D test of S_mud.
ww3_tbt2.1 Wave-mud interaction, 2D test of S_mud.
ww3_tpt1.1 Tests for alternative spectral partitioning methods.
ww3_ta1 ww3_uprstrt, update the restart file of homogeneous conditions (1 point model).
mww3_test_01 Test for expanded grid mask with wetting and drying, etc.
mww3_test_02 Two-way nesting test with single inner grid.
mww3_test_03 Overlapping grids and two-way nesting tests (6-grid version with beach in high-resolution
mww3_test_04 Current or sea-mount test for two-way nesting with stationary swell conditions.
mww3_test_05 Three nested hurricane grids with moving grids test.
mww3_test_06 Tests for irregular grid(s) w/ ww3_multi.
mww3_test_07 Tests for unstructured grid(s) w/ ww3_multi.
mww3_test_08 Tests with wind and ice input.
ww3_ufs1.1 Tests for global 1deg grid w/ ww3_multi/ww3_multi_esmf; wind/current/ice inputs; ww3_grib outputs; grid integration (ww3_gint); restart/MPI task/thread reproducibility (ufs app).
ww3_ufs1.2 Tests for GFSv16 setup w/ ww3_multi/ww3_multi_esmf; wind/current/ice inputs; ww3_grib outputs; grid integration (ww3_gint); restart reproducibility (ufs app).
ww3_ufs1.3 Tests for GEFSv12 setup w/ ww3_multi; wind/ice inputs; ww3_grib outputs; grid integration (ww3_gint); restart reproducibility (ufs app).

The above tests are contained in directories with the indicated tags within the regtest directory. A description of each test is summarized in the info file and required materials are in the input directories. A separate set of large input files should be downloaded from NOAA ftp site using the following script:

./model/bin/ww3_from_ftp.sh

Regression tests are run using the run_test script in the regtests/bin directory. How to run this script, including options, is shown by running run_test -h.

Note, in the transition from GNU to CMAKE regression testing an additional run_cmake_test exists. It is recommended to use this instead.

The Matrix: Comprehensive Regression Testing

A matrix of regression tests can be prepared in order to run a full suit of regression tests with different compiler options and HPC environment. This framework provides a way to run a larger set of regression tests, using a wider range of configurations. The practice is encouraged to ensure that changes made to the code will not unintentionally affect other parts of the code.

The list of required test can be modified in regtests/bin/matrix.base. Up to this point, there are two matrix_<machine> (matrix_ncep and matrix_datarmor) in regtests/bin/ to generate a matrix for the selected tests, defined in matrix.base. So far, these scripts are compatible with supercomputers used at NCEP and Ifremer/Datamor. Trusted institutions are encouraged to create their own versions of the matrix, which we will add to the bin directory in the future. The matrix_* scripts generate a batch-queue-ready script with the required modules for compilers, corresponding NetCDF and MPI libraries, and paths to METIS/PARMETIS/NC4, as well as the number of required cores used to execute the tests.

As part of the development framework in WAVEWATCH III®, prior to reintegrating feature/bugfix branches to the main or develop branches via a pull request, code managers at trusted institutions should check proposed code changes, comparing a matrix run made with the new branch with the stable develop branch, to ensure reproducibility of results and compilation. A set of scripts is available to assist in this process.

The following steps should be taken s part of the matrix comprehensive regression testing procedure:

  1. Clone the WW3 repository in (i.e. bf_branch or develop).
  2. Checkout the desired branch (develop/main if the control or the feature or bugfix branch to be tested. Make sure the feature or bugfix branch is up to date with Develop.
  3. Download the large binary files from ftp site by executing ./model/bin/ww3_from_ftp.sh
  4. If you do not have access to a machine with an existing matrix file, create your own by copying and existing matrix_<machine>.
  5. Edit the new matrix_<machine> to select the test list, required modules, set path and compiler option.
  6. Generate matrix by executing matrix_<machine>.
  7. Submit the matrix.

The matrix takes a while to complete, so it is recommended to split the matrix into multiple subsets and then run the matrix to shorten the total runtime of regression tests.

During runtime, a matrix log file is stored in a file determined by the wrapper script (default: matrix.out). This log file can be checked for compilation or other runtime errors by using a command such as “grep nri error matrix.out”. In case errors are found, which can be labeled “error”, “Error” and “ERROR” within matrix.out, the code manager/developer should debug the problem and repeat the process until a successful outcome is achieved.

After a successful matrix run, regression test outputs for the new feature/bugfix branch should be compared with the develop branch matrix outputs. The following steps are available:

  1. edit the path to the output of the regression test s for the develop branch in ./bin/matrix.comp
  2. execute ./bin/matrix.comp all, which compares all the regression tests and generates summary and full comparison.

If the differences are justified as a consequence of the new developments/changes in the feature branch, the code is ready for a pull request to the NOAA-EMC repo, or if from the latter, for a new release tag. If differences are unexpected, the proposed changes should be debugged until a successful outcome is attained.