NOAA Fisheries Integrated Toolbox

Corinne Bassin, Christine Stawitz

ECS Federal in support of NOAA Fisheries S&T

Toolbox Overview

The NOAA Fisheries Integrated Toolbox (NOAA FIT) is an interdisciplinary, web-based portal of operational tools that can be used for a variety of applications, including stock assessment modeling, forecasting, and data preparation for fish and protected species, as well as economic and ecosystem modeling.

Goals
  • Sharing and communication across regions
  • Tool comparison, benchmarking, and testing
  • Support for tool developers and users
  • Platform infrastructure (shared interfaces, ensembles)

Steps

Forming the architecture (onboarding)

  • Getting the tools in one place
  • Enforcing good coding practices
  • Making tools usable and sustainable

Building on the foundation

  • Input/output standardization
  • Ensemble modeling framework
  • User interface and visualization tools
  • Cloud computing

Continuous support

  • Supporting users and developers
  • Domain-specific validation
  • Versioning and releases

Caveat: we are a team of three people, and the UI is still a work in progress

Forming the architecture

  • What is included?
  • Metadata form about tool
  • Version control (git)
  • NOAA Github policy
  • Code review
  • ..and more
Tools to include

We seek to be inclusive; tools that are of use in NOAA products are our primary focus. There must be at least one tool owner we can work with to implement version control, etc. We recommend including the latest stable version in the toolbox. Research and development can continue elsewhere on new tool features.

Metadata Form

Initial authors of a tool fill out a meta-data form about their tool which includes background informtation that can be made public on the tools landing page.

Version Control Implementation

All code needs to be under version control, locally and on a server. Basic information about version control and best practices for the toolbox are available.

NOAA GitHub Policy

NOAA has specific guidelines that cover the use of GitHub for version control . We can fork from existing Git repos, create Git repos from Vlab, or help you set up git. Only NOAA affiliated personal will be able to directly push to an NOAA GitHub repository but the code is publicly available. Outside collaborators will be able to suggest changes and perform a pull-request to have their changes accepted.

Code review

After these steps have been completed, the toolbox team will review your git repository and file issues about specific items that need to be resolved for inclusion in the toolbox.

Initial Process Completed

Link to tool: The tool receives a landing page and link on the toolbox website

Basic support level determined : Depending on the collaborators affiliations and development state of the tool, each tool recieves differing levels of support from the NOAA National Modeling Team.

Building on the foundation

Input/output standardization

When tools are all shared and under version control, we can more easily work with individual tool developers to standardize and simplify.

Example: lookup table for stock assessment model names; automate writes to SIS

Ensemble modeling framework

Standard formats for inputs and outputs make it easier to average model results. Instead of custom code, pulling results from multiple output files with consistent formatting makes it easy to add to ensembles.

User interface and visualization tools

Common inputs will also allow us to develop common user interfaces that write model inputs. Standard output files make it easier to visualize output and generate reports.

Example: A standard package for plotting stock assessment output; creating new web-based UI for legacy desktop applications.

Cloud and high performance computing
  • Increasing reproducibility (binder, the TAF)
  • Running models in the cloud via Azure, AWS

Continuous support

Supporting users and developers

Management by FIT reduces the burden on tool developers since we can monitor issues and help support users.

We're also working on support blog and trainings (at NSAW) to teach about git, software best practices, etc.

Domain-specific validation
How do we ensure tools are not just functional but scientifically accurate?

Software peer review a.la rOpenSci?

Steering committee, or leverage existing committees

Versioning and releases

The toolbox uses badges to communicate the most up-to-date versions of tools. We can also monitor the Github repositories of tools and encourage splitting into versions during the code review process. (e.g. GMACs)

Contact
  • Corinne Bassin: corinne.bassin@noaa.gov
  • Christine Stawitz: christine.stawitz@noaa.gov