FIT Tool Quality Standards: Reviewer Checklists

Checklists for FIT tool quality standards

The below are checklists for reviewers assessing the FIT tool quality standards

Checklist for FIT coordinator

The FIT coordinator will fill out the following checklist, which must be met for tool inclusion (i.e., no badge assigned for basic check).

• Metadata complete, as determined by running against the json schema
• Links in metadata work
• A license is included (where appropriate, an open source license).
• For NOAA developed products where the source code is linked, there is a NOAA disclaimer on the readme.
• After review: Add FIT badges to metadata based on reviewer’s work.

Checklists for reviewers

Thanks for reviewing this tool! Please use the information submitted by the tool authors to fill out the checklist.

If you have any questions, you can ask them directly on this thread by using the @ in front of their github username in a comment.

Documentation

Check off all that are complete. A badge will be assigned based upon how many are checked off:

• 0-2 checked = red
• 3-6 checked = orange

• All 7 checked = green

• Background text includes a description of the tool and motivation or scope of tool. If appropriate, it also includes a link to examples where the tool has informed science-based decision making.
• Installation instructions that the reviewer can run. If this is a web app, check this off (Reviewer, please try installing and only check off if install is verified).
• A getting started example that the reviewer can run (e.g. R vignette; Reviewer, please try running this example and only check off if it’s verified that it can run)
• How to cite the tool
• Documentation of how to use the tool in an appropriate form (e.g., a user manual, function reference: roxygen, doxygen, Sphinx)
• Example demonstrating advanced features or functions
• Web-hosted documentation (e.g., pkgdown site, doxygen site, readthedocs)

Tests

Check off all that are complete. A badge will be assigned based upon how many are checked off:

• 0-1 checked = red
• 2-4 checked = orange

• All 5 checked = green

• Integrated tests have been done (manually or within a testing framework)
• Unit testing framework used (e.g., testthat, googletest, unittest) that allows running tests with a single command
• Test coverage acceptable (>40%)
• Test coverage excellent (>70%)
• Tests set up on a continuous integration service to run automatically upon code changes or on a schedule (e.g., on GitHub Actions, Travis, Jenkins)
• For web apps: manual user testing has been done. If not a web app, check this off.