Helmholtz Quality Indicator for Data and Software Publications Introduced

Research Software is well on its way to becoming a first class citizen in the Helmholtz community. Starting 2025 the Helmholtz Quality Indicators for Data and Software Products will be used to count research software as well as data publications within Helmholtz for the annual reporting of research output. Several requirements need to be met for a software (or data) publication to be included (just as papers need to get through peer-review).

While HMC is your partner for data publication questions, HIFIS is here to support you with everything needed to make your research software count! The indicators are developed by the Task Group Helmholtz Quality Indicators for Data and Software Products and your feedback is very welcome there.

How the Software Indicator Works

Initial Requirements

For a software to be considered in your center’s report, it needs to:

• have someone in your center listed as an author,
• be a research software, and
• have a release in the reporting year (e.g. the initial publication, a new major version, or an update fixing bugs and updating dependencies).

Quality Dimensions

Your software’s quality is assessed in six dimensions (the first four dimensions are based on the FAIR4RS principles):

• Findable
• Accessible
• Interoperable
• Reusable
• Scientific Basis
• Technical Basis

Attributes Detailing Each Dimension

To judge the software’s quality along those dimensions, specific attributes in each dimension are evaluated. For instance to determine whether a software is Findable, the indicator looks at whether it is published in an open repository, whether some sort of versioning is applied, whether it can be found via an identifier (e.g. a DOI) and whether metadata is provided. Each attribute has several maturity levels: For versioning some software might only count releases, others might employ structured versioning and some software might even come with a description of its versioning scheme and a documented release cycle.

Minimal Requirements

Not all attributes and maturity levels need to be covered by a software to count. For each dimension a minimal score is defined, which needs to be met on average among its attributes and the indicator will start out slowly and increase those values over time to give everyone the chance to develop their software project accordingly. One can depict the 6 dimensions in a radar plot as examplified in the figure and your software will be counted if it covers at least the entire minimal polygon (red in the figure). In the example publication A and B would count, publication C, however, is not meeting the minimal requirements in all dimensions.

→ The indicator will also be introduced at the next HiRSE Seminar.

The seminar takes place on February 21, 2025, 11am CET, via zoom. Details on the event page.

How to Make Your Research Software Count

Check back in here, we will publish the details of the indicator once available. Until then you can already have a look at the FAIR4RS principles and make sure your software uses a version control system like git, has a licence, a README and relevant meta data information available.

HIFIS Supports You

…with Tools

If your project is at home in the Helmholtz Codebase you already tick important boxes in the dimensions “Findable” as well as “Technical Basis” without your code leaving Helmholtz servers. You can additionally publish your software in the Helmholtz Research Software Directory (RSD) to make it easier to find for potential users and improve your score for findability at the same time. Additionally, various tools to evaluate software quality will be added to the RSD in the upcoming months and the RSD will be an important work horse for automating the evaluation of software at Helmholtz following the quality indicator.

…with Trainings

If you feel like you don’t really know how to go about some of the requirements the indicator poses, some of the HIFIS courses might be helpful for you. Check the HiDA Course Catalog for the next dates of our courses on Introduction to Git and GitLab, Foundations of Research Software Publication and more.

…with Knowledge

You prefer to help yourself? The Awesome RSE lists curated by HIFIS gather some helpful resources to make your software FAIR and give it a strong technical foundation.

…with Personal Advice

If you need additional help with getting your project into shape, the HIFIS Consulting Team is ready to help you out!

Roadmap

From 2025 until 2027 the indicator will be tested and feedback can be incorporated. Starting 2028 it will become part of the evaluation for round five of the program-oriented funding (PoF V), so it’s a good idea to already get your software up to speed and share your feedback on the indicator with the task group.