Computational tools are indispensable in almost all scientific disciplines. Especially in cases where large amounts of research data are generated and need to be quickly processed, reliable, carefully developed software is crucial for analyzing and correctly interpreting such data – e.g., in biology, physics, or, very recently, in epidemiology. However, the development and maintenance of scientific software are often characterized by uncertain funding and time pressure, which can lead to quality deficiencies. To evaluate software quality in an automated way, computer scientists at Karlsruhe Institute of Technology (KIT) and Heidelberg Institute for Theoretical Studies (HITS) have designed the SoftWipe tool. They published their findings in the Nature Scientific Reports journal.
“Adherence to coding standards is rarely considered in scientific software, although it can even lead to incorrect scientific results,” says Professor Alexandros Stamatakis, who works both at HITS and at the Institute of Theoretical Informatics (ITI) of KIT. The open-source SoftWipe software tool provides a fast, reliable, and cost-effective approach to addressing this problem by automatically assessing adherence to software development standards. Besides designing the above-mentioned tool, the computer scientists benchmarked 48 scientific software tools from different research areas, to assess to which degree they met coding standards.
“SoftWipe can also be used in the review process of scientific software and support the software selection process,” adds Adrian Zapletal. The Master's student and his fellow student Dimitri Höhler have substantially contributed to the development of SoftWipe. To select assessment criteria, they relied on existing standards that are used in safety-critical environments, such as at NASA or CERN.
“Our research revealed enormous discrepancies in software quality,” says co-author Professor Carsten Sinz of ITI. Many programs, such as covid-sim, which is used in the UK for mathematical modeling of the COVID-19 disease, had a very low quality score and thus performed poorly in the ranking. The researchers recommend using programs such as SoftWipe by default in the selection and review process of software for scientific purposes.
How Does SoftWipe Work?
SoftWipe is a pipeline written in the Python3 programming language that uses several available static and dynamic code analyzers (most of them are freely available) in order to assess the code quality of software written in C/C++. In this process, SoftWipe compiles the software and then executes it so that programming errors can be detected during execution. Based on the output of the code analysis tools used, SoftWipe calculates a quality score between 0 (poor) and 10 (excellent) to compute an overall final score.
Original publication:
Zapletal, A., Höhler, D., Sinz, C. et al.: The SoftWipe tool and benchmark for assessing coding standards adherence of scientific software. Sci Rep 11, 10015 (2021).
https://doi.org/10.1038/s41598-021-89495-8
Being “The Research University in the Helmholtz Association”, KIT creates and imparts knowledge for the society and the environment. It is the objective to make significant contributions to the global challenges in the fields of energy, mobility, and information. For this, about 10,000 employees cooperate in a broad range of disciplines in natural sciences, engineering sciences, economics, and the humanities and social sciences. KIT prepares its 22,800 students for responsible tasks in society, industry, and science by offering research-based study programs. Innovation efforts at KIT build a bridge between important scientific findings and their application for the benefit of society, economic prosperity, and the preservation of our natural basis of life. KIT is one of the German universities of excellence.