Because scientific software is used to support mission-critical situations, decision making, and scientific findings published in research literature, faults are problematic. In extreme cases, scientists have retracted publications due to faults in scientific software. Therefor, proper testing is essential for scientific software. The literature review identified two classes of challenges for testing scientific software. First, the characteristics of the software result in challenges to (1) test case development, (2) identifying expected outputs, (3) test case execution, and (4) interpretation of results. Second, the cultural differences between scientists and the traditional software engineering community result in challenge because of limited understanding of (1) testing concepts, (2) testing process, and (3) how to apply testing methods. The literature also described the following techniques for overcoming these challenges: pseudo oracles, analytical solutions, experimental results, measurement values, simplified data, and professional judgement. The authors claim that software engineering research can provide effective solutions for testing software with oracle problems.
Scientists typically spend 30% or more of their time developing software. However, almost all of these scientific software developers are self-taught. Therefore they typically do not employ many basic software development practices such as version control, issue tracking, writing maintainable code, and task automation. This paper describes a set of easy-to-adopt practices, such as, (1) write programs for people, not computers, (2) Let the computer do the work, (3) Make incremental changes, (4) Don’t repeat yourself (or others),(5) Plan for mistakes,(6) Optimize software only after it works correctly, (7) Document design and purpose, not mechanics, and (8) Collaborate. These practices have proven effective in many research settings. The authors recommend these based on their own experience in building scientific software, reports from many others groups, and software development in general.
Despite the importance and usefulness of software in the scientific literature, the methods for making such software visible are still unclear. In addition to formal citations, authors use less formal approaches, such: web page URLs in footnotes, parenthetical notes akin to those used for purchased scientific consumables, and simple mentions in the paper text (with no formal citation). This paper describes how authors currently make software visible in the biology literature and discusses the extent to which this visibility contributes to achieving the normative ideals of scientific practice. Specifically, the authors performed a content analysis of a random sample of 90 Biology journal articles to develop a reliable scheme for identifying mentions of software. They also analyzed how well these mentions achieve the goal of providing a systematic motivation for improving the visibility of software in scientific literature.