Workshop Notes

Workshop Proceedings

Workshop Proceedings (4 week FREE access)

Overview

This workshop is concerned with identifying and applying appropriate software engineering (SE) tools and practices (e.g., code generators, static analyzers, validation + verification (V&V) practices, testing, design approaches, and maintenance practices) to support and ease the development of reproducible High Performance Computing (HPC) & Data Analysis/Analytics (HPCDA). Specifically:

• HPC applications, which include large parallel models/simulations of the physical world running on HPC systems.
• HPDA applications that utilize HPC systems (e.g., GPUs computing, compute clusters, cloud computing, containers, or supercomputers) to analyze large amounts of data.

Despite the increasing demand for utilizing HPC for such applications, software development for HPC historically attracted little attention from the SE community, while paradoxically, the HPC and HPDA communities have increasingly been adopting SE techniques and tools. Indeed, the development of HPC and HPDA software differs significantly from the development of more traditional business information systems, from which many SE best practices and tools have been drawn. These differences appear at various phases of the software lifecycle as described below:

• Requirements:
  • Risks due to the exploration of relatively unknown scientific/engineering phenomena;
  • Supporting reproducible science, particularly on non-deterministic systems;
  • Constant change as new information is gathered;
• Design
  • Data dependencies within the software;
  • The need to identify the most appropriate parallelization strategy for HPC and HPDA algorithms;
  • Software engineering challenges related to applications running across different architectures;
  • Balancing the cost of resource utilization and developer productivity;
  • Integration with modern frameworks and libraries, including machine learning libraries;
  • The presence of complex communication among HPC nodes that could degrade performance;
  • Challenges in designing unit and system tests at appropriate scales;
  • The need for fault tolerance and task migration mechanisms to mitigate the need to restart time-consuming computations due to software or hardware errors;
• V&V
  • Results are often unknown when exploring novel science or engineering areas, algorithms, and datasets;
  • Challenges in applying unit and system tests at appropriate scales;
  • Challenges in retrospectively designing and implementing tests for legacy code;
  • Challenges in validating models on large scale systems;
  • Popular tools often do not work on the latest HPC architectures; they need to be tuned to handle many threads executing at the same time.
• Deployment
  • Failure of components within running systems is expected due to system size;
  • Continuous integration on platforms with high available and infrequent downtimes;
  • Easy-to-use build systems and packaging, including containers;
  • Long system lifespans necessitate porting across multiple platforms.

Therefore, in order to identify and develop appropriate tools and practices to support HPC and HPDA software, members of the SE community, the HPC community, and the HPDA community must interact with each other. This workshop aims to provide a platform to facilitate this interaction by encouraging paper submission and workshop participation by people from all three communities. Following keynote addresses and presentations of accepted papers, we spend time in interactive brainstorming to identify the most important questions at the intersection of SE and HPC and HPDA in need of additional study, as an agenda for future research. We will also collectively assemble a list of important work in the area, including case studies, that can form the basis of a reading list for newcomers.

Previous editions of this workshop have identified interesting topics related to research HPC software, including: bit-by-bit vs. scientific validation, reproducibility, unique characteristics of research HPC software that affect software development choices, major software quality goals for HPC software, crossing the communication chasm between SE and HPC, measuring the impact of SE on scientific productivity, SE tools and methods needed by the HPC community, and how to effectively test research HPC software. Topics identified in one workshop have often been the subject of papers submitted to a subsequent workshop. Given recent changes in both hardware platforms and software frameworks, this year we want to find new and changed topics that can be used to springboard the field forward.

Motivated by the outcomes of previous workshops, this edition of the workshop emphasizes data analysis/analytics software as a partner of computational science and engineering. In addition, we will place special emphasis on experience reports (including positive, negative, and neutral) of applying software engineering practices to the development of HPC and HPDA software. It is important to document those successes and failures for the community, and, as quality assurance is a challenge for HPC and HPDA software, which was specifically discussed previous workshops, we will also recruit papers describing quality assurance techniques for HPC and HPDA software and their use in practice focusing specifically on the challenges of unit testing, system testing, and continuous integration for HPC and HPDA software, addressing both legacy code and testing at scale on different architectures and platforms.

For questions contact Jeffrey Carver.

---

Last Updated on April 30, 2019 by Jeffrey Carver