Exploring Reproducibility in High-Performance Computing Publications with the Chameleon Cloud

Hello everyone,

I’m Klaus Kraßnitzer and am currently finishing up my Master’s degree at the Technical University of Vienna. This summer, under the guidance of Sascha Hunold, I’m excited to dive into a project that aims to enhance reproducibility in high-performance computing research.

Our project, AutoAppendix, focuses on the rigorous evaluation and potential automation of Artifact Description (AD) and Artifact Evaluation (AE) appendices from publications to this year’s Supercomputing Conference (SC). Due to a sizeable chunk of SC publications utlizing Chameleon Cloud, a platform known for its robust and scalable experiment setups, the project will be focused on and creating guidelines (and potentially, software tools) that users of the Chameleon Cloud can utilize to make their research more easily reproducible. You can learn more about the project and read the full proposal here.

My fascination with open-source development and research reproducibility was sparked during my undergraduate studies and further nurtured by my role as a teaching assistant. Hands-on projects and academic courses, like those in chemistry emphasizing precise experimental protocols, have deeply influenced my approach to computational science.

Project Objectives

1. Analyze and Automate: Assess current AE/AD appendices submitted for SC24, focusing on their potential for automation.
2. Develop Guidelines: Create comprehensive guidelines to aid future SC conferences in artifact submission and evaluation.
3. Build Tools (Conditionally): Develop automation tools to streamline the evaluation process.

The ultimate aim of the project is to work towards a more efficient, transparent, and reproducible research environment, and I’m committed to making it simpler for researchers to demonstrate and replicate scientific work. I look forward to sharing insights and progress as we move forward.

Thanks for reading, and stay tuned for more updates!