Bailey, David and Borwien, Jonathan and Colbry, Dirk and Corcho, Oscar and Deelman, Ewa and Dietze, Michael and Fahey, Mark and Gilbert, Benjamin and Harkes, Jan and James, Doug and Keele, Seth and Kooper, Rob and Kumar, Praveen and Lee, Jong Gun and Linke, Erika and Loewe, Laurence and Marciano, Richard and Marini, Luigi and Mattman, Chris and Mattson, Dave and McHenry, Kenton and McLay, Robert and Miguez, Sheila and Minsker, Barbara and Santana-Perez, Idafen
Standing together for reproducibility in large-scale computing: report on reproducibility@XSEDE.
In: "XSEDE14 Workshop", 14 Jul 2014, Atlanta, Estados Unidos. pp. 1-16.
This is the final report on reproducibility@xsede, a one-day workshop held in conjunction with XSEDE14, the annual conference of the Extreme Science and Engineering Discovery Environment (XSEDE). The workshop's discussion-oriented agenda focused on reproducibility in large-scale computational research. Two important themes capture the spirit of the workshop submissions and discussions: (1) organizational stakeholders, especially supercomputer centers, are in a unique position to promote, enable, and support reproducible research; and (2) individual researchers should conduct each experiment as though someone will replicate that experiment. Participants documented numerous issues, questions, technologies, practices, and potentially promising initiatives emerging from the discussion, but also highlighted four areas of particular interest to XSEDE: (1) documentation and training that promotes reproducible research; (2) system-level tools that provide build- and run-time information at the level of the individual job; (3) the need to model best practices in research collaborations involving XSEDE staff; and (4) continued work on gateways and related technologies. In addition, an intriguing question emerged from the day's interactions: would there be value in establishing an annual award for excellence in reproducible research? Overview