“We provide our resources in a dedicated repository”: Surveying the Transparency of HICSS publications
Pekaric, I., Apruzzese, G., Hawaii International Conference on System Sciences, 2024 Conference
Oneliner: Only a tiny fraction of the HICSS papers published in 2017--2024 have a functional and publicly available repository.
Abstract. Every day, new discoveries are made by researchers from all across the globe and fields. HICSS is a flagship venue to present and discuss such scientific advances. Yet, the activities carried out for any given research can hardly be fully contained in a single document of a few pages—the “paper.” Indeed, any given study entails data, artifacts, or other material that is crucial to truly appreciate the contributions claimed in the corresponding paper. External repositories (e.g., GitHub) are a convenient tool to store all such resources so that future work can freely observe and build upon them—thereby improving transparency and promoting reproducibility of research as a whole.
In this work, we scrutinize the extent to which papers recently accepted to HICSS leverage such repositories to provide supplementary material. To this end, we collect all the 5579 papers included in HICSS proceedings from 2017–2024. Then, we identify those entailing either human subject research (850) or technical implementations (737), or both (147). Finally, we review their text, examining how many include a link to an external repository and, inspect its contents. Overall, out of 2028 papers, only 3% have a functional and publicly available repository that is usable by downstream research. We release all our tools.