Research Infrastructures and Tools for Reproducible Data Communication Research

The Non-Paper Session will take place at Room Mondego.

Experimental research on networked systems requires suitable research infrastructures (RIs). For many years, the dominating approach of research groups was to create a specific experimental setup tailored to the needs of a specific set of experiments, e.g., in the context of a PhD thesis. The community is well aware of the obvious shortcomings of this approach. To overcome such challenges, the networked systems community built testbeds FABRIC, Chameleon, CloudLab, and Fed4Fire. While these initiatives demonstrated significant progress, many areas for improvement remain. Artifact evaluation committees need to spend significant effort to evaluate papers with artifacts (software and data with metadata) as experimental artifacts, and the involved tools are highly customized and heterogeneous. At the same time, experimenters have to invest significant effort to prepare their artifacts for evaluation and reusability. This approach clearly does not scale, in particular in the context of data-driven science powered by AI/ML. An area for improvement is experiment control. So far, scientists who use a large testbed for specific experiments need to solve how to orchestrate their experiments, how to collect, process, and store the data produced by the experiment, and how to add metadata to support other scientists. Consequently, there is a high heterogeneity concerning the artifacts of specific experiments. The SLICES Research Infrastructure introduced several concepts for the evolution of networking testbeds into scientific instruments, with comprehensive support for experiment control and data management. The recent Dagstuhl seminar 24462 entitled "Research Infrastructures and Tools for Collaborative Networked Systems Research" focused on bridging the gap between the services provided by large-scale testbed infrastructures and the needs of researchers conducting cutting-edge experiments, and concluded with a set of recommendations, which will be shared and discussed with the SIGCOMM community. Conclusions address strategic investment and community engagement, open access and data sharing, amplifying impact, and network effects. Recommendations include scientific objectives, common abstractions, reproducibility and repeatability, FAIR principles (Findability, Accessibility, Interoperability, and Reusability), flexibility and adaptability, interoperability and openness, user experience, international collaboration, and support of sustainable development goals (SDGs).