ACM SIGCOMM 2023 Tutorial: Designing Networks for Testing

Tutorial Program

Background

Unlike modern software systems, most networks are not "designed for testing." To be clear, it's not that testing is never used in the context of networking. On the contrary, command-line testing tools like ping and traceroute are widely employed by operators to check the healthiness of network paths, and there are entire industries devoted to validating individual network devices (e.g., Keysight). But modern testing techniques, like property-based testing, coverage-driven test generation, mutation testing, etc. are rarely applied to network hardware and software. In practice, the limitations of current testing methodologies not only lead to increased faults and outages, it also makes network owners conservative, since it's hard to know whether new features will interact badly with existing features, leading to new problems.

Call For Participation

This half-day tutorial will present approaches for bringing testing methodologies to bear on practical problems in networks. Building on the foundations provided in prior work in software engineering, as well as systems and tools such as CrystalNet [SOSP '17], Switch-V [SIGCOMM '22], p4testgen [SIGCOMM '23], Yardstick [SIGCOMM '21], and others, we will show how high-level abstractions (e.g., the P4 language) and open-source platforms (e.g., the mininet network emulator) can be used to build powerful approaches to testing. A common theme will be how emerging abstractions can be used to tame the inherent complexity of networks and develop precise models and interfaces that can be used as a basis for testing. The tutorial will be presented as a mixture of lectures and hands-on exercises, and culminate with a panel with invited experts to identify promising directions for future work in this space.

Draft Program

• Introduction (30 minutes)
  • Welcome
  • Motivation
  • Software Engineering background
  • Key "gaps" in networking
• Building Network Models (60 minutes)
  • Control Planes
  • Data Planes
  • Hands-On Activity: Modeling a "mystery" device
• Testing Network Models (60 minutes)
  • Symbolic Execution
  • Fuzzing
  • Property-based Testing
  • Hands-On Activity: Finding a bug in a "mystery" device
• Scaling Up (30 minutes)
  • Emulation
  • Simulation
  • Coverage
  • Hands-On Activity: Using test-driven methodologies to enable rapid evolution
• Expert Panel (45 minutes)
  Moderator: Jonathan DiLorenzo (Google)
  • Hongqiang "Harry" Liu (Uber)
  • Ratul Mahajan (UW & Amazon)
  • Rob Sherwood (NetDebug)
• Conclusion (15 minutes)
  • Current research themes
  • Next steps

Audience Expectations and Prerequisites

The primary goal of this tutorial is to nurture and grow the inter-disciplinary research community at the intersection of networking and software engineering. As such, we plan to make the material accessible to the SIGCOMM audience and will assume no background in testing or software engineering, other than undergrad-level programming experience. We also do not require any special computer requirements other than network connectivity and (optionally) the ability to run a virtual machine. We will provide a Linux virtual machine image with all necessary software pre-installed.

All tutorial materials will be made available on GitHub (https://github.com/cornell-netlab/sigcomm23-tutorial) and released under an open-source license (Apache License, Version 2.0). No special software is required to participate in this tutorial. However, participants who wish to follow along with the live exercises are encouraged to install Docker and download the ~328MB Docker image jnfoster/sigcomm23-tutorial:squashed prior to the conference. This can be accomplished on the command line as follows: docker pull jnfoster/sigcomm23-tutorial:squashed

Organizers

• Nate Foster

  Cornell & Jane Street

• Fabian Ruffy

  NYU

• Rob Sherwood

  NetDebug

• Jonathan DiLorenzo

  Google