Abstract

Session

World Wide Web

Paper

3-1

Full Paper

ps.gz

Title

A Protocol-Independent Technique for Eliminating Redundant Network Traffic

Author(s)

Neil T. Spring (University of Washington)
David Wetherall (University of Washington)

Abstract:

We present a new technique for identifying repetitive information transfers and use it to analyze the redundancy of Web traffic. Our insight is that dynamic content, streaming media and other traffic that is not cached by today's Web caches is likely to derive from similar information. We have therefore adapted existing similarity detection techniques to the problem of building a system that eliminates redundant transfers. The result generalizes other approaches such as delta-coding and duplicate suppression. Our technique is lightweight enough to run at better than T3 rates (45 Mbps), and trace analysis predicts that a system based on it can be highly effective in practice. In our traces, after Web proxy caching, an additional 40\% of Web traffic is found to be redundant. Moreover, since our system makes no assumptions about HTTP protocol syntax or caching semantics, it provides immediate benefits for other types of content, such as streaming media, FTP traffic, news and mail.