One of the fundamental difficulties with trying to characterize the Internet's behavior lies in its immense diversity [Danzig92, Paxson94]. To date, efforts to characterize end-to-end Internet dynamics such as packet delay and loss have studied at most a handful of traffic traces and Internet paths [Mogul92, Sanghi93, Bolot93]. This limitation makes it difficult to assess the generality of the findings.
To try to tackle this problem of gathering representative traces in the face of widespread diversity, we've developed a distributed framework for conducting end-to-end measurements. The framework consists of a number of Internet sites running a Network Probe Daemon (NPD) that accepts authenticated requests to make measurements and send back the results. We earlier reported on an end-to-end routing study conducted using the framework, in which 37 sites participated, yielding about 40,000 routing measurements along 1,000 Internet paths [Paxson96]. We characterized routing in terms of pathologies, stability of time, and symmetry, finding that (1) pathologies appreciably increased during the course of 1995, (2) most paths through the Internet are strongly dominated by a single path, (3) about 1/3 of the paths have lifetimes on the order of minutes to hours, while 2/3's have lifetimes of days or weeks, and (4) about half the paths at the end of 1995 exhibited a major asymmetry (a different city visited in the two directions).
We are now analyzing 20,000 TCP bulk transfers measured using the same framework. Each measured connection transferred 100 KB (typically) from one NPD site to another. Using tcpdump, the NPD's recorded all packets generated by the sender and receiver at both endpoints. The dual traces allow us to determine the fate of each packet: whether it successfully reached its destination, and how long (relatively) it took to do so. We can then determine both how the TCP implementations behaved and the network dynamics encountered by the packets. Our goal is to devise plausibly- representative characterizations of Internet dynamics such as packet loss and delay, congestion levels, and available bandwidth.
We will discuss (1) measurement errors introduced by the packet filters; (2) dealing with unsynchronized and skewed clocks; (3) how we separate out TCP behavior from network behavior; (4) some seriously broken TCP's; (5) findings concerning pathological network behavior; (6) preliminary characterizations of available bandwidth, packet loss patterns, packet delay, and congestion levels.