Research in Distributed Computing

Comparing broadband services on users' terms

Led by Zachary Bischof

Recent studies on broadband services (such as those published by SamKnows in collaboration with the FCC in the US and Ofcom in the UK) focus primarily on comparing the performance of ISPs in traditional networking terms, such as throughput, latency and packet loss.

While increased throughput rates or lower latency should improve user perceived performance, it is unclear how strongly these two are related. For example, for some applications, such as web browsing, subscribing to a faster Internet service will not always result in a better user experience. A range of other factors, such as latency over the last-mile and to the DNS server, as well as the agreements the ISP has with content providers, will also have a significant impact on performance.

In this post, we compare users' download throughput rates with the time it took to load a web page. For this experiment, we include results from bing.com, craigslist.org, and paypal.com, three sites in the top 20 most popular sites (according to Alexa.com). In the figures below, we look at how page loading time is affected by download throughput rate for two ISPs, Verizon and Comcast, across these websites.

First, we look at Bing, a search engine optimized to decrease loading time. Here we see that for users in Verizon's network, page-loading times are fairly consistent for all users (all are less than 500 ms). In other words, increasing download throughput rate does not result in significantly faster page-loading times. Since this website has been optimized to reduce response time, we expect latency to be a bottleneck in improving performance, not download throughput. For users in Comcast's network, on the other hand, page-loading times are higher for many users and show a much wider range, with 33% of users seeing page load times over 500 ms. Again, increasing throughput does not result in faster page-loading times.

Next, we look at Craigslist, a classified ad site as an example of a minimalistic design (few objects to retrieve). Here we see that for Verizon, increasing the download throughput rate appears to result in a decrease in page loading time, but only to a point. Users with a download throughput rate above approximately 5 Mbps see similar page loading times. When looking at Comcast, we again see that increases in download throughput do not necessarily result in faster page-loading times and see a wider variation in page-loading time (as compared to Verizon).

Last, we look at page-loading times for PayPal. Compared to the Craigslist example, Paypal has a larger number of objects to be retrieved and therefore makes significantly more requests (about 34 for PayPal and 8 for Craigslist). For users in Verizon's network, we again see that increasing download throughput leads to faster page-loading times. However, similar to Craigslist, beyond a point, further increases in throughput rates do not result in faster page-load times. For Comcast, we again see a much wider range in page-loading times. This time, the range in page loading times is about twice as high as the Craigslist example. This is likely due to the fact that for most Comcast users, download throughput is not a bottleneck in improving page-loading time. Instead, other factors, such as latency and DNS performance may be negatively impacting performance.

In future posts we'll be further investigating how different metrics (throughput, packet loss, latency, and DNS performance) affect the performance of network applications such as web browsing, video streaming, and P2P, as well as comparing the performance of services offered by ISPs. Be sure to check back for more posts.

• < Prev
• Next >