Mobile Network Performance and Content Delivery - Part 2
Mar 20, 2014: Most Content Delivery Networks (CDNs) attempt to pick the closest replcia server to your location. We see large inefficiencies in replica selection for cellular networks, with remote Public DNS resolvers retrieving better replicas a significant fraction of time.
Led by John Rula
This series looks at the current state of network performance and content distribution on cellular networks. Download Namehelp Mobile for Android in the Play Store to find which DNS give you the best performance on your phone. In Part 1 we looked at the impact DNS performance and compared your default DNS service on your mobile device to several public options. In this part we look at how Content Delivery Networks (CDNs) attempt to optimize content delivery on mobile networks.
Content delivery networks (CDNs) are the unseen engine which drive the Web. Companies like Akamai and Limelight operate a global network of replica caches, with the intention of delivering content from the closest (and thus fastest) server to its users. For instance, when you go to the New York Time's website, your browser resolves www.nytimes.com to its IP address by its locally configured DNS, typically provided by your internet service provider. This DNS tthen asks the authoritative DNS server (in this case run by its CDN Akamai), which trys to choose the closest server. Since the CDN can only see the origin of the last request, it has to approximate your location based on the location of your DNS server.
When selecting replica servers, CDNs try to choose the server with the lowest latency, typically the closest geogaphically, to its intended destination. Latency is important factor for web performance since the typical content request include several round trip times before the user receives data, including a DNS resolution, a TCP connection and HTTP header exchange. The multiple round trips mean that any latency improvements to the content server are compounded throughout the transfer.
Conventional widsom in replica selection is that the closer you are to your DNS server, the closer your content servers will be. It is very hard to improve upon the DNS server of your internet service provider when it comes to replica server selection. For example, our previous investigation into the impact of Public DNS on replica server selection (paper here )found that replicas selected using public DNS resolves had over twice the latency when compared to those chosen by your ISP's DNS.
Replica selection in cellular networks seems to contradict these traditional beliefs. In Part 1 of this series, we saw that public DNS resolvers like GoogleDNS and OpenDNS are typcaily farther away from clients resulting in longer DNS resolution than the ISP configured resolver. However, when we look at the latency to the replica servers returned by each DNS, we see that public DNS resolvers returned servers which had equal or lower latency 74% of the time in the case of OpenDNS! Figure 1 below shows the percentage of time the replicas returned by public DNS servies had equal or improved latencies for all hostnames we tested. It is also interesting to note that both public DNS resolvers returned servers with a 20% latency improvement around 20% of the time, and a 40% improvement 10% of the time.
(larger is better)
When we look at replica selection on a per domain level, we see that it varies depending on the hostname chosen, and on the CDN they employ. The greatest example of disparity we saw was with both facebook.com and answers.com, where OpenDNS performed as well or better 95% of the time. Figure 2 shows the percentage of time replicas returned had latencies which had equal or greater performance than the Local DNS options. The figure displays the wide disparity between public DNS services as well as across different hostnames. It seems clear that the current system of replica selection for cellular networks is far from optimal.
(larger is better)
We have released a tool called Namehelp Mobile which measures the performance of your ISP provided DNS service against several public DNS options. In addition, it also measures the role that your DNS service has with Content Delivery Networks (CDNs) in mobile networks, and how your ISP provided DNS service can actually deliver worse performance in many cases.
Mobile Network Performance and Content Delivery - Part 1
Mar 20, 2014: Web performance is determined by more than just SpeedTest's reported “Network Performance”. The video you watch, the DNS service you choose, and the server it's delivered from have a much larger impact on your perceived performance.
Led by John Rula
This is Part 1 of a blog series which looks at the current state of network performance and content distribution on cellular networks. This work attempts to challenge conventional wisdom on the state of mobile content distribution, specifically that little can be done to improve outside of the cellular network. Part 1 looks the performance on mobile devices of Domain Name Services (DNS), an intergral often overlooked service required for the internet to function.
The Domain Name ServiceThe Domain Name Service is a globally distributed system that translates a human readable hostname (i.e. facebook.com) to its routable IP addres (e.g. 173.252.110.27). Nearly all requests form a device must first resolve a domain name in order to download content, and the importance of domain name services is only increasing. According to a report by Arbor Networks last year, the number of DNS requests sent across mobile operators has quadrupled between 2010 and 2012. Aside from an increase in mobile network use, mobile websites have begun to incorporate an increasing number of unique domain names in their sites for items such as ad networks, analytics providers and social networking plugins.
Public DNS services like GoogleDNS or OpenDNS provide an alternative to ones Local DNS service provided by their internet service provider. These services popularity has grown recently due to claims of higher reliability, better performance, and a more secure infrastructure. A low performing DNS service can kill your performance and any DNS failures can render your device practically useless. Current cellular operator policy prohibits users from selecting their own DNS service without first rooting their device.
Performance of Public DNS on Cellular NetworksWe investigated the performance of public DNS services GoogleDNS and OpenDNS against the cellular operators provided Local DNS. Figure 1 shows the median, 75th and 90th percentile times taken to perform a DNS resolution for each of the public DNS services with the provided Local resolver. We see the Local DNS service performing better around 80% of the time, with a long tail which peforms much worse than the public DNS services (almost twice the time at the 90th percentile). These results seem to follow those done on the wired internet, where the public DNS services are typically located farther away from clients, and therefore have a longer latency between them.
(smaller is better)
Another factor affecting network performance including DNS performance is the radio that your phone is currently using. Most phones have several different radios which it can use at any time (e.g. LTE, EVDO (3G), or CDMA 1x (2G)). Each of these radios have different performance capabilities and cause different levels of network performance. Figure 2 below shows the DNS resolution time to each users Local resolver for each radio technology used.
(smaller is better)
While 4G technologies like LTE clearly display the best performance, we are not always connected with 4G radios. We looked at the percentage of time users are connected to each radio type. Figure 3 shows the proportion of time CDMA users are connected to each radio type: LTE, EVDO Rev A, EHRPD and CDMA 1x. While 75% of the time is spent in LTE for our users, we suprisingly see that nearly 10% of time is spent connected to CDMA 1x, which has an average DNS resolution time of 1.25 seconds! It seems that regardless the number of signal bars you have, the type of signal is even more important for your network performance.
We have released a tool called Namehelp Mobile which measures the performance of your ISP provided DNS service against several public DNS options. In addition, it also measures the role that your DNS service has with Content Delivery Networks (CDNs) in mobile networks, and how your ISP provided DNS service can actually deliver worse performance in many cases. Read more about this in the second part of this series.
Comparing broadband services on users' terms
Feb 12, 2013: Paying for higher download speeds may not improve your web performance; it also depends on the pages you visit.
Led by Zachary Bischof
Recent studies on broadband services (such as those published by SamKnows in collaboration with the FCC in the US and Ofcom in the UK) focus primarily on comparing the performance of ISPs in traditional networking terms, such as throughput, latency and packet loss.
While increased throughput rates or lower latency should improve user perceived performance, it is unclear how strongly these two are related. For example, for some applications, such as web browsing, subscribing to a faster Internet service will not always result in a better user experience. A range of other factors, such as latency over the last-mile and to the DNS server, as well as the agreements the ISP has with content providers, will also have a significant impact on performance.
In this post, we compare users' download throughput rates with the time it took to load a web page. For this experiment, we include results from bing.com, craigslist.org, and paypal.com, three sites in the top 20 most popular sites (according to Alexa.com). In the figures below, we look at how page loading time is affected by download throughput rate for two ISPs, Verizon and Comcast, across these websites.
First, we look at Bing, a search engine optimized to decrease loading time. Here we see that for users in Verizon's network, page-loading times are fairly consistent for all users (all are less than 500 ms). In other words, increasing download throughput rate does not result in significantly faster page-loading times. Since this website has been optimized to reduce response time, we expect latency to be a bottleneck in improving performance, not download throughput. For users in Comcast's network, on the other hand, page-loading times are higher for many users and show a much wider range, with 33% of users seeing page load times over 500 ms. Again, increasing throughput does not result in faster page-loading times.
|
www.bing.com
|
|
|
Verizon
|
Comcast
|
|
Download throughput rate (Mbps)
Loading time (ms)
|
Download throughput rate (Mbps)
Loading time (ms)
|
Next, we look at Craigslist, a classified ad site as an example of a minimalistic design (few objects to retrieve). Here we see that for Verizon, increasing the download throughput rate appears to result in a decrease in page loading time, but only to a point. Users with a download throughput rate above approximately 5 Mbps see similar page loading times. When looking at Comcast, we again see that increases in download throughput do not necessarily result in faster page-loading times and see a wider variation in page-loading time (as compared to Verizon).
|
www.craigslist.org
|
|
|
Verizon
|
Comcast
|
|
Download throughput rate (Mbps)
Loading time (ms)
|
Download throughput rate (Mbps)
Loading time (ms)
|
Last, we look at page-loading times for PayPal. Compared to the Craigslist example, Paypal has a larger number of objects to be retrieved and therefore makes significantly more requests (about 34 for PayPal and 8 for Craigslist). For users in Verizon's network, we again see that increasing download throughput leads to faster page-loading times. However, similar to Craigslist, beyond a point, further increases in throughput rates do not result in faster page-load times. For Comcast, we again see a much wider range in page-loading times. This time, the range in page loading times is about twice as high as the Craigslist example. This is likely due to the fact that for most Comcast users, download throughput is not a bottleneck in improving page-loading time. Instead, other factors, such as latency and DNS performance may be negatively impacting performance.
|
www.paypal.com
|
|
|
Verizon
|
Comcast
|
|
Download throughput rate (Mbps)
Loading time (ms)
|
Download throughput rate (Mbps)
Loading time (ms)
|
In future posts we'll be further investigating how different metrics (throughput, packet loss, latency, and DNS performance) affect the performance of network applications such as web browsing, video streaming, and P2P, as well as comparing the performance of services offered by ISPs. Be sure to check back for more posts.
