Characterizing the impact of Google's Accelerated Mobile Project

The short story

Over the last decade, the number of mobile subscriptions has grown rapidly, surpassing 7.7~billion by late 2017. Since October 2016, more websites have been loaded on smartphones and mobile devices than on desktop computers as users spend up to three times more hours browsing on their smartphones than on any other device.

Despite the shift toward mobile browsing, much of the web has been designed for desktop machines on wired connections. This ``disconnect'' typically translates into complex websites that offer poor quality of experience to mobile users, something that has recently received a significant amount of attention from researchers and industry alike. The impact of low quality of experience (QoE) on user retention has led to a number of efforts studying mobile performance and exploring ways to reduce page load time (PLT) and related proxies of QoE. Projects such as Facebook Instant Articles rely on formats and infrastructure to speed up browsing, while Amazon Silk and Opera Mini do it through specialized web browsers.

Accelerated Mobile Project (AMP) is a recent effort started by Google with a similar goal of improving the mobile browsing experience. Announced in October 2015, AMP provides content creators with what is essentially a stripped-down and optimized version of standard web development tools. While there has been anecdotal evidence of its benefits, we are not aware of prior, independent efforts to quantify its impact on users' browsing QoE.

This paper presents the first characterization of the performance impact of AMP on user experience. We use a set of common metrics to characterized AMP's impact on QoE -- Page Load Time (PLT), Time to First Byte and SpeedIndex (SI) -- with a corpus of over 2,100 AMP webpages and their corresponding non-AMP counterparts, collected using trending keyword-based search results.

We show that AMP significantly improves SI, yielding a 60% faster SI on average, not accounting for prefetching, compared with non-AMP pages. The performance improvement comes primarily from AMP pages dramatically reducing the number of objects and DNS resolutions made, minimizing render blocking, as well as removing requests to third party sources that contribute significantly to latency on regular sites.

We find that the prefetching of AMP pages pushes this advantage even further, with prefetched pages loading over 1,100ms faster than non-prefetched AMP pages. This clear boost may come, however, at a non-negligible cost for users with limited data plans as it results in over 1.4 MB of additional data downloaded on average, unbeknownst to users.

You may want to look at our paper for the details!

People

Publications

Resources

Soon we will make the collected data, trest frameworks, corpus of AMP pages and measurement results available to the community here.