Web performance matters, and on a mobile phone, it matters even more since a badly designed web site can be barely useable over a radio connection. Having to stare at a blank page because of a blocking script is a terrible experience.

Besides performance, the amount of transferred data also matters a lot, since cheap and really unlimited data plans are still pretty rare.

Having to wait and pay for non-visible content is irritating. Being forced to load something that could have been deferred is upsetting.

On a mobile phone, users don’t give a shit about how visually sophisticated a web site is, especially if that requires waiting for a shitload of images to load before any actual content can be read.

mod_pagespeed to the rescue

mod_pagespeed is not a magic wand, and it’s not a substitute for manual optimization. But it still does an amazing job at automatically improving the overall performance in most cases (which is, actually a pretty sad state of affairs).

The project is very active, with improvements and new exciting features being added every day. If you haven’t checked it out recently, you may be surprised at all the work that has been done since the first release.

mod_proxy is typically used as a reverse proxy. However, it can also be used as a forward proxy, improving the performance of web content before it is delivered to the client. Like, a mobile phone.

Setting up mod_pagespeed

A mod_pagespeed proxy designed to be used from a phone needs a public IP. So, go get yourself a cheap VPS. mod_pagespeed is fast, and doesn’t need much horsepower.

mod_pagespeed is currently designed to work as an Apache module (and it now works with Apache 2.4 if you like).

So, install Apache and follow the mod_pagespeed installation instructions.

Configure Apache’s proxy module to listen to your public IP address, and configure mod_pagespeed to accept any domains:

ModPagespeedDomain *

And test from a desktop browser that your proxy works as expected.

Now, go install a DNS cache. When using a proxy, don’t forget that the proxy is going to issue the DNS queries, not the client. This, also helps a lot in order to reduce latency. DNS lookups on a radio network are very expensive. And by following the best practices and using multiple names in order to parallelize downloads, a lot of web sites are actually slow like hell on a 3G network due to DNS lookups.

So, go install something like Unbound. Unbound is fast, supports DNSSEC (something that no mobile phones support yet), and can preemptively refresh a record before it expires. In a sad world of insanely low TTLs for DNS records, this is a more than welcome feature.

Here is a simple unbound.conf file to get you started:

server:
    auto-trust-anchor-file: "/var/lib/unbound/root.key"
    prefetch: yes
    prefetch-key: yes

Create the root.key file with:

unbound-anchor -a /var/lib/unbound/root.key

Start Unbound, update your /etc/resolv.conf file, and you’re good to go.

Want to block phishing web sites even when using a 3G connection? No problem, just tell Unbound to use OpenDNS as an upstream resolver. Ain’t it cool? Yet another thing that can’t be done without a proxy or a jailbroken phone.

How to configure a proxy on iPhone

How to set up a proxy for 3G and other radio (non-Wifi) connections on iOS?

Unfortunately, that can’t be done directly on the device. You need to prepare a profile, but this is actually dead simple, and it perfectly works on non-jailbroken devices.

The first thing you need is to get the APN settings for your carrier, and one of the best place to get this information is the Unlockit web site.

Then, install the iPhone Configuration Utility and create a new configuration profile.

Fill some random junk in the “General” section, and directly jump to the “APN” section.

Enter the APN, user name, password and proxy.

Export the profile (it doesn’t need to be signed), send it to yourself by email or download it from somewhere and open it from your phone.

Don’t forget to disable the Wifi in order to check that everything works as expected. If this is the case, you’re done. Mostly.

Wut? No security?

Here’s the catch: the proxy doesn’t have any kind of authentication. If you’ve been actually following all the previous steps before reading this: stop your proxy. Now. You’re running an open proxy, and within 5 minutes, spammers are going to take advantage of it.

Unfortunately, an APN doesn’t provide any way to authenticate a connection to a proxy. Even a cleartext, basic HTTP authentication is not an option. Damn.

I wrote a simple app that periodically pings the proxy and pokes a temporary hole in the firewall in order to allow the current phone IP address to connect to the proxy. Barely secure, especially since some carriers are NAT’ing 3G connections, but better that nothing.

A much, much much more secure way is to use a VPN, since an iOS VPN configuration not only allows specifying a proxy, but also its credentials. But setting up a VPN between iOS and a Linux/*BSD system happens to be a real pain in the ass (if anyone ever achieved to do that on OpenBSD, please drop me a line).

If you go the VPN route, never use a “transparent” proxy, or else DNS queries will still need to be made by the client.

mod_pagespeed configuration

mod_pagespeed has a fast release pace, but I highly recommend to use the latest and greatest code from the repository, which is usually very stable, thanks to an extensive test suite.

Here is my current filter set:

ModPagespeedRewriteLevel  CoreFilters
ModPagespeedEnableFilters rewrite_images

ModPagespeedEnableFilters combine_heads
ModPagespeedEnableFilters combine_javascript
ModPagespeedEnableFilters convert_jpeg_to_webp
ModPagespeedEnableFilters convert_png_to_jpeg
ModPagespeedEnableFilters inline_preview_images
ModPagespeedEnableFilters make_google_analytics_async
ModPagespeedEnableFilters move_css_above_scripts
ModPagespeedEnableFilters move_css_to_head
ModPagespeedEnableFilters prioritize_visible_content
ModPagespeedEnableFilters resize_mobile_images
ModPagespeedEnableFilters sprite_images

ModPagespeedEnableFilters defer_iframe
ModPagespeedEnableFilters defer_javascript
ModPagespeedEnableFilters detect_reflow_with_defer_js
ModPagespeedEnableFilters lazyload_images

ModPagespeedJpegRecompressionQuality 75

Update: Added convert_jpeg_to_webp and ModPagespeedJpegRecompressionQuality. Thanks to jmarantz@google for the feedback!

See the mod_pagespeed documentation or the source code to see what each of these are doing, but to sum up:

• javascript and css content is minified.
• images are recompressed, and, when a mobile user agent is detected, resized.
• cached (proxy-side) resources are served as cacheable (client-side) resources, as their URL are rewritten to include a hash of their content.
• whatever can be inlined (images, scripts, css) is inlined. This can significantly reduce the number of requests, which is always a big win on a high-latency radio connection.

The last set of filters is considered dangerous, but drastically improves performance and reduces the amount of data you need to download.

Consider a typical web site viewed on a mobile phone. Due to the small screen size, only a fraction of the content is visible at a given moment.

Why load (and pay for) non-visible content? Yes, it may become visible, if you decide to scroll down the page and go through everything. But this is not what the typical mobile user experience looks like.

People go back and forth between pages, and glancing at the first visible elements is enough to know if the content is what you’re looking for.

The lazyload_images filter lazily loads images as needed, ie. when they become actually visible. Recompressing an image is sweet, but not having to load it at the first place is obviously way better.

Practical gains

mod_pagespeed doesn’t make it easy to measure data savings, even though they are obvious and everything feels instantaneously more reactive.

So let’s take a look at some typical web sites.

http://www.over-blog.com, Edge connection

Waterfall - Waterfall with mod_pagespeed

Over-Blog have a version of their web site optimized for mobile. When a mobile user agent is detected, http://www.over-blog.com issues a 302 redirection to http://www.over-blog.com/m/ which is the mobile version.

I unintentionally included the redirection in this test, but it’s statistically insignificant.

mod_pagespeed does an amazing job at deferring scripts and images loading. A pretty decent demonstration that optimizing a web site for mobile devices takes more than making the content fit a small screen.

http://www.skyrock.mobi, Edge connection

Waterfall - Waterfall with mod_pagespeed

Yet another site explicitly designed for mobile devices. The home page only displays a header, and a form with 2 fields and 2 buttons. Oh, and a static banner ad, too. That shouldn’t take much data to transfer, right?

How come a simple page like this one requires so much data? What is not obvious is that scrolling down the page shows extra content below the ad banner. Links, images, and other non-essential content one wouldn’t expect on a login/signup page.

The mod_pagespeed optimized version lists 38 requests, but there’s a catch: half of them are actually inlined images. Because these images are small, mod_pagespeed didn’t defer their loading, but directly inlined the content instead.

Thumb up for using zepto.js. But thumb down for having to wait for 4 blocking scripts before anything can be rendered. mod_pagespeed fixes that easily and the content shows up after 1 second instead of 35 seconds.

http://cookingwithfrank.wordpress.com, 3G connection

Waterfall - Waterfall with mod_pagespeed

This is a standard wordpress.com blog. When accessed using a mobile device, the site directly displays the first articles. Because of the nature of the blog, there are a lot of photos. On a phone, it takes a lot of swipes to scroll down an entire page.

Ouch! Visiting only the first page of this site sucks 3.15 Mb out of your data plan. This is huge especially since all one can see without scrolling is plain, old, black and white text.

mod_pagespeed’s lazy image loader doesn’t load an image until it is actually visible. Hence the drop from 3.15 Mb to 340.16 Kb. Pay only for what you actually want to see.

340.16 Kb still seems huge considering the fact that no images are loaded. Blame backbone + underscore + jquery + mustache and a bunch of jquery modules. None, absolutely none of them is required to display this boring static page. But mod_pagespeed can hardly spot what is bloat and what is not.

http://www.opendns.com, 3G connection

Waterfall - Waterfall with mod_pagespeed

This site doesn’t claim to be optimized for mobile devices. It’s rather been designed for a wide viewport, and as a result, Mobile Safari rescales the whole page.

Unlike previous examples, almost all the content has to be visible, thus loaded and rendered.

And unlike previous examples, www.opendns.com uses sprites and a javascript loader. And images are already well compressed. Thumb up. Scripts and images are also served from a CDN. But unfortunately, the latency induced by DNS requests outweights the gain.

mod_pagespeed does a good job at minifying the CSS, but the gain is not stunning compared to the previous examples. mod_pagespeed does everything it can to speed things up without changing the layout.

So, is it worth doing?

Using a mod_pagespeed proxy can drastically improve your mobile experience, and save you a couple bucks by requiring less data transfers.

After using this setup for a while, going back to a non-proxied setup feels incredibly painful and sluggish.

Considering the fact that most “optimized for mobile devices” web sites are actually just considering the screen size without paying much attention to performance, mod_pagespeed makes wonders. And running it as a forward proxy for your phone brings you this experience for virtually any web site.

So, is it worth doing? The short answer is: yes.

If you already have a server / VPS with a public IP address, go do it now. If you don’t, consider the fact that you can probably get a cheaper data plan with the savings this setup offers.

Note: Vincent Bernat also blogged about this quite some times ago.