How a CDN works

“Downloads from Apple are excruciatingly slow”. “Don’t use Google DNS or OpenDNS, or else iTunes will be slow” “LOL, OpenDNS claims it makes the internet faster. Yeah, by telling Akamai you are in the US and getting 200K/sec downloads in Oz…”

Fact: these claims are true.

Apple uses Akamai, a popular CDN. The whole point of a CDN is to make your downloads faster: CDNs have servers at multiple locations, and when you request some content, CDNs are supposed to serve it from the closest (and hopefully fastest) location.

There are different ways to achieve this, but a very common one, that Akamai also relies on, is to leverage the DNS system.

When your client wants to resolve www.apple.com, Akamai DNS servers are going to give different replies according to the source IP address.

But what source IP address? Yours? Not always. What Akamai consider is actually the IP address of the DNS resolver hitting their servers. If you’re using a local resolver, that’s cool. Akamai will know your exact IP address, and hopefully redirect you to the closest server.

If you’re using your ISP’s resolver, this is the IP Akamai will consider in order to pick the server that will process your query. The result may be the same as if you had used a local resolver. Or not. If the DNS resolver is on a different subnet, Akamai can get confused and you can be redirected to a server that actually is way off base.

If you’re using OpenDNS, Google DNS, Norton, Level 3, any other public DNS service, or any other remote DNS resolver (for example, through a VPN connection), Akamai will see the source IP address of your remote resolver, too. Nor yours.

And the server they will redirect you to would probably be an excellent choice if you were Google or OpenDNS. But the very same server can be a very poor choice for you.

From the same origin, packets can take a totally different route in order to reach Google, OpenDNS and Akamai. DNS queries sent to Google or OpenDNS from Paris to Amsterdam can be super fast. But then, downloads from Paris to an Akamai server in Amsterdam can be super slow. Just because the routes can be totally different.

What about other CDNs?

How come this problem occurs with Akamai but not with other CDNs?

Some other major CDNs don’t have this problem because instead of considering the resolver IP address, they actually pay attention to the actual client network, in order to pick the closest/fastest server.

OpenDNS and Google DNS have been supporting the edns-client-subnet extension for a long time. This mechanism was designed by Google specifically to address this problem. And it works beautifully. CDNs can send a redirection to the best server no matter what resolver you use.

Unfortunately, Akamai still don’t support this mechanism.

Make an exception

It’s totally possible to use Google, OpenDNS or any other public DNS service and still avoid being redirected to a slow server by Akamai.

Just make an exception.

If you happen to run OSX, your operating system already provides a way to use specific DNS resolvers in order to resolve specific domains.

If you want queries to *.apple.com to go to your ISP’s resolvers instead of Google, OpenDNS, or any other default resolver, just create a file named /etc/resolver/apple.com with the following content:

nameserver a.b.c.d
nameserver e.f.g.h

Replace a.b.c.d and e.f.g.h with your ISP’s resolvers. You can have as many resolvers as you want, and as many exceptions as you want. Just create files as needed. One per domain name.

If you’re running another operating system, you should be running a local DNS cache. The Windows DNS cache is a total joke and most Unix-like systems don’t provide any cache at all.

Running a cache like Unbound is the best way to reduce latency due to DNS queries, even if queries are forwarded to Google or OpenDNS. In addition to caching queries, modern resolvers have nice features like the ability to automatically prefetch records before they expire. For CDNs and social networks using stupidily low TTLs, it can really make a difference.

Once you are running a cache, and it has been configured to forward queries to OpenDNS, Google, or anything else, you can configure exceptions.

With Unbound, it’s as simple as adding two lines to the configuration file:

forward-zone:
    name: "apple.com"
    forward-addr: a.b.c.d
    forward-addr: e.f.g.h

The edns-client-subnet mechanism has been around for quite some time now, so all modern CDNs are eventually going to support it, and manually adding exceptions won’t be required any more.

But as we speak, this is still required, at least for Akamai.

Fortunately, adding exceptions is easy. So you can totally work around CDNs that don’t support a DNS mechanism that was designed for them, and still enjoy the benefits of fast and reliable DNS services in order to access content served by other services.