Update 2017-01-28: I released a copy of the tools run your steam cache in this manner. Requires a Squid proxy, and a NodeJS application running somewhere.
While a DNS override is trivial to implement, it is not the only way to get a Steam cache working. DNS overrides also require mainteance - occasionally the list of hostnames used for content distribution changes, and that means Steam clients will bypass your cache if you don’t catch that.
How the Steam client discovers its CDN
The Steam client is configured with a number of Content Servers (CS) which
it bootstraps from. You can see the list in
%YourSteamDirectory%\config\config.vdf, and look for the
Each of the addresses in this semicolon separated list can be used to
The first entry in my list is
220.127.116.11:80. Before beginning a
download, Steam will make a request to an address like this:
http://18.104.22.168:80/serverlist/53/2/. This returns a server list.
The first number (53) is the download region ID,
Australia - VIC in
this example. The second number is the maximum number of servers to
Here’s an example response:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25
There are two entries returned, since we asked for two in the request. The
first entry is a
steamcontent.com is a new domain
that Valve has just started using in the last few weeks. Some of these are
owned by Valve, some are owned by ISPs. The distiction is irrelevant to us.
It is part of cell 52 which is the
Australia - NSW region.
The second entry is a
is a Content Distribution Network (CDN) which Valve uses to efficiently
distribute data around the globe without needing to deploy their own infrastructure.
Once the Steam client has a list of content servers, it makes an
initsession HTTP request, which is an authentication step and is not cached.
It then downloads a manfiest file
for each depot in the package. For example, if we were to download
Swordy, it has one depot.
All data for a depot lives in a folder beginning with
where the number is the depot ID - Swordy’s only one in this case.
The current manifest can be found at
The manifest lists all of the files for the depot. For example, one file
from the above Swordy depot is
Steam downloads and decrypts and/or decompresses, and boom - your game is downloaded.
Using this knowledge to run a cache
If you were paying attention, you may have noticed in the server list response
that each entry had two fields named
vhost, and that
set to an RFC1918 address,
10.0.0.194. This is not what Steam returned to us
- in fact, this is how we’ve been running our Steam cache since SteamPipe
was first implemented in late 2012.
All traffic leaving our LAN goes via a
transparent Squid proxy which does not itself do any
caching for Steam. Instead, we use Squid to rewrite the URL of requests that
match the regex of
/\/serverlist\/(\d+)\/(\d+)\/ and redirect them to our own
server list generator.
The rewriting is done using a url_rewrite_program. The URL of every request is passed to this program and the program returns either the original URL if it’s not a Steam serverlist request, or a URL that points to our cache if it is a serverlist.
Our own server list generator will receive the request. It takes an original
genuine server list from Valve and replaces the
Host field with the address
of our own cache, before returning that to the Steam client.
At present, it fetches the original upstream server list every
time, but it could also cache the lists as they do not change regularly.
vhost field remains in tact, the Steam client will then make a
HTTP request to our cache’s address, but it will include in the HTTP headers
a Host header, such as
Host: valve5.steamcontent.com. This way, our cache
server can easily identify where the request was meant to be sent to, so in
a cache miss situation we can forward the request on with no issues.
And there it is
And that’s how Steam downloads your games! This alternative method of caching games has some extra complexity and overhead - it requires running a transparent proxy and has more moving parts, but it means you do not need to keep a list of DNS entries up to date. Isn’t that lovely?
The scripts used to perform this magic haven’t been released yet, but will be “soon”. If you’d like a copy of them in their current state, get in touch via the links at the top of the page and we’ll organise something.