HTTP is great
First off I'll say hello, as that's traditional for new guys - I'm @robashton and I hail from codeofrob.com where I've written a few posts on WebGL and node.js since picking up those technologies, much to the confusion of my regular readers. I have sneaked in here to write for a (hopefully) more engaged audience on those subjects, lay down a few opinions and hopefully receive feedback from some of the minds in this space in order to further massage my brain waves.
Today's entry will be a quick introduction to HTTP in the context of content delivery in games development.
I am personally writing code for WebGL (which sits in the browser), but this is generic enough that it can be applied to any browser-based game - and in fact, I'd put forward that it is generic enough for most mobile games too if you lot would hold back on re-inventing the wheel once in a while ;-).
[Insert tech here] is just yet another way of writing cross platform code that runs slower than everything else.
Before launching off on one, let's understand why I ended up wanting to talk about this - one of the biggest questions I get asked by people is "What new do any of these browser based technologies really bring to the table?" (and why are you wasting your time with hovercraft when you could be making money?)
And they have a point, apart from the supposedly cross-platform nature of these technologies and having content delivery sorted out for you, there really isn't much differentiating them from say, cross platform C* with OpenGL.
So let's look at content delivery and see why this is a (nearly) automatically solved problem in our browser-based games.
*(if you can ever call it that)
Our old 'new' content delivery mechanism
HTTP is old and it was originally built to provide interaction with documents to various network clients. It is also much more than simply "GET/POST url host: example.com" (Although judging by most of the awful abstractions built around it in the past decade you'd be forgiven for thinking otherwise). It has built in support for caching, re-directs, updates, deletes and this has has been more widely realised of late - even in the largely enterprise IT world that I reside in (well, to an extent).
This greater realisation has brought with it a suite of fresh-faced, lightweight new web frameworks such as Sinatra, Node.js, etc, which rather than distort HTTP beyond recognition instead seek to provide a low level abstraction over the protocol itself, and augment that with helpful libraries and supporting code.
We also find ourselves with more developers potentially targeting the browser for their games with emerging technologies such as WebGL and Canvas coming into fruition, and with those games getting larger and more complex we're entering the territory where we need to know about content delivery and actually think about what is going on when we include files and textures into our games.
These two trends are rather supportive of each other as we'll see.
So, in games development, I have been led to believe that unless you're doing everything procedurally, that games require assets - and indeed without exaggerating too much, I think that most games require an awful mega huge amount of assets.
This is interesting, because you get an awful lot for free here, as browsers support (most) of the HTTP protocol and a good server supports more than enough of it to give you all the content delivery you need.
Here's the thing though, you don't have an awful lot of control over this process (sorry guys), compressing all of your assets to a single packed and compressed proprietary format and shipping it with your game via some magically efficient wizardry is quite impractical, although not entirely impossible.
What we're saying really, is that if you want to get content into your browser-based-game you're going to be downloading those assets via HTTP.
The content itself
There are only a few types of content you can download into your browser when creating a browser based game with the current technology stack and these take the form of:
- Models/data about your entities (JSON)
- Textures (Images)
- Sounds (WAV/whatever)
Taking advantage of HTTP
If you're writing a mobile game for any of the various mobile devices, you've probably hit the size limit for whatever application store you're deploying to and realised you're going to have to pull content down from a server somewhere (Or start to compress, compress, compress).
This is quite interesting, as there are many parallels to be drawn between delivering content for those games, and delivering content for our browser-based games - except that with browser based games we haven't got to do any of this work ourselves because the client and server protocols are not just defined but we're running in a sandbox which natively supports them.
So, for example:
I want to pull down resources I don't have yet
In our client's terms, this is an HTTP GET, and our server just needs to return the data requested - nothing more elaborate than that. This is the request the browser will make whenever you download the initial page hosting the game, include a script, attach an image to a resource or well, anything else in fact.
I don't want to pull down resources I already have, but I do want new versions of those resources
In simplest terms, the HTTP protocol supports this, you make a request for a document with a specific version/date/etag/whatever and the server returns "304 not modified" - job's a good'n.
I want to compress the data coming down
Surprise surprise, HTTP supports this too, and most browsers and servers implement gzip, which most clients will opt in to by sending "Accept-Encoding: gzip" to the sever on initial request. This means that you can tell your content providing server to compress content as it sends it, saving band-width, time and money.
Implications for browser based games
If you haven't got any multi-player and you don't need to generate anything dynamically, then you can just stick your assets behind a minimal web server, and serve a static page which downloads those static resources and allows the user to play the game at their leisure.
If you have multi-player, or some other need to run code server-side, then you can use something more dynamic for this - I am personally using paperboy for all my static files in node.js, and I make sure that when delivering dynamic content from the server (such as world-data that gets modified over time) that caching headers are set appropriately.
In a more intensive deployment I'd probably separate the two out, and have static files served via a CDN (Content distribution network), and dynamic files served from the node application. The technology itself is irrelevant as it's just an illustration.
You don't need to implement any client-side content caching yourself (I've seen people suggest using Local Storage for this - no, no and no), you don't need to implement any transfer protocols of your own and you don't need to get all elaborate with caching and e-tags - it's all right there for you in the way the browser talks to the web server.
Other (non browser and specifically mobile) games
Given all of this, is there any real reason to go and write heaps of code for content-provision in your average mobile game? I would put forth the answer 'no', because HTTP seems to do quite nicely for the rest of us and wait for the inevitable "But I really needed X or Y so I wrote it all myself".
Given the amount of off the shelf software available for doing HTTP both on the server and the client, it surely doesn't make financial sense to be spending any more time on this than absolutely necessary.
(That's Mike's "Challenge your ignorance" challenge undertaken then) - I genuinely would be interested to hear if there are any reasons why HTTP isn't good enough (raw performance is a hard sell, but feel free to try), as trying to come up with answers for any of these problems in a pure browser-based-world will be a useful exercise.
I am going to post the second half of this entry (I split it in two because it got too big), and talk about specific browser-based stuff - like strategies for loading resources, unloading resources and in general being a good citizen to both your users and to your bandwidth. I might even find time to talk about constant-loading/streaming worlds and how I think that works in a purely browser + standards based world.
Oh, and unlike this entry, there will be some code snippets.