John McCutchan is a Software Engineer at Google focused on Dart and better games for the web. John started contributing to open source software in high school. He developed inotify- the Linux kernel file system event notification subsystem used by software in every Linux distribution while an undergraduate. After grad school, John joined Sony Computer Entertainment of America where he optimized the Bullet Physics library for the PlayStation 3. John developed Move.Me which gave hobbyists, students, and researchers an open source SDK for the PlayStation Move controller on their PCs. Move.Me is used for rehabilitative physical therapy, freestyle Jazz performances, and controlling a giant water painting robotic arm (amongst other things).
Posts by John McCutchan
  1. Bringing Regal OpenGL to Native Client ( Counting comments... )
  2. Bringing SIMD Accelerated Vector Math to the Web Through Dart ( Counting comments... )
  3. Controlling Your Game Engine Over WebSocket ( Counting comments... )
  4. Writing Your Own WebSocket Server ( Counting comments... )
  5. Intrusive Lists ( Counting comments... )
  6. A blob array ( Counting comments... )
  7. A Programmer and His Blobs ( Counting comments... )
Technology/ Code /

In my previous article I introduced WebSocket and detailed building your own WebSocket server. I also explained that I have embedded a WebSocket server into my engine and use a web application to control, configure and monitor my engine. Check out my previous article for some concrete examples of what I have been doing. In this article I will show you how I do this by explaining the remote procedure call system I designed and implemented.

RPC

What exactly is a remote procedure call system? Simply put, it’s a function call from one process to another. The processes do not have to be running on the same machine or share the same architecture. To help understand, first consider a local procedure call in C/C++. The function arguments are pushed on to the stack, then the program branches to the address of the function being called. When the called function returns the result is stored on the stack for the calling function to access. This works because the caller, callee and data live in the same address space and agree on calling conventions. In a remote procedure call system, the caller, callee and data do not share an address space and thus can not share data by memory address. In order to make a remote procedure call the parameters must be marshalled for transmission over the network and packed into a message for the remote system. The return value(s) from an RPC are returned in a similar way. Still with me? Don’t worry, I will not spend too much time in the plumbing for this article.

The first step in building a RPC system is to pick the data format that messages will be exchanged in. Since we are talking about the web, naturally, I chose the Java Script Object Notation, better known as JSON, as the data format for my RPC system. JSON is a simple, human readable data format. It supports numbers, strings, booleans, arrays and key value maps. Here is an example:

1
2
3
4
5
{
  "type" : "command"
  "command" : "Memory.Stats"
  "id" : "50"
}

Another benefit of JSON is that web browsers natively support it. Your browser can convert almost any Javascript data structure into JSON with a single function call. You can also go the other way, from JSON to a native Javascript data structure just as easily.

Now that the data interchange format has been settled, we need to come up with the RPC message framing. That is, the mandatory portion of the RPC message. Each RPC is a JSON map with two required keys, the message type and the message serial number.

1
2
3
4
{
  “type” : “”
  “id” : “”
}

The type field is used to indicate the type of message. My system supports three types, “command”, “result” and “report”. I will explain these shortly. But first, the id field is used to connect related messages together. For example, this command has an id of “4”:

1
{ “type” : “command”, “id” : 4,}

The result of the command sent back from the engine also has an id of “4” allowing to easily connect calls with their return values.

Getting back to the type field:

  • command” : Commands initiate some sort of action or state change.
  • result” : Result of a command. Linked by the id field. Commands are not required to reply with a result.
  • report” : A regular, repeating message from a subscription. Clients can subscribe to a data stream and set an interval between reports.

The other, non-mandatory, fields in a message are type specific.

Examples

Some example commands that my system supports:

  • “Echo” - Replies with a copy of the “message” field.
  • “Subscribe” - Subscribes the caller the “reporter” field.
  • “Unsubscribe” - Unsubscribes the caller from the “reporter” field.
  • “Memory.Stats” - Replies with the state of all registered allocators.
  • “Config.Set” - Sets “variable_name” to “variable_value”.
  • “Config.Delete” - Removes “variable_name” from the configuration variable set.
  • “Config.Get” - Returns the value of “variable_name”.
  • “Config.Dump” - Returns all registered configuration variables.
  • “Object.Create” - Creates a game object
  • “Object.Set” - Sets “property_name” to “value”. “render_model” is an example property.
  • “Message” - Sends a message to another connected client.

I have two reporters:

  • “Memory” - Regularly reports the state of all registered allocators.
  • “Config” - Sends configuration variable change notifications.

Here is a screenshot of a live graph of memory allocators:

Quake Style Console

What developer interface would be complete without a quake style console triggered by the ~ key? For this, I used JQuery Terminal. It is a great library which implements a command terminal. Adding custom commands to the terminal window is really simple. I have also made it possible for the user to construct messages by hand from within the terminal, making the terminal window very powerful. The following screenshot shows a user sending a custom Echo message by using the “dcc” terminal command. It also shows the response from the server.

C++ implementation notes

Connection Management

I started with the explicit goal of supporting multiple connected clients. My previous article discussed the importance of decoupling the code that waits for a connection over TCP from the code that manages a WebSocket connection. Because of that decoupling, supporting multiple connections was practically free.

Commanders

Support for commands is added by implementing the commander interface:

1
2
3
4
5
6
7
8
9
class doCommandCenterCommanderInterface {
public:
  doCommandCenterCommanderInterface() {};
  virtual ~doCommandCenterCommanderInterface() {};
 
  virtual const char* CommanderName() const = 0;
  virtual bool CanProcessCommand(const doCommandCenterCommandPacket* packet) const = 0;
  virtual void ProcessCommand(doCommandCenterConnection* connection, const doCommandCenterCommandPacket* packet) = 0;
};

A commander can process one or many commands (CanProcessCommand). Each commander is registered with the central command center, which does the routing of messages from connected clients to the appropriate commander. doCommandCenterCommandPacket just contains a parsed JSON object and doCommandCenterConnection has the WebSocket connection and various buffers in it.

Reporters

Support for reports is added by implementing the reporter interface:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
class doCommandCenterReporterInterface {
public:
  doCommandCenterReporterInterface();
  virtual ~doCommandCenterReporterInterface() {};
 
  void Subscribe(doCommandCenterConnection* connection);
  void Unsubscribe(doCommandCenterConnection* connection);
  bool ShouldRefresh(palTimerTick tick);
  void ChangeRefreshDelay(doCommandCenterConnection* connection, palTimerTick delay);
 
  virtual const char* ReporterName() const = 0;
  // Update internal state
  virtual void Refresh() = 0;
  // Report state to all subscribed connections
  virtual void Report() = 0;
};

Each reporter is responsible for generating a single type of report. Similar to commanders, reporters are registered in the central command center.

Client commands to subscribe and unsubscribe are processed by a commander, like all other commands.

Javascript implementation notes

Connection

My web application has a communication class ‘Connection’. The interesting methods are:

FireAndForget(command) - Send a command. No callback function is registered.
Fire(command, callback) - Send a command. Register a callback to be called when the result arrives.
Subscribe(reporter_name, report_callback) - Subscribes the client to the reporter. Registers a callback to be called whenever a report comes in.
Unsubscribe(reporter_name) - Unsubscribes the client to the reporter. Unregister report callback.

Message Processing

When a message arrives the behaviour depends on the type of message. If it is a result and the caller registered a callback, the message is passed to the callback and the callback is removed from the callback table. If it is a report the report callback is called.

Modules

My web application consists of many modules. Each module covers a set of related tasks. For example, the Memory module can query for all open allocations and updates a live graph of memory used by each allocator. Each of the modules registers subscriptions and command callbacks with the connection to update itself when new messages arrive. Each module is given an HTML

element to render itself into.

UI

As you have already guessed, the UI for my web application is written with the help of JQuery. A couples months ago I only had a vague idea of what JQuery is, but after using it I say- it is awesome and makes working with the DOM a cinch.

Why a web application?

There are many advantages to building development tools as a web application in concert with an actual instance of a game. First of all, deployment is trivial. Rolling out updated tools to the team happens transparently. Tools are platform independent running on Mac, Linux, Windows and even smart phones. Separating the tools UI from an actual game engine will force a less coupled design and require a clean design on how the tools communicate with the engine. Tools UI can be used with multiple games, so long as they agree on the RPC mechanism and a common set of commands. Forcing the tools to communicate with the engine over the network allows for a developer to connect to an instance of the engine from anywhere, view the output, and make changes at run-time. When working with console development, the game is not running on your computer and interacting with it may be cumbersome. Moving the UI into the web browser solves the problem. Imagine showing a colleague the latest feature you have implemented, but she is all the way across the building. With a remote viewer that runs in a web browser, she can just connect to the engine running on your development kit and see what is going on. Finally, as I have discovered, modern web development is a great platform for developing applications. Did you know that Chrome has a debugger, profiler and other tools built in? HTML5, WebGL, WebSocket, and Canvas make rich web applications possible. These are just some of the advantages to using a web application.

Conclusion

In order to control my game engine with a web application I chose to use WebSocket as the medium. I built an RPC system that supports commands and subscriptions to updates using JSON as the data interchange . I chose JSON because it is the native web data structure. The UI was built with JQuery and some off the shelf JQuery libraries. My engine offers a rich set of commands and allows for multiple simultaneous connections. I am very happy with this system because it is really easy to add new commands and reporters on the C++ side and Javascript is so easy it feels criminal. Next week I will be back with my final article in this series explaining how I was able to add simple streaming video to my game engine. Below is a screenshot of a screenshot of my game engine:

Don’t forget that I am giving a related talk at AltDevConf on Saturday, February 11th at 14:00 PST. I hope you check it out.