Managing the player data deluge

Let’s consider the established players. They have their own problem, one that they themselves create.

You see, the BattleCell map is big. Really big. It won’t take long for a new player to realize that they cannot singlehandedly conquer a map having 55 million cells. Cue the automation? Nope. Not even then. Much too big. Federations? Possibly, but no. They would have to be ridiculously large. But let’s not jump to the end-game just yet.

Our challenge is the information volume. The sheer number of timed actions that each established player initiates usually creates logistical tracking and organizational challenges. One’s daily routine demands a simple answer to a complex question. When and where does my next significant event occur?

Some actions do complete instantaneously. If they buy a cell, it’s instantly theirs. If they buy some troops, they are immediately available. In fact, if they buy most anything, a building, an empire upgrade, ships, stalkers, missiles, whatever… almost anything, it’s instantaneous.

On the other hand, most big ticket items require time to obtain. It takes a few days to build each missile silo. A city? Wait 6 hours, or build it yourself. Oh, and all movements require time to complete. Each player can have thousands of timed actions that all complete at different times, and at different coordinates throughout the world.

We organize each player’s in-progress actions into groups by both geographic location and completion time. If two troop groups arrive at the same destination, one in an hour and the other in 4 hours, then that is two events. Likewise, if two troop groups arrive at the same time, but different destinations, then that is also two events. The end result looks something like this:

TIME_ACTIONtab

But, I do want to say something about how we do it. It is the fun in what we do. We take all the events for each user and graph them in 3 dimensions: latitude, longitude and time. Then we perform hierarchical (agglomerative) clustering analysis on the dataset. We do this in Javascript, on the browser side, to avoid the computational hit to our servers. We also avoid impacting the player’s experience by running the analysis in a web worker.

Here is a nice Javascript library that does all the cluster analysis work for us: https://github.com/harthur/clusterfck

And here is a quick intro to web workers:
https://developer.mozilla.org/en-US/docs/Web/API/Web_Workers_API/Using_web_workers

Then we test by generating some 3 dimensional sample data. We “pre-grouped” our test data to simplify the process of confirming that the clustering analysis works as intended. For sanity checks, we verified our test data by graphing it in 3 dimensions. Not much to look at, but does provide instant data validation.

One massive persistent map

In 2008, I set it in my mind to create a single massive game map able to host hundreds of thousands of active players.  What I wanted, I had not seen before; and I wanted to build it as an AJAX website using the Google Maps API.  Cue June of 2008.

I expected the biggest challenge to be the system architecture. Boy was I wrong.

The real trick is balancing new player experience with established players in a long-lasting multi-year game.  Compounding that, BattleCell’s map can host well over 100,000 active players, all eager to pounce on those most vulnerable –the newbie who takes their time deciding whether they even want to join, years behind their biggest competitors.

I’ll discuss our experiences, challenges, and solutions as we work our way through this project.  This has been, and continues to be a learning experience.  Most solutions have not worked well.  Though, some have.

BattleCell is now in open beta.  You’re welcome to try it out. But the gameplay is not complete. We’ll be adding the finishing touches over the next few months, and we want to hear from you.  Give us your thoughts, suggestions, and concerns.

http://www.battlecell.com