Managing the player data deluge

Let’s consider the established players. They have their own problem, one that they themselves create.

You see, the BattleCell map is big. Really big. It won’t take long for a new player to realize that they cannot singlehandedly conquer a map having 55 million cells. Cue the automation? Nope. Not even then. Much too big. Federations? Possibly, but no. They would have to be ridiculously large. But let’s not jump to the end-game just yet.

Our challenge is the information volume. The sheer number of timed actions that each established player initiates usually creates logistical tracking and organizational challenges. One’s daily routine demands a simple answer to a complex question. When and where does my next significant event occur?

Some actions do complete instantaneously. If they buy a cell, it’s instantly theirs. If they buy some troops, they are immediately available. In fact, if they buy most anything, a building, an empire upgrade, ships, stalkers, missiles, whatever… almost anything, it’s instantaneous.

On the other hand, most big ticket items require time to obtain. It takes a few days to build each missile silo. A city? Wait 6 hours, or build it yourself. Oh, and all movements require time to complete. Each player can have thousands of timed actions that all complete at different times, and at different coordinates throughout the world.

We organize each player’s in-progress actions into groups by both geographic location and completion time. If two troop groups arrive at the same destination, one in an hour and the other in 4 hours, then that is two events. Likewise, if two troop groups arrive at the same time, but different destinations, then that is also two events. The end result looks something like this:


But, I do want to say something about how we do it. It is the fun in what we do. We take all the events for each user and graph them in 3 dimensions: latitude, longitude and time. Then we perform hierarchical (agglomerative) clustering analysis on the dataset. We do this in Javascript, on the browser side, to avoid the computational hit to our servers. We also avoid impacting the player’s experience by running the analysis in a web worker.

Here is a nice Javascript library that does all the cluster analysis work for us:

And here is a quick intro to web workers:

Then we test by generating some 3 dimensional sample data. We “pre-grouped” our test data to simplify the process of confirming that the clustering analysis works as intended. For sanity checks, we verified our test data by graphing it in 3 dimensions. Not much to look at, but does provide instant data validation.