Game Analytics Reports

Context

I liked playing RTS games competitively as my main hobby, and even though StarCraft 2 was my primary game, I kept my eyes on other upcoming RTS games for a potential switch. Since I also was part of the community, I wanted to share my insights regarding interesting stuff with other people. This led me to establish a github.io space with the standalone interactive artifacts and reports I made.

What I Built

I've built a few public-facing reports, which were mostly HTML files with the D3.js viz library for charts. All of these efforts were based on the data I personally collected and processed via the RTS Data Manager project. Technologically, there were Python analytics pipelines and DuckDB carrying the data-engineering part. Only the resulting view datasets were exported to the web presentation where the visualisations utilized them.

The important part for me was that these were not just notebooks left on disk; they were meant as public, explorable artifacts for the communities around those games.

• AoE4 multiplayer quickmatch activity report — the first project was interested in the shape of player activity after the release of Age of Empires IV. The analytics dealt with player retention (new players vs players going inactive), which game mode people played first, how player populations of different game modes overlap, and how retention differs for people playing 1v1 exclusively, team games exclusively, or a mix of modes. Kaplan-Meier charts and set visualization were used in addition to other common visuals, because the main question was less “how many players are there?” and more “what shape does the player base decay and split into?”

• Stormgate 2024-02 Playtest Observations — this report was mostly focused on matchmaking quality, as Stormgate tried to have a unified matchmaking system instead of the usual split into multiple regions. I tried to observe custom quality metrics of MMR diff (estimated skill difference) and ping diff (disadvantage due to input delay), together with activity per MMR bands, to see whether the ladder could be considered healthy.

  Since the configuration of the matchmaking system was hidden, I also estimated the Elo system configuration and observed that it took 42 games on average to stabilize the in-game rating (MMR). The fun visual showed which players managed to play against each other based on their primary region, further showing that even worldwide, all players were within 2 degrees of separation (via Floyd-Warshall), thus making the rating system fair and not creating regional artifacts.

• Battle Aces July 2024 Playtest 1v1 Observations — this report was mostly concerned with unit pick rates in the deck-building stage of the game, and the daily trends throughout the short testing period. I tried to provide interactive visuals for users to explore the timelines and compare trends. I also looked at unit combos and not just individual units, that is looking at popular picks, what their prevalence is, and how they shape the meta.

Interesting Problems

• API issues — as usual in data analytics, data collection and data quality tend to make or break the analysis. Some noteworthy issues were that I didn't have complete player population information. With AoE4, only players with more than 10 games in a given mode are reachable. This made the numbers a bit more uncertain and several assumptions had to be disclosed, but it seemed that the trends remained visible even then. All the games had API endpoints available with a few days of delay after launch, so the early days cannot be fully analyzed.
• Having to utilize snapshot data — in most cases, I had to derive the activity information from comparing snapshots between days or hours. There is rarely an endpoint in games that gives the info that player X played a game at a time T1, T2 and T3. Instead I need to prepare a crawler, that gets the entire leaderboards and calculates which players have their games played stat increased. This somewhat limits the breadth of viable analytic questions and mostly I initially gathered the data and based on the available information I thought about how to utilize them for a meaningful analysis.
• How to provide interesting visuals and storytelling — the reports are text heavy, which makes them hard to sell on quick-dopamine platforms such as TikTok or Twitter, so I tried to think of a hook to get the readers interested in the story and spend their time on that. I tried to do that via unusual interactive visuals, which in my mind could satiate a novelty-seeking brain and from my experience people like to see stuff dynamically change on pages. This led to creating a visualization with playback, with on-hover applied filters and highlights or just a helper line that shows the data point at a given day.
• Inability to discern a user with multiple accounts — the analytics usually shows the player numbers, but that is also just an estimated fact. In reality people (especially the better players) tend to have multiple accounts for various reasons. The APIs don't provide any means to easily link the accounts to a single entity and any type of secondary analytics is impossible to do for all the accounts. If someone needs to investigate a player and find all their accounts, it's more of a social detective job with a bit of help via game replay analytics (to see if the playstyle matches) or via activity analytics (in most cases, it would be highly unusual to play games from the same account at the same time).

Related blog posts

The project pages show the final artifacts, while the posts below go more into the messy analytical bits: rating-system poking, snapshot-derived activity, deck parsing, and the visual decisions behind the reports.

• Analyzing StormGate 2024-02 playtest — deeper notes on Elo/MMR parameter estimation, rating convergence, matchmaking quality metrics, ping-based region inference, and the opponent-network analysis.
• Crafting of Battle Aces 2024-07 report — notes on inferring activity from rating changes, unpacking deck arrays into unit pick rates, wildcard/combo analysis, and a few dead-end analytics ideas like association rules.

Related projects

• RTS Data Manager — data collector, parser, cleaning, preprocessing