Stormgate Replay Collector

Context

Stormgate was an RTS game with big ambitions to be the next big thing, but those ambitions were unfortunately not fulfilled. One of the areas where it was lacking was support for competitive players—the in-game leaderboard was offloaded to a third-party provider and rendered in the game via a website renderer, for example. Similarly, replay sharing and acquisition were pain points. I tried to gain a slight advantage (and save time) by smartly utilizing existing API tools from the external provider.

What I Built

I've built a web dashboard that enables efficient lookup of public replay files. With the default Untapped.gg setup, a user needs to access a specific player's profile page, hope they have the match history and replays set as public, find an interesting match, and download the file. I decided to create a Python crawler, which needed to do several things:

• check access to recent match history;
• store matches and discover unseen opponents;
• fetch explicit history/replay visibility;
• download eligible replays and retain local copies.

There are plenty of constraints and annoyances along the way, which make the process non-trivial. Those constraints turned the crawler into a stateful update process rather than a one-off scraper. It stores visibility states and check times in SQLite, refreshes public and private profiles on different schedules, and prunes replay requests when a permission result makes further attempts pointless.

The web dashboard provides two views. One shows players with public match histories and their ranks on the ladder, so users can more easily pick their favorite player and gather their replays. Showing rank on the ladder is important: from the last top 500 snapshot (2026-04-30), only 76 rows had public match histories and 58 had public replays, so even though someone can appear to be high in the abridged table, they might only do so because better players are hiding their information. The view can be filtered by faction and activity (time since the last game played).

The other view is a chronological list of all known public replays. Yet again, it has filters per faction, so the user can find only the subset relevant to them. Other filters are the map, difference in skill between players, and game length, so the user can filter out imbalanced matches that would be hard to learn something from, or seek replays showing quick aggression leading to the game ending in the early game.

Unfortunately, the game is dead, so the utility of this has come to an end; however, I believe this was a much-needed aggregate page for the community to better engage with the competitive aspect of the game.

Interesting Problems

Discovering active accounts outside the leaderboard

The only straightforward player listing is the top 500 leaderboard. However, that is sorted by ever-growing points instead of MMR (an Elo-based rating), meaning that new accounts need to play quite a lot of games (depending on when the last reset happened) to appear in there.

We can try to discover other accounts in various ways. Remembering them from previous leaderboard seasons and snapshots is the straightforward solution, which doesn't yield all the accounts, though. Another option is to look at individual player profiles that have stats including frequent opponents (this part cannot be made private), so we can learn other player accounts by crawling through these. Another option is to get the player IDs from individual players' match histories, if public. It's impossible to do the full scan every day, though, because there are about 50k known accounts (based on the exhaustive search of all historical leaderboard snapshots taken via RTS Data Manager) and only 20 requests per minute are allowed on the API.

Which players have public histories/replays

An additional annoyance is that the information about replay availability and match history privacy is hidden behind an extra API call and cannot be gained via the profile scan call used to find other accounts. The interesting logic is that at least one player needs to have their replays public for the replays to be downloadable. A successful replay download proves that at least one participant exposes replays; it does not guarantee that both do. A 403 means both participants keep their replays private.

Technically, it is possible for some replays from a player's match history to be public while others are hidden, which is something that once again needs to be "brute-forced." Since the visibility can change at a player's whim, I decided to store local copies of the replays and try fetching the previously unavailable ones from time to time.

Related projects

• RTS Data Manager, which collects snapshots of various RTS games and was partly used to establish the list of known accounts in Stormgate.