Tournament Format Simulator

Context

I've played in quite a few tournaments (mostly RTS games) and have also done some work from the other side as the tournament organizer. Over time, I've noticed that each tournament system and game plan has some pros and cons, but there is rarely any debate on which system would strike an optimal balance between them, and the decision to use a particular structure is usually not communicated to the players and viewers. I suspect some (usually amateur or community) organizers don't even think about the options and just pick a format they have seen in another competition recently. This lack of public debate also somewhat stunts the development of the community's understanding of this issue, which makes advocating for a well-meaning change a messy struggle.

With this project, I hoped to create a tool or exploratory study material that could be used to demonstrate the concept to the general public and explain the benefits and issues of particular systems, partly borrowing from the game theory field and my competitive experience. This way, I could show a tournament host that choosing a single-elimination bracket for a fun community tournament is a poor choice, as half the players will only play a single match and the match will usually be imbalanced due to how seeding works.

What I Built

This project idea has been iterated repeatedly over the years. It started as a project with a group of students in 2017, which got stalled and didn't create any meaningful artifact. The next attempt was around 2021 as a solo project, which also got shelved before being publicly presentable. The current iteration started in 2026, and it was initially meant as some sort of benchmark and exploration of an agentic coding approach.

The core of the project is a runnable experiment framework in which match resolution, player lineups, competition structure, and observable metrics are configurable inputs. The experiment artifacts and results are shown via a web-based dashboard.

It is currently in MVP shape, with round-robin and single-elimination systems implemented, manual player seeding, game resolution based on Elo-logistic probabilities, and six derived analytical metrics. The metrics are win probabilities (empirical frequency across repeated tournaments), rank distribution (as a per-player histogram), average number of matches played, an excitement proxy metric (the mean absolute Elo difference between opponents—a smaller difference is assumed to indicate a more exciting match), pair-order agreement of final rankings, and mean rank shift against expected Elo order. Experiments are run as batch jobs that can be set up and monitored via a web interface.

Future Plans

I have quite a few ideas on where to go with this project so it has some value.

• Support for additional tournament formats and their chaining — primarily, I want to model the commonly used combination of a group stage followed by an elimination-bracket playoff, as that is a widespread format that can serve as a baseline. Following that, I'd love to add the custom formats I used in my experimental tournaments (as discussed in SC2 Tour System).
• Prepare curated experiment runs to demonstrate classic pitfalls of given formats.
• Improve the dashboard view to easily showcase performance of the same player cohort in different systems, with the possibility of exploring the individual player runs.
• Collaborate with community tournament admins to get ideas about new useful metrics and empirically validate their ideas and hypotheses about experimental tournament systems.
• Utilize public progaming statistics and prediction models (such as Aligulac) to better simulate the outcomes and provide concrete what-if scenarios based on players people recognize.

Interesting Problem

One of the interesting issues that arose due to the framework still being in its early stages is the limited number of options for tie-breaker resolution in the round-robin format. Traditionally, there are secondary scoring metrics such as sets won, head-to-head performance, extra matches played, or initial seeding. To simplify, I went with the last one, as that was the most straightforward to implement (we already had the seeding number of each player, so we could resolve the ties this way). This led to a circular dependency on the players' Elo ratings, which were used both for seeding and for the expected final ranking of the tournament. Thus, round-robin tournaments have, by design, a natural bias towards better agreement with the expected truth. This is something that will go away with further work, but it felt like a nice nit-picky example of how simplification can lead to weird biases in experiments. To further illustrate, in the initial experiment for a four-player tournament, this showed up as 87.2% (round-robin) vs 78.3% (single elimination) pair agreement, with a non-trivial contribution from this design artifact to the higher metric. The contribution from the Elo tie-breaker was approximately 8.5 percentage points, which would put it at a value similar to that of single elimination.

Related Projects

SC2 Tour System was my playground to experiment with tournament system design and seek formats that would be fun not only for the top players, but also for the casuals.

Fotbal Nadstavba Sim was an analytical project examining the suggested structure of the Czech football league and challenging the hypothesis that teams would not be incentivised to throw matches to get a paradoxically better result by finishing in a worse spot.