Quality of Tie breakers

A lot of discussion has been going on at Sensei's Library (and other places) on the relative merits of several tie breakers, such as SOS, (Iterative) Direct Comparison, SODOS, Prior rating, and many others. Not much attention has been payed to what makes a tie breaker "good" or "fair" however.

There is some general agreement on the following:

• Breaking ties is often not necessary.
• A good way to break ties is by playing a play-off.
• More rounds make for a more reliable ranking.

However, it is quite often infeasible to organize extra rounds due to time constraints.

What is the purpose of tournaments?

For most players, the purpose of a tournament is to have fun. In the context of this discussion, however, that is not relevant. In general we can say that the relevant purposes of a tournament are:

1. To find the strongest player
2. To rank the players by playing strength.

In this context, we take strongest player and higher playing strength to mean "performed better than others in this tournament". This is not necessarily the player that is generally the strongest, as performance in a tournament may depend on many additional factors, such as time limits, player well-being, or playing conditions (noise level, lighting, etc).

For finding the player that is generally the strongest we use rating systems.

Depending on the relative importance of these two purposes, different tournament systems are preferred. Single Elimination (knock-out) tournaments can achieve only purpose 1. Double Elimination can achieve purpose 2 as well, but for a limited set of players, McMahon/Swiss can achieve purpose two for all players.

How does a tournament achieve its purpose?

If we wanted to know, for each pair of players, which one is the strongest, then the best way to do it is to have each pair of players play one or more games. This is what is know as round robin. Usually, there is not enough time to do this however, as the number of games to be played grows rapidly with the number of players.

Several tournament formats use several different methods to overcome this obstacle. Tournament formats other than round robin try to achieve their purpose by only taking a limited number of games and using those to calculate an overall ranking. It is generally agreed that playing more games (taking a large sample of games) will improve the performance of most tournament systems. Some systems, such as Swiss or McMahon (as well as Round Robin), allow ties. Other systems, such as Knockout or Double Elimination, do not allow ties.

Some systems, such as McMahon or multi stage seeded Knockout use the prior rating of the player and considers it as a valid component of the playing strength (in the context of the tournament), other systems such as Swiss or Round Robin do not take prior rating into account.

It is up to tournament organizers to choose a system that they feel best represents playing strength in the context that they want to measure it.

Why break ties?

Breaking ties is an extension of the purpose of the tournament. To find the strongest player and to rank the players by performance strength. If the answer "these players are equally strong" is not acceptable in certain circumstances (single plane ticket to be won, for example), then ties need to be broken.

What makes a final placement tie breaker good?

From the above, we can formulate a number of proposals on what makes a tie breaker good:

A tie breaker can be good if:

1. It is likely to predict the result of a play-off
2. It is likely to predict the result of the tournament when one or more additional rounds are played.^[1]
3. It is likely to fully or partially order tied players by their playing strength.
4. It is easy to understand and calculate.

(here likely is used relatively, as in "more likely than a pure lottery", and is not meant to reflect absolute likelihood)

These can be considered separately, but can also be combined.

Given a choice of the above proposals, we can compare tie breaking systems by saying that system A is better than system B if system A is more likely than system B to perform the given task.

Other factors to consider

One must consider for which purpose a tiebreaker is being used: for ordering final places, for determining a round's pairings, for seeding to a next stage of a or another tournament

Tiebreakers behave differently for different purposes of usage, in different tournament systems, for different numbers of rounds, different numbers of players etc. It can be that a particular tiebreaker that is relatively good under some circumstances of usage is relatively bad under other circumstances.

The proposals in the sections above are based on the assumption that the tournament in question is an honest sampling of games. This is not always the case however. Although it is rare, cheating can happen. It is an additional quality of a tie breaker if it is not vulnerable to cheating (eg: by collusion between players). Examples of cheating:

• A player might deliberately lose in order to improve the SOS of a friend (who played against the opponent) or decrease the SOS of an enemy (whom they played).
• A player might deliberately lose in order to decrease the Direct Comparison value of an enemy.^[2]
• A group of players might submit a faked event to the rating list to improve the prior rating of one of the players.

In Swiss and MacMahon systems, collusion can also occur when one of the schemers is paired down against another: the lower placed player can intentionally lose to improve the chances of the other. Therefore, in these tournament systems, collusion must anyways be discouraged by a means other than the tie breakers, so this property is not so important as it might sound at first.

[1]: This is not the same as a play-off, as a play-off allows that players are paired against each other again, whereas additional rounds will be paired normally, and thus avoid pairing the tied players against each other unless they have not yet played.

[2]: If three players A, B and C are tied for first place with 3 points after 4 rounds in a 5 round tournament (quite possible if the top group is around 8 players), and they have already played all matches amongst themselves (winning in a circle with A>B, B>C, C>A), then they will all be paired down. If B and C are friends, B might lose on purpose to lower A's DC value, thus giving C tournament victory (provided A and C both win the last round).