KGS Escaper Design
The easy answer is "no" since this debate has been going on for so long, and we are all tired of it. At some point enough is enough, and you just gotta drop it and move on. However, I'd like suggest that in fact there is some room for improvement, and that due to the apparent increase in escaping behavior, it's time to take another look. Also, we have technology that did not exist before, which may be relevant. Clearly it is a nontrivial problem.
The problem is worth solving correctly, not only because escaping behavior pollutes the rating system, but because it makes the user experience unpleasant and leads to excess rambling in discussions (perhaps this article is case in point) :) . Merely making the system more strict will cause an increase in false positives (people with bad connections labeled as chronic escapers).
The key is to engineer a solution with careful consideration of all aspects of the problem. No solution is perfect, and for any solution, or component of a solution, there may be a rebuttal. When we play go, every move is a careful balance between conflicting goals, and the same is true when implementing a game server. With several years experience as a core developer on a major online chess server, I am somewhat familiar with the issues surrounding escapers. I suggest that the reader try to find a useful idea or two in all of this even if some of the specifics do not seem beneficial.
- Avoid negative effects of accidental disconnects on both players and protect the rating system from incorrect results due to false positives.
- Reduce negative effect of escaping on the remaining player.
- Minimize occurrence of escaping behavior through disincentive (without emphasis on punishment). Ideally, the disincentive should apply earlier instead of later, subject to other considerations. This goal is formally met if, in all cases, resigning is at least slightly favorable to escaping when the game is lost.
- If escaping is rewarded (regardless of the degree to which it is rewarded), the behavior will continue.
- Most players are honorable, so it is likely (though not certain) that the remaining player is honorable.
- Fails to distinguish between accidental disconnects and intentional escapes. Both are just as likely to be chronic, so both are equally capable of triggering the forfeit policy.
- Fails to provide any disincentive to escaping behavior. The forfeit is not a disincentive at all, since the game would otherwise have been lost anyway. In fact, escaping is rewarded if there is a possibility that some losses are forgiven as a result. (seehttp://senseis.xmp.net/?KGSIssueEscapers%2Fdiscussion#toc8 "Chronic Escaper Doesn't Care", From Marathon, July 31, 2009).
Also, a non-escaper with a flaky connection will tend to be unduly penalized by the policy. This also has a side effect of polluting the rating system with incorrect results. Inevitably, a player who accidentally loses connection frequently will trigger the escaper policy.
So, wouldn't it be nice if there was a good way to distinguish between escapers and accidental disconnecters? The current policy makes no such distinction (frequency is not a reliable correlate, since a bad connection is just as likely to be chronic as escaping behavior). There are at least two reasonable ways to do obtain useful correlates (both are probabilistic, but much better than just guessing).
- Get the opinion of the remaining player. The downside is that this player might be wrong, however it seems reasonable to expect a positive statistical correlation between the opinion and reality. Escaping behavior is often accompanied by significant circumstantial evidence such as a blunder followed by a requested for take back (denied) followed by sudden disappearance. Credibility metrics may be useful here.
- Use a bot to adjudicate the game. Not for an immediate rated result of course, but to provide a correlate for the escaping hypothesis. To be very clear, we do /not/ want rated game results based on the adjudication (the reasons should be obvious). The downside is that the bot might be wrong, however it seems reasonable to expect a positive statistical correlation, since it is unusual for an escaper to escape when ahead (though hypothetically possible in a handicap game that starts out poorly for the escaper).
Using this additional data, there are many possible ways to adjust the current policy to be more aggressive against escaping behavior while at the same time being less harmful against people with bad connections.
The purpose of weighting the rating of the game is to provide a mild incentive to resign instead of escape. A weight of 1.1 should be sufficient. Using a variable weight based on other factors such as credibility may be useful. One may be tempted to use a weight of less than 1.0 on the grounds that a disconnected game is less significant than a real completed game, but that would miss the whole point. The key is to make sure that the prospective escaper is motivated to resign rather than to escape. By making the incentive fairly mild, we protect innocent players from false positives.
After a disconnect, offer the remaining player the following options:
- Wait for player to return.
- Resign. (i.e. the remaining player is honorable and realized that the game is lost)
- Adjourn normally (resume when both players come back).
- Adjourn, and express the belief that opponent escaped.
If 3 is selected, nothing special happens (i.e. no escape points would be tallied, and the game will not be rated unless it is resumed). This is really very important because it will greatly reduce false positives in cases where there is no dispute. I believe this is the majority of cases.
If 4 is selected, a modern Go bot plays out the game in the background. If the result is against the accused player, the following applies:
- The remaining player's credibility increases in proportion to the exiting player's escaper score.
- The escaper score of the exiting player increases in proportion to the remaining player's credibility.
- If the escaper score exceeds a threshold, all adjourned games for that player are tallied as a loss for the escaper with a rating weight as a function of the escaper score and the credibility of each opponent.
Admittedly this puts a player who accidentally disconnects somewhat at the mercy of the remaining player. However, the remaining player is more likely honorable than not. Nevertheless, this suggests a policy to encourage good behavior on the part of the remaining player: If the remaining player selects 4 and the system concurs (disconnecting player is deemed a likely escaper either currently or retroactively in the future), the credibility of the remaining player increases. Otherwise it decreases. The credibility of the remaining player would included in the forfeit weight formula. So selecting 4 speculatively (when the opponent is not likely to be an escaper) is likely to result in a decrease in credibility for the remaining player, with no rating benefit, hence honesty and accuracy in selecting 4 is rewarded.
- Allows the remaining player to express an opinion. This helps bring closure to the abused victim of the horrible crime.
- Protects users with bad connections from false escaper label.
- Provides victim with some consolation in the form of increased weight of game rating if the game ends up tallied as an escape.
- Reduces frequency of escaping behavior by implementing mild disincentive
- Distorts the rating system.
- The incentive is unlikely to be great enough to address what is fundamentally a psychological issue with a subset of players.
- It is not shown that Robots are likely to give an accurate result for the majority of games, nor will Robots be able to take into account abusive or insulting behaviour by an opponent.
- Distortion is very minimal (maybe 10% increase in weight of game), and in any case this is certainly less distortion than the escaping behavior. It is sufficient that it will cause less distortion than the current system.
- The incentive is small, but what matters is that it exists. I believe that most escapers do so to gain a slight advantage. If this slight advantage becomes a slight disadvantage, the behavior will stop.
- The accuracy of the robot is not particularly important. The point is to have some correlation with the correct result. Please note: As stated above, I do not propose the adjudication result to be used directly as the game result. Instead the adjudication result is meant to correlate with escaping behavior. The vast majority of escaped games will certainly be evaluated against the escaper, and therefore is a good statistical correlate for identifying escaper behavior. It is sufficient that it is a better correlate than none.