(editor's note: Usagi wrote 2 books, he is/was? known also as Taiji)
I am usagi. I am currently rated 1 dan on KGS. I live in China.
You should check out the Korean-Style Insei League. It is everything I had ever hoped to do with the old ASR League, and more. The Advanced Study Room still exists today (with different management) and is as popular as the day I left it, but I think that breakfast's system is a little more interesting to me right now.
- My Alexandre Dinerchtein number is 1.
- My tartrate number is 2 (through both solaris and breakfast).
- My Cho Hun-Hyun number is 2 (through Kim Jung-Jin).
- My Shusaku number is therefore 5 through various sources; additionally I have beaten Steve Fawthrop on 9x9 and 19x19 a few times, and his Shusaku number is 4.
- I will give an ounce of gold to the charity of the player's choice (or to the player) for a game which can confirm for me a Shusaku number of 4. I would appreciate if the game would be commented afterward but this is not a requirement. Please contact me here or on KGS, or in e-mail if you like.
I think this discussion belongs right here, because over the past 15 or so years no one has really taken anything I've said to heart about online ratings. Everyone admits there are serious problems. Among them..
- rank drift due to decay
- rank deviling (rank being affected by games others play after the fact)
- winning 20 games in a row and not getting promoted.
- taking 180 days for the system to promote you (i.e. weak 3d to weak 4d).
- if you lose the game after you get promoted it takes 3-4 wins in a row to get it back.
- blitz games being calculated at the same % as slow games causes the rating system to lose credibility
- the handicap system favors white by 0.5 stones
- games are worth more to new players than old ones
- having a blitz acct or starting a new acct is not a solution.
- it's also not even possible for a KGS plus member (admins will not transfer old lectures to new accounts)
and so on and so forth.
The basic premise seems acceptable; that there is some P such that Pw = 1-Pb. The first mistake which is made by KGS is to assign weights to games. This is a poorly thought out solution. Consider the matrix of possibilities:
- a game has a low weighting and is an important game for calculating the player's rating (bad idea to lower it's weighting)
- a game has a low weighting and is not an important game for calculating the player's rating (bad idea to give any weighting at all)
- a game has a high weighting and is an important game for calculating the player's rating (bad idea not to weight it fully)
- a game has a high weighting and is not an important game for calculating the player's rating (an obscure case I suppose, but assuming it, why is the game being weighted at all?)
In the above four cases, the only one that seems to work is the case where we assume that a game is important in calculating a player's rating. But then why does such a game decay over time? Either the game is important in deciding a players rating, or it is not. Games five months old to six months old account for approximately 10% of a player's result. This is enough to prevent promotion from one rank to another for as many as 150 days or more. This is ridiculous and undermines confidence in the ratings system.
Thus we discover the fundamental flaw of the KGS rating system: it continuously re-weights and then re-calculates it's players ratings.
No real world system operates this way. It is completely un-necessary. It is only being done this way because it CAN be done this way and for no other reason. There is no reason to apply Pw and Pb in this fashion. As a programmer - even as a Java programmer of considerable experience myself, I understand the desire to do things this way, but it is the sort of intuitive solution that doesn't make sense.
Essentially, once a game is used to calculate a player's rating it should never again be used to re-calculate that player's rating. Even from the perspective of good programming this is a horrible kludge of a "solution". Why not just do it the way every other rating system does in the world? Once players are rated, they stay at that rating until they complete the next rating period. In the real world this is a tournament, a league system, or whatever. There's no reason to assume this can't be adapted to online ratings. Just consider a player's last 20 to 50 games, calculate his rating, then let that rating sit until he's played another 20 or 50 or however many games. There's no need to continuously recalculate weights and games every five minutes. That can't be correct.
One easy way out is to keep the system the same but to cut the half-life in two and not keep games over three months. Three months is already a large amount of time, perhaps too large; but at least if a player focuses for a week and tries to promote he will have a decent shot at it, instead of the current system where players bust their butts winning 9 and losing 2, every day, only to not promote for a month or longer.
I've come to another conclusion. No one will ever make a serious change to the rating system if they haven't by now. Defenses such as 'it has been thought out extensively', or 'it has been serving the community well for decades', that 'it isn't intended to accurately rank players' are all at their core intellectually dishonest statements. There are serious problems here, with known solutions, that aren't getting implemented. A ratings system could be designed and tested in shadow to the existing live system and switched to without any end user being aware of a transition. There's not even any reason asking why this hasn't been fixed. We know why.
Even just tweaking the system so that people who are promoted a rank start at halfway through the rank they're promoted or demoted to, would solve an entire class of problems.
But hey, no one listened to Go Seigen over rule disputes, why should anyone listen to me?