jasiek: Too Great Influence of a Player's Game History (2006-11-27 07:18) [#2446]

On 2006-11-27, the previous discussion is moved from the parent page here while the parent page shall get a WME for this topic.

• Also see Forum for CGoban 3 / Experience with new rating system.

• cocoon? When you have a lot of game on your account, the rating changes too slowly. Why consider the games played during the last 160 days ? if someone plays regularly, his rating will inevitably get stucked. It would be better imo if the limit for a game to be taken into account wasn't the number of days since it was played, but instead simply the number of games played after that one : for example only the 160 previous games of a player would count for his rating. Then ppl would n't change they account so often. +?

• Iago I'd like to second cocoon on this, if you played a lot of rated game, your ranking can get stuck which does not seems logical. why not keep the "160 days" limit, but also add a "last 25/100 games" limit. this way ranking would not get stuck. hope this has not yet been answered elsewhere... couldnt find it.

• anonymousPerhaps game history could "decay" by days AND/OR number of games played, whichever happens faster. And there ought to be a way to reset your rank (start afresh as "?" without being forced to change your account name). Now people are registering several accounts just to do this - and if you're a KGS Plus subscriber, you can't just dump the old account either.
  • Phelan: I like the idea about resetting the rank. It seems to solve the problems I read about here, although I'm not sure how much it would mess with the rating system already in place

• Mef: Just a couple comments - First, the weight games have on your rank does have a decay factor in it so that older games count less than newer ones. Second, the reason it is easy before your rank to become "stuck" is because let's say you've played 200 games as a 10k, and those 200 games have anchored you at mid-10k, you now go on a 15 game winning streak, which is more likely, you've gone up in rank? Or are you just having a good day? Winning streaks are not uncommon against people of a like rank, and I hardly think the results of a few recent games should overrule the findings of an entire history. There are several threads on RGG where someone did the math of how often winning streaks should occur so I won't repeat it all here, but let's just say they're much more common than people would like to think. Besides, if all else fails, you can challenge people to games at the rank you think you are, and the server will compensate for that.
  • Aimless?: Regardless of what the statistics say about the probability and length of winning strengths (on which I agree with you completely), the ratings on KGS are generally less volatile than they should be. Players' strengths may vary as much as two stones (for the SDK range; more in the lower kyus, less in the dan range) depending on whether or not they are having a good day. The rating system should be capable of reflecting that. Even if a 10 game winning streak is nothing more than a statistical outlier out of a sample of 200 games, it should be rewarded. Likewise, a 10 game losing streak should be demerited. If it is an actual long term change in player strength, then they'll keep winning at their new rank. If not, then they will eventually regress to the mean, and all will be well with the world. So why not let someone feel good about winning a few in a row by giving them a promotion and a chance to prove they can hack it?

• Iago : Well i understand what you are saying about winning streak, but isnt there a limit to how solid we want a rank to become ? some people play 10 rated games a day, so... we are talking around 2000 games, 500 of which could have been 3months away, when he was 3k weaker... I still think a reasonable limit on the number of game taken into account should be used.

• Mef: Someone may want to double check my math but I think It's at least close: Let's assume you play 10 games a day. That's 1825 games in 6 months. Worst case scenario (mainly for upper ranks) the halflife for a game's weight is 45 days. The first half of the games you've played (i.e. the 913 games you played 3 - 6 months ago) account for 1/5 of your rank. games that came in the next 2 months (608 games) will account for another 2/5 of your rank, and the last 2/5 of your rank will be based on games in the most recent month (304 games). That sounds pretty reasonable and it's a worst case, for lower ranks the halflife is shorter so the more recent games would have even more weight. Best case scenario, The most recent month would account for more than 3/4 of your rank.

• Iago: I agree with the math but i feel we are not talking about the same thing... what if we have someone who play 15games a day ? or 20 ? my point is, your rank may not be stable for 200 games (especially for low/mid kyu). or maybe it's 1000 or... but adding a limit on the number of games taken into account would make the ranking system more reasonable. why let a player who is very motivated (he played 15 games a day for month) and who got from 20k to 10k in 2 months get stuck at 10k because he lost 500 games when he was 20k ? with time, when the game disapear due to timout, he will skyrocket in rankings, but in the meanwhile, he is stuck to be a sandbagger. or urged to make a new account.

• Bottomline is : people DO get stuck, and make new account to bypass it. so i feel this is a problem. please let me know if i misunderstood.

• Mef: From my experience, weaker kyu ranks are usually rather volitaile, and the main problem is people who think a winning streak constitutes them being grossly underranked. Also alot of people like to cite stories from back when the ranking system was admittedly flawed and no one's rank could increase, but this has been fixed now. With only a few exceptions, most of the people I've seen do the new account thing may have gotten a nice jump at first, but usually settled back down to where they would've normally been (The exceptions being people who hadn't been playing regularly rated games, and they knew they were sandbagging because they hadn't let their rating keep up.). If all else fails, like I mentioned earlier, you can play games at the rank you think you are, and they will be scored as such, and they can help you correct your rank faster, and always remember: patience is a virtue, and go is about fun,not rank.

• GammaTau?: Yes, people do get stuck and this is a real problem in KGS. In this context, it is important to understand what "getting stuck" actually means and what is the fundamental problem. In my opinion, the fundamental problem is that the more games you have in your history, the more games you need to win to raise your rank. The most important and perhaps the most annoying direct consequence is that if you have been an active player, you need to play many games slightly underranked in order to reach the proper rating. Once the number of the games you need to play as an underranked player is more than you are willing to play, it is fair to say that the rating has "got stuck". It's then only natural and reasonable to make a new account. It doesn't require dozens of games and you don't have to play underranked when you don't want to.

• GammaTau?: I'm not a mathematician so it's not easy for me to make the right formalism for describing the systematic error in the system but I'll try. Let's say that there is a player who plays on a 13 kyu level in January, 12 kyu in February, 10 kyu in March and 9 kyu in April. When the rating is calculated for May, the data from January has become basically worthless. It doesn't represent the true strength of the player and thus it should be discarded entirely. The KGS algorithm has a method for discarding outdated data, that is, it puts less weight on older games and more weight on new ones. But this method is a hack and it has obvious flaws. The weight of old data should be determined by comparing it with recent data. If old data and new data give a different representation of player's strenght, the old data should not be used to measure player's strength. On the other hand, if the old data and new data give the same representation of strength, it's perfectly fine to use it. Basically, the time it takes for old games to expire should ideally not depend on time alone but also by calculating how well the old data fits with new data. If you are gaining strength by studying go and you have gained one or two stones in strength, you can't win 100% of your games against people who play with the player's old rating. Even 70% might be optimistic. It is extremely frustrating to continue with above-50% win rate when the rating is basically stuck. Constant above-50% win rate should be enough for raising the rating and this should not take 50 or 100 games when you have hundreds of games in your history.

• Iago: Well, this is about as far as i will go, my ranking is not stuck, as i hardly play 2 or 3 rated games a month. I still think that making tons of new account could be avoided by adding a limit ot the number of games, but if no one else feels that way then... I would appreciate to have wms point of view sometime, but nothing urgent i guess.

• EdwardHammerbeck: I'm much less concerned about what my rank is than that pesky question mark. If I feel like I am a 25k, I will challenge a 25k to an even game and settle the question like a gentleman, but if I have a question mark, s/he might not even respond. A win against a 25k? or whatever hardly counts as a win at all. I have something like sixty rated games with about a 20%-80% win-loss ratio. I'm 27k? last time I checked. My problem is that I don't play that often. Needless to say I think the time factor should be scrapped.

• cyberain?: I think that rating is growing too slow... For example, look guys how many games this user win and when I'm writing this words he's still 15k... http://www.gokgs.com/gameArchives.jsp?user=keiru
  • You may well have a point. There are 4 ways around this.
    1 Change handicaps so you are 14k
    2 play very few rated games indeed to make your rating lighter
    3 Make a new account
    4 get a picture

• Harleqin: Let the games' weights decay for a player not only by time, but also by number of games played since by that player. This reduces the problem that it is more difficult to change their rank for players who have played a lot of games. There has been a discussion at the forum. I propose the additional factor X/(X+games) to the weight, where X would be an analogon to the halflife by time. +
  • wms: A game's weight must be the same for both players in the game. Otherwise the system becomes unstable. It would have to be games played by either player, which would make it less meaningful. Furthermore, I don't see any advantage of this over simply hastening the decay rate of a game (if what you really want is to speed up the rate of change in people's rank).
  • Harleqin: If your first statement is true, then of course this proposal makes no sense (replace "furthermore" by "then"). However, as far as I know, there is already an assymetric weight factor in use: the certainty of the opponent's rank. Can you elaborate on how the system would become unstable and how this differs from the other assymetric factor?

• What I really want is more independence of the possible rank change rate from the number of games a player has played in the past. The numbers would have to be tweaked to give a sensible result, e.g. a time halflife of 90 days and game halflife of 60 games would (rough estimate) result in a player with about 5 games per week not feeling much change. It should be possible to adjust both halflifes in such a way that the possible rank change rate is more or less independent from a player's past playing rate.