Internet Ratings Database

Note: This page details the concept of an Internet Ratings Database. This concept has not be realized as of January 2009.

We have a situation where there are many smaller go communities that all have different rating systems. As evidenced on the worldwide rankings discussion page it is very difficult to make comparisons between differing go servers/organizations because there is only anecdotal evidence for comparisons available.

The idea is this, a central repository of "rated" games played.

Jim creates an account at the database and are assigned an ID (243)

Jim the logs on to a participating go server account (angry_lemur) and enters 243 in his profile.

That night, when the participating go server sends updates to the central db it sends information about the association of angry_lemur/243.

The next day Jim logs into his account on the central database's page and notices that angry_lemur on go server X is claming to be him. He can approve or reject this assertion.

Lets say Jim signs up on another go server under the name Jimmy, He follows a similar process as before.

The next time go server #2 gets data from the central database it notices that there Jimmy has played many games as angry_lemur on another server and using this data it can come up with a good rank right away. Next week when Jim logs in again he notices he has a solid rank.

Note that the central database itself dosent force its rankings on server admins, it simply offers an exchange of data. The server admins agree to send the database game wins/losses played on their server, and in return they can get the wins/losses for their users that play on other servers. The server admin can then apply whatever crazy ratings math that they prefer to come up with their own ratings/rankings.

For this to work it would require cooperation, youd need at least a few server admins to decide to support the database. And of course there would have to be users that play regularly on multiple go servers and that would bother to actually sign up with the central server.

Eventually if some of the larger asian go servers decided to sign up we could come very close to being able to calculate a true worldwide amature rank. One could argue that tournament games are a better indicator, but I think that the sheer number of internet games played would overcome the difficulty of some people playing too quickly or not as serious on the internet.

A tool could be written for go organizations as well that would submit their tournament results.

Malweth: This may be possible currently to some degree, however, it would require user participation or server participation. It should be possible for users to add their own game results to a central database if the server does not accept them. It may also be possible to harvest game rating information from other servers.

One possibility that could happen now (if KGS, IGS, and other admins allow this) is for the IRG admin to verify users affiliations with other servers and automatically update the IRD database daily.

For example: IRD user "Malweth" signs up the KGS account "MalwethKGS" and IGS account "MalwethIGS." Both are verified by the IRD admin on the two servers (perhaps a code phrase). The AGA user ID "12345" is also registered. (etc)

The IRD server connects to IGS daily and runs "stats" on the user "MalwethIGS." Additionally, the IRD server connects to KGS and checks the rating of "MalwethKGS" (even better if this were possible via web). Finally, the IRD server queries the AGA ratings database for 12345.

IRD would be able to do the following:

1. IRD could have a ratings conversion table that is more accurate than the one available on senseis.
2. IRD could extrapolate a user's probable rating on other servers.

Using actual game results from other servers to find a rank on the local server is unlikely at best. It would likely be highly problematical and it would open up security and accuracy issues on the local server.

Check out UGS for more discussion on similar topics.

As long as there are no time-dependent ratings on those servers, even the consistency for the same player on the same server is missing, when using two accounts for two different time settings. A cross-server rating comparison makes more sense (if at all) after having established normalized time-dependent ratings on each server. tderz

Malweth: Certainly true, except that time-dependent ratings on any current (internet) server seems like a lot to ask. There is always the risk that a speed-go account from one server and a slow-go account from another server are used to corrupt the IRD.

Is it really a lot to ask? Most chess servers (FICS, ICS, etc.) have at least three different ratings per account that pertain to three different time controls. (less than 5 min., between 5-15 min., and more than 15 min.) for Go the times would probably be different but certainly having different ranks for different times seems very simple, and indeed, I wonder why they don't have this already.

The questions I have are:

• How accurate does this IRD need to be?
• What chance of such corruption exists?
• Can't a standard deviation limit be used to reduce such corruption?

The problem also exists that some go ratings databases are not queryable by some third party. I know EGF and AGA are available, but I'm sure others (some Japanese, for example) would be hard to obtain.

The final problem I see is that low data points result in low accuracy. Without a large population of entrants in this database, the information becomes less useful - the only fix for this is time.

I hadn't thought about harvesting data, I guess the issue with that is that there is no way to prevent me from claiming to be some account on another server which is not mine. I think we would need some sort of minimal server cooperation if for nothing more than authenticating the username associations.

As for the time issues, I was thinking we could include the time information, with the game result. So if you feel that blitz games are innacurate, when you calculate the ratings you could simpy filter out games which were not played with suitable time controls.

Malweth: My idea for standard deviation "throw-out:"

Server XXX:	IGS	KGS	AGA	(Standard Deviations are primed)
Rank:		Ai	Ak	Aa
Rank:		Bi	Bk	Ba
Rank:		Ci	Ck	Ca

User:		a	b	c	(a is close to Ai, b is close to Ck, c is close to Aa)

IGS-centric:
Server:	IGS	KGS	AGA
User:		Ai-a	Ak-b	Aa-c	deviations (absolute)
			****		Indicates large deviation

KGS-centric:
Server:	IGS	KGS	AGA
User:		Ci-a	Ck-b	Ca-c	deviations (absolute)
		****		****	Indicates large deviation

AGA-centric:	IGS	KGS	AGA
User:		Ai-a	Ak-b	Aa-c	deviations (absolute)
			****		Indicates large deviation

All large deviations are thrown out. Other ranks are factored into the table.

The "Server XXX" table exists for each server/association in the database.

The acceptable deviation can be adjusted as necessary, but this deviation should probably be based on the standard deviation of the average for all users (for any given rank and server base).

For example:

IGS, at 5k has 50 users. 35 of these users have a KGS rank. 10 of these users have an AGA rank.

The sum(KGS)/35 is the average for all KGS ranks corresponding to IGS 5k. The standard deviation is the sum(|sum(KGS)/35-KGS|)/35 for all users (e.g. the average difference for all users).

This is where it pays to "affiliate" with the go servers (which I doubt is possible) to get a rank with decimal places: A user's information is accepted into the true average iif their rank (Ak) is within the standard deviation of the average to the average. In other words: |Ak-b|-Ak'+k > 0 (k is a factor).

EdwardHammerbeck: I want to get away from the go-based tech talk and focus on the programming/systems design stuff. If I may add my "duh" factor comments, I think this is a good candidate for some sort of XML web service type thing. We have disparate systems using disparate data models, but everybody is essentially all doing the same thing. The semantics are all that is really different. So, if we could all agree to some sort of XML based standard for the server admins to provide a web service for, then all this central ratings body would have to do is agree on the maths required to crunch all the different data streams. The data would not have to be real time. I mean, so what if your rating data is a week old or two days old. Leave it to the server admins to set their schedule based on their resources available.

So they, weekly or twice weekly or whatever, publish an updated stream. Our users, when they query their stats, get the most recent data available. Nothing too fancy.

Of course I am oversimplifying to a great degree and relying on a whole mess of assumptions (endless free time, altruism, etc.) but I think the high-level idea is sound.

Ectospheno: I suppose this page is an interesting intellectual discussion but is anyone here under the delusion this will ever happen? From what I can gather the admins for the various popular go servers don't exactly like each other. One server in particular will never do anything that would encourage players to learn about other servers much less play on them. Forget technical problems - the political problems alone will ensure this never happens. Sorry to be the wet blanket but I'm afraid that is the reality of the current situation.

Stormer: I don't think its that dire. The server admins in question are investing a lot of time running free services for the community. If they feel that having some sort of unified ranking system would be beneficial (or at least interesting) I think it could happen.

Edward Hammerbeck: I don't want to be a wet blanket either, but my enthusiasm for the project is tempered by the political realism Ectospheno mentions above. I've been around the online go scene less than a year, and I already know better than to mention two or more go servers in the same sentence. Or rather two particular go servers. Yikes. I hope I don't inadvertently start a flame war.

Fabian?: Obviously one needs a server independent system then... I would propose an independent page where everyone who has two (or more) up to date ranks can enter his rank. The page could calculate the average conversion factors then. This way one person can also enter his ranks more than once when he progresses. Of course this solution as well would need someone to actually develop the webpage for it...

Tas: Well Senseis is trying, in rank worldwide comparisons and similar pages. We just need more users to fill in their various ranks.