European Go Database / Wishlist

Wishes, requests, FAQ... and similar for the European Go Database.

Table of contents Answered questions & resolved requests Open questions & requests Presentation Chronological order of games in games list Sorting by placement broken Sum of GoR in statistic and graphs... Total # games against same opponent Opponent list Show Number of new Players for a Tournament. Informative page titles: Show details in drop-down list of previous pages Plus Sign Possible queries Unified Rating List Name searching Comparing populations URL to specify players to compare Programmatic access Data access Get data via AJAX More GET requests Data organisation Four letter club abbreviations break functionality 4-letter clubs, 3-letter clubs... Club abbreviations (unnecessary to distinguish lower-case / upper-case there) Automatically split events Data quality Apparent duplicates Nationality Additional data Class Z tournaments European Go Calendar Shusaku Number Ranking system Declared rank updates between tournaments Does order of submission of results affect ratings? Epsilon Universal treatment of Rank Resets

Answered questions & resolved requests

See EuropeanGoDatabase/WishlistOld

Open questions & requests

Presentation

Chronological order of games in games list

Charlie: Right now when you look at a player's list of games they appear to be in chronological order with the most recent game at the top of the list, which makes sense. However, on closer inspection it turns out that they are listed with the most recent 'tournament' at the top of the list and then in an opposite chronological order within the tournament, starting with round 1 etc. There is currently no way to put the 'games' in chronological order. This makes for a messy list IMO, and it seems to me the intention was to put them in chronological order. Can we at least make it possible to order them like that? That would be helpful if one wants to monitor progression over time or write a script to calculate how one is doing in a national rating system that is not based on GoR. Thank you.

Sorting by placement broken

tapir: Sorting according to placement evidently just compares strings thus making 110th place a better placement than 14th and that one better than 2nd place. it would be better to cut the string with the "/" and process the remaining placement as integer for sorting purposes.

Sum of GoR in statistic and graphs...

I think it could measure strength of go club (or country) better than average GoR... Thanx!

Total # games against same opponent

It would be nice to just have one more column in the table where you can see the list of all opponents one player met during his go career. The 3 last columns are "won", "lost", "jigo". I would add a column called "total" which would just be the sum of these 3 columns. The idea is that when you sort descending by this column, you can see which player(s) you played the most and can be called your "rival(s) in tournament".

Opponent list

tapir: It would be nice if there would be some indication in the list, in case there where handicap games between the players. Like 4(1) - 4 games of which 1 was handicapped.

Show Number of new Players for a Tournament.

Simply show the Number of new Players (first Tournament)

Informative page titles: Show details in drop-down list of previous pages

I often go into the pages for particular players – myself, family members, & players I know. Then sometimes I want to go back to one of these previous pages – maybe to compare. However, it's not easy to see which page to go back to, as they all say things like:

E.G.D. - European Go Database | Main panel

Now, what I'd suggest is that instead they should read "EGD - 1674 John Doe", "EGD - 1652 Jane Doe", "EGD - 1632 Johnnie Dow", "EGD 1502 - Freda Bloggs", where the nnnn is the GoR.

This way, it'd be easy to compare players you know, & flick between their pages. Also, I'd imagine that this would be a simple change? Does anybody support this idea? Many thanks, Martin, a BGA Go-captive.

Request dated the 5th of January, 2014.

Patrick Traill: Seconded, 2018-12-17 (though I think the club would be better than the GoR) – This would be helpful, not just in one’s browser history but also for creating bookmarks.

Similarly, tournament pages should include the tournament name and other pages could probably be enhanced as well.

Plus Sign

GoR variation shouldn't omit the plus sign.

Possible queries

Unified Rating List

hyperpapeterie: Hope this isn't my ignorance, or a request someone else has made on here, but is there no way to show all players (including non-european players) in a single list, sorted by current rating?

tapir: The main list is basically a list of all players (with ratings according to European tournaments or tournaments with significant European participation). That's why WAGC etc. data is included. As far as I know this is the standard list shown to you on the EGD page.

hyperpapeterie: If I go to http://www.europeangodatabase.eu/EGD/EGF_rating_system.php, the option for all european players excludes some players (e.g. Kyoungnang Kang). The only other option I know of is to sort under the players page, which produces too many extraneous results (i.e. professionals who have never played in Europe). Were you thinking of a different list?

tapir: Just click on the submit button without any option and sort by GoR. ( here) Add or don't add the EGF only option. Seems you don't want this as well. But the reason is http://www.europeangodatabase.eu/EGD/Player_Card.php?&key=14733873, that she gives Korea, Seoul as residence. So there is no data in the database to divide her from those unwanted extraneous results.

hyperpapeterie: Thanks for letting me know that this option is the closest to what I want. I believe that there is a workable way to get the result I want based on the data in the database. You could filter for all and only players who have played a tournament in an EGF country. Perhaps there would be complications in implementing this, but I believe it would achieve the desired result of showing all players who compete in Europe.

Name searching

kaiou: I wish to be able to search in the whole name+surname string, because composite names, like "AA van CC", are sometimes split as AA+van_CC, sometimes as AA_van+CC. It is often a problem to search for people with spaces in their names. It would be nice to use spaces instead of underscores (or both), too. Of course, these are not critical issues at all.

Comparing populations

• tapir: I would like to compare populations – say Finnish–Dutch, London–Istanbul (for this the data may be insufficient) by calculating the overall GoR change due to the matches of a certain period. This would permit to spot rating differences and follow their changes over time. The kind of study mgoetze refers to above.

URL to specify players to compare

Patrick Traill: When one makes a comparison chart for various players, the result still has the URL of the query page ( http://www.europeangodatabase.eu/EGD/chart_comparison.php) so one cannot bookmark a comparison of particular players. Could this be adjusted, so we can bookmark specific comparison charts? The page title should ideally also include the players’ names.

Programmatic access

Data access

• Herman: I would like to be able to experiment with extending the EGD's functionality. Is there any possibility to get remote access to the data in a convenient way, e.g. SOAP or similar RPC protocol?
  • Flatline: We haven't developed such interfaces yet. However we are willing to do it if this can drive to improve EGD's functionality. At the moment, for instance, we are studying how to integrate EGD and OpenGotha for submitting results. Please share your ideas with me, we shall see what we can do together.
    • Herman: OK, here's some ideas: (note, some of these can equally well be done with a simple copy of the data, other would be more interesting if up to date)
      1. I would like to generate additional statistics. E.g.: Generate a histogram that shows how many games a player played against different ranks since a certain date, and what percentage of those games he won.
         • Flatline: That sounds nice. I think it would be useful to add such histogram in the player's card. If you'd like to take care of it, we can continue this discussion privately.
      2. I would like to be able to experiment with the GoR formula, for example changing the a and con parameters and seeing how that impacts the reliability of predictions down the line, etc. E.g., from a certain period take randomly 90% of games, feed them into GoR to update ratings, then calculate how reliably those ratings predicted the results of the remaining 10%. Things like Jeff Sonas did for the chess Elo ratings (See: http://www.chessbase.com/newsdetail.asp?newsid=562)
         • Flatline: To do this, you need full access to the complete tournaments tables. I can grant it to anybody upon simple request, since the only concern is about bandwidth.
         • Andeol? How about providing the possibility to download the tables as csv? That would allow anyone to perform their own analysis, and shouldn't be heavy on the bandwidth. It's basically the same as downloading them as text, like the option that is already provided. Just with a bit of extra formatting.
      3. I would like to try and find interesting things. For example, can I find a group of three players, A, B and C, who defeat each other in a circle (A defeats B 60% of games, B defeats C 60% of games, C defeats A 60% of games).
         • Flatline: Help me to better understand your idea: you mean a published toolset of predetermined interesting-things-detection for the general audience, or the possibility to dig off-line into raw data for specific researchers ?
    • geoff When you say integrate EGD and Open Gotha I wonder exactly what you mean. My system produces a file from GoDraw in the correct format for uploading to EGD and it all works very well.
      • Flatline: I mean no need to enter the EGD backend to upload data, just clicking a button from OpenGotha. Also, using EGD web service support when registering players for a tournament.
        • Yes Flatline, This looks to be a nice feature! I am ready to take on OpenGotha's part of the job

Get data via AJAX

• Flatline OK, it's a little weird that I'm opening a "wishlist" topic, but I'd like to know your opinion about this matter: <GEEK MODE ON> I've developed a little tool which can be accessed as a web service: you call it with http://www.europeangodatabase.eu/EGD/GetPlayerDataByPIN.php?pin=XXXXXXXX and it gives back a JSON structure containing player's data from EGD database (try it, if you like). Do you think that it is useful, so that it can be regarded as a new feature ? Do you think that would be worth to implement also a GetPlayerPINByData? (i.e.: last name [and country [and name and/or club]] ), given that names are often misspelled, so the returned data will be an array of possible PIN's ? <GEEK MODE OFF>
  • tapir: I guess this is useful to cross-check (e.g. as a tournament organizer). Though I won't use it that much personally.
    • Flatline Frankly my idea was not only to let cross-checking, but also to specifically allow to setup webpages (or also desktop applications) for tournament registration with a direct link to the EGD database (just to get an idea, have a look at this registration form: http://goski.agi.go.it/index.php?option=com_wrapper&view=wrapper&Itemid=65 , and try typing the PIn OR the last name )
      • tapir: That's great!! No chance for tournament directors to mess up :)
  • TeeSushi Maybe it's possible to make a request by country and club? Thus a club can easily generate a list of its members and their actual status. Though I think it's not that important since most clubs use the lists that you can download at EGF ratings system.
    • Flatline: As explained in the Updates page, it is already possible to get data by country and club:

http://www.europeangodatabase.eu/EGD/GetPlayerDataByData.php?lastname=XXXXXXXXXX [&name=XXXXX][&country=XX][&club=XXXX]

N.B.: lastname OR country are required

• Sec? I came to this page to request for an AJAX interface only to find it is already there. Great! – Can we also have access to the tournament data via AJAX? I wanted to play with generating different GoR-graphs and it would be nice to be able to get that data on a player-by-player basis.
  • Flatline: I'm glad you find this feature useful :-) For tournament data, my only concern is that requests of this kind can be easily automated and can generate a huge bandwidth consumption... I'm thinking of allowing requests on a single-tournament or on a single-player-history basis, and limiting the number of requests from the same IP.
    • Sec? If it is of any interest, the way Twitter does it is to limit access to its API to "n requests per IP per hour" and people needing more need to request them stating why – they get a relaxed (higher) limit. — For me this sounds like a valid approach. — If you implement it, the ability to get all tournaments of a single player would be the most interesting, followed perhaps by a listing of all games within one tournament.

More GET requests

Herman: Could more of the EGD be implemented as GET (as opposed to POST) requests? Right now, there are many pages that could easily be linked to directly with URL parameters if only GET were used instead of POST (e.g.: http://www.europeangodatabase.eu/EGD/Stats_Country.php?country_code=NL&club=Delf doesn't work, but there is no reason not to let it work).

Also, on a more formal note, the HTTP standard says that GET request should be safe (should not alter the state of the server), whereas POST requests can be unsafe (can alter the state of the server). Because of this, browsers should (and do) consider it unsafe to reload pages that were the result of a POST request (e.g.: go to the stats page for any club, then click reload, your browser should pop up a warning, asking for confirmation).

• Flatline: Try this: http://www.europeangodatabase.eu/EGD/Stats_Country.php?country_code=NL&club=Delf&ricerca=1

Herman Ahah, there is a secret parameter! :-)

• Flatline: EGD is a chamber of secrets ;-)

Data organisation

Four letter club abbreviations break functionality

Herman: With the introduction of four letter club abbreviations, a lot of the functionality for clubs is currently broken. If, for example, I want to generate a list of members and ratings for my club, Delft, I will find 12 of them under the abbreviation "Del" and four of them under "Delf". If I generate statistics for NL/Del, I get a nice graph of active players. But generating the same statistics page for NL/Delf, the graph is broken (due to there only being one year of data, I guess). But the bigger issue really is that there shouldn't be two pages for this at all. "Del" and "Delf" are the same club, so I'd like to see the statistics for players marked with either "Del" or Delf" together. Is there any plan to fix this? Perhaps by automatically converting old club abbreviations to new ones (so change "Del" to "Delf" for all older results, and convert it automatically for new results entered into the database). I would be more than happy to provide a list of Dutch clubs with "Old" -> "New" abbreviations, and I'm sure there's people all over Europe willing to do the same for their own country.

• Flatline: This sounds good: please do. I apologize for the inconsistencies introduced by the transition from 3-chars to 4-chars city codes, but this must be regarded as a temporary, unavoidable pain. We have already an automatic converter, so that with an "Old => New" table it is easy to fix everything. It is only necessary to find people willing to populate such tables (BTW: I've already fixed "Del" => "Delf").
  • Herman Thank you very much! I have sent you a file with (most of) the other Dutch conversions.
  • geoff My understanding is that club names can go up to 4 chars but EGD will continue to accept 3 chars. If so what is the convention you use to convert 3 to 4 chars?
    • Flatline: No. EGD does accept 4 chars.
    • PeterHB: For clarity, Flatline, can you confirm you are saying that from now on the EGD only accepts 4 character club codes? Or is that a misunderstanding? I'm hoping you will say that providing a once off file with matching of 3 to 4 character club codes will provide compatibility, and then results can be submitted in both 3 or 4 character format for a transition period of a year or two. A clear description of the situation would be helpful.
      • Flatline: EGD accepts up to 4 characters (i.e.: 1, 2 or 3 are accepted as well). However, the most widely used pairing program (MacMahon) used to export only 3 characters, so incoming data used to have this limitation. In May 2009 this program has been changed, and since then EGD strongly encourages to download the version 2.50, so that the city codes now can range as far as 4 characters. That's all. So, there is no need of a transition period of a year or two in which both 3 and 4 chars are used. Nevertheless, I urge every submitter to switch a.s.a.p. to 4-characters (i.e.: to download the latest version of MacMahon). Also, it would be very appreciated that some people from every country do the same job that Herman did for the Netherlands, i.e. provide a "3-chars" => "4-chars" conversion table AND act in their own country in order to make this same table available to every tournaments directors, and urge them to use it.
      • geoff: Are you planning to publish the club-code/club-names for each country? If this is not done then I am not convinced of the necessity of re-inventing 4 letter club codes. How are people in other countries supposed to know what the codes mean? One could just add a trailing underscore if you really want this for formatting purposes. In the UK all tournament results are submitted to me for processing and uploading to the EGD - we do not use MacMahon.
        • tapir: My impression was that a conversion table is needed because new versions of MacMahon will export 4 letters while old versions have only 3 letters - the problem isn't that the club has to have 4 letters, but that new 4 letter input should somehow compare with old 3 letter input which is still used, so the database can recognize A from Del is from the same city as B from Delf. Concerning conversion tables, can you give an idea for which countries they are still lacking, that we can spread the word?
          • isd: Clubs is not a very accurate field sometimes. It often happens that a player, living in a country (say Germany) will be from another country (say UK). He will use his club in the UK - COL, but this will be interpeted as a German Club. (or vice versa). I see this quite often. It would be nice to have a club conversion list (Full name to code-4 to code-3) published for each country to help out in such cases.
• Ellyster: The only player in "Spain, Gran" (me) should be in "Spain, Gra"... In Spain, "Gra" is stands for Granada meanwhile "Gran" stands for Gran Canaria... I agree with @isd this only abreviation system couses lot of trubles.

4-letter clubs, 3-letter clubs...

tapir: It is just a big mess. (I sometimes want to look for clubs, compare clubs, player populations, look for players... this doesn't work anymore with the degree of randomness in club abbreviations right now.) Or more constructively, how can we help to correct this mess?

jon: Please can we display the name of the club rather than the club code whenever it's displayed?

We should also use the name of a club for a lookup, rather than the code, although this is rather more difficult to implement.

The 3/4 character code should not appear on the user interface, except for admin purposes.

Club abbreviations (unnecessary to distinguish lower-case / upper-case there)

tapir: I just looked at some statistics, it seems that it divides lower-case and upper-case spellings of the club abbreviations doubling some entries. E.g. "Win" in CH and "win" in CH, doubtless there are more elsewhere.

Automatically split events

Herman Sometimes, is useful for an event to be split into multiple events in the database. This already happens, for example, with the European Go Congress. The EGC is split into "first week" and "second week", which allows the weekend tournament to be processed between those two events. This has the disadvantage, however, that the EGD does not contain the wall list for the whole 10 round main tournament, even though it is a single event from a McMahon/Pairing point of view. It would be nice if the EGC (and possibly other events), could be entered as a single event, but with the added option to specify that it should be split over multiple dates. So in case of the EGC, some option to say "rounds 1-5 should count on date A, rounds 6-10 should count on date B". So in the database, it is visible as a single event, with 10 rounds, but for ratings calculation, it is split into two events in the background...

Data quality

Apparent duplicates

tapir: I don't know these players and everything might be alright, but ... not to mention the famous clubs xxx, and xxxx. (Of course it is easy to err on the other side as well as happened to me in my first tournament.)

15262148	Brakmann	Nils	DE	BN	9k	1125	4	0
15237728	Brackmann	Nils	DE	BN	13k	811	1	0

10686126	Groenefeld	Martin	DE	BN	2k	1735	41	2245
12998975	Gruenefeld	Martin	DE	BN	2k	1836	1	0

Nationality

tapir: I noticed another far more important data quality issue, I understand nationality is meant as the country of the go club, as there is no way to search for clubs regardless of country, i.e. both fields belong together, however, I know people who (will) start to give their local club but their real nationality. This will mess up EGD data, as soon as you try to look at club level or how many clubs there are in a country etc.

Can some guidance be offered on the correct value for the Country Field. Should it be

1. The federation a player is a member of.
2. The nationality of a player.
3. The nationality of the player's club.
4. The country in which a player currently lives.

Country is sometimes used for calculating the amount of money a Federation should pay to the EGF, and sometimes used for calculating the PairGo points a Federation will receive. I think this means it should be treated in a standardized way during the tournament submission process. There are problems though, since a player can have multiple nationalities and multiple federation memberships.

Patrick Traill: Certainly applies to me ( 10433489), a Briton living in Germany who plays just across the border in the Netherlands (de Go Splinter in Venlo); I am not currently a member of any federation, but am inclined to join the Dutch and maybe German federations. Come to think of it, I have German nationality as well since 2018-09-13. In fact, all these data can vary with time.

Additional data

Class Z tournaments

Herman: At the moment, the EGD contains tournaments that are ranked A, B or C, depending on their thinking time. Is it an option to have a tournament class for tournaments with shorter thinking times, which would not count for the rating at all, but could still be uploaded? The purpose would be to have an archive of such tournaments. The reason I am asking is because someone on the Dutch go mailing list asked about the Dutch tournament archive (on the website of the Dutch go association). I would point him to the EGD, where he can easily find any tournaments played in the Netherlands, but realized that that is not actually true. Certain events will not be in the EGD because of their limited thinking time, despite being noteworthy tournaments. I would like, for example, to add the results of the Dutch lightning championships to such a category.

• kaiou: I agree with Herman, that would be nice. There are a lot of club tournaments, blitz tournaments etc that a lot of people would be happy to show on their EGD profile.
• Flatline: mumble mumble...
• Sec? oh yes. A comprehensive database of all go games would be a dream come true! :)

European Go Calendar

• kaiou: It would be nice if there was a calendar of the upcoming tournaments, updatable in the way as the wallist uploading works. The official EGF calendar is updated rarely and only lists the major tournaments. The wiki page here is more complete, but only a few organizers are aware of his existance, or care about updating it (many time I added in bulk the Italian tournaments). I often hear people on KGS who already look at the EGD as a reference for the past tournaments, and expect to find a calendar for the future ones.
  • tapir: I guess most people look at national pages for finding tournaments. and maybe that one of a neighbouring country. the player population that goes to tournaments across different countries regularly may be very limited.

Shusaku Number

The Shusaku number is known for some players in the EGD. Therefore, it is possible to calculate a (maximum) Shusaku number for other players based on tournaments automatically. Would be nice to have.

Ranking system

Declared rank updates between tournaments

Jouni: It would be nice if there were a possibility to have declared ranks updated between tournaments. This would help pairings because up to date info could be retrieved directly from EGD with OpenGotha. Now direct usage of GoR for pairing is little tricky because we need to adjust possible pre-tournament resets by hand and there are possible cases where this is even impossible. Pairing with GoR should be essential especially in EGC because there are lots of Japanese and others who participate with nonsense ranks, but often their GoRs are already quite close after few congresses.

This would be a great feature, because the problem is quite significant at the EGC. I think that there would be enough control if registered and trusted EGD users could adjust declared ranks if needed. With Finnish ranks it would be easy because we have a very practical common Wiki which is always kept up to date with official rank and tournament info. ( http://www.suomigo.net/wiki/Luokituslista) A similar wiki approach would work with EGD too, I think. Even though there are various degrees of rank liberalism in different European countries. (2009-12-01)

• Flatline : As far as I know, OpenGotha doesn't show the rank picked up from EGD; instead, it recalculates the correct rank from the actual GoR and displays this result. So, this feature would be useless. Furthermore, I don't think that EGD should allow ranks changes between tournaments, since it is basically a tournaments repository, and stores information about players only when they attend tournaments.

• Jouni: Perhaps it is difficult to find a good solution for this, because of variations in rank liberalism. I wonder if there is a good method to introduce pre-tournament resets automatically? Of course the development of this feature should be done in cooperation between Aldo and Luc, if it is the case that people are willing to go for using GoR instead of declared rank in tournament pairings. I do not see why this is not the future of pairing, because GoR is superior in pairing accuracy compared to declared ranks. (2009-12-20)

• LovroF: You simply recalculate the rating yourself when starting the tournament...

RobertJasiek: What is called the "declared rank" is not the declared rank but the rank that is either the player's declared rank or the tournament organisers' declaration of what they want / enforce to declare as the rank for the players. This should be made clear in the players' rating graphs.

Does order of submission of results affect ratings?

Lovro Furjanic – Sorry, I did not know where to post this. Is there any possibility that, when someone posts a tournament late, you lose rating? Let me explain. Suppose, in tournament A you get x points, but it is not (yet) submitted to the EGD. Then you play tournament B and get y points, where both x and y are positive, and B **is** submitted to the EGD. After that, A (which was held earlier) is submitted to the EGD. Is there any chance, that although you won in both tournaments, your GoR drops after A is submitted because your profit on B got smaller as your GoR increased from A, and the margin by which it decreased is bigger than one by which it increased after A? Sorry if I confused you...

Phelan: From what I understand, it doesn't matter when the tournaments are sent to the EGD. I don't really know how to explain it, but when the system runs after getting the new tournament(A), it will calculate the GoR change in tournament B based on the GoR the player has after A.

Lovro Furjanic - That’s exactly what I am talking about. If can your GoR from A increase less than your GoR decreases after you get lesser points B. I don’t really know if you understand me :)

tapir: As far as I understand the system, the answer is a clear ‘yes’. I doubt however that if you won more than a handful of points you will end up losing points, but if you won 3 rating points in the first computation you may well end losing some points after some older tournaments are computed as well. But usually you add new tournaments to the database not old ones.

Epsilon

Since a winning expectancy below zero makes no sense, epsilon should be removed from that formula and be applied directly to the rating adjustment formula.

Universal treatment of Rank Resets

This is not something which can be implemented by a computer program, but instead something that should be done consistently by all EGF federations. Imagine the scenario where a 10kyu enters a tournament and beats all his opponents who we will imagine to be 10kyu, 8kyu, 7kyu, 5kyu, and 4kyu. There exist at least 3 different ways in which this 10kyu is treated, each of which has a different effect on the ratings.

1. No adjustment is done, the player just earns his rating points, he plays in his next tournament at 8 kyu.
2. The player is promoted post tournament to a new rank, let us imagine it is 5kyu here. He plays at 5kyu in his next tournament.
3. The player is promoted to 5kyu before submitting the tournament results. He plays at 5kyu in his next tournament. His opponents in the tournament he just played lose fewer rating points.

Normally, or perhaps ideally, there should be only 1 treatment of this player's rank.