KGS Issue - Kogo's Joseki Dictionary

    Keywords: Joseki, Online Go

This is the discussion of the wish to allow to use Kogo's Joseki Dictionary with CGoban2.

Andre: Whatever that exactly means. Do you mean an upload of KJD onto KGS or simply opening the file by the program from your local harddisk while playing? I do neither favor nor oppose such wishes. My wish would be: to establish a KJD Mistake Search Room, where strong dan players pick out a branching of KJD and tell me what's wrong or new developed about the given variations by leaving such files there which I could download from a webpage of this room to correct mistakes in KJD.

wms: The problem is that Kogo's is generated with WinMGT, and WinMGT produces bogus SGF files. CGoban 2 is very careful to only be able to write valid SGF; in fact, its internal data structures are not capable of holding invalid SGF. When it reads a file that is invalid, it reports errors. CGoban 1 (and most other apps) aren't as careful and don't notice the mistakes in the file. I have exchanged email with the author of WinMGT about this, and his answer was that the SGF standard is stupid and he won't fix his program. I could probably make CGoban 2 do a better job of fixing the problems as it reads in Kogo's, but as you can imagine, writing code to repair files written by programs whose authors have that attitude isn't real high on my list of things to do. My advice: Please complain to the author of WinMGT, and tell him to make his program emit valid SGF. Once he fixes his program, and Kogo/Gary Odom start using a fixed version, then these errors will disappear.

Andre Ay from Leipzig: Since I took the job to edit KJD and entered a lot of new developed jeongseok, KJD is written with CGoban1.9.8. That won't change soon.

  • Fwiffo: I've found a workaround for those who want to use Kogo's Joseki Dictionary with cgoban2... There is a command line tool on the [ext] SGF home page called [ext] SGFC for checking and fixing SGF files. I've run Kogo's through it and it looks like it fixes it up nicely. It would still be nice if WinMGT got fixed, but this is a good alternative in the mean time.
    • Cheyenne: just a little note.. the SGFC tool is the reference implementation for the SGF file standard.
      • pel?: After fixing the sgf-file, it still takes a huge amount of time to load the file compared to, for instance, glgo. It is almost as if the program actually renders (without displaying) the board for every position in the game tree.
      • wms: Nope. Doesn't. What does take a lot of time is validating the whole file. The internal structure I use to record SGF checks every single property that gets added to make sure that it doesn't conflict with any other property in the node. This slows things down a lot. Not a big deal in any case except when reading truly massive SGF files, of which exactly one is at all popular, and that's Kogo's. Because exactly one file is actually affected by this slowness, and that file has real problems anyway, I don't plan on changing this - this thorough validation has caught a lot of times that I accidentally coded CGoban in a way that would have produced illegal SGF.

Bill: I have also found a remedy. Replace every "AW[" with ";AW[" and every "AB[" with ";AB[". Easy. :-)

erislover: The newer version seems to give my CGoban2 some new problems besides the one Bill found to fix. Line three, "RE[]", should be deleted, and from line four (now line three because we just deleted a line) "BR[]" and "WR[]" should be deleted.

Andre Ay: I did that, but couldn't find any different behavior of cgoban1.9.8. I don't know if these lines will be reincluded by cgoban if I once edit the Game Info. Doesn't seem important.

Ithmatic: Here's how to execute Bill's fix from the command line in a Unix-like OS (assuming you have sed):
sed -e 's/AW\[/;AW\[/g' -e 's/AB\[/;AB\[/g' Kogo\'s\ Joseki\ Dictionary.sgf > Kogo\'s\ Joseki\ Dictionary_Fixed.sgf

Dronak: For what it's worth, I just tried the fix Bill gave and it removed the initial warning message CGoban 2 was giving me. The file still took a while to load up, but I guess that's just the validation process wms mentioned. It looks like it has to do this every time the file is loaded. It would be nice if the program could know that the file was already validated once and not validate it again on every load, as long as the file isn't changed at all. But since this is apparently the only popular file big enough to show delays in loading, I suppose it's not really worth it.

fractic: I have run into the same problem. I normally use glGO but it just crashes when trying to open the sgf. I tried to fix the file with the SGFC tool, but it wouldn't help. Bill's fix also didn't work. I managed to get it to work with CGoban though but I normally work with glGO

I tried breaking down the file into one for each main branch (33 point, 34 point) thinking it could be the size. All except the one with the 34 point variations work.

Andre: Could still be the size, as the 34 branching is nearly 50% of the content due to Nadare and many extensive pincer variations.

Has anyone managed to get it to work with glGO?

wayward: Here's some extra for Bill's semicolon fix. A semicolon (";") in SGF means start of a new node, something that WinMGT seems to ignore in the case of AW, AB and PL nodes ("add White stones", "add Black stones", "player to move"). The rationale seems logical when we assume that any such action means that a new game node starts, but an SGF reader program is not supposed to think about node and property semantics. Two other errors detected by CGoban regarding LB property seem to have something to do with unbalanced parentheses, something that's not super-easy to fix considering that parens are used for grouping of variations. Here's the slightly modified version of Ithmatic's script:
sed -e 's/AW\[/;AW\[/g; s/AB\[/;AB\[/g; s/PL\[/;PL\[/g' Kogo\'s\ Joseki\ Dictionary.sgf > kogo.sgf

Andre: Did something similar. Nonetheless just executing this line isn't enough. I found one case of a ;AW already included and one of ;AB. I don't know yet if cgoban will include these tags with ; or not. I have found several superflous PL, which now appear as empty steps in the variation tree, which have to be deleted ba hand. Don't know why this is better than before.

jhnc: This line consolidates fixes for the problems noted above:

  sed -e 's/\(^\|[^;]\)\(PL\|A['"BW"']\[\)/\1;\2/g; s/PL\[[12]\];//g' <kogo.sgf >fixed.sgf

The first s/// does the AB/AW/PL replacements and the second one removes redundant PLs (that happen always to be preceded by ";" so it doesn't have to be retained). If you elide the final semicolon, then ALL PLs are removed, which doesn't appear to cause any problem when I view with CGoban 3.

The two sets of quoting around BW is optional. It's just to stop the wiki misdisplaying the line.


This is a copy of the living page "KGS Issue - Kogo's Joseki Dictionary" at Sensei's Library.
(OC) 2009 the Authors, published under the OpenContent License V1.0.
[Welcome to Sensei's Library!]
StartingPoints
ReferenceSection
About