SGF Wishlist

This page discusses new proposals for SGF.

Table of contents Properties Things to think about TO - Topology of Board SD - Special Definition MM - Move Marker PT - Problem Type PD - Problem Difficulty BE, WE - Elapsed Time / or TS - Time Spent, var. 1 T - Absolute Elapsed Time, var. 2 KK - Collection OM - Open Marker LI - Link Redirect property RA, WA - Right answer and wrong answer markers IM - Illegal Move Menu, Table of Contents Text Formatting, Mark-Up Hyperlinks and Labels Reordering of Nodes xx - Bookmark xx - Home Multi-Language Support Existing Properties (extend specification) DT - Date of game Format Compression Collection Transfer Protocol (Single node or branch) Go Stone Photographs Retracted wishes VO - Voice

Table of diagrams example 1 same position different order rotated position circle=IM

Properties

SGF allows applications to freely define its own properties. However, discussing new properties helps achieve standardization.

Things to think about

We must remember that SGF stands for Smart Game (not Go) Format and that it serves 40 different games. Anyone with a knowledge of SGF and the specification changes over its development should know that properties that have been too specific to one game (and underused then) have not survived revisions. So bear this in mind.

Willemien What is the size of CR?. In FF4 it is not specified. what has the result that some sgf editors use it as a big circle (almost the size of a stone) while for other editors it looks more like a dot. i think the first interpretation is correct but it should just be described

TO - Topology of Board

Property

TO

Propvalue

simple text

Propertytype

game-info

Function

Discrimingnating different topologies from each other. (e.g., "standard", "wrapp-around", "wrap-x", "wrap-y")

Proposed by

Bugzaren

Explanation This might be an odd request (and maybe very specific for Go) bur the fact that the playing field is defined solely through its nodes and links mean you can play around with the topology. For instance letting the board be periodic so that you identify the left and right sides with each other.

SD - Special Definition

Property

SD

Propvalue

SimpleText : SimpleText : SimpleText : !Text

Propertytype

root

Function

to make it easier for sgf readers to detect errors and convey full information to the viewer

Proposed by

LithiumTwo

Explanation

a sample of sgf (to apply to the PT tag below) might be SD[PT:SimpleText:game-info:The type of problem posed in this sgf]. Where the first value is the tag to be defined that is outside the file's FF specification, the second to be the type of variable for the property (composite types of "(simpletext:number)" may be possible), the third to be the type of property, and the fourth would be a description of the property. The reason for this would be exactly that if people needed a new property for their program, then this file would be perfectly portable, and error checking (and correcting) would be made much easier as standard sgf programs know what to look out for. In addition, for those intending to create new properties: sgf editors should have the capability to easily add these new properties to the sgf (even if they can't interpret them as the specific program will), and they can be displayed to the viewer of these sgfs in any normal editor.

Discussion

MM - Move Marker

Property

MM

Propvalue

simple text

Propertytype

game-info

Function

the appearance of the marker for the current move in a node (e.g., "none", "cross", "reddot", "greensquare")

Comment

unifies the appearance of the board over all sgf viewers. There should be several standard shapes.

Proposed by

Harleqin

Discussion

When this property is used, all sgf viewers mark the current move in the same way, so confusing text ("the stone with the red dot" "what red dot?") is avoided.

The motivation is to finally get wms to abolish his "CR" hack. :) (See also KGS Issue CR Marks On Moves.)

LithiumTwo: This is a bit redundant... people usually say "the current move" or it's coordinates... or even "this move"... I've never heard anyone refer to "the stone with the circle on it". However, the main reason it's redundant is that wms has already abolished the CR hack =)

axd: Abolished? You mean it's no longer there? 2.6.12 still has it, btw.

PT - Problem Type

Property

PT

Propvalue

simple text

Propertytype

game-info

Function

the type of a go problem contained in the SGF file (e.g., "Tsumego", "Fuseki", "Capturing", "Connecting", "Yose", "Semeai" etc.)

Related

PD

Comment

Problem solvers recognizing this property could automatically group problems into categories and let the user search for them (e.g., "give me all yose problems")

Proposed by

maruseru

Discussion

The main motivation comes from the existing practice of storing tags such as the problem type and difficulty outside the SGF file. This makes it difficult, when renaming, moving or copying the SGF files to keep them in sync with the tags. I'd really like to see this information being kept inside the SGF file.

I've added support for editing and searching PT[] and PD[] properties to kombilo and submitted the patched version to kombilo's author, Ulrich Goertz. I've also added a rather basic problem-solving mode to kombilo. Now it would be nice to have support for these properties in problem solvers such as GoGrinder and Go Album. That way you could edit your problems on the desktop and deploy the content to another application (possibly running on a mobile device).

Ithelog: Goproblems.com use tag GE[] for this purpose

PD - Problem Difficulty

Property

PD

Propvalue

simple text

Propertytype

game-info

Function

the difficulty of a go problem contained in the SGF file (e.g., "Beginner", "Easy", "Medium", "Hard", "Pro")

Related

PT

Comment

Problem solvers recognizing this property could automatically group problems into difficulty levels and let the user search for them (e.g., "give me all hard tsumego problems"). Obviously classifications such as "easy" and "hard" are subjective, but they might denote rank intervals. For example, "beginner" = 30k-20k; "easy" = 19k-10k; "medium" = 9k-1k; "hard" = dan; "pro" = pro (suprisingly; for example, the Igo Hatsuyoron problems would have a PD[Pro]).

Proposed by

maruseru

Discussion

Benjamin: I would propose more fluent way of representing a problems difficulty (e.g. something like problem-{EGF}"rating") to give problem databases a way to further divide the probs according to user statistics.

IanDavis: how do you propose accurately finding the correct EGF rating for a problem ;-)

maruseru: I've created a page for discussing problem ratings.

Ithelog: Goproblems.com use tag DI[] for this purpose

BE, WE - Elapsed Time / or TS - Time Spent, var. 1

Property

BE, WE

Propvalue

floating point number (number of elapsed seconds)

Propertytype

game-info, move

Function

document how much time elapsed while the current mover was on the clock.

Related

BL, WL

Comment

Depending on the time system used, you frequently can not deduce the elapsed time from the BL and WL properities.

Proposed by

jfc

Discussion

jfc: This property is useful for playing back a game at the speed and tempo it was originally played. This is particularly useful for games that spend most of their time in Japanese byo-yomi as the BL and WL provide next to no information about the tempo of play during byo-yomi.

it could also be called TS = time spent (no need to differentiate on colors black/white), the value would be a floating point number in seconds, like above, specifying how long the player took to think about the move in the current Node. Very useful property!

T - Absolute Elapsed Time, var. 2

Property

T

Propvalue

floating point number (number of elapsed seconds)

Propertytype

- (node annotation)

Function

time stamp, documenting how much time had elapsed since game start when the corresponding set of properties was created or revisited.

Related

BL, WL

Comment

One node may contain more than one T property, but a file must not contain any two T properties with identical time value.

Algorithm to replay a file along time stamps:

     (0) Starting from T[0.0] at game info,
     (1) the viewer searches for the stamp which temporally (not necessarily
         textually) follows the current one.
     (2a) If no stamp specifies a later time:
              Replaying is stopped.
     (2b) Otherwise:
              As soon as that next stamp's point of time has passed, the viewer
              jumps to there and executes all subsequent properties until it
              encounters another T property or EOF, then does (1) again.

For convenience, sgf recording and/or replaying software should be able to pause.

Proposed by

blubb

Discussion

blubb: I'd regard such absolute (cumulative) time stamp tags as more promising, because they would allow to replay even highly branched reviews in original order and timing, while also providing all the information BE and WE could define.

LithiumTwo: this would be very nice for complicated reviews.

jfc: In the interest of recording the game flow it might also be useful to have an 'adjourn' property that shows when a games was stopped for any reason (e.g. when I lose my connection).

Ithelog?: This would be very useful for reading lectures recorded in the sgf-files afterwards. Currently there is no way to see in which order the lecturer has visited the nodes of the game.

KK - Collection

Property

KK

Propvalue

simple-text

Propertytype

game-info

Function

Apparently used by the GoGoD PBase to indicate problem collection (e.g. KK[Guanzipu]).

OM - Open Marker

Property

OM

Propvalue

none

Propertytype

node

Function

Indicate which position should be shown first after sgf-file has been opened

Comment

Currently application decide whether to show start of the game or end of the game when sgf-file is opened. With this marker, application could show first defined at the file. In my application I would place the marker at the position shown on the screen when file is saved.

Proposed by

Ithelog

Discussion

blubb: To ensure consistency, you'd either need to limit OM to one per file, or to provide a rule how to deal with multiple OM occurrences (e. g. due to merged files).

LI - Link Redirect property

Property

LI

Propvalue

to be discussed

Propertytype

node

Function

To make a position independent of the order of moves were possible

'Comment: Currently different order of moves lead to different nodes tn the gametree while the resulting position is the same. I would like to be possible to have an redirect property that when you change the order of moves where applicable you gat to the same node. So that the following comments and followup moves only have to be added once.

Proposed by

Willemien

Example

The following games lead to the same position

example 1  

same position different order  

rotated position  

Maybe even more advanced would be that you also rotated / mirrored symetries can be redirected to the same nodes (athough then there are problems with coordinate changes.

[edit]

RA, WA - Right answer and wrong answer markers

Property

RA or WA

Propvalue

none

Propertytype

node

Function

Indicate whether the variation which follows is correct (useful for go or chess problems).

Arno: There are already many markers for good moves, bad moves, good and bad positions, interesting and doubtful moves. See the SGF user guide for an overview.

IM - Illegal Move

Property

IM

Propvalue

list of point

Propertype

setup

Function

Indicate that the listed points may not be played on this move.

Comment

Mainly of use in composing tsumego in SGF format that begin with the other player having just taken the ko. IM would then be used to indicate the point the ko capture occurred on:

circle=IM  

Here the circled point would be marked with IM in the SGF.

Proposer

TheBigH

[edit]

Menu, Table of Contents

I was just reviewing the counting lectures by Dr. Straw. While it's awesome that a 5 dan makes these SGF files up for anyone to read, I found the SGF file itself a little hard to deal with. For the counting lecture, Dr. Straw started with a finished game to show the counting, then had the viewer return to previous positions in the game that he had anotated.

I'll say here that the result was completely useable, if one was willing to work at it. The jumping around was a bit confusing but not overwhelming. I just found myself thinking "Hmm, a little programming could make this really slick." So this is not some huge need that I'm discussing here, but rather something that would be nice to have.

The menu (or Table of Contents, if you like) should work something like the TOC in an Adobe Acrobat document. The user should see a menu, in order, of positions in the SGF file. Clicking on a menu item will take the view directly to the spot in the SGF file.

Its important that the menu have a definate order. Its no good, for example, for the creator of the SGF file to make a menu that says "Lesson 1," "Lesson 2," and "Lesson 3" if the user sees them in the reverse order. Likewise, the menu items themselves have to be able to "reference" arbitray points in the SGF file. Remember that Dr. Straw's lecture started at the end of the file.

For example, here a simple implementation that take a "list" of pairs. The "list" is in order that the user should see. The pairs consist of a menu entry, which the user will see, and the reference which the user won't (probably) see.

MENU["Lesson 1":LB[_START], "Lesson 2":FG[FIG1], "Lesson 3":VW[A]]

Here the first menu entry points to label (LB) "_START". The second menu entry points to figure (FG) "FIG1", and the final menu "Lesson 3" points to view (VW) labeled "A".

Yes I know this isn't strict SGF format, I'm just puting this here to spark ideas. If this looks like someone might implement it, we'll nail down the format better at that time. Ok folks there it is, rip it to shreds. (By Gojira.)

Ithelog?: Elapsed time proposal 2 above would solve this issue. For solving this issue it would we adequate to have optional node tag for order of the nodes, where each order would be allowed only once and each node may have any number of orders. The elapsed time proposal 2 above is however more powerful in form it is proposed.

[edit]

Text Formatting, Mark-Up

And while I'm at it, Dr. Straws SGF file was readable, but I also found myself wishing for some sort of mark-up. So how about putting HTML in the comment section of an SGF file? Normally I'd be against including such a "heavy-weight" library (After all, heavy shape is bad; sabaki is good ^_^). But HTML libraries are so ubiquitous now I think it's reasonable to just add one to an existing SGF editor.

For SGF editors that don't wish to support HTML, it should be easy to write a simple parser that ignores and strips out the HTML, leaving the user with a reasonable comment to read.

Lots of other suggestions could be added here, but I'll wait to see if there is any interest first. (By Gojira again ^_^)

[edit]

Hyperlinks and Labels

Hyperlinks in sgf's would be very useful. I don't mean hyperlinks to websites or html pages, but rather to labeled nodes in the sgf file. This would also allow the easy construction of the 'Table of Contents' feature described above. Of course, this would require that labels be added to the standard as well. I've been working on an sgf file containing the basic corner shapes and their status. It would be very helpful if I could create a table of contents in the top node comments, with links to the base nodes of each of the shapes. And of course it would be really cool if the comments for a particular corner shape could have links to any related corner shapes.

Proposed by gnos?

additional comment by gnos?: The labels feature permits the sgf viewing program to have a 'labels browser'. This would allow the user to jump around the sgf file and know where he is going and be able to get back. This is in itself an extremely important feature that is currently lacking in the standard. Please implement this!

[edit]

Reordering of Nodes

Nodes should be reorderable. This would facilitate e.g. the creation of dictionaries with multiple possible ways of ordering the data. Basically, the idea is that there would be a method for changing the position of nodes within a given branch. If there is a vertical difference between the node's positions, then obviously this should not be possible, or if it is it would be a different feature. But if they are on the same plane laterally, then they should be reorderable. The way to do this is to create sytematic labels for nodes on the same level of a certain 'branch'. Basically, nodes could be labeled with an ordering property. So for example, you might have one ordering label called alphabetical?, and individual nodes would be labeled [alphabetical [1]] [alphabetical [2]] etc. and if the user selects the 'alphabetical' ordering, (from a link in the comments, or from a button or menu item in the sgf viewer, or by any other means), then he will have the nodes on that particular branch ordered alphabetically (this would of course allow for the author of the sgf to err, and accidently place some nodes out of order within the alphabetical ordering, and I consider this a good thing more or less.). The same set of nodes could also have other ordering labels, e.g. [SizeofShape[1], SizeofShape?[2]], etc. Of course, if there are some nodes left unlabeled within a certain ordering system on a given branch, they will simply not appear when the user selects that ordering system. And again, I consider this a good thing, it's up to the author to make a sensible sgf file after all. The user must always be able to select "no ordering", and just get ALL the nodes entered by the author and in the order the author entered them.

suggested by gnos?

[edit]

xx - Bookmark

MultiGo already implements a BOOKMARK property (visually represented by an asterix on the bookmarked node) that allows to quickly jump between bookmarked nodes (handy when you want to e.g. compare positions).

Ithelog: MultiGo use tag MULTIGOBM[] for this purpose

Can be implemented using HO property?

[edit]

xx - Home

Relating to bookmarks HOME property would be usefull. Basicly home is like normal bookmark, but there should be only one home. Meaning for home is to be the position that should be opened when the file is opened. So, in case bookmark is included in the sgf standard, home could be simply special case of it, like BM \[ HOME \] for example.

[edit]

Multi-Language Support

HandOfHair: It could be useful if a single SGF file could have each text property supported in multiple languages. For example:

C[The default language is English.]C[ja:このコメントは日本語であります。]

(The Japanese version of that comment reads "kono komento wa nihongo de arimasu," which, if I'm not mistaken, means "This comment is in Japanese.")

huh?!? KGS already supports Multi-Languages. Its UTF-8

BM (for bad move) and IT for tesuji, (i.e., good move) are to provide language independent comments. Love to see more of the same, GENIUS; or OOPS; or BECAUSEOF[bc][cd][ef]; for because of this sequence. I believe that I read there has been pressure to move away from comments, because they are language dependent. That way, HandofHair, your brilliant, insightful, wonderful comments on go games stored in SGF can be viewed in over a hundred languages, translated automatically by SGF viewers and editors, for centuries to come. Every go book I've every read uses phrases like "this/these stones are too far away to help," "the marked stone supports this move," "too close to opponents wall," "should have been either," etc.. It should be possible to combine more then one such item per node. Go is about placing stones in RELATIONSHIP to other stones. Maybe BM[ad][ae][af]; and IT[cb]; labeling the points that make it a good or bad move would be a first step. SABAKI[dc]; HONTE[dd][de]; SENTE[df][tt]; MIAI[dc][de]; ALIVE[][][]. I'll tell you one thing, I sincerely appreciate you getting my gray cells into gear, and I'm going back and skim my go books for the type of language they use. Best wishes, Kbuny. psfrl (at) sbcglobal.net

Existing Properties (extend specification)

DT - Date of game

Problem

The format for full timestamps of begin/end of a game (especially on realtime servers) is not specified. Also, no timestamps for adjourn/continue of game are possible.

Suggestion

The following format is suggested:

• DT[2010-01-14 12:13:13 +100], full timestamp with timezone information. To convert to UTC time one hour must be subtracted (thus, 11:13:13 +000).
• Specify begin and end of game DT[<begin>/<end>] with two timestamps, separated by a slash.
• Or DT[begin <t>/adjourn <t>/continue <t>/end <t>] with full adjournment information

Format

Compression

A standardized compression scheme for SGF files would be useful, especially for programs making liberal use of flash media (Palm pilot, etc).

Compression scheme should maximize (in order of preference)

1. (Minimize) Processor requirement
2. Speed
3. Size

Malweth

AshleyF: You seem to be talking about compression of some binary data (voice, player photos, etc.) but I thought I'd mention that Anders Kierulf (creator of SGF) has implemented a simple compression scheme in SmartGo that applies to game records. It greatly reduces file size for large collections while still maintaining an SGF-like syntax. For example, 40.1MB for all of GoGoD as a single SGF collection compresses down to 15.7MB as an SGC. Check out: http://www.smartgo.com/HTML/index.html?compressedgamecollections.htm

Malweth: I was talking about compression of the text data. Text such as an SGF can be compressed by 60-80% using RAR or ZIP algorithms. I wasn't aware of the simple method being used by Smartgo (I did realize there was a different file format used by this program). I do see problems with this, however - instead of a bulky compression (complete file) that is done and un-done by a compression scheme - once variations are added in the middle of the SGC sequences, the group must be split - a simple variation could actually end up adding more to the file size than the variation sequence actually requires. In any case, some type of method would be useful if added to the specification (not just as a recommendation).

Collection

A standardized SGF collection file would help put multiple SGF files into a single SGF collection. These files should contain a table of contents.

Such a standardization would be useful for SGF databases and collections. The (optional) compression scheme should also apply to SGF Collection files.

Malweth

AshleyF: The SGF spec already includes the idea of collections; just by including multiple game trees in one file (see http://www.red-bean.com/sgf/sgf4.html#ebnf-def). Not all clients support it though. SmartGo, to name one, has nice support for it and builds a 'table of contents' view as you suggest above: http://www.smartgo.com/HTML/index.html?gamecollections.htm.

Transfer Protocol (Single node or branch)

Malweth: A standard protocol to transfer a single node or branch of an SGF file would be useful. This would make collaboration in SGF format possible. It seems likely that the Ultimate Go Server project will define such a protocol.

This need have no firm basis in SGF, but staying as close as possible to the SGF standard would be useful for databasing these collections.

Go Stone Photographs

Photographs of some black go stones, and especially of a few dozen good quality white shell go stones, all lit from the same angle. Place in a zip or rar or ?? and placed on Source Forge or Flickr? or some such. Copyleft or Gnu License or ? Easy enough to come up with pictures of wood for go boards, but the white shell go stones are a challenge. Thanks for your attention to this.

RueLue: are you sure, this is the right page for your wish?

Retracted wishes

VO - Voice

Property

VO

Propvalue

simple text

Propertytype

game-info, move

Function

contains URL or relative address of sound file to be played when game is opened or the move is displayed respectively.

Related

GC, C

Comment

Voice-over-IP is already supporded by some Go Servers. This property would allow to save it.

Proposed by

Sebastian

Discussion

Cheyenne: I have a concern about increasing the size of the SGF file (think of trying to download the SGF from say KGS and then importing into PilotGOne). A suggestion that this facility use two parts. The first part is handle for the sound file. The second part would be indexes into the sound file (start/end times). There could be provisions to handle multiple segment sound files (say within a variation branch). So you would have initial specification of where to find the sound file and then at each point you would only have a small index value to be used to indicate where to start and end the sound clip.

The sgf viewer resolves the connection between the handle for the sound file and the external file (that is either local or online). This should keep the size of the SGF down.

The way that I could see this functioning under say KGS would be that if you downloaded the SGF file you would be prompted if you wished to also download the corresponding sound file for the SGF. The file name would match the file name of the SGF and the file extension would be different (foo.sgf & foo.wav for example)

(Sebastian:) You're raising a good point. The standard could certainly include this in the form

VO[relative\path\voicefile.wav#60-95]

In order for it to become an accepted standard, it should be really easy to implement. I'm not sure yet how easy or hard it is to implement start/end times.

Another way to reduce the size of the SGF file would be to use smart defaults. E.g. an empty property VO[] could be interpreted as sgffilename\movenumber.wav. This would however require that each move description be kept in an individual file, which will certainly create more overhead.

Cheyenne: I think having the actual file name within the sgf file is bad. It makes an assumption that the path to the file will never change. What happens if I transfer the file from one platform to another, or from where the path to the file is a URL to where it is now a local file? who or what is going to keep it in sync. I think what would be better is to simply create a tag that the viewer/player then associates with the sound file (with external hints maybe). For example:

 VO[A#60-96] ... VO[B#0-90]

Then let the viewer create the association between A and /relative/path/to/the/file.wav

Juan An alternative is to include the actual sound binary, not just a pointer. Of course it would increase the size of the sgf file, even if encoded as MP3 and depicted as base64 chunks in the file...

LithiumTwo: Zarlan's comment "This would of course only be useful for reviews and lessons" sums the useless of this idea up quite nicely - because currently lessons and reviews CANNOT be stored in sgf - because teachers have the freedom to go back and forth, and such tags are not in the current sdf specification. Also, to keep the game in sync with the sound file, every node of the game would need a <B/W>L (<black/white> time left) property. unless the sound files were going to be stored separately, which someone has already said would be a stupid idea... now that we've established that this would be useless for teaching and reviewing and in this regard it remains to say that people don't usually talk during internet games. If employed this tag would damage the longevity of sgf files either because urls would change or because, for an archieve service (such as KGS), much fewer games could be supported because of the tripling of the actual sgf file sizes (because of the <B/W>L props.) and because of the sound files added on top.

Ithelog?: Original purpose of SGF standard has been to make game files very portable. Sound files in this sense are a problem. How ever, I have proposal to address this issue. Let's define separate higher level file that has both the sgf-file and the sound files indexed within it. This does not solve the issues about file renaming, but it would make easy distribution of the sgf-file as it stand separate to the sound information. If the sound files exists, the usual distribution would be some sort of compressed package and having index file + files in that makes sense. I object the idea of having this index information in sgf-file itself. It does not belong there, but the sgf standard could be extended to support higher level index file with more versatile properties.

(Sebastian:) I like Ithelog's alternative, and I'm therefore retracting this wish.