SGF Wishlist

    Keywords: Go term

This page discusses new proposals for SGF.

Table of contents Table of diagrams
example 1
same position different order
rotated position
circle=IM

Properties

SGF allows applications to freely define its own properties. However, discussing new properties helps achieve standardization.

Things to think about

We must remember that SGF stands for Smart Game (not Go) Format and that it serves 40 different games. Anyone with a knowledge of SGF and the specification changes over its development should know that properties that have been too specific to one game (and underused then) have not survived revisions. So bear this in mind.

Willemien What is the size of CR?. In FF4 it is not specified. what has the result that some sgf editors use it as a big circle (almost the size of a stone) while for other editors it looks more like a dot. i think the first interpretation is correct but it should just be described

TO - Topology of Board

Property
TO
Propvalue
simple text
Propertytype
game-info
Function
Discrimingnating different topologies from each other. (e.g., "standard", "wrapp-around", "wrap-x", "wrap-y")
Proposed by
Bugzaren

Explanation This might be an odd request (and maybe very specific for Go) bur the fact that the playing field is defined solely through its nodes and links mean you can play around with the topology. For instance letting the board be periodic so that you identify the left and right sides with each other.

SD - Special Definition

Property
SD
Propvalue
SimpleText : SimpleText : SimpleText : !Text
Propertytype
root
Function
to make it easier for sgf readers to detect errors and convey full information the the viewer
Proposed by
LithiumTwo

Explanation

a sample of sgf (to apply to the PT tag below) might be SD[PT:SimpleText:game-info:The type of problem posed in this sgf]. Where the first value is the tag to be defined that is outside the file's FF specification, the second to be the type of variable for the property (composite types of "(simpletext:number)" may be possible), the third to be the type of property, and the fourth would be a description of the property. The reason for this would be exactly that if people needed a new property for their program, then this file would be perfectly portable, and error checking (and correcting) would be made much easier as standard sgf programs know what to look out for. In addition, for those intending to create new properties: sgf editors should have the capability to easily add these new properties to the sgf (even if they can't interpret them as the specific program will), and they can be displayed to the viewer of these sgfs in any normal editor.

Discussion

MM - Move Marker

Property
MM
Propvalue
simple text
Propertytype
game-info
Function
the appearance of the marker for the current move in a node (e.g., "none", "cross", "reddot", "greensquare")
Comment
unifies the appearance of the board over all sgf viewers. There should be several standard shapes.
Proposed by
Harleqin

Discussion

When this property is used, all sgf viewers mark the current move in the same way, so confusing text ("the stone with the red dot" "what red dot?") is avoided.

The motivation is to finally get wms to abolish his "CR" hack. :) (See also KGS Issue CR Marks On Moves.)

LithiumTwo: This is a bit redundant... people usually say "the current move" or it's coordinates... or even "this move"... I've never heard anyone refer to "the stone with the circle on it". However, the main reason it's redundant is that wms has already abolished the CR hack =)

axd: Abolished? You mean it's no longer there? 2.6.12 still has it, btw.

PT - Problem Type

Property
PT
Propvalue
simple text
Propertytype
game-info
Function
the type of a go problem contained in the SGF file (e.g., "Tsumego", "Fuseki", "Capturing", "Connecting", "Yose", "Semeai" etc.)
Related
PD
Comment
Problem solvers recognizing this property could automatically group problems into categories and let the user search for them (e.g., "give me all yose problems")
Proposed by
maruseru

Discussion

The main motivation comes from the existing practice of storing tags such as the problem type and difficulty outside the SGF file. This makes it difficult, when renaming, moving or copying the SGF files to keep them in sync with the tags. I'd really like to see this information being kept inside the SGF file.

I've added support for editing and searching PT[] and PD[] properties to kombilo and submitted the patched version to kombilo's author, Ulrich Goertz. I've also added a rather basic problem-solving mode to kombilo. Now it would be nice to have support for these properties in problem solvers such as GoGrinder and Go Album. That way you could edit your problems on the desktop and deploy the content to another application (possibly running on a mobile device).

Ithelog: Goproblems.com use tag GE[] for this purpose

PD - Problem Difficulty

Property
PD
Propvalue
simple text
Propertytype
game-info
Function
the difficulty of a go problem contained in the SGF file (e.g., "Beginner", "Easy", "Medium", "Hard", "Pro")
Related
PT
Comment
Problem solvers recognizing this property could automatically group problems into difficulty levels and let the user search for them (e.g., "give me all hard tsumego problems"). Obviously classifications such as "easy" and "hard" are subjective, but they might denote rank intervals. For example, "beginner" = 30k-20k; "easy" = 19k-10k; "medium" = 9k-1k; "hard" = dan; "pro" = pro (suprisingly; for example, the Igo Hatsuyoron problems would have a PDPro).
Proposed by
maruseru

Discussion

Benjamin: I would propose more fluent way of representing a problems difficulty (e.g. something like problem-{EGF}"rating") to give problem databases a way to further divide the probs according to user statistics.

IanDavis: how do you propose accurately finding the correct EGF rating for a problem ;-)

maruseru: I've created a page for discussing problem ratings.

Ithelog: Goproblems.com use tag DI[] for this purpose

BE, WE - Elapsed Time / or TS - Time Spent, var. 1

Property
BE, WE
Propvalue
floating point number (number of elapsed seconds)
Propertytype
game-info, move
Function
document how much time elapsed while the current mover was on the clock.
Related
BL, WL
Comment
Depending on the time system used, you frequently can not deduce the elapsed time from the BL and WL properities.
Proposed by
jfc

Discussion

jfc: This property is useful for playing back a game at the speed and tempo it was originally played. This is particularly useful for games that spend most of their time in Japanese byo-yomi as the BL and WL provide next to no information about the tempo of play during byo-yomi.

it could also be called TS = time spent (no need to differentiate on colors black/white), the value would be a floating point number in seconds, like above, specifying how long the player took to think about the move in the current Node. Very useful property!

T - Absolute Elapsed Time, var. 2

Property
T
Propvalue
floating point number (number of elapsed seconds)
Propertytype
- (node annotation)
Function
time stamp, documenting how much time had elapsed since game start when the corresponding set of properties was created or revisited.
Related
BL, WL
Comment
One node may contain more than one T property, but a file must not contain any two T properties with identical time value.
Algorithm to replay a file along time stamps:
      (0) Starting from T[0.0] at game info,
      (1) the viewer searches for the stamp which temporally (not necessarily
          textually) follows the current one.
      (2a) If no stamp specifies a later time:
               Replaying is stopped.
      (2b) Otherwise:
               As soon as that next stamp's point of time has passed, the viewer
               jumps to there and executes all subsequent properties until it
               encounters another T property or EOF, then does (1) again.
For convenience, sgf recording and/or replaying software should be able to pause.
Proposed by
blubb

Discussion

blubb: I'd regard such absolute (cumulative) time stamp tags as more promising, because they would allow to replay even highly branched reviews in original order and timing, while also providing all the information BE and WE could define.

LithiumTwo: this would be very nice for complicated reviews.

jfc: In the interest of recording the game flow it might also be useful to have an 'adjourn' property that shows when a games was stopped for any reason (e.g. when I lose my connection).

Ithelog?: This would be very useful for reading lectures recorded in the sgf-files afterwards. Currently there is no way to see in which order the lecturer has visited the nodes of the game.

KK - Collection

Property
KK
Propvalue
simple-text
Propertytype
game-info
Function
Apparently used by the GoGoD PBase to indicate problem collection (e.g. KK[Guanzipu]).

OM - Open Marker

Property
OM
Propvalue
none
Propertytype
node
Function
Indicate which position should be shown first after sgf-file has been opened
Comment
Currently application decide whether to show start of the game or end of the game when sgf-file is opened. With this marker, application could show first defined at the file. In my application I would place the marker at the position shown on the screen when file is saved.
Proposed by
Ithelog

Discussion

blubb: To ensure consistency, you'd either need to limit OM to one per file, or to provide a rule how to deal with multiple OM occurrences (e. g. due to merged files).

LI - Link Redirect property

Property
LI
Propvalue
to be discussed
Propertytype
node
Function
To make a position independent of the order of moves were possible

'Comment: Currently different order of moves lead to different nodes tn the gametree while the resulting position is the same. I would like to be possible to have an redirect property that when you change the order of moves where applicable you gat to the same node. So that the following comments and followup moves only have to be added once.

Proposed by
Willemien
Example
The following games lead to the same position
[Diagram]
example 1  
[Diagram]
same position different order  
[Diagram]
rotated position  

Maybe even more advanced would be that you also rotated / mirrored symetries can be redirected to the same nodes (athough then there are problems with coordinate changes.


RA, WA - Right answer and wrong answer markers

Property
RA or WA
Propvalue
none
Propertytype
node
Function
Indicate whether the variation which follows is correct (useful for go or chess problems).

Arno: There are already many markers for good moves, bad moves, good and bad positions, interesting and doubtful moves. See the [ext] SGF user guide for an overview.

IM - Illegal Move

Property
IM
Propvalue
list of point
Propertype
setup
Function
Indicate that the listed points may not be played on this move.
Comment
Mainly of use in composing tsumego in SGF format that begin with the other player having just taken the ko. IM would then be used to indicate the point the ko capture occurred on:
[Diagram]
circle=IM  

Here the circled point would be marked with IM in the SGF.

Proposer
TheBigH

Menu, Table of Contents

I was just reviewing the counting lectures by Dr. Straw. While it's awesome that a 5 dan makes these SGF files up for anyone to read, I found the SGF file itself a little hard to deal with. For the counting lecture, Dr. Straw started with a finished game to show the counting, then had the viewer return to previous positions in the game that he had anotated.

I'll say here that the result was completely useable, if one was willing to work at it. The jumping around was a bit confusing but not overwhelming. I just found myself thinking "Hmm, a little programming could make this really slick." So this is not some huge need that I'm discussing here, but rather something that would be nice to have.

The menu (or Table of Contents, if you like) should work something like the TOC in an Adobe Acrobat document. The user should see a menu, in order, of positions in the SGF file. Clicking on a menu item will take the view directly to the spot in the SGF file.

Its important that the menu have a definate order. Its no good, for example, for the creator of the SGF file to make a menu that says "Lesson 1," "Lesson 2," and "Lesson 3" if the user sees them in the reverse order. Likewise, the menu items themselves have to be able to "reference" arbitray points in the SGF file. Remember that Dr. Straw's lecture started at the end of the file.

For example, here a simple implementation that take a "list" of pairs. The "list" is in order that the user should see. The pairs consist of a menu entry, which the user will see, and the reference which the user won't (probably) see.

MENU["Lesson 1":LB[_START], "Lesson 2":FG[FIG1], "Lesson 3":VW[A]]

Here the first menu entry points to label (LB) "_START". The second menu entry points to figure (FG) "FIG1", and the final menu "Lesson 3" points to view (VW) labeled "A".

Yes I know this isn't strict SGF format, I'm just puting this here to spark ideas. If this looks like someone might implement it, we'll nail down the format better at that time. Ok folks there it is, rip it to shreds. (By Gojira.)

Ithelog?: Elapsed time proposal 2 above would solve this issue. For solving this issue it would we adequate to have optional node tag for order of the nodes, where each order would be allowed only once and each node may have any number of orders. The elapsed time proposal 2 above is however more powerful in form it is proposed.

Text Formatting, Mark-Up

And while I'm at it, Dr. Straws SGF file was readable, but I also found myself wishing for some sort of mark-up. So how about putting HTML in the comment section of an SGF file? Normally I'd be against including such a "heavy-weight" library (After all, heavy shape is bad; sabaki is good ^_^). But HTML libraries are so ubiquitous now I think it's reasonable to just add one to an existing SGF editor.

For SGF editors that don't wish to support HTML, it should be easy to write a simple parser that ignores and strips out the HTML, leaving the user with a reasonable comment to read.

Lots of other suggestions could be added here, but I'll wait to see if there is any interest first. (By Gojira again ^_^)

Hyperlinks and Labels

Hyperlinks in sgf's would be very useful. I don't mean hyperlinks to websites or html pages, but rather to labeled nodes in the sgf file. This would also allow the easy construction of the 'Table of Contents' feature described above. Of course, this would require that labels be added to the standard as well. I've been working on an sgf file containing the basic corner shapes and their status. It would be very helpful if I could create a table of contents in the top node comments, with links to the base nodes of each of the shapes. And of course it would be really cool if the comments for a particular corner shape could have links to any related corner shapes.

Proposed by gnos?

additional comment by gnos?: The labels feature permits the sgf viewing program to have a 'labels browser'. This would allow the user to jump around the sgf file and know where he is going and be able to get back. This is in itself an extremely important feature that is currently lacking in the standard. Please implement this!

Reordering of Nodes

Nodes should be reorderable. This would facilitate e.g. the creation of dictionaries with multiple possible ways of ordering the data. Basically, the idea is that there would be a method for changing the position of nodes within a given branch. If there is a vertical difference between the node's positions, then obviously this should not be possible, or if it is it would be a different feature. But if they are on the same plane laterally, then they should be reorderable. The way to do this is to create sytematic labels for nodes on the same level of a certain 'branch'. Basically, nodes could be labeled with an ordering property. So for example, you might have one ordering label called alphabetical?, and individual nodes would be labeled [alphabetical [1]] [alphabetical [2]] etc. and if the user selects the 'alphabetical' ordering, (from a link in the comments, or from a button or menu item in the sgf viewer, or by any other means), then he will have the nodes on that particular branch ordered alphabetically (this would of course allow for the author of the sgf to err, and accidently place some nodes out of order within the alphabetical ordering, and I consider this a good thing more or less.). The same set of nodes could also have other ordering labels, e.g. [SizeofShape[1], SizeofShape?[2]], etc. Of course, if there are some nodes left unlabeled within a certain ordering system on a given branch, they will simply not appear when the user selects that ordering system. And again, I consider this a good thing, it's up to the author to make a sensible sgf file after all. The user must always be able to select "no ordering", and just get ALL the nodes entered by the author and in the order the author entered them.

suggested by gnos?

xx - Bookmark

MultiGo already implements a BOOKMARK property (visually represented by an asterix on the bookmarked node) that allows to quickly jump between bookmarked nodes (handy when you want to e.g. compare positions).

Ithelog: MultiGo use tag MULTIGOBM[] for this purpose

Can be implemented using HO property?

xx - Home

Relating to bookmarks HOME property would be usefull. Basicly home is like normal bookmark, but there should be only one home. Meaning for home is to be the position that should be opened when the file is opened. So, in case bookmark is included in the sgf standard, home could be simply special case of it, like BM \[ HOME \] for example.

Multi-Language Support

HandOfHair: It could be useful if a single SGF file could have each text property supported in multiple languages. For example:

C[The default language is English.]C[ja:このコメントは日本語であります。]

(The Japanese version of that comment reads "kono komento wa nihongo de arimasu," which, if I'm not mistaken, means "This comment is in Japanese.")

huh?!? KGS already supports Multi-Languages. Its UTF-8

BM (for bad move) and IT for tesuji, (i.e., good move) are to provide language independent comments. Love to see more of the same, GENIUS; or OOPS; or BECAUSEOF[bc][cd][ef]; for because of this sequence. I believe that I read there has been pressure to move away from comments, because they are language dependent. That way, HandofHair, your brilliant, insightful, wonderful comments on go games stored in SGF can be viewed in over a hundred languages, translated automatically by SGF viewers and editors, for centuries to come. Every go book I've every read uses phrases like "this/these stones are too far away to help," "the marked stone supports this move," "too close to opponents wall," "should have been either," etc.. It should be possible to combine more then one such item per node. Go is about placing stones in RELATIONSHIP to other stones. Maybe BM[ad][ae][af]; and IT[cb]; labeling the points that make it a good or bad move would be a first step. SABAKI[dc]; HONTE[dd][de]; SENTE[df][tt]; MIAI[dc][de]; ALIVE[][][]. I'll tell you one thing, I sincerely appreciate you getting my gray cells into gear, and I'm going back and skim my go books for the type of language they use. Best wishes, Kbuny. psfrl (at) sbcglobal.net

Existing Properties (extend specification)

DT - Date of game

Problem
The format for full timestamps of begin/end of a game (especially on realtime servers) is not specified. Also, no timestamps for adjourn/continue of game are possible.
Suggestion
The following format is suggested:
  • DT[2010-01-14 12:13:13 +100], full timestamp with timezone information. To convert to UTC time one hour must be subtracted (thus, 11:13:13 +000).
  • Specify begin and end of game DT[<begin>/<end>] with two timestamps, separated by a slash.
  • Or DT[begin <t>/adjourn <t>/continue <t>/end <t>] with full adjournment information

Format

Compression

A standardized compression scheme for SGF files would be useful, especially for programs making liberal use of flash media (Palm pilot, etc).

Compression scheme should maximize (in order of preference)

  1. (Minimize) Processor requirement
  2. Speed
  3. Size

Malweth

AshleyF: You seem to be talking about compression of some binary data (voice, player photos, etc.) but I thought I'd mention that Anders Kierulf (creator of SGF) has implemented a simple compression scheme in SmartGo that applies to game records. It greatly reduces file size for large collections while still maintaining an SGF-like syntax. For example, 40.1MB for all of GoGoD as a single SGF collection compresses down to 15.7MB as an SGC. Check out: [ext] http://www.smartgo.com/HTML/index.html?compressedgamecollections.htm

Malweth: I was talking about compression of the text data. Text such as an SGF can be compressed by 60-80% using RAR or ZIP algorithms. I wasn't aware of the simple method being used by Smartgo (I did realize there was a different file format used by this program). I do see problems with this, however - instead of a bulky compression (complete file) that is done and un-done by a compression scheme - once variations are added in the middle of the SGC sequences, the group must be split - a simple variation could actually end up adding more to the file size than the variation sequence actually requires. In any case, some type of method would be useful if added to the specification (not just as a recommendation).

Collection

A standardized SGF collection file would help put multiple SGF files into a single SGF collection. These files should contain a table of contents.

Such a standardization would be useful for SGF databases and collections. The (optional) compression scheme should also apply to SGF Collection files.

Malweth

AshleyF: The SGF spec already includes the idea of collections; just by including multiple game trees in one file (see [ext] http://www.red-bean.com/sgf/sgf4.html#ebnf-def). Not all clients support it though. SmartGo, to name one, has nice support for it and builds a 'table of contents' view as you suggest above: [ext] http://www.smartgo.com/HTML/index.html?gamecollections.htm.

Transfer Protocol (Single node or branch)

Malweth: A standard protocol to transfer a single node or branch of an SGF file would be useful. This would make collaboration in SGF format possible. It seems likely that the Ultimate Go Server project will define such a protocol.

This need have no firm basis in SGF, but staying as close as possible to the SGF standard would be useful for databasing these collections.

Go Stone Photographs

Photographs of some black go stones, and especially of a few dozen good quality white shell go stones, all lit from the same angle. Place in a zip or rar or ?? and placed on Source Forge or Flickr? or some such. Copyleft or Gnu License or ? Easy enough to come up with pictures of wood for go boards, but the white shell go stones are a challenge. Thanks for your attention to this.

RueLue: are you sure, this is the right page for your wish?

Retracted wishes

VO - Voice

Property
VO
Propvalue
simple text
Propertytype
game-info, move
Function
contains URL or relative address of sound file to be played when game is opened or the move is displayed respectively.
Related
GC, C
Comment
Voice-over-IP is already supporded by some Go Servers. This property would allow to save it.
Proposed by
Sebastian

Discussion

Cheyenne: I have a concern about increasing the size of the SGF file (think of trying to download the SGF from say KGS and then importing into PilotGOne). A suggestion that this facility use two parts. The first part is handle for the sound file. The second part would be indexes into the sound file (start/end times). There could be provisions to handle multiple segment sound files (say within a variation branch). So you would have initial specification of where to find the sound file and then at each point you would only have a small index value to be used to indicate where to start and end the sound clip.

The sgf viewer resolves the connection between the handle for the sound file and the external file (that is either local or online). This should keep the size of the SGF down.

The way that I could see this functioning under say KGS would be that if you downloaded the SGF file you would be prompted if you wished to also download the corresponding sound file for the SGF. The file name would match the file name of the SGF and the file extension would be different (foo.sgf & foo.wav for example)

(Sebastian:) You're raising a good point. The standard could certainly include this in the form

 VO[relative\path\voicefile.wav#60-95]

In order for it to become an accepted standard, it should be really easy to implement. I'm not sure yet how easy or hard it is to implement start/end times.

Another way to reduce the size of the SGF file would be to use smart defaults. E.g. an empty property VO[] could be interpreted as sgffilename\movenumber.wav. This would however require that each move description be kept in an individual file, which will certainly create more overhead.

Cheyenne: I think having the actual file name within the sgf file is bad. It makes an assumption that the path to the file will never change. What happens if I transfer the file from one platform to another, or from where the path to the file is a URL to where it is now a local file? who or what is going to keep it in sync. I think what would be better is to simply create a tag that the viewer/player then associates with the sound file (with external hints maybe). For example:

  VO[A#60-96] ... VO[B#0-90]

Then let the viewer create the association between A and /relative/path/to/the/file.wav

Juan An alternative is to include the actual sound binary, not just a pointer. Of course it would increase the size of the sgf file, even if encoded as MP3 and depicted as base64 chunks in the file...

LithiumTwo: Zarlan's comment "This would of course only be useful for reviews and lessons" sums the useless of this idea up quite nicely - because currently lessons and reviews CANNOT be stored in sgf - because teachers have the freedom to go back and forth, and such tags are not in the current sdf specification. Also, to keep the game in sync with the sound file, every node of the game would need a <B/W>L (<black/white> time left) property. unless the sound files were going to be stored separately, which someone has already said would be a stupid idea... now that we've established that this would be useless for teaching and reviewing and in this regard it remains to say that people don't usually talk during internet games. If employed this tag would damage the longevity of sgf files either because urls would change or because, for an archieve service (such as KGS), much fewer games could be supported because of the tripling of the actual sgf file sizes (because of the <B/W>L props.) and because of the sound files added on top.

Ithelog?: Original purpose of SGF standard has been to make game files very portable. Sound files in this sense are a problem. How ever, I have proposal to address this issue. Let's define separate higher level file that has both the sgf-file and the sound files indexed within it. This does not solve the issues about file renaming, but it would make easy distribution of the sgf-file as it stand separate to the sound information. If the sound files exists, the usual distribution would be some sort of compressed package and having index file + files in that makes sense. I object the idea of having this index information in sgf-file itself. It does not belong there, but the sgf standard could be extended to support higher level index file with more versatile properties.

(Sebastian:) I like Ithelog's alternative, and I'm therefore retracting this wish.


SGF Wishlist last edited by 185.66.193.33 on April 15, 2015 - 23:55
RecentChanges · StartingPoints · About
Edit page ·Search · Related · Page info · Latest diff
[Welcome to Sensei's Library!]
RecentChanges
StartingPoints
About
RandomPage
Search position
Page history
Latest page diff
Partner sites:
Go Teaching Ladder
Goproblems.com
Login / Prefs
Tools
Sensei's Library