Page difficulty discussion

Dieter: Four and a half year after the discussion below, the results for difficulty level searches are:

• 112 introductory pages, among which some systematic joseki
• >500 beginner pages
• 280 intermediate
• 480 advanced
• 213 expert

The two highest ranked players known to regularly publish something here are unkx80 and Minue. They have concentrated on beginner exercises and haengma for beginners. So, our experts seem to concentrate on beginner stuff. OK, other experts such as Bill and Robert have tackled more specific stuff, such as CGT or rules. Still, we don't seem to have the stretch of expertise to warrant a division into 5 levels. I for one cannot choose between intermediate and advanced, ever.

Much of the pages listed with a difficulty level should have another, or none (homepages e.g.). It's too much stupid work to manually change these. Either we find a tool to mass change, or we accept the low relevance of the difficulty level.

Herman Hiddema: My own personal guide in this is:

• Expert: Stuff I (3d) can learn from. Stuff that is difficult for me. This should generally be by professionals. Should be interesting study material for 1d and up.
• Advanced: Stuff that I generally know, but do not consider easy. Should be interesting study material for 3k-3d
• Intermediate. Stuff I mostly know and do not find challenging anymore. Should be interesting study material for 10k-1k
• Beginner. Basic stuff that most players know. Study material for 10k and weaker.
• Introductory. Things that can be learned immediately by any player. Mostly examples of term (ladder, snapback, etc). Good for those players that know the rules but not much more.

Dieter: In my opinion the big difference between a 3d and a 6k is the consistency and willingness with which he applies what he knows, while keeping the versatility required to be a good go player, rather than the theoretical knowledge he is familiar with or finds challenging. "Advanced" and "intermediate" players have about the same theoretical framework, but vary in the number of mistakes they make against it. For instance, it is not difficult to understand why one should play away from thickness. You can explain that to a beginner, using only the rules and some first concepts such as territory, influence and life. Yet, you will find a 3d to apply this principle much better than a 6k.

So, in terms of theoretical level of difficulty, I see the following categories:

• Introductory: only the few articles someone with absolutely no idea of what Go is, can read. E.g. Stone, Board, rules of go - introductory, Sensei, ...
• Beginner: everything an dan player can explain with full confidence. territory, moyo, hane at the head of two ...
• Advanced: everything a dan player has the framework to understand, but not the maturity to explain to others. I'm thinking of atsusa, the magic sword, ...
• Expert: intricate CGT and rules stuff, pro game discussions by pros for pros, igo hatsuyoron elaborate discussions

Herman Hiddema: I agree to a point with your assessment of the difference between a 6k and a 3d, but I would say that there are a lot of things that a 6k knows theoretically, but does not truly understand. Playing away from thickness is a good example. Many 6k players know this rule, but have a very hazy understanding of what thickness really is and what too close is in that context. There is, in my experience, also a gap in knowledge. Most 3d know a lot more joseki than a 6k, especially in the broadest sense of the word, where it includes joseki sequences resulting from capping plays, shoulder hits, 3-3 invasions, etc. This is mainly, I think, because the only way to truly learn those things is to apply them in games, and most kyu players have simply not played as many games as the dan players.

Your definition of expert is more narrow than mine, and probably better, but there is very little material on sensei's that falls into that category then.

Bill: What strikes me as curious in the stats are the relatively few intermediate pages. You would think that the numbers for intermediate pages would be larger than those for advanced pages. Now, I have no problem with considering dan players experts. We know they are not pros. So maybe it would be a good idea to merge the intermediate classification into the advanced classification. That would leave us with Dieter's four classifications: Introductory, Beginner, Advanced, Expert, but not necessarily defined in any crisp way.

unkx80: Regarding Dieter's comment on me, I guess that popularity of the beginner exercises was a big motivation for me to post more, heh. My knowledge in Go is heavily biased, so basic tsumego is where I am most confident in, while other topics such as joseki and opening are rather fuzzy to me and I do not have much confidence in these areas. But that is just me, I think most others have a more balanced knowledge in Go. In any case, I believe in the ability in recognizing basic patterns in tsumego and also in reading, hence my focus in the "easy" stuff.

As for page difficulty classification, I admit I don't really have a good idea what intermediate level should be like, and usually end up classifying pages as either beginner or advanced. However, I am not sure whether merging the intermediate classification into the advanced classification is a good idea. Also, I think Dieter's rough classification scheme sounds good, except it seems to break down when it comes to classifying difficulty of tsumego problems. Dan players should have no problems explaining both crane's nest and two-stone edge squeeze, but I don't really think the latter belongs to the beginner classification. In a way, I agree with Bill's suggestion that classifications need not necessarily be defined in any crisp way.

Bill: We now (Nov. '03) have 5 difficulty levels, from Introductory to Expert. However, I see that Pages for Beginners still talks about the Beginner level as though it were the lowest, and says nothing about the Introductory level.

What do we do about this now? I suppose that page levels could be changed automatically, but that's no guarantee that they would be right. Inspecting and changing the level indicators seems like too much to ask of one person, or only a few people. Maybe we can make changes gradually, as people notice that a level needs to be changed.

John F. Does anyone ever look at difficulty levels,

Charles Moi - all the time

and if they do, do they care? I'm trying to visualise a typical page as I type this, and I haven't a clue where the level is shown.

More dogmatically, I wonder whether setting difficulty is worth the effort - could even be counter-productive?

Charles Disagree.

While there are lots of people who like to be prescriptive and tell others what is good or right for them, there are far more who hate being pigeon-holed. If you read something you don't understand, you either assume it's badly written - which you can often tell at a glance - or assume you have not reached that level yet. Isn't that all you need to know?

Charles F'rinstance a joseki. They in practice come in at one of the top three levels. I think it's well worth the effort of marking the top (Expert = dan) ones; and so on. Most joseki books award stars or otherwise flag importance/complexity.

Also 'forbidden fruit': I'm sure people look at Expert pages for a guilty thrill, too.

John F. Sorry, can't see. I may get a thrill if a page is marked expert, but if I'm looking up joseki I'd look at all pages anyway. As to the fr'instance, I'm sure there's a correlation between length and difficulty in joseki - again that's all I need to know. 99 times out of 100 I'm sure you can adumbrate the difficulty in the title - "Basic Joseki", "Liberties for Beginners", "Advanced Zen breathing exercises for go players."

Bill: John, you make some good points. But it does not feel to me like grading the page difficulty pigeon-holes the readers. After all, they can read what they want to. And they may find the level helpful. When I had been playing go for a year or two, Igo Club was just right, while Kido was too difficult. Later, Kido was just right. (For other readers, those were go magazines.)

And the difficulty level can help the writer. Take, for instance, the number of liberties in a Big Eye. I'm not going to mention them at all at an Introductory level; I'll address it for Beginners, showing them examples; I'll link an Intermediate page to the appropriate page; at higher levels I won't bother to explain anything, assuming that they know already. Who is my audience?

Charles I've just gone to Advanced Find Page and searched for Intermediate level pages with the Strategy keyword. The results were quite gratifying: 16 pages came up, and they included for example box formation, shinogi and motare, all of which would be helpful to a player around 10 kyu who wanted some idea how strategy is discussed.

So in the light of that, I think the levels are very useful here.

tapir: 5 level of difficulties are fine with me, but they are not widely enough used. Less than 1 in 6 pages has a difficulty set.

Older discussion, from the days when the Difficulty levels were fewer

Alex Weldon: There are essentially three difficulty levels for pages on Sensei's, not counting professional (never used, since none of us are, as far as I know), and n/a ("not applicable", not really a difficulty level). Dan level is self-explanatory, but there seems to be a great deal of confusion about the difference between "beginner" and "advanced." Judging from some pages, beginner is maybe 30k-16k and advanced is 15k-1k. But other people (probably dan level people) seem to label pages as if "beginner" is anything weaker than 5k.

I've noticed this many times, but the inconsistency I noticed today, sparking me to write this, is Snapback Problem 26 and Snapback Problem 27. To me, they seem about the same level of difficulty; I got them both right on the first try, and each took me maybe 45 seconds to read out before I was sure of the answer. Yet, on Snapback Problem 27, Bill comments that he'd estimate it as a 2k problem (which, as I say there, is probably an overestimate, since I'm 5k and it wasn't hard for me, but still), and it's rated "advanced" (which is what I would have called it). Meanwhile, Snapback Problem 26 is rated "beginner." To me, a "beginner" is someone who is still learning the basics, like what a snapback is, or how to read out a ladder. I would not expect a "beginner" (in my sense of the word) to be able to really understand the solution to that problem after having seen it, let alone be able to find it themselves.

This is just an isolated example, but there are others. Everyone knows what "Dan level" means, but "beginner" and "advanced" are so ambiguous and inconsistently used that they are, IMHO, essentially useless.

Perhaps what we need is to change it to a four-level system with more objective level names: 30-21 kyu, 20-11 kyu, 10-1 kyu and Dan. Still not perfect, since obviously not all 15 kyus, say, know and don't know the same things, but better than what we have now.

If this weren't a Wiki, a good system would be if every registered user specified their rank, and every page had an option to click "above my level / can't understand," "just right / useful for my level," or "below my level / mastered it already." The site could collect this information and assign a range to each page, e.g. "Best studied by: 12-15 kyu." One could then search for pages by appropriate skill level, as well as by name. Most obvious problem (aside from the fact that it doesn't really go well with the Wiki nature of the site) is that it would likely result in a lot of misinformation at the lower ranks, because most players would only be reading, and contributing to, pages around their own level.

Anyway, what is everyone else's opinion? Any way for the little drag down menu labelled "difficulty" to be made useful, or should we just continue assigning difficulties arbitrarily and then ignoring them as most of us seem to be doing now?

mAsterdam: As far as I read the people who did write about this somewhat earlier agree with you that the levels and the way it works, especially with problems and exercises need refinement. (See ^[1]).

The problem is to get consensus on how. Any solution that would require extensive or involuntary moderation is no good. The draft proposal below does not address the whole problem but it would be a small step in the right direction.

Bill: I agree that both Snapback 26 and 27 are far from Beginner level, however that is understood. I am not sure that we want evenly divided levels, however. People are considered newlyweds during the first year of marriage. People who take to go and play a good bit typically advance to IGS 15-kyu or above in a year. That might be a good place to put the upper limit on Elementary (with an Introductory level below that).

But there is a good bit of guesswork to rating difficulty level. Can we have a self-rating system? For instance, put a counter after each level and let people click what they think is the appropriate level for the page. So we would see on a problem page the current tally. E. g.,

Elementary - 13
Intermediate - 35
Advanced - 2

I realize that that could be easily abused, and that, sooner or later, some vandal will do so. Would such a system be workable?

I think that we might make an exception for introductory material which is labeled as such in the title.

Charles Seems elaborate to me. I guess it's agreed that two kyu levels isn't enough. An 'ultimate' system would have levels 4 kyu apart, I believe. We should be able to move to a splitting of the current Advanced rating without prejudice to the longer term. I haven't noticed a single 'difficulty level edit war', so I don't see that any of this is really contentious, currently.

Hu: A counter is an imaginative idea. However, not only would it be prone to abuse, but the abuse would be hard or impossible to repair.

Charles So, my idea is hardening to this: make three ranges 1k-8k, 9k-16k, up to 17k now. Divide those in two, one day, when we have some more idea of levels.

Hu: I began writing "It appears some agreement", (2) below, and then a self-dissent (trying to understand it from all angles), but I have moved it up first (1), because maybe we are too quickly coming to agreement without brainstorming first.

(1) How about a rank range at the top of most pages as a text string in a standard format that a deshi could edit simply. Then the search engine could find all pages that meet a rank criteria. For example a page rated "15k - 5k" would be found by searches for "10k and weaker" and "10k and stronger", but not found by searches for "strictly 10k and weaker" and "strictly 10k and stronger". As I read more carefully above, I see that Alex has proposed something similar but more automatic that I agree is probably not practical for this Wiki, due to programming complexity, and difficulty of fixing up after abuse.

Using a rank range has important advantages: (a) It is flexible and bridges over rigid boundaries; (b) It does not require defining fuzzy terms such as "intermediate".

The text string could be something like "[range 15k - 5k]" positioned right at the front. It would be interpreted by the Wiki engine and reformatted for positioning on the page and for accessibility by the search engine.

mAsterdam: I like the semi-free format. Rank range. No split necessary. I have nothing to add. I want to take back my proposal.

(2) It appears some agreement is coalescing on four levels, roughly: beginner, intermediate, advanced, and dan (pro to be eliminated since it's unused). To move the debate along, let's take those as the "proposal", and discussion fine points. If more or fewer levels are desired, they can be discussed in dissenting opinions. Two issues I think need to be defined are the names to be used and the rank split.

Names should be short (number of letters), meaningful (M), and easily understood (U). Several interesting alternatives have been proposed, which are mentioned without attribution.

"beginner": in use, 8, M, U
"elementary":  10, M, U
"introductory":  12, M, U
"double-digit kyu":  16, M, U
"single-digit kyu":  16, M, U
"DDK":  3, M
"SDK":  3, M
"intermediate:  12, M, mostly U
"advanced": in use, 8, M, mostly U

novice, novitiate, initiate, adept, master, apprentice, journeyman, expert, virtuoso, learner, neophyte, tyro, greenhorn, tenderfoot, proficient, skilled

Ranks could be split as: up-to 20k, 20k to 10k, 10k to 1k, 1d and up. There should be some recognition that any split is going to be fuzzy and therefore some overlap should be expected and perhaps made explicit.

Bill: I think that it's important to have a category for true novices, with introductory material. For instance, anything more difficult than killing a group by playing in the center of an eye with 3 points would probably be too advanced for that category. If I had to give a rank range for it, I would say 30 - 40 kyu.

unkx80: I agree. The categories "Introductory", "Beginner", "Intermediate", "Advanced" and "Dan"/"Expert" would be good. =)

2003 July 10

mAsterdam

There is clear consensus that a measure would be an improvement, there are just differences in the details. I suspect that as soon as there is enough agreed upon detail for implementation, Arno will not hesitate to build it. While I also could accept all alternatives above, this is what I guess has most support. I like the range idea but I also like small steps. One small step that

• would not hurt later further differentiation/more advanced mechanisms in the future
• would immediately help address the stated problems

Lets's ask Arno at guinea pigs feedback to create these levels:

proposal

• n/a
• 40-30 kyu
• 29-21 kyu
• 20-10 kyu
• 9kyu - 1kyu
• 1dan - up

Meaning:

• 40-30 kyu - Introductory
• 29-21 kyu - Beginner
• 20-10 kyu - Intermediate (a.k.a. DDK)
• 9kyu - 1kyu - Advanced (a.k.a. SDK)
• 1dan - up - Dan

The levels would not be exclusive, so a page could be aimed at e.g. 9kyu - up.

Current 'Beginner' pages may start as "20-10 kyu", the rare real beginner pages will be edited manually. Dan and Pro pages start as "1dan - up".

Please comment, as no comment would probably mean: no implementation (this does not mean: please agree, though lots of subtle differences would not exactly help or speed up things).

[1] Moved from Metadiscussion (from here to below):

2003 June 8

BobMcGuigan There has been some discussion about keeping things simple on beginner pages, something I support. So when I read Charles's new splitshape page I did not add a comment that the split shape often arises as a result of an ignored ko threat. But that seems a useful thing to know so I'm not sure whether it is "telling them too much" or not.

unkx80: I vote "no". A lot of times people will add tidbits of information here and there now and then (which I am also guilty of), and after some time, the page gets too bloated for a beginner to handle. Or rather, make a separate page for the comment and link it from split shape?

mAsterdam: I do appreciate those tidbits very much. My vote: Deal with the problem once it arises. Just tell (too) much and split it into chunks whenever the chunks get too big.

Charles In this case there are a few examples scattered over several pages. It is much harder to collect those up, than to re-order material if it's too much (really the work of a moment). Probably "Bisecting a Knight is a Big Cut Indeed" should be the central page with other things hanging off it (since it is a proverb). But that could usefully be renamed, too (now Cutting right through a knight's move is very big).

Bill: Beginner is still a large category. I thought that we are planning to have new difficulty categories that will distinguish between utter novices and experienced but still unlearned players. That will help.

Hu: I'm with Grauniad: beginner, double-digit kyu, single-digit kyu, and dan make for easily defined categories. Having the double-digit kyu category will take some burden away from beginner. Four is not too many, rather like initiate, novice, acolyte, master. There is no reason why a topic can't be treated at several levels by having separate pages. As material becomes available pages can be master-edited. The three-word level designations may be a little unwieldy and the abbreviations DDK and SDK would be opaque to some. Single-word designations, if chosen, would also have to be explained somewhere.

mAsterdam: Bill, Hu and Grauniad like to see more differentiation in the skill levels, especially for beginners. I agree. Let's get specific. Will this do?

Draft proposal: 5 (4 of which will be used) levels for the audience a page is aimed at:

1. aspirant: completely new to go up to say 24 kyu
2. double digit kyu
3. single digit kyu
4. dan: shodan - 4dan
5. master: 4dan and up (current pro)

The levels would not be exclusive: like with keywords, one page can have more levels. Initial:

• Current 'Beginner' pages will start as ''double digit kyu" (most are, I think), some will be edited real soon to become "aspirant"
• Current 'Advanced' pages will start as ''single digit kyu"

If after this there is still too much in one category maybe guild/monk-terms (novice, acolyte) or military rank-terms will be needed to split even further.

Please add your comment.

2003 June 6

Bill: Do we need more difficulty levels? The current discussion about ''liberty'' makes me think so. Below dan level we have only advanced and beginner. I have always referred to double digit kyu players as beginners, but some object to that. And that's something I understand and respect. Currently, a lot of SL pages with "beginner" difficulty seem (to me) to be aimed at that group, rather than at complete novices. Anyway, material for complete novices is really special, and if we aim "beginner" material specifically at them, there is a huge gap between that and "advanced".

What about a few more categories, such as Novice, Elementary, Intermediate? All that plus Beginner is too much, but we need at least one more category, it seems to me.

Dieter: Yes, I second that. The "advanced" category IMHO may also be abolished in favour of dan. I can't qualify myself as an advanced player or a theorist who has grasped the advanced subjects, but if there is discussion about technical stuff among the dan players then that says something about the level of the stuff. "Advanced" could stay but then only if some pro or a reknowned author has written it. In the kyu levels I wouldn' use grade categories as subject levels so there my vote goes to novice - elementary - intermediate.

Charles I could agree to that structure: not necessarily under those names. 'Introductory' is more consistent than 'novice'. If you take out 'dan level' you need to explain rather more: 'advanced amateur' is maybe possible. I'm not sure I agree with the comment about author - is it compatible with wiki ideals?

Bill: I like the idea of Introductory. But I would abolish Beginner, then. If you have been playing for 2 years and just broke the 20 kyu barrier, it is demeaning to be referred to as a beginner.

Also, how about putting "Introductory" in the page titles? E. g., Liberty - Introductory, Rules - Introductory?, Endgame - Introductory?. Then if and when the player wants to know more he can look at the reference pages. The reference pages might be simplified, as well, putting some material on the introductory pages. The introductory and reference pages would be cross-linked, OC.

Arno: I think having separate pages for some topics is a good idea. About the nomenclature for difficulty: instead of "Beginner, Advanced, Dan Level, Professional" you'd like to see "Introductory, Elementary, Intermediate, Dan Level, Professional", correct?

Charles Sounds good. A more neutral way to put Dan Level would be Expert.

Dieter: Well, if it must be advanced or expert then so be it. I am a dan player but I'm not at all an expert.

Grauniad: Why not: beginner, double-digit kyu, single-digit kyu, dan? DDK and SDK are clearer than elementary and intermediate. Actually, even four levels may be too many. Why not simply: beginner, kyu, dan? Any boundaries are going to be very fuzzy.

Bill: That's pretty much what we have now, with advanced instead of kyu. It seems to me, however, from recent discussions, that that puts too much strain on beginner. There really is too much of a range.

If we had the beginner category for complete novices, and then only had kyu up to dan, that would put too much strain on kyu. Anyway, that's how it seems to me.

Dieter: In response to Arno's proposal for the page difficulty header: the proposal is ok but I can't agree with the spirit of mapping dan & pro onto master. I'm a dan player but by no means a master of the game. I can grant myself the "advanced" level though.

unkx80: Since none of us are Go masters here, I guess the "expert" level should be used instead.