Forum for MoGo
MoGo playing strength [#1286]
I was curious about MoGo's playing strength, so I decided to play a few games against it. I downloaded MoGo and used Drago as a client program.
The hardware I used is a laptop with a single 1.73 GHz processor, 1Gb of RAM, running Windows XP.
I played against MoGo 9 times, giving it 9 stones (I am 3 dan EGF). I gave it 12 seconds per move, and played as quickly (usually quicker) myself.
Of these 9 games, I won 3 and lost 6, so 9 stones seems just a little too much.
Given these results, I would estimate MoGo's playing strength at 6 kyu under these conditions. MoGo will perform better under Linux (under Windows it is 30% slower), will perform better with more processing power, and will perform better with more time.
Has anyone else performed similar tests?
Updated to give an estimate of 6 kyu instead of 5 kyu, I made a basic calculating error here, thought 9 stones was the normal handicap for a 3 dan against a 6 kyu (3+6=9), but its the normal handicap against a 7 kyu. My estimate puts MoGo as 1 stone stronger than that, so 6 kyu.
I've always found the additive property of handicap to be under most pressure in the low dan - low kyu range. Very difficult to give a 5 kyu 6 stones (which is what I should do as a 2d). Your estimation may hence be a little optimistic. Most 5k will porbably be able to beat the program.
It is indeed hard to give a realistic playing strength estimate with such high handicaps.
Generally, I can beat 7 kyu players with 9 stones if there is time pressure involved (no more than 30 minutes per person), but have trouble if this pressure is removed.
I do think that most 5 kyu could beat the program in an even game under the conditions I used, but might have trouble against MoGo when it is running on a fast quad-core machine under linux and has a little more time.
The "rise of the machines" in go has arroused my great interest, since I have been very persistent in claiming that go cannot be played at a good level by computers. I still maintain that viewpoint.
I have played three games against mogo. My intention is to play nine games, as Herman did, but the preliminary results already support my hypothesis. I will update this thread when I play more games.
I played mogo on a windows machine (i.e. -30%), with the drago client, giving mogo a nine stone handicap. My system has a 3400+ sempron processor and 2 GB of RAM. I set the engine to 12 sec/move. Also, I enabled pondering, which should allow the engine to think while I am thinking, however I am not sure whether this actually worked. I played much faster myself and I am a dutch first kyu (1k EGF, 1d KGS).
The results where 2-1 for me. I lost the first game, although I was winning, however I made blunder that cost me a group. I could not recover this loss. The second game I won by 8.5 points, although I am not sure about the counting. Mogo played 6 times within his own territory, while I was filling dame points. It seems to me that mogo lost 6 points here and the actual victory should have been by only 1.5 points. The third game mogo resigned after 203 moves. At that time I was ahead by more then 100 points.
Although three games is not sufficienct for statistics, I would like to carefully draw the conclusion that mogo might not be as strong as stated before.
The reason might be that a slight change in tactics is required. In my games i played several moves like 10-6, or 5-5 over mogo's 4-4 stone. Plays like these should be bad, I think, even in handicap games. However, in one game the following sequence was played:
![[Diagram]](diagrams/47/feace00851dd5af68b29fc7e36019f36.png)
I like this result very much ;). Mogo seems to play somewhat influence oriented, possibly because most of the random plays are in the center, especially if he knows not to play on the first line. However, when one plays to make stable groups at the edges mogo gets too much centrum influence. The weird "light" moves in the centre eventually help, even if they are sacrificed.
To conclude, I think Mogo is definitely a step forward in the progress of machine go. However the estimates of how strong the program really is are probably too high. Based on my preliminary study I would say mogo is as strong as 9 kyu on my machine and than I am giving him the benefit of the doubt.
Above is my post, could someone change that?
"The second game I won by 8.5 points, although I am not sure about the counting. Mogo played 6 times within his own territory, while I was filling dame points. It seems to me that mogo lost 6 points here and the actual victory should have been by only 1.5 points."
You need to set it to area scoring. Then the 9 handicap stones will count as points. So instead of beating Mogo by 8.5, in fact Mogo won by 0.5.
Yes, this is important. MoGo does not understand territory scoring and will play in its own territory if it thinks it is ahead.
I have never beaten MoGo on points, it will always resign if it sees no way to win.
Thanks for the reply, i knew something was wrong.
Interestingly, The Drago release notes for version 3.02 (2007/11/24) have a Fix note for Area counting doesn't take handicap into account. ![[ext]](images/extlink.gif)
1
I have version 3.01, which version do you have?
I don't think beating Mogo with with 9 stones allows you to conclude that Mogo is weaker than 9k. UCT programs play lots of weak and/or bad moves when they think they are ahead on the board, which should be the case if you are starting it 9 stones. Possibly the initial 9-stone handicap would equate to at most a 1-3 stone advantage for UCT, since the algorithm would waste all of the advantage before beginning to "play seriously". The sequence you showed in the corner is strong evidence, since Mogo should never play this kind of sequence unless it is ahead by a lot.
It would be best playing it in an even game a few times. If you can beat it at 9 stones 1 out of 3 times as you did, you can probably beat it in an even game reasonably consistently, but you might be surprised at the program's strength.
I have tried playing it in even games, but it is simply no match for me then. If some mid-range SDK players could try this, that would be good.
I've played another two games resulting in a both a win and a loss. (correct scoring this time ;) although probably it doesn't matter, you lose if mogo keeps playing and you win if he resigns)
I am susceptible to the argument that UCT wastes the 9 handicap stone advantage. Possibly Mogo is stronger than 9 kyu. I would really like to see 5k-3k players take up Mogo on in an even game.
I've started giving it 8 stones, but have so far won all three games.
I am a solid 5kyu KGS, 8kyu EGF & IGS and I can beat Mogo everytime with me playing White, no komi, in an even game. It beat me the first game, then I adapted to its strange way of playing. Beating with ease since. Pentium 4, 1.4Ghz 512Mb, XP, 19x19, 12s a move.
Ah, great to see more people participating ,especially in that range!
Could you list what hardware you use and what system (EGF, AGA, etc) you are 5k in?
I have run MoGo on Windows with less than spectacular results, even with very long thinking times, yet when I hooked up a version of it to KGS through GNU/Linux it performed much better. A recent version of it is up as "CzechBot?" and maintains a 3K KGS rating. I believe the official version stayed at 2K.
Also, I've found that it plays best when it manages its own time in sudden death, or sudden death-like settings (e.g., 30:00 + 1 x 21s, given the way KGS reports byo-yomi time). The 64 bit version I'm running uses almost 80% of my memory and 95%+ of the processor's time to make a move in some arbitrary amount of time. It is lightning fast for the first few book moves, slows down a bit for the opening, and slows down a lot later in the game. Giving it a fixed time per move might make it weaker than it should be when it can give up time when it can't use it well early on and use it later when it can really crunch some moves.
Try giving it ~45 minutes absolute or more under Windows, maybe you can take the 30% hit in calculations by giving it more time.
Also, I'm not sure if enabling pondering will help it or hurt it. If you're hitting your hard disk a lot for virtual memory because of pondering, it will probably make it worse than if you hadn't enabled it at all. (Not 100% sure on this.)
![[Welcome to Sensei's Library!]](images/stone-hello.png "Sensei's Library")