Monteo/Superhuman Performance Analysis

Sub-page of Monteo

xela: I've moved this conversation from Cho Hun-hyeon, because we're no longer talking about a specific player, we're now discussing the technique in general. Monteo, putting it as a subpage of your page seemed like the most logical place. But if you'd prefer to have it somewhere else, I can move it again.


Monteo In the 1st Ing Cup Final against Nie Weiping, Cho Hun-hyeon performance is at peak and has been assessed with a superhuman performance analysis technique, his Go skill is a little stronger than Honinbo Shusaku’s (Move Strength = -86.7 for Cho and -87.7 for Shusaku]. In terms of best moves, both Cho and Shusaku play equally well (44% best moves).

kmr: Is this some kind of pseudoscience or just marketing? No serious scientist would draw conclusion based on 1 game! If we check one Jowa game against Gennan Inseki we could draw conclusion that Jowa was below 6 dan ama (in this game Jowa played really poorly, blundering heavily).

When serious chess scientist would tried to compare all World Chamions they analyzed thousands of games yet conclusion was still not 100% sure (there is always some doubt).

Monteo That is the most efficient way, given my limited free time. You can use as many games as you like and would be even more confident in my conclusion. I analyzed another game of Shusaku, when he was younger, his move strength is worse (-98.8); this analysis is quite plausible, a new scientific method which has been validated. I also checked the performance of AlphaGo Zero and AlphaGo Master. Results show that Zero is stronger than Master, which are both much stronger than Ke Jie, consistent with the reality.

kmr - To assess AGZ strenght with enough confidence you need bot much stronger than AGZ, i really doubt you have one.

Monteo I used a modified AlphaZero process and architecture bot, and a new calculation technique. No matter how strong the AI is, even we had the God of Go, game results can be computed correctly. That’s why I used both %best move and move strength to define Go performance. Move strength is a powerful algorithm that can reveal the strength of AGZ’s moves.

Oh so now you have God of Go? Funny, but i have Saint Almanach of Divine Go Moves, and there is written that Cho Hunhyun is slightly weaker than Shusaku and Dosaku (who is also stronger than Ke Jie). The error rate and move weakness is bigger for both Ke Jie and Cho Hunhyun than Dosaku.

Whatever algorith you have, you still need bot stronger than AGZ to assess it strenght, and i guess many ppl would still heavily doubt that.Sorry.

Monteo You don’t need to believe me. I will stop explaining this science to you. Anybody else please feel free to add comments.

kmr - pls do not ninja-edit your comments.

xela: Hi Monteo, I have a couple of questions. When you say his performance "has been assessed", did you do the assessment yourself, or did you find it somewhere else? Can you tell us more about how the move strength is calculated? I'm wondering if there should be a separate page for describing the analysis technique, instead of having it here on a player page where it's not so easy to find (presumably the same technique can be applied to other players).

Monteo I did the assessment. Move strength calculation has been described in Who is the Strongest Player in History?: Superhuman Performance Analysis, a PDF book by WGA. Results really surprise me. I also checked who is stronger between Honinbo Shusaku and his teacher, Honinbo Shuwa, which has long been unknown since Edo. Now it’s very clear to me. Shuwa is clearly weaker. I also checked the performance of Shusaku when he was younger. Results show that he is weaker than when he is older and at peak performance, which also validates the technique.

kmr Sorry but this is wrong. You assume that Shusaku when younger was weaker than when he was older, and then you say it validates your technique. If you assume sth, then you completely does not need to prove it, so your technique is useless to do this. What you now said is logicall fallacy, unfortunately. Either your technique validates claim that Shusaku is stronger, or you assume first that Shusaku was stronger later but then it cant validate your technique. Cant have both.

Monteo You don’t understand and don’t try to understand but try to falsify, so you will never understand.

kmr Results that young Shusaku is weaker than older Shusaku cant validate your techinique, unless you know A PRIORI that younger Shusaku was weaker. Try to understand this, and maybe then you will see that i am not hostile to you or your method. Anyway, its true that i dont understand your method, and nobody else except you here can understand it cuz you didnt explained your method AT ALL, you only said conclusions which can be drawn from your method and application of it.

Did your method is significally different than method of Matej Guid and Ivan Bratko? If yes, this differences may be very interesting, but if no, then your lack of explanations is perfectly understandable.

Monteo I did explain the technique and more validity tests in the book. I don’t have to explain it here. You may try other methods and can ignore the book. I have no problem with that. I am sure AGM is weaker than AGZ, and results are as such without my attempt. I also found that Lee Changho at peak is still weaker than his master at peak and he is weaker than Shusaku. Other great players at peak were also studied, namely Go Seigen, Cho Chikun, and Lee Sedol. Results will not be shown here.

Part II

kmr

  • Sure you dont need to expain method of your technique here, but then you should just make it clear its advertisement.

Monteo There is a thin line between ad and news. I don’t invite anybody to buy my book. I don’t tell you it is a good thing to buy. I tell you what I found, so that people can discuss if they have any better answers.

kmr - I think the best way to follow Sensei Library style would be to make a page about your book like many others authors, where you could reveal anything you want. This page could be linked to players your research refers, so i think everybody would be happy. If nobody will do it, i will do it soon.

  • Also all go world is pretty sure that AGZ is stronger than AGM so this is not big surprise.

Monteo If you very carefully read, you will find that the purpose I mentioned AGZ and AGM is NOT to make you surprised. I am not surprised either that you are not surprised. “AGZ is strong than AGM” is a well-established fact and I used it as a fact to validate my method, which can generate the right results.

kmr - Yes, if we assume a priori fact that AGZ is stronger than AGM (or Shusaku is stronger older than younger), then same result validate your method a bit, but we need to know this beforehand. I hope you see my point.

  • You claimed that Shusaku was stronger than Shuwa - well its perfectly possible, but did you take into consideration that they had pretty different strenght of opponents? Shuwa had opponents like Gennan Inseki (he was a bit past his peak when he played against Shuwa), Yasui Sanchi (he was around peak when played against Shuwa) etc, while Shusaku had either same opponent as Shuwa past his peak or new players which were weaker than Shuwa opposition?

Monteo No need to consider the opponent’s strength. SPA (Superhuman Performance Analysis) differs from Elo rating that considers the tournament scores. SPA considers move quality in terms of move strength. Make sure that the game being analyzed was played by the actual player when he was actually at his peak and he played seriously. That is the key I did concern. SPA results have been standardized by the external benchmarking superhuman neural network, not the human opponent of each match. That’s why SPA can compare the performance of different great players from different times throughout the history.

kmr Sorry to say that, but it sounds very suspicious. Imagine position which white leads solid +50 points and all groups are safe, and rest play is smaller than 3 points.Lets say white play dame. White still leads a lot, and has sure win. Do your method consider this move as same good as 3 points (biggest possible) play?

  • There are many questions (like how you took komi games into consideration,do you took fuseki/joseki development into consideration, what is strenght of AI you used, what AI you used etc).

Monteo Komi doesn’t determine the strength of any player. Komi just affects the game results, which is NOT an indicator of players’ strength—you can win all games when you always play with the weaker players and that doesn’t imply that you are the strongest player in history. No need to consider komi when assessing Go performance. Fuseki development doesn’t determine the player’s strength either. Note also that Shusaku fuseki has not been developed to perfection as many have thought. It is almost perfect and represents merely the strength at the early opening. I found that Shusaku did some big mistake in the middle game but his opponent is even weaker and cannot see it. From a superhuman perspective, Shusaku is a weak player and much weaker than Ke Jie. The strength of the AI used in SPA is at the superhuman level, more than enough to assess human performance. I mentioned the AI name in the book, not here.

kmr - Komi does not determine players strenght thats pretty obvious - but determine players approach to game and their play in this game. When playing white (in no komi game) players can tend to play risky or try psychological suboptimal play(from their pov) or many others things. About fuseki development i feel that we didnt understood each other - problem is that lets say player now plays 20 moves of AI prepared fuseki (which is near perfect according to current knowledge) but Shusaku plays his 20 moves of what then was considered near perfect fuseki (but now we know that its not so good) - i think its clear now what i meant.

  • I am not pushing you to reveal content of book you want to make some profit i think, but if you are familiar with method used in chess i mentioned [AI analyzed position before and after each move and they summed up all errors, and then divided by number of games(thats basic pattern).

Monteo Believe or not, really really good book is not free. I have different worldview about the book: to help kind people solve Go problems. Profit is not my goal. My goal is to produce quality books that give new insights to those who need them. I don’t need to be familiar with chess method which can limit your creativity. I focus on the results. If the method generates plausible results, it is already okay.

kmr - there are no free books in this world. I may be only weak sdk player myself, but i really know obvious things :).

  • After that, they took into consideration factors like complexity of position etc, and they get some number which represents strenght of player], you should know that its not empty field you exploring. Therefore some precisely thinked description would be much better for you and your book. The methods you used can be hidden, but then if you could add precise and detailed info of your unknown to ppl AI could greatly help your cause. I am sure some ppl would be very interested in your research then (well, maybe me not, i am too much fan of Shusaku). Also if your method is really accurate for judging player strenght, it could be used maybe as test for "AI cheating detection", after some modifications (just a thought).

Monteo I don’t care whether people are interest in my research. I want to notify people that there is a new better way to knowing who the strongest player in history really is. I am a fan of Shusaku too, and not of Cho Hunhyun, but the results make me rethink about it. The method is already accurate enough for assessing human Go players’ performance.

kmr - Wait a sec, you just said that you are not familiar with method i mentioned and now you are saying there is new better way? Sorry, but am i missing something?

This is a better way because if other ways were better or equally good, people might have known for sure the name of the Go player who is really the strongest human player in the history of Go. SPA can tell you that name for sure.

Part III

xela: Please remember that SL is not Wikipedia. It's OK for people to have "hobby projects" on this site, and they do not have to live up to standards of scientific best practice. kmr, I'm as confused as you are, but I'm keen to see where this is going. The last time Monteo wrote something I didn't understand, I checked it out myself and the conclusions stood up to scrutiny better than I expected. Monteo, is there a place we can download your books to learn more?

Monteo Thank you. The book excerpts are available at the [ext] book website after more proofreading and editing to ensure its highest quality and reproducibility. The book may be very controversial if I don’t explain the methodology clearly enough because I assert the findings are plausible, not just for fun although fun is a primary reason why I tried to check the comparative strength of the great players, which had long been unknown.

kmr - My last 2 cents (i hope) - If you explain your methodology in all details that it can be repeat by others, then its all OK for scientific book. All you reveal here is up to you, the better way you do it, more ppl would try your book . The only downfall i see is that you decided to not reveal AI you used (but i guess you have your good reasons to not do it, since when you publish your book it will be revealed anyway),and you didnt said how many games you analyzed.

Monteo The AI name is also a key finding after trying several AIs and can be considered a business secret, not downfall I see but a deposit of gold. I will tell people who support my work only or I may not publish the book anyway. It is up to me. The number of games I analyzed are important but not urgent.

kmr I used wrong word, downfall is not what i really meant - i should say its sth which shouldnt cost you anything, so its from pov of ppl interested in your research a "weak point" (its pretty standard to name external tools anybody use in his researches). You say its buisness secret, and you may not even publish your book - now that is really strange.

"The number of games I analyzed are important but not urgent" - this sounds like bad calque from your native language, it does not make sense in context of our discussion, i think you meant sth different.Did you meant that "number of games i analyzed is important but revealing this number is not urgent"? I am pretty lost here.

Monteo It means the number of games is important but not urgent. It is important because it will make the results even more plausible, particularly to those who are under the influence of statistical approach to sciences. But it might take a lot of times and effort and yield little new findings. More urgent is to report the initial key findings now to see how people think about it or whether they have already had a better way to decide who the strongest Go player in history really is, so that I will not reinvent the wheel. More games can be analyzed later by other people (if any), not necessarily by me, to falsify this method and the underlying theory. It is urgent because I want to be the first to reveal that Cho Hun-hyeon is a little stronger than Honinbo Shusaku. Also Go Seigen at his peak is stronger than Ke Jie when he was the World’s #1 and played against AGM.

kmr In what languages you intend to publish your book?

Monteo English

kmr (to Xela mostly, and all interested) - imagine there was no 1927 match between Alekhine and Capablanca. 2008 research using method i mentioned earlier clearly shows that Capablanca was stronger player than Alekhine (if we assume that method was correct ofc). If we would exclude this match from this research, then Capa would be seen even stronger. Match to 6 wins is a lot a this level. So Alekhine chances at this match were pretty low. Yet he won that match convincingly. Imagine sth else - there is 16 players knock out tournament, all players are equally stronge except 1 who has 60% chances to win game against others.They all play 1 game and winner prevails. What is chance that best player win this tournament? Doing this easy calculations shows why strongest player according to WHR rating Shin Jinseo didnt won any title till just now (LG cup).And now imagine that they play not 1 game, but best of 5 match. Now this chances looks different, arent they? Now if you compare this result and result of Capablanca-Alekhine match (Alekhine won 6-3, rest draws) you may see my point.

Monteo Game results don’t imply the winner is the strongest player in the history. SPA uses the move quality-based performance assessment, not the results-based performance assessment. For example, I used data from the match Ke Jie defeated by AlphaGo Master; SPA shows that Ke Jie is stronger than many other top title holders.

Part IV

Monteo I will end my discussion on the strongest player in history after the following note. Results in the extreme condition tests are consistent with the known facts. That is, AGZ is the strongest and Michael Redmond is the weakest among the selected 12 Go players. Beside, AGM is the second strongest. All of these findings confirm that the method is quite reliable. Then, I applied the method to the nine greatest players in their peak times, namely Honinbo Dosaku, Honinbo Shusaku, Honinbo Shuwa, Go Seigen, Cho Chikun, Cho Hunhyun, Lee Changho, Lee Sedol, and Ke Jie. Results are given in the book, not here, which provides more information about the games selected, superhuman performance analysis methodology, the modified AlphaZero Go engine, the superhuman neural network, the full results, the name of the strongest human Go player in history, and new insights.

See the [ext] Google Site of this book for more information and book excerpts.


Monteo/Superhuman Performance Analysis last edited by 124.122.94.235 on February 27, 2020 - 16:31
RecentChanges · StartingPoints · About
Edit page ·Search · Related · Page info · Latest diff
[Welcome to Sensei's Library!]
RecentChanges
StartingPoints
About
RandomPage
Search position
Page history
Latest page diff
Partner sites:
Go Teaching Ladder
Goproblems.com
Login / Prefs
Tools
Sensei's Library