Moyo Go Studio DevBlog

Almost none of my customers buy the high-ranking amateur games.

Moyo Go Studio shows you the replies of other players in identical local positions in your game.

My customers all think they play just like a pro, otherwise they would purchase, for a few measly bucks, a few hundred thousand amateur games, containing hundreds of millions of positions that occur in non-pro games. What makes you think that, as a Kyu player, your games are full of positions that occur frequently in pro games?

When you’re a Kyu player and you use the pro-only version of Moyo Go, you will not be able to benefit of the program’s most powerful feature: Showing you what a much stronger player played in the exact same local situation (ALL local situations EVERYWHERE on the board). If you’re not getting the strong ama games, it hardly makes any sense to buy Moyo Go at all.

This is a picture of the new computer I built.
It will be used for number-crunching.

I have a water cooling system, but I am worried one day a rupture will ruin the RAM, so I used fans instead.

The fan on CPU1 (a dual-core Opteron) had to be smaller because a bigger one simply didn’t fit.

The beast makes too much noise to keep it in the room so I put in in the cellar - nice and cold there. I remote it with Remote Desktop over WiFi, works great. You should see the looks of visitors when I show them, on the external monitor of my laptop, the “My Computer” properties of 64 GB RAM :-D

For those wondering about where to get a mobo with 16 DIMM slots: It’s the Iwill DK88. 64 GB is an expensive proposition - 14 kilobucks at the time of writing. Luckily, my company is an authorized hardware dealer, so I get stuff cheaper.

Remember I blogged about cleaning up alternatively-spelled Chinese and Korean names and such?

The result is finally included in the latest version of Moyo Go.

More than two thousand new pro games have been added as a bonus.

A good 2008 to you all!

Some days ago I received a bug report from Germany.

The issue: Moyo Go did not parse a SGF2MISC file properly and thought it contained illegal SGF. I found the cause and fixed it. This fix will be available in the “2008″ version (will still take a long time before that one will be released, I will stop selling the 2007 version several months before that - yes, shutdown all sales such as not to disappoint those who just bought the 2007 version).

Whilst doing that, I discovered a glitch in the ProHandicap database regarding adding new games, and fixed that one as well - it’s downloadable here.

(Unzip in the Moyo Go Studio root install folder.)

In a previous posting I mentioned Dariush’ pattern system, and how it was on a par with Moyo Go’s in terms of pro-prediction.

(But Moyo Go gives accurate statistical data on moves, Dariush doesn’t and there are many more big differences that make Dariush, in my opinion, not a serious competitor YET).

Of course I have many ideas on how to improve the pattern system, but I never bothered because it was the only pattern system in the world. This has changed and it’s time to do something about it.

It’s not neccessary to modify the pattern database itself or anything -the only thing that has to be improved is the “value” of the patterns. MGS uses 16,777,216 patterns (no idea how Dariush manages to squeeze a pattern in a single bit of memory, MGS uses 8 bytes). If it’s true that Dariush uses 60 million patterns and MGS almost four times less, then there is enormous improvement potential for the latter, as with almost 4 times more patterns, Dariush only gets about the same results as Moyo Go.

The key to improving MGS’s pattern system is to go back to my initial “learning” method. My initial learning method is the one that Microsoft copied after studying my previous blog and website. It’s called “Bayesian Learning” or something like that. The mathematics seem to be complex if one reads their paper, but I intuitively re-invented it and have no clue how the maths work. It provided great results but it took ages to finish, so I devised a very much faster method (hundreds of times faster), which yielded very good results too. I never compared the results of both methods but the Bayesian method, given enough time, will surely produce better results. I discarded it not only for its extreme slowness (months of number crunching), but also because it destroys the statistical data, harvested from the game records.

It seems, however, that not many care much about accurate statistical move percentages, and that a higher pro-prediction percentage might be nicer to have than accurate statistics. Dariush does not provide meaningful data on suggested moves (they appear to fake a percentage), and because they have about as strong a pro-prediction as MGS, I’ll see if I can outdo them, abandoning the statistical part of the pattern system. I could easily provide two versions, one with exact statistics and one without, but with better pro-prediction.

This is next on my to-do list, so I expect to report back about progress in a while. It will take a long time to crunch the data in a Bayesian way, but I expect the pattern system to get quite a bit stronger, especially because I have now tens of thousands of extra pro games to work with. The cool part is that I’ll include the Bayesian-cruncher as a part of the retail version of Moyo Go Studio, so that people can keep optimizing their own pattern database. (Because it may be that years of number crunching is required to achieve the optimum).

Translating the Chinese names took days of hard work, and I still didn’t manage to do the ones with morwe than 3 characters. I discovered that almost all names I had translated belonged to amateurs, so I later deleter more than a hundred games.

I found a couple of name lists on the web and used them for the auto-transliteration stage.

One thing Mace Lee already mentioned is that some Chinese-spelled names actually belong to Korean players, so the phonetic Chinese still needs to be matched to a slightly differently phoneticized Korean name. I am slowly learning the names of the well-known players..

I’ve been “SGF hunting” as well, making the net result of my efforts of the past weeks a net gain of around 1000 games, totalling some 49350 19×19 pro-pro games. After the illlustrious secret Nihon-Kin collection, easily the largest pro-game collection in the world. And pretty accurate too, after spending days of disambiguating alternatively-spelled names. That work is likely not completely finished yet (and I hope I did not make mistakes). GoBase lists AKA’s, but I hardly used them. The database looks very different now, with zero games that have only Chinese-spelled names and just a few games where one players’ name still is Chinese. And literally hundreds of slightly-differently-spelled names are now unified, making it finally practicable to search for a player and get ALL his/her games. I am making the search functions more robust too: Automatic substitution of the “normalized” name when you specify a known “misspelling”. For some players, Moyo Go will know over ten alternative spellings, and that’s excluding the Chinese ones!

About the new SGF collection: It’s being exported as SGF’s now. For some reason that takes 20 hours or so, no idea why the SQL database I use is so sluggish with compressed fields. They aren’t even encrypted, for crying out loud.

Another feature I’m making for MoyoGo 2008 is the automatic display of the Gobase biography of a player when you right-click the name. I bet that this feature will become standard in all serious Go software.

For those who are still reading: Do you think I should make that game database a free download? Because even though I am at the receiving end of a lifelong boycot for all and any Go products I will ever make due to me using previously commercialized game records, I still believe game records should be free as in -speech.

I realize that it’s hardly a groundbreaking feature, but I have taken to translating those players’ names that are not yet done. I do Google searches on their Chinese names, and search for English pages only. When I find at least two sites that state them with the same name, I use it, otherwise I use Google Trans - that works just fine because Chinese names are not like “Johnny Vanderbildt” but they are a string of sounds, literally and phonetically represented by their characters. And in case of names, you are only interested in the transliterated representation of the sound, latinized in a standardized way.

The result will be the largest latinized game database available. I hope that nobody will steal it ;-)

Someone told me that the game packs he paid for & downloaded don’t install properly. Turns out I used the wrong setup files. That’s what happens when you migrate stuff you haven’t worked on in over a year to a different computer and don’t test rigorously..

Three people have bought game packs (two in the US and one in Israel) and I am burning a DVD for them as we speak, with nice Moyo Go printings and all available update packs to make up for the delay. I am sending it by express airmail.

All is now fixed, the corrected and tested setups have been uploaded to the web-shop.

I thought I’d use http://en.wikipedia.org/wiki/Go_players to enlarge Moyo Go’s automatic Asian-to-English spelling converter. Well, that’s useless, because the Wikipedia article is full of blatant errors:

Look behind the name of Yasuro Kikuchi. You know what the Japanese name between braces is? “Kato Tomoko”! The link to Yasuro’s page has his correct name.

Funny how someone like myself, total outcast in the Western Go world due to boycots, not a Go player and not able to read any SE-Asian language, has to fix the errors in Wikipedia articles on Go players. Someone is actually doing his homework here.

Remember my post about the duplicate Korean names due to non-uniform spelling?

Here is part of a list of players that have games in MasterGo’s database: (MasterGo must bear the brunt today because AGA’s webmaster is also the programmer of MasterGo and Slate & Shell is the company of AGA’s leadership, and AGA is boycotting Moyo Go Studio).

All nonsense. Chen Delong and Chen Dilong don’t exist. Hence their meagre number of games. The dude is called Chen Qiulong. The same with Chen Huifeng. No such player. His name is Chen Huifang.

I’m cleaning up this shit now, including the shit of those people who’s game records I “stole”. Nobody else is doing it, because it’s too much work. I have been working on this for days already, and the end is not in sight. I have to look up all names that seem ambiguous and verify that there indeed exist two different Go players with indeed those generally accepted transliterations of their name. And if not, I have to obtain a high degree of certainty that they are in fact the same name. This requires multiple Googlings per name, and we’re talking about hundreds of dodgy names here.

So here I am: I don’t even speak any SE-Asian languages or play Go for that matter, reviled and ostracized by AGA, Sensei’s Library, the BGA and GoBase - putting another few dozen hours into cleaning up the mess of player-fragmentation. I am amazed that this work has never been done! It might be that only GoGoD and GoBase ensure the uniformity of player’s names, but it seems that no Go software except Moyo Go Studio cares about this issue.