Difference between revisions of "Talk:Openings on 19 x 19"

From HexWiki
Jump to: navigation, search
(KataHex up-training)
 
(Trying to get my signature to work.)
 
(2 intermediate revisions by 2 users not shown)
Line 1: Line 1:
 
Thanks for creating this resource. While KataHex is very strong, I think one should take its opinion on 19x19 with a grain of salt. According to its website [https://zhuanlan-zhihu-com.translate.goog/p/476464087?_x_tr_sl=auto&_x_tr_tl=en&_x_tr_hl=en&_x_tr_pto=wapp], KataHex has mainly been trained on 13x13 (for 20 days), and then up-trained on sizes of up to 19x19 for only one day. Up-trained means that the model that was already trained on 13x13 was then allowed to play 19x19 a bit. This method of training means that like a human, KataHex will mostly try to apply whatever lessons it has learned from 13x13 to the larger boards, and then learn a little bit extra from experience.
 
Thanks for creating this resource. While KataHex is very strong, I think one should take its opinion on 19x19 with a grain of salt. According to its website [https://zhuanlan-zhihu-com.translate.goog/p/476464087?_x_tr_sl=auto&_x_tr_tl=en&_x_tr_hl=en&_x_tr_pto=wapp], KataHex has mainly been trained on 13x13 (for 20 days), and then up-trained on sizes of up to 19x19 for only one day. Up-trained means that the model that was already trained on 13x13 was then allowed to play 19x19 a bit. This method of training means that like a human, KataHex will mostly try to apply whatever lessons it has learned from 13x13 to the larger boards, and then learn a little bit extra from experience.
  
Even with that in mind, KataHex on 19x19 is probably still much stronger than most human players, so we can learn from it. But the advice may not be very specific to 19x19. 20:38, 30 March 2023 (UTC)
+
Even with that in mind, KataHex on 19x19 is probably still much stronger than most human players, so we can learn from it. But the advice may not be very specific to 19x19. [[User:Selinger|Selinger]] ([[User talk:Selinger|talk]]) 20:38, 30 March 2023 (UTC)
 +
 
 +
Thanks for your comments. I agree this article might be better named "Openings on large boards", and the advice for 18×18 wouldn't (and arguably shouldn't, for humans) differ very much from 19×19.
 +
* I think KataHex on 19×19 is likely stronger than even top players but don't have proof. (I believe much stronger; if I had to bet on KataHex opening with a1 against a top player, I would bet on the bot winning.) I've analyzed tens of games among top players/bots at LittleGolem, and usually when the bot strongly disagrees with a human's move (which is often), I think there's a good reason to prefer KataHex's move.
 +
* KataHex does have weaknesses. I've seen its win probability for two provably equivalent positions (such as before and after a bridge intrusion that is useless for both players) differ by over 10-20% occasionally. The win percentages are very inaccurate when you introduce pass moves, though I imagine these rarely occur in the training data, so KataHex has no incentive to get it right. Therefore, I think KataHex is very far from perfect play.
 +
* KataHex seems excellent at local play for various reasons, like agreement with leela_bot/gzero_bot in acute corner joseki. Local play is something I expect to transfer well to 19×19, and the focus of much of the article.
 +
* It's hard to say much about the 20 days on 13×13 vs. 1 day on larger boards (except that playing strength in Elo probably looks way more linear with respect to the logarithm of training time, rather than training time itself); while it's likely that a version of KataHex trained for a year would be at least several hundred Elo stronger, I think humans are currently so far from perfect play that it hardly matters.
 +
* I created this guide to try to fill in the holes of existing articles/strategy guides. I think time will tell whether the advice in this article is good (if people who tend to follow the advice empirically win games more often), but I think it's worth the try.
 +
[[User:Hexanna|Hexanna]] ([[User talk:Hexanna|talk]]) 00:47, 31 March 2023 (UTC)
 +
 
 +
I agree. The kind of inferences I am most suspicious about are the ones like this image: [https://pic4.zhimg.com/80/v2-ed52d7b720e4fef5159d0aab51716b9b_720w.webp]. That pattern could be an artifact of uptraining, rather than an actual insight. But of course you didn't use this in the article, so it is only marginally related. [[User:Selinger|Selinger]] ([[User talk:Selinger|talk]]) 01:49, 5 April 2023 (UTC)

Latest revision as of 01:49, 5 April 2023

Thanks for creating this resource. While KataHex is very strong, I think one should take its opinion on 19x19 with a grain of salt. According to its website [1], KataHex has mainly been trained on 13x13 (for 20 days), and then up-trained on sizes of up to 19x19 for only one day. Up-trained means that the model that was already trained on 13x13 was then allowed to play 19x19 a bit. This method of training means that like a human, KataHex will mostly try to apply whatever lessons it has learned from 13x13 to the larger boards, and then learn a little bit extra from experience.

Even with that in mind, KataHex on 19x19 is probably still much stronger than most human players, so we can learn from it. But the advice may not be very specific to 19x19. Selinger (talk) 20:38, 30 March 2023 (UTC)

Thanks for your comments. I agree this article might be better named "Openings on large boards", and the advice for 18×18 wouldn't (and arguably shouldn't, for humans) differ very much from 19×19.

  • I think KataHex on 19×19 is likely stronger than even top players but don't have proof. (I believe much stronger; if I had to bet on KataHex opening with a1 against a top player, I would bet on the bot winning.) I've analyzed tens of games among top players/bots at LittleGolem, and usually when the bot strongly disagrees with a human's move (which is often), I think there's a good reason to prefer KataHex's move.
  • KataHex does have weaknesses. I've seen its win probability for two provably equivalent positions (such as before and after a bridge intrusion that is useless for both players) differ by over 10-20% occasionally. The win percentages are very inaccurate when you introduce pass moves, though I imagine these rarely occur in the training data, so KataHex has no incentive to get it right. Therefore, I think KataHex is very far from perfect play.
  • KataHex seems excellent at local play for various reasons, like agreement with leela_bot/gzero_bot in acute corner joseki. Local play is something I expect to transfer well to 19×19, and the focus of much of the article.
  • It's hard to say much about the 20 days on 13×13 vs. 1 day on larger boards (except that playing strength in Elo probably looks way more linear with respect to the logarithm of training time, rather than training time itself); while it's likely that a version of KataHex trained for a year would be at least several hundred Elo stronger, I think humans are currently so far from perfect play that it hardly matters.
  • I created this guide to try to fill in the holes of existing articles/strategy guides. I think time will tell whether the advice in this article is good (if people who tend to follow the advice empirically win games more often), but I think it's worth the try.

Hexanna (talk) 00:47, 31 March 2023 (UTC)

I agree. The kind of inferences I am most suspicious about are the ones like this image: [2]. That pattern could be an artifact of uptraining, rather than an actual insight. But of course you didn't use this in the article, so it is only marginally related. Selinger (talk) 01:49, 5 April 2023 (UTC)