H. G. Muller wrote on Sun, Mar 19, 2023 06:27 PM EDT:
Well, the default setting ('2.5 ply') of the ID is quite weak; it already has difficulty in recognizing mate-in-1 threats there. In an end-game with just a few pieces it would play very fast. An AI that is controlled by time rather than depth would automatically start to search deeper in the end-game. Humans would think farther ahead in a simple end-game than in a complex middle-game, so with a fixed search depth at some point in the game the AI will start to look stupid, even when it initially would be percieved as a tough opponent that knows what it is doing. I could of course control the playing strength through specifying seconds, or a number of nodes, instead of ply. The advantage of ply is that it provides an absolute measure for the tactical capabilities (e.g. how far ahead it will recognize a checkmate), independent of the complexity of the variant.
To seriously compare the strength to that of an AI controlled by thinking time it would be necessary to keep the time used by the ID (which it reports after every move above the board) approximately at the same level of that of its opponent during the game. Which would require cranking up the depth as the game simplifies.
It seems that the Ludii AI is simply buggy. A good search should not make blunders of the kind it does, even if it is not especially optimized for Chess variants. Jocly is also uses a very general (MCTS) search, because of its general scope. But I have never seen it make blunders like this. Ai Ai is quite prone to blundering, though.
Well, the default setting ('2.5 ply') of the ID is quite weak; it already has difficulty in recognizing mate-in-1 threats there. In an end-game with just a few pieces it would play very fast. An AI that is controlled by time rather than depth would automatically start to search deeper in the end-game. Humans would think farther ahead in a simple end-game than in a complex middle-game, so with a fixed search depth at some point in the game the AI will start to look stupid, even when it initially would be percieved as a tough opponent that knows what it is doing. I could of course control the playing strength through specifying seconds, or a number of nodes, instead of ply. The advantage of ply is that it provides an absolute measure for the tactical capabilities (e.g. how far ahead it will recognize a checkmate), independent of the complexity of the variant.
To seriously compare the strength to that of an AI controlled by thinking time it would be necessary to keep the time used by the ID (which it reports after every move above the board) approximately at the same level of that of its opponent during the game. Which would require cranking up the depth as the game simplifies.
It seems that the Ludii AI is simply buggy. A good search should not make blunders of the kind it does, even if it is not especially optimized for Chess variants. Jocly is also uses a very general (MCTS) search, because of its general scope. But I have never seen it make blunders like this. Ai Ai is quite prone to blundering, though.