Comments by HGMuller

Joker80 does game/time very well, but you have to tell it that it should do game per time for that to be apparent. This time control is known as 'Incremental', and you have to select it in the Time Control dialog, and then enter 15 min per game + 0 sec per move.

All you have discovered is that the code for reading the time control does not test the input for reasonability. I should simply make Joker80 exit immediately when someone specifies more than 60 movesin a conventional time control. Conventional time controls are never played with more than 40 moves per session; in Chess that would be quite pointless. You would get a huge amount of extra time at a point where in 95% of the cases the game is already decided, if not finished.

9999999 moves per 15 minutes is just not a valid way to enter game / time for WinBoard engines.

That depends on how you define reasonable. I guess it would only reduce its strength by about 150 Elo compared to non-extreme time controls of the same average time per move. (Of course the opponents would likely also suffer, and it is difficult to predict if they would suffer more or less.) That is at least more reasonable than reducing its strength by 2500 Elo through making it search 1 ply only.

The reason is that engines function optimally if they can allocate time to those moves where it is needed. This results in a strongly variable time per move: there are natural points where they could stop, and time spent after that is largely wasted, until you go all the way to the next point where to stop. Unless they discover that they are really in trouble, i.e. the move they focused on so far suddenly turns sour at larger depth. Then it pays to allow them extra time until they found at least a reasonale alternative. After all, time is only a factor if you plan to fight on. So playing a move that you know to be immediately losing just to save time is never an attractive proposition.

By harrassing the engine through facing it with a time control on every move, you largely make sensible time management impossible. In particular, Joker calculates its nominal search time as T/(N+3), where T is the remaining time on its clock, N is the number of moves it still has to do in that time. So it keeps 3 moves worth of time reserve to be able to solve inadvertant trouble on the last move before the TC. But if N=1 always, this would make it do the initial moves very fast.

I would advice against trying extreme time controls, but use the engines in modes for which they are designed, rather than see how much you can stress them before they break. At least, I assume you are interested in how well they play Chess. If you put two grand masters to a game where someone is constantly knocking them over the head with a pillow, and hosing them down with a fire hose... Well, they might be able to complete the game, but who wins is likely to reflect other properties than their skill at chess. Just play game/time, game + small increment per time or 40 moves/time. So 15+0, 10+5 or 40/10 would be good controls to test them at (i.e. 15 min sudden death, 10 min + 5 sec/move or 40 moves / 10 min). Trying extremes only tells you something about the robustness of their time management outside the range of its design parameters, and has little to do with chess.

Well, I guess if some participants only can play with some fixed time per move, which does not accumulate when they are fast and do not use all their time, this will severely handicap those engines. So it would only be fair if you would handicap the other engines likewise. The quality of the play for a given invested amount of CPU time would go down by this, but you might consider that less of a drawback than favoring the engines with more versatile time management.

An engine playing at 1 move / N sec would be able to quickly build uo some time reserve during the first 10 moves (in which they are unlikly to lose the game already, because both sides will be underdeveloped in this phase), and then have the advantage of being able to alot more time on difficult moves, which the others can't. So perhaps it is not as bad as I first thought to use the 1 move / N minute mode, and certainly fairer towards the opponents.

WinBoard does have a TC mode where you can set fixed, non-accumulatng time per move, but it is cannot be set from the TC dialog, and my engines do not support it well. (Joker80 would play, but probably use less than 30% of its time, while Fairy-Max would sooner or later forfeit every game, because it does not have an emergency break to cut extreme flukes of its highly variable search time.) But I notice that you do not play Fairy-Max anyway (although I was under the impression that it is a lot stronger than ChessV).

It depeds a bit on your philosophy, I guess. Is it fair that engines with more advanced time management have a better chance of winning? Some would say yes, because time management is part of the game. Others would be interested purely in the quality of the search engine.

As to the docs: This might be an idea, although the help file of WinBoard is already very long. As I did not change any of the basic TC modes, which have been around for decades, I did not change the docs on it either. I had to add enough for describing all the enhancements I made, and concentrated on that. But now that I look at it I see that the description is not very clear. In addition, I added time-odds controls to this dialog, and tey are not described in the menu section at all. So I guess I should make some additions here anyway.

OK, sounds good. Did you consider SMIRF? It can play under WinBoard with my Smirfoglot adapter, and I am pretty sure it should be able to do Schoolbook. In last years 10x8 tourney I held, SMIRF finished between Joker and TJ. But I think TJ has been much improved; I was playing 0.96 last year, and now we have 0.121.

The past day I have been playing Joker80.np vs TJchess10x8 0.121 at game/15 min in Schoolbook. That is, to make sure I got all different games, I started from the Schoolbook opening, and then 16 positions that had a (different) first move made by white. Each engine played each position with white and black, for 34 games in total. Joker80 beat TJchess10x8 by 22 vs 12. 

Most games were very exciting, lots of fireworks. (There were only 2 draws.) Interesting is that the engines differ very much in evaluation for most of the game, usually by 3-4 Pawns, so that they can each think they are 2 Pawns ahead! If you are interested: the PGN file can be downloaded at:

http://home.hccnet.nl/h.g.muller/Schoolbook.zip

From watching the games, it seemed to me that the Schoolbook opening array is a bit awkward; in many games the engines had grave difficulty developing the Queen. Often this was the decisive factor leading to their demise.

I found a page with a download link for SMIRF here:

http://www.chessbox.de/Compu/schachsmirf.html

Do you have Smirfoglot to run it under WiBoard? For the latest WinBoard version, downloadable from WinBoard forum, I include Smirfoglot with the (optional) Chess-Variants profile of the install.

I never noticed a big advantage for white in any opening array, but my method of testing might hide it. White bias in normal Chess is 3-4%, so to measure changes of that you really have to measure at the 1% level, which means a 1000+ games. When I test for piece values I play that number of games, but usually I shuffle the array, and any particular aray is only played a few times. In addition, Fairy-Max is programmed to do its first 3 moves more or less randomly, to create even more diversity. So if there is a particular line that would give white a better advantage, it would not be played often enough to stick out of the statistical noise in the result.

I guess what would be needed is really avoid randomness as much as possible, and let the positio play by 1000 different engines. But I don't have that many engines. As an alternative I could try to program more randomness during the game, and less in the beginning. But the problem is that engines might prefer some opening line then, which need not be the best at all. Especially at fast time controls. In normal Chess most engines would have grave difficulty in rediscovering major opening lines known from theory, when they have to play without book. Some make an absolute mess of it even at long TC. Joker is actually pretty good at this, because it was designed to play (normal Chess) without book, and I tuned its evaluation to not make silly opening errors. But if this also works well in 10x8 is anybody's guess. I discovered a very bad misadjustment of Joker's way to score castling w.r.t. King Safety after castling, which in Schoolbook led it to destroy its own castling rights (because it could not find a way to castle within the horizon fast enough, like it always would in normal Chess).

I guess that the best way to test such things would be to create a small opening tree of likely moves by hand. Or perhaps run an engine in multi-PV mode to analyze early opening positions for a few minutes each, to get the best few moves in every position, and construct a tree from that. And then evaluate the leaves of that tree by using it as starting position for a deep analysis, or for a thousand fast games. And then propagate the scores or winning probabilities along the opening tree towards the root.

It is my personal opinion that immediate mate threats do not impact the white beas very much, anymore than the existence of fool's mate or shephard's mate have a big impact on openings of ordinary Chess. In fact threatening those latter mates are considered very poor opening moves. To my eye threatening mate with the Marshal looks much the same. It does not look like a good opening move, (which you would likely wanted to play when it did not make the mate threat), and the opponent will simply defend against the mate with a good development move, like advancing the attacked Pawn, or defending it with a Knight, or blocking the Marshall's path with a Knight

Even the arguments I have seen that the Capablaca is a bad because of its high white bias are strongly based on the assumption of piece values, considering many lines as 'losing' because they contain a (B+N) vs (R+N) trade. While in fact these trades might be favorable in reality, and just misevaluated because of underestimation of the (B+N) value. Part of what makes the Joker80-TJchess10x8 games so interesting is that these engines do have a very different opinion on the relative values of the superpieces, so that unequal trades occur quite often (e.g. A vs M or A vs. R+B). This gives several nice examples of an Archbishop slaughtering a Marshall or Queen.

P.S. after spending most of the day by being captivated by the Schoolbook match, I hosted the monthly on-line computer blitz tourney on my Internet Chess Server in the evening. And it really struck me how incredibly boring ordinary Chess was compared to the 10x8 games I had been watching in the afternoon...

Indeed, the number of required games increases fast. But the WinBoard games are not so much a problem, they can be run automatically overnight. So Zillions and ChessV are the participants that really count, in terms of effort. Perhaps it is an idea to first play Mat's improved version of Zillions against all the other engines. Then, if it loses all games, it might be pointless to play the weaker version of Zillions. ;-)

Btw, Gregory Strong is working on the follow-up of ChessV. It is called Quadrox, and it already plays normal Chess. It participated in my on-line blitz tourney yesterday. It is planned to be a WinBoard engine as well, and soon will be able to do 10x8 Chess.

Not all engines are deterministic, and Joker80 is mildly non-deterministic. (At very long time control it might become more deterministic, though.) It adds a pseudo-random term to its evaluation, determined from the starting time of the game and the hash signature of the position. This way of implementing randomization acts as a kind of poor-man's mobility evaluation, and thus should increase playing strength compared to the purely determiistic version.

Another method of introducing randomness is to add a small pseudo-random number to the score of every move in the root. When several moves have nearly equal score, this should randomly choose between them. This does not increase playing strength, but it stays effective at arbitrary large search depth.

Put this multi-PV mode (givin scores for several moves, rather than just the best) such as you show on ChessV can definitely be very useful for constructing a book. Perhaps I should implement it in Joker80 too. The idea of a book is that you are willing to play theoretically sub-optimal moves to provide diversity, so that the opponent cannot easily prepare, and thus will eventually play even more sub-optimal moves. If you are striving for a variabilty of 1000 lines, and you can choose between 10 moves on each ply, you only need a book of depth 3. (Well, perhaps 4 because many lines are transpositions of other lines.) But you would be choosing the 10th best move on some of these lines, and its score might be really a lot lower than that of the best. So in practice you only choose on average between 2-3 moves on each ply, and your book had to be 10 deep before it fans out to 1000 lines (assuming the opponent always plays his best move). But then all these lines can be very close in score to the best line. So there is a trade-off between depth of the book and how much score you have to sacrifice to get a given variability.

It would be interesting to develop (and implement) an algorithm that would create the minimal tree that would have a given number of end leaves with a score (as determined by an engine search of a given quality) within a preset margin from the best line. E.g. how much variation would it buy us if we are willing to sacrifice 30 centiPawn after 1, 2, 3, 4, 5 moves, against optimal play.

> My plan ('plan' being the operative word) is to modify Winboard to 
> implement free castling, having it so, if playing Schoolbook, one of 
> the engines doesn't recognize the free castling move, Winboard will 
> simply rearrange the pieces for that engine and continue the game. 
> This will give engines that know how to do free castling an edge. 

There might be more problems with that then you think. Some engines might consider loading a new position the start of a new game, and would get out of sync with WinBoard for counting moves to the next time control. If you play an incremental TC that should not matter, though.

Perhaps what you propose could be added as a general option to WinBoard: when an engine refuses a move through the prescribed illegal-move error message, WinBoard could reset the engine and force the position after the move into it, and restart it.

I think that the current WinBoard actualy does understand all King moves that step more than one file as castlings, putting the Rook located in that direction to the other side of it. (Except when you do a wrap-around move for Cylinder Chess.) So the engine could send f1d1, f1c1 or f1b1, and they would all be performed as queen-side castlings in -variant capablanca, provided you switch Test Legality off.

> In the tournament, Zillions will probably only achieve a draw, at
> most, but this doesn't matter if it plays good and interesting chess.
> These are factors which have been tragically left out of computer
> chess.

I don't understand how you can say that. Have you looked at the Joker80 vs TJchess10x8 Schoolbook games at all? I think they are all extremely interesting games, of dazzling brilliance.

And I don't follow your reasoning. What is good and what is poor Chess is determined by the rules of the game and the aim to win. If that good Chess is not aesthetically appealing to your personal taste, it cannot be blamed on the players. The players are bound by the rules, and must do the moves that lead to the best possible result these rules allow. If the result of this optimization is not to your liking, it can only be blamed on the rules.

If a program can see that a Pawn storm will be losing, it won't engage in a Pawn storm. Programming it such that it will start the Pawn storm anyway as a pathetic attempt, so that it willingly runs into the knife and gets slaughtered, might be entertaining to you, but it is simply poor Chess. If you want to see Pawn storms, you should design a Chess variant where it pays to do Pawn storms. Mad Queeen's in general does not seem to be one.

Of course it can be derived from algorithms. Tal and Petrosjan are also nothing but algorithms. Exceedingly complex and unpublished algorithms, perhaps, but algorithms nonetheless. Your argument on classical music is unconvincing. What computers cannot do now, might be common place in a decade. Thirty years ago one could have said exactly the same about computers lacking the creativity to play Chess at grand-master level.

If Tal and Petrosjan subdued their opponents by theoretically unsound moves, they were only able to do it because they were much better than their opponents. This is quite normal in Chess. Speculating on that your opponent will not see the refutation but in stead will fall into a trap you set is a very efficient way to speed up winning a game against a weaker player. And that might look brilliant to one who is not able to recognize the trap either. Computers can be very easily programmed to play that way. But it will get them slaughtered by their peers, of course. It is easy to look brilliant and creative when you are _much_ better than your opponents. But when you are hardly better, it calls for realism. This has nothing to do with computer vs human algorithms. It is just the computer's bad luck that they always meet their match, in terms of search depth. So they cannot afford fancy-looking strategies that can be exposed as unsound at their own level. When your traps are exposed and backfire, they suddenly don't look so brilliant anymore. If you win by sacrificing a Queen everyone is in awe. But when you lose by it, they shrug and say, ' he blundered his Queen away again. Will this guy never learn?'...

OK, thanks for this info. But where can this 0.9.0 version be obtained?

In the version I have posted now I was able to work around the
color-switch problem: this seems to occur when there is  fail-high in the
root. The research then seems to be done with a messed-up move list. By
doing the initial search with a fully open window in every iteration, there
can never be a fail high, and the problem does no longer occur. In fact it
seems ChessV plays a lot stronger by this! I guess that even in the cases
where there was no color switch, a re-search would make it do a random
move.

Major remaining problem is that ChessV seems to be blind for promotions in
its search. It takes no measures to stop an enemy promotion, and its score
stays positive up to the very moment that the opponent promotes. After that
it is of course at -9000...

The download of the 0.9.2WB is already available at the link I gave below.
Gregory Strong is going to look at the remaining issues himsef, though, so
perhaps we should wait for that. (
http://www.talkchess.com/forum/viewtopic.php?t=29828&start=25 )

OK, this is the Joker80 analysis of Schoolbook array. I let it run for 12 ply from the opening, printing every position with an exact score searched to a depth of 10 ply, keeping the window open from -0.10 to + 0.20 (white POV) on all searches of 10 ply or deeper. Both the 11-ply and 12-ply search came up with c2-c4 as best move. At 11 ply e2-e3 was deemed as good, but it dropped a lot in the 12ply search. But f2-f4 did exactly the opposite.

The following table gives the move path, followed by the score/depth of the position after these moves. Because I searched 12 ply, and printed everything with d>=10, it did not only print the white moves but also the black replies. elow I did list some replies to the best white moves.

c2c4  0.10/11 (0.05/10)
f2f4  0.05/11 (0.00/10)
g2g4  0.04/11 (0.02/10)
c2c3  0.04/11 (0.04/10)
e2e3  0.01/11 (0.00/10)
e2e4  0.00/11 (0.05/10)
f2f3  0.00/11 (0.01/10)
h1i3  0.00/11 (-0.00/10)
g2g3  0.00/11 (-0.02/10)
d2d4  0.00/11 (-0.03/10)
h2h3 -0.00/11 (0.00/10)
c1b3 -0.01/11 (-0.01/10)
h1g3 -0.01/11 (-0.01/10)
d2d3 -0.03/11 (-0.04/10)
e1d3 -0.03/11 (-0.05/10)
h2h4 -0.03/11 (-0.06/10)
c1d3 -0.04/11 (-0.07/10)
a2a4 -0.06/11 (-0.09/10)
i1h3 -0.09/11 (-0.06/10)
j2j3 -0.09/11 

c2c4 c7c5: -0.10/10
c2c4 g7g6: -0.11/10
c2c4 c7c6: -0.14/10
c2c4 g7g5: -0.14/10
c2c4 h8i6: -0.15/10
c2c4 e7e5: -0.16/10
c2c4 f7f6: -0.16/10
c2c4 e8d6: -0.16/10
c2c4 e7e6: -0.18/10
c2c4 i8h6: -0.18/10
c2c4  0.10/11
12      10  2582729  104628430 ? c2c4 c7c5 f2f3 i8h6 c1b3 d8c7 h1g3
e2e4 c7c6:  0.00/10
e2e4 c7c5: -0.04/10
e2e4 e8f6: -0.09/10
e2e4 e7e5: -0.10/10
e2e4 h7h6: -0.10/10
e2e4 f7f6: -0.13/10
e2e4 e7e6: -0.14/10
e2e4 g7g6: -0.15/10
e2e4 g7g5: -0.16/10
e2e4 h8i6: -0.18/10
e2e4  0.00/11
11       4    25428   83877288 ? e2e4 f7f6 d2d4 c7c6 f2f3 d8c7 i1h3 e8i4
12       0   426852 1429007351 ? e2e4 c7c6 i1h3 e8d6 f2f3 f8e8 d2d4 h7h6 e4e5
f2f4 f7f6: -0.05/10
f2f4 c7c6: -0.07/10
f2f4 f7f5: -0.08/10
f2f4 h7h6: -0.10/10
f2f4 d7d5: -0.11/10
f2f4 e8d6: -0.11/10
f2f4 h8i6: -0.13/10
f2f4 d7d6: -0.13/10
f2f4 e7e6: -0.14/10
f2f4 c8b6: -0.14/10
f2f4 g7g6: -0.15/10
f2f4 j7j5: -0.16/10
f2f4 b7b6: -0.16/10
f2f4 c8d6: -0.17/10
f2f4 i7i5: -0.19/10
f2f4  0.05/11
12       5   692665 -1980185374 ? f2f4 f7f6 e2e3 g7g5 h1g3 c7c6 f4f5 d8c7 d1g4
g2g4 c7c5: -0.04/10
g2g4 f7f6: -0.05/10
g2g4 g7g6: -0.07/10
g2g4 h7h6: -0.08/10
g2g4 e7e5: -0.08/10
g2g4 d7d5: -0.08/10
g2g4 c7c6: -0.08/10
g2g4 g7g5: -0.09/10
g2g4 e8d6: -0.10/10
g2g4 c8b6: -0.13/10
g2g4 h8g6: -0.14/10
g2g4 i8h6: -0.14/10
g2g4 h8i6: -0.15/10
g2g4 d7d6: -0.16/10
g2g4 b7b5: -0.16/10
g2g4 i7i6: -0.17/10
g2g4 a7a5: -0.18/10
g2g4 e7e6: -0.18/10
g2g4 j7j6: -0.19/10
g2g4  0.04/11
c2c3 c7c6: -0.04/10
c2c3 c7c5: -0.04/10
c2c3 g7g5: -0.07/10
c2c3 g7g6: -0.07/10
c2c3 e7e6: -0.10/10
c2c3 c8b6: -0.14/10
c2c3 h8i6: -0.15/10
c2c3 i8h6: -0.15/10
c2c3 h7h5: -0.17/10
c2c3 h8g6: -0.18/10
c2c3 e7e5: -0.18/10
c2c3 d7d5: -0.18/10
c2c3  0.04/11

Sam: The most obvious difference is that Joker80 likes c2-c4 quite a lot, while ChessV thinks it a pretty poor move.

Mats: I think this technique could be useful to reveal flaws in the array that would cause an unusually large white bias. E.g. essential weaknesses due to undefended Pawns or critical mate threats.

To create an opening book, a better approach would probably be to play a few hundred thousand ultra-fast games, with an engine that randomizes through a root bias of 10 centiPawn or so. This should provide Monte-carlo sampling of the opening tree, fully taking into account strategic effects like those you mention.

With 40 moves/10 sec one could do 120 games/hour on a single CPU. So a quad would make you about 500 games/hr or 12,000 games/day. Keep that up for a month, and a very nice data set would emerge.

There is a new version (4.4.0) obtainable from WinBoard forum:

http://www.open-aurec.com/wbforum/viewtopic.php?f=19&t=50387

This new version supports the -noGUI option, which supresses updating of board and clocks completely. The Animate Moving checkbox should be in the Options -> General... dialog. I guess I misremembered a little; I looked it up and Joker80 was playing 15 sec/game against 24 min/game for ArcBishop. But at that speed (which should be equivalent to 40 moves / 10 sec, as the average 10x8 blitz game takes about 60 moves), it was able to win games without forfeiting on time. This was on a 2.4GHz Core 2 Duo.

Since we want self-play with absolutely equal engines, it should also be possible to make a version of the engine that does this without GUI. ChessV has such an option to play itself. Both sides would use the same hash table, which is equivalent to having a ponder-on tournament with 100% guarantee for a ponder hit, but only using a single CPU! The first 30 moves of each game could then be saved together with the result in binary format, to build a book from it later.