Comments/Ratings for a Single Item

That is purely a Fairy-Max problem. It has a pretty sloppy CECP parser, which doesn't bother to recognize most commands that require no processing (e.g. accepted, rejected, computer). 'Result' is also a no-op for an engine that doesn't implement learning. A GUI should not worry if an engine complains against non-essential commands. I think WinBoard only reacts to rejection of time/otim commands (concluing that it must be ealing with GNU Chess, and refraining from sending these commands for the rest of the game), and 'analyze' (informing the user that the engine doesn't support it, for the benefit of WB v1 engines that haven't announced this in advance through a feature analyze=0).

Ok, good to know that I don't need to worry about that.  I was wondering because I have been experiencing some stalls when running ChessV vs. Fairy-Max games in batch mode and it only happens when Fairy-Max declares a draw and then only sometimes.  I don't think it's related to this issue though.  So I'll have to keep digging.

Regarding playing CwDA through CECP, this game presents some interesting challenges, most notably army selection.  Given the current state of affairs, there are two obvious approaches, but neither is great...

1.  We could consider each army match-up to be a different variant - FIDEs vs. Rookies is one variant and Rookes vs. FIDEs is another.  This is a little cheesy and leads to variant explosion.  Essentially CwDA becomes two to the power of the number of armies different variants.  That said, this is probably the easiest.  I think ChessV can do this now with no modifications to the code at all.  We could always start here and be up-and-running in no time and just back out these "dummy" variants later.

2.  We can do what you are doing now - have engine paramters for army selection.  This is arguably better, but I don't love this option either.  We are adding engine-level settings for things that only apply to one game.  This is somewhat cheesy, unintuitive, and doesn't scale well.  What if we have a different variant that also has army selection but different armies for choices?  What happens when we start adding configuration parameters to other variants?  (ChessV already has configurable game-specific options for other things, such as the castling rule in effect in Capablanca variants.)  I don't mind altering ChessV but when I add things I always try to add them in a general way.  I'd rather not add code just to "hack" this one specific variant.

I think what we want - ultimately - is a new concept.  ChessV has "game variables" which are configuration options that apply only to specific games.  Conceptually, this is exactly what we are trying to accomplish.  Ideally, CECP would be given a new feature which engines could declare support for.  If an engine announces support for game variables, the GUI could then ask it, for any given game, what options it supports.  But I realize that this requires significant changes to XBoard/WinBoard, and it is probably no small undertaking.  But I think it's something worth striving for in the long term, and would pay dividends down the road.

Actually, we might be able to get there with the first step just being naming convention.  If we make "cwda" to be the Chess with Different Armies variant, maybe the options could be named like "@cwda:White Army", for example.  At first, WinBoard could just treat them like any engine options.  But in future, it might parse the option name to understand that it only applies to variant "cwda".  Just thinking out loud here ... haven't considered it deeply.

Fairy-Max currently also uses approach (1), which doesn't require engine or GUI to be aware that there is some relation between the 'sub-variants'. It uses the general 'engine-defined variant' mechanism for this, where it can just make up any name it wants and tell to the GUI that it plays it, after which the GUI makes it available for selection by the user. And when it gets selected the engine sends details like board size, pieces and moves to the GUI. Before that it did something similar by using 'fairy' as wildcard variant, where an engine-defined option then could select what 'fairy' actually meant (and a similar flow of rule info to the GUI when it got selected). Like you, I am not very happy with the resulting variant explosion. (Although this is not so bad as you say, just N^2 instead of 2^N.) Even with the 4 'standard' armies it overflows WinBoard's New Variant dialog with 'FIDE-Nutters' and similar variants. So I put them at the end, and kept in the old selection mechanism through an option.

So for the new engine I try to find something better. For an engine dedicated to CwDA the per-player option for army selection is a logical choice. But, like you say, if it really is a general multi-variant engine, it could lead to an explosion of options meaningful in only a single variant. I also found it quite annoying in practice that I have to set the option on both engines, when doing engine-engine games.

An explicit method for defining variant-dependent options like you propose would be a possible solution. An even more general case would be where options presented to you by the GUI would depend on the setting of other options. Processing the option definitions that the engine sends at startup in XBoard was not limited to the handshake phase, but originally it was not possibe to redefine or remove an option once it was defined, as newly received option features were always appended to the option list. But to make this behavior useful I made it such that receiving 'feature done=0' clears the entire list of options for that engine in the GUI. So an engine can redefine its options at any time. The application I had in mind was that engines could send a minimal set of options by default, including a button 'Advanced', and when receiving the activation signal for that option, resend a more extensive list of options (perhaps including a 'Minimal' button to switch back to the minimal set). Or even have several 'submenus' selectable by buttons from the base dialog (e.g. Eval Parameters, Search Options, EGT Options) each with a button in them to return to the basic list. Other applications would be 'collective' options that alter the setting of other options (e.g. 'Restore defaults'); the engine could then resend the option list not with different option names, but with different defaults, to make the GUI aware of the new settings. Main problem with this in the GUI is that the settings dialog gets redefined while it is open in such cases, which I think I tentatively solved by simply closing it (not sure anymore). If a change of option list occurs due to selection of another variant, that problem would not exist, as the settings dialog would be closed at that time.

Now from a protocol point of view this is a rather inelegant solution. But it is very general, and it works. A multi-variant engine at startup can only declare its general options (e.g. 'Resign Threshold'), but on reception of a 'variant' command resend those (starting with done=0) plus all options specific to the specified variant. If the user would then open the settings dialog he would see the new options. The engine would have to be smart enough to suppress resending of the option list if the variant did not really change, though, to prevent that clearing the list would make the GUI forget the previous setting of these options. Or be smart enough to make the previous setting the new default when it resends the list. (Not entirely trivial, as the 'new' command formally switches you back to 'variant normal' at the start of every game. So it should remember the argument of the latest 'variant' command, and on reception of a new variant command compare with that, and redeclare options only if there is a change that would lead to a change in options.)

It also seems CECP is in need of a mechanism for engine-defined options that work on both engines. Normally the corresponding features would be controlled by standard options, which might even use a different syntax (like 'feature memory=1' to set the hash size). But for features the GUI is oblivious of this cannot be used. But if the engine could define an option as 'slave', the GUI could send any change of the value to the other engine as well (provided both engines declared it). You could then change their value in the dialog of either engine, while the GUI would guard the consistency of the settings.

Sorry it has taken so long to get back to you on this.  A lot to chew on here.

First you mentioned engine-defined variants the GUI doesn't understand but still allows to be played.  ChessV doesn't do this, and it's not high on my list of things to do, but there is certainly some value to it so it would be best if whatever we settle on supports this.

Then you mention my game variables concept and the more general concept of allowing engines to suppy additional options as things change (variant selected, other options selected, option buttons clicked ...)  Certainly the more general concept has value, but I also think there is value in being able to specify that an option is a game-specific variable.  This would make clear that, not only does the option only apply to a specific game, but that the option affects the rules of the variant being played and not just the internals of the engine AI (although it could certainly do that too.)  This tells the GUI that, in order to play, both engines must support the same options, and both should be automatically set to the same values after the user selects them.  And I think it makes sense that these options should be presented differently because they are an essential part of selecting what game is being played.  In ChessV when you select Chess with Different Armies, the first thing you get is a dialog box to pick your armies with random armies pre-selected.  (The same thing happens if you pick Fischer Random Chess - you get a dialog with a random position number selected, with a preview, but you can change it if you want to play a specific position.)  I think game variables can offer a better user experience - if you don't care about, or even don't understand hashtable size, etc, who cares, but you better understand what the armies are.  Picking the armies is part and parcel of picking what variant you want to play.  And, also, some engines might play CwDA and offer extra armies that others do not.

Another thing I like about game variables it the potential they offer to allow an engine to play variants it wasn't specifically designed for.  ChessV has some understanding of this now.  If the user selects Corridor Chess, it is smart enough to offer as options any engine that supports both orthodox chess and the setboard option.  Likewise, if it offers variant capablanca and has setboard, it will be offered as an option for Modern Carrera, but not for Schoolbook because of the flexible castling.  (You did propose a clever workaround for this but I haven't got around to implementing it.)  But engines that support flexible castling should offer the "Castling" option of "Flexible" in variant capablanca.  Then, even if they don't know Schoolbook, Grotesque, or Ladorean, they can still play them, along with new variants that meet the same rules.

10x8 is popular, in the sense that there are lots and lots of variants, but almost all use one of a few castling rules, which I have named "None", "Standard", "Long", "Fexible", "Close-Rook", and "CRC".  Likewise, on a 10x10, there are only a handful of castling rules, and a handful of rules for the pawn's multi-step moves that cover most games.

About a year ago you mentioned working towards a standard for defining variants so you don't have to configure Fairy-Max in one way and SjaakII in another.  I think this is also worth pursuing - it would be awesome if we could acheive it, but this is a steep mountain.  I respectfully suggest that this would make a good first step.  This is the low-hanging fruit...

The castling issue is a good example of the different design philosophies that are possible, so let me respond to that first. If an engine supports Capablanca Chess, it is unlikely to support flexible castling. The only reason why the author would care to support such castling is that he is aware of Scholbook Chess, and wants the engine to be able to play that too. But if he wants that, why would he want the user to alter the start position (through setboard) and the castling rules (through an option) separately? It would be much simpler, for him as well as the user, to just add Schoolbook as an additional variant the engine can play. If there are other 10x8 variants that use flexible castling, but have a different start position from Schoolbook, these could then be played as sub-variants of Schoolbook (through setboard). Variants without castling are sub-variants anyway, as you can simply specify a start position without castling rights.

Having a castling option is a step towards defining the variant for the engine through option settings. If you have an option for castling, then why not one for how the super-piece moves (Queen / Chancellor / Archbishop)? Or options for how any piece moves? Or how large the promotion zone is? Or what you can promote to? (To play Janus as sub-variant of Capablanca.) This would all be possible, of course; it would converge on a design where the GUI manages all game descriptions (be it hard-coded or through a configuration file that the user can edit), and then instructs an engine how to play it.

This is not how existing multi-variant engines such as Fairy-Max, Sjaak and Nebiyu work, however. These manage the file with game definitions themselves. Which seems the logical way to do it, as then the file can also contain engine-specific info. (Like which pieces have mating potential, value of pair bonuses etc.)

Because of the way XBoard started (as merely a graphical front-end for an already complete ASCII-based user interface) things tend to be 'engine-centered' in WB protocol. From the early days on XBoard could already be used to play variants (as long as they were 8x8 with no more than 6 piece types, such as Shatranj) by switching legality testing off. (In fact in the earliest version it could not even be switched on, as legality testing was not implemented at all. Even when I first started working on WinBoard myself in 2008 it did not know about castling rights, and would always accept any castling.) Historically it was up to the engine to refuse moves it considered illegal, on which the GUI would retract the move and relay the info to the user. Downsite of this is that the GUI cannot indicate target squares of a move when pieces might move different from normal, and that it will generate crappy SAN (with missing or redundant disambiguation, but which it would still understand itself on reading it back).

I generalized the 'legality off' capabilities a bit by attaching a meaning to some otherwise unused combinations of mouse clicks; e.g. any sideway move of a King by more than 2 spaces is now interpreted as an attempt to castle with the corner piece in that direction (whatever it is), while dragging the King on top of the corner piece would be castling with it where the King steps one. (In all cases the corner piece ends up on the other side of the King.) Unless Fischer castling is specified by a GUI option (which by default is set for FRC and CRC), then capturing an own backrank piece with the King indicates castling with that. This covered all cases I was aware of, so the GUI doesn't really have to know the game rules to interpret the user move; assuming that what the user tries is legal, it will interpret it correctly. The engine watches over the latter, by forcing the GUI to take back whatever illegal thing the user might have done.

For better GUI performance w.r.t. SAN and target-square highlighting, however, it is essential that the GUI is also aware of the rules. Logically there are 3 ways to accomplish this:

1. Both the GUI and the engine are programmed / configured for the same rules independently.
2. The Engine transfers the rules to the GUI
3. The GUI transfers the rules to the engine

Your castling option is an example of the latter; through engine-defined options any info can in principle be transferred in this direction, but some standardization would have to be developed to make it possible for the GUI to do this automatically (rather than having the user decide on a per-engine basis what is necessary). For the start position the protocol has the setboard (or edit) command. For case (2) I extended the protocol with a 'setup' command as a reverse setboard, (plus a list of participating pieces) and 'piece' commands for defining piece moves.

Fairy-Max and Sjaak || use method (2); their game-definition file specifies how castling is to be performed, and (if this is non-standard) they inform the GUI of it as part of a King-move definition, e.g. "piece K KisO4", where the isO4 is the XBetza notation of a castling where the King moves 4 squares. Through this info XBoard itself can then already refuse any attempt of the user to move the King other than 1 step or slide it 4 (if still virgin etc.) without first consulting the engine. (And does this by not even highlighting other King destinations, and in general refusing any move to a non-highlighted square.)

I guess we agree that method (1) is undesirable; this was basically the situation that existed before introduction of the setup and piece commands, where the user had to provide a FEN (and even in an indirect way, as a .fen file with a GUI option referring to it) and a -pieceToCharTable option to specify the FEN's meaning, and switching off legality testing when standard glyphs were used to represent exotic pieces not in the standard set. For the case of castling it seems to me that method (2) is more direct than (3). In all cases the engine would have to announce something to the GUI at startup, either that it has a castling option that features 'flexible' as a choice, or just that it supports Schoolbook. In the case of the option the GUI will have to recognize it, look up in a list of known variants which variants would need any of the available settings for that option, and enable selection of those variants by the user (if their other requirements are also satisfied). In the other case it can just enable selection of 'Schoolbook' without having to worry about anything, or even know what 'Schoolbook' means. When the user decides to select Schoolbook, in the case of the option the GUI would now have to set the castling option in the engine to the value it knew to be required for Schoolbook, as well as enable that type of castling in its own move generator used for legality testing / square highlighting / SAN parsing and generation. In method (2) it would just configure its move generator likewise because the engine tells it so through a piece command. Again, it did not need to have any pre-programmed knowledge for that. The 'piece' command can be considered in this case just as a (somewhat cryptic) command to set a GUI option for the type of castling. This is much less of a hassle for the GUI.

I am perfectly happy with XBoard being a 'dumb' GUI, which leaves the engine in charge. But if you want ChessV to be a GUI that knows about many variants, and makes conclusions about which variants not explicitly supported by the engine can still be played as sub-variant of some other, supported variant, method (2) would still allow you to do it. In all cases the GUI would look which variants the engine supports, and what aspects of variants the engine allows to be changed in general (startposition by setboard, possibly other options), and then can run through its list of known variants to see which can be run as sub-variant. It doesn't really matter if a 10x8 variant has to be run as Capablanca + setboard + castling=flexible or through Schoolbook + setboard. If you want the GUI to have intrinsic knowledge of what Schoolbook is, it can simply ignore setup or piece commands the engine sends to specify start position and castling type.

As to the selection of armies:

We definitely agree that it is useful to have options that are common for both engines. Putting some common prefix, and perhaps an agreed-upon special sysmbol in front of a set of option names is always possible, and doesn't interfere with any interface that would not treat it as special.

The main thing I worry about is that army selection in CwDA, and even more clearly selection of the 'guest pieces' and where they have to go (as replacement or through gating) in Superchess and Musketeer Chess, setup of the super-pieces in Metamachy, are all really parts of game play. According to the official rules of those games there is a prescription for how the players negociate an initial position. In the XBoard implementation of Superchess I dodged that problem by implementing it as a shuffle game; my justification for that was that this 'game prelude' is really a work-around for human-human games, where there is no third party that could make the decision for them in an unbiased way. When a computer is involved, it might as well make the decision for them. If they don't like the proposed setup for a human-comp game they can just keep pressing 'New Game' until they do. If comp-comp games need a specific start position you just play them as games from a setup position, and provide the desired FEN to the GUI that runs those games. Jocly, OTOH, takes these game preludes seriously, which makes the implementation of the mentioned variants enormously more complex, involving a lot of non-standard code. (Where other variants are implemented by simply giving a list of piece properties and their start location.)

For CwDA the issue is a bit more subtle, as there are potentially so many armies (even the Wikipedia lists many armies I had never heard of) that it is impossible to give the pieces a unique 1-letter ID. So which army the engine has to play with cannot be unambiguously specified through the FEN in a setboard command, even if the GUI would make the army selection at random.

Historically WB protocol contained a feature that would allow user-engine dialogs as needed in a prelude: the askuser command. This is equivalent to a single string option: in response XBoard will pop up a dialog where the user has to enter a text, which will then be sent to the engine on 'OK'. In the latest XBoard I tried to expand the possibilities by allowing the engine to request opening of its entire settings dialog (as defined at that time) 'spontaneously'. So it could redefine its option list to define a dialog for the purpose, starting with feature done=0 (to temporarily remove its normal options), and end with done=2 (the forced popup request). This could have two comboboxes in it for the available armies, or (for Musketeer chess) comboboxes for selecting piece types and the files at which they would gate, etc. Problem is that such things would not work in automated engine-engine games unless the GUI itself would be able to set the presented options. But in WB protocol the engine can know if the opponent is a computer, and then refrain from requesting the popup, and use the default values, or the values previously given for those options.

Things are really ugly if the rules for the prelude require the players to take turns on selecting things; in a human-comp game you would then have to go through a sequence of dialogs where the engine each time says "I did this, now what is your next step?", before we can actually start moving pieces. And what if it makes sense for the engine to really think hard about its decision? Should its clock be running?

I got the impression that you would like to have a separate dialog for setting the variant-specific options of the engines, which perhaps pops up automatically after you selected the corresponding variant. This would be feasible for CwDA, but in view of the above I wonder if it would be sufficient in general.

Regarding army selection, and other setup options:

I got the impression that you would like to have a separate dialog for setting the variant-specific options of the engines, which perhaps pops up automatically after you selected the corresponding variant. This would be feasible for CwDA, but in view of the above I wonder if it would be sufficient in general.

Yes, this is what I do.  It populates the dialog with any game variables that are not yet specified.  There are reasons why they may already have values.  The game may provide values, in which case they are there just as a configuration options, but the user isn't normally prompted.  Or the user may have selected a game that is derived from the game which fills in the options. (You could make a new variant derived from CwDA where the armies are always both Nutty Knights plus whatever other changes.)  Or the options may already be specified because the user has opened a saved-game file, which of course must contain this information along with the game.

You raise a number of other things though which could certainly be considered related.  For games where players take turns making choices or placing pieces, I view this as part of the actual game.  Ideally, the GUI prompts you when it is your turn to make a decision.  And the computer's clock should certainlly be running while it is considering its own choices.  That said, I have not implemented anything like this yet.  I haven't implemented any of the games you mention...  And for some choices, it may not be obvious how they would be communicated in the protocol.  For placing pieces, though, the existing @-notation for dropping pieces seems like an obvious choice.  This does, of course, require a lot of new code...  I agree it could be considered an "ugly" problem from a code-writing perspective.

In the latest XBoard I tried to expand the possibilities by allowing the engine to request opening of its entire settings dialog (as defined at that time) 'spontaneously'. So it could redefine its option list to define a dialog for the purpose, starting with feature done=0 (to temporarily remove its normal options), and end with done=2 (the forced popup request). This could have two comboboxes in it for the available armies, or (for Musketeer chess) comboboxes for selecting piece types and the files at which they would gate, etc.

Having this come from the engine just does not make sense to me.  There are potentially two engines but only one GUI, so it seems to me the GUI should be "driving the bus" here.  Which engine makes the request?  What if the other engine doesn't support the same armies (or other options)?  You could just pick one engine and stipulate that to run a match between engines they must support the same things (or that the use will only pick options supported by both.)  But then the GUI needs to tell the second engine, so the options would need to be named the same... It seems this approach still has game variables, and the GUI still must have an understanding of this concept to facilitate the hand off.  I think it is more straight-forward for the GUI to know which game variables and associated options the engines support, (the engines announce this anyway), and present the user a choice of the options both support.

I'll hold off on addressing the other topic for the moment.  I don't have enough time at the moment to think through it all and draft an intelligent response.

Note that I don't want to argue that this is an ideal solution for everything; I just indicate what is already possible in XBoard. The possibility to pop up complex dialogs was inspired by Evert Glebbeek's remark that he would like an engine to be able to use the GUI for design of a new variant (rather than requiring the user to edit config files), through an interface similar to the Design Wizard I put on the Interactive Diagram article here. The described method was trivial to implement in the GUI, as everything was already there. So it was just a matter of allowing the engine to invoke it (rather than a menu click).

Some steps occurring in typical preludes could indeed be covered by regular moves, so that they would not need any special provisions at all on the GUI side. E.g. variants that start with setting up the army turnwise on an empty board could use drop moves, and the engine can simply refuse premature attempts to move around pieces in the setup phase. This can be extended by adopting the conventon that a drop move on a square that is occupied by a friendly piece indicates a swap, and takes the original occupant back into the hand. That way a Superchess prelude could be implemente as: white substitution 1, black substitution 2, obligatory white symmetrizing substitution 2, obligatory black symmetrizing substitution 1, white substitution 3, etc. In Musketeer Chess you don't have to pick just pieces, but also assign them to a file. (Very bad idea, btw...) This could be done by making extra (initially empty) board ranks to drop the selected pieces on. But this would then turn their later gating into a double move, where they have to step forward as a side effect of the piece in front of them moving away. Of course if the protocol extension is used that allows general multi-moving, (as in the WinBoard Alien Edition), this would be no problem. It does seem a bit unclean, though (and for clarity the extra ranks should be differently colored than the normal checkering, so a lot of GUI support). Some people would attach value to the piece and its gating location being visible at all times, but I don't consider that an absolute necessity. (There is also no optical clue for whether you are allowed to castle, so not all game state is always visible.)

I am dwelling on this to decide whether there is any market for a protocol extension that could exchange arbitrary messages between the players (whether engine or human), which would then also switch the clocks and appear as a move in the PGN. E.g. "pseudomove <string>" both for engine->GUI and GUI->engine, depending on who's turn it is. At first glance this seems of little use if the GUI does not know what <string> means. But it could be used to change hidden game state, for instance to assign a file to a piece in hand, to play Musketeer as a modified Seirawan Chess (where the GUI assumes you would always be allowed to gate anywhere). The engines could then communicate the allowed gatings to each other through the pseudomove command, send the actual gating in S-Chess format (e.g. f1g3c to gate a Cannon), which would be understood by an S-Chess-supporting GUI. GUIs especially designed for Musketeer could apply some visible clue to indicate the gating pieces and files, which could go as far as displaying a piece just beyond the board edge. But you would not be dependent on such a GUI feature for playing engine-engine or human-engine games. (If you are willing to put up with the hidden game state.) In the latter the GUI would have to present the <string> to the opponent, and allow the user to enter one by typing. (The general mechanism.)

Note that playing different engines against each other is really something that almost no one does. TalkChess is really a very unrepresentative environment in that respect, full of engine developers and testers. But engine developers, and especially variant-engine developers, are a rare breed. In XBoard the 'second engine' is only used in 'Two Machines' mode. But XBoard is very tightly coupled to the 'first engine', taking it along with everything you do (e.g. in Edit Game mode). This because it originally was dependent on that engine for legality checking of the entered user moves. So what XBoard does is ignore the second engine completely (it is not even started up) until the user presses 'Two Machines'. Then that engine is started, and if it then turns out to not support the current variant, the mode switch is refused, and an error popup appears reporting "Second engine does not support this variant". For variants that are not fully specified (such as shuffle games) the first engine is in charge, and any start position it specifies will be loaded through setboard into the second engine. The engines should better be compatible w.r.t. how the pieces are called and how they move, but that always applies. If they don't agree on the initial setup (like the probably would, in shuffle games) it is no problem, the position indicated by the first (if the GUI or user had no desires on this) would be force-fed to the second to setboard. If they would disagree about piece names or moves this would wreck things, and XBoard currently doesn't check this. But this would always be the case, even in standard variants known to the GUI, where there isn't anything that could be checked. It just means one of the engines is non-compliant.

I guess the real problem is that different CwDA flavors are in fact different variants, so that treating them as a single variant leads to all kinds of unnatural problems. In a single variant, only partially supporting it (like: "I can play Chess, but the Knight is not implemented") is not an option. But there is nothing wrong with supporting CwDA except the Nutters. The only real problem is the unmanageable proliferation of variants this causes when you support many armies. Perhaps the GUIs should just provide a smarter selection method for variants in general, not just allowing ticking of one radio button from a list of 'full' variant names, but allowing a variant name to be synthesized from two parts, each selectable by a radio-button from a list. E.g. the engine could specify it plays rookies~CWDA and CWDA~nutters, where the ~ in the variant name would be recognized by the GUI as having the special significance of defining a partial variant name, where the all-capitals part then would be considered a place holder as well as a group identification. It could then present lists of such 'left' and 'right' partial names in the New Variant dialog, each with their own group of radio buttons and allow selection of two compatible (i.e. belonging to the same group, here 'CWDA') partial variants (and refuse 'OK'ing the dialog for incompatible selections), and combine the partial names to 'rookies~nutters' to get the selected variant. Depending on the preference of the GUI developer this could also be two combo-boxes (for left and right partial name), or even an arbitrary number of pairs of comboboxes (one for each variant group). The list of 'full' names would probably list 'selected below' as one of the possible choices to enable the comboboxes for selecting the partial names. Or one radio button for each variant group. E.g.

o normal

o xiangqi

o shogi

o CWDA: [combobox1] ~ [combobox2]

[Cancel] [OK]

The (first) engine would have full control (through its variants feature) over how this New Variant dialog would look. (And pedantic GUIs could ignore things in the list that they don't like or don't know; e.g. the standard edition of WinBoard would ignore variant tenjiku.) If a match with the second engine is attempted, the GUI will check if the current variant is in that engine's variant feature directly, or as a combination of partials. And refuse the request to play the engines against each other when it isn't.

This moves the selection of armies from the engine's private option settings to the GUI's variant-selection dialog, which is what you want. The variant is a common setting for both engines, so it also removes the problem of  'engine coherence', which is what I like about it.

I have some misgivings about engine-defined options that should be standardized, recognized by the GUI, and displayed in a separate dialog. This is just not what the option feature was desigintended for. For comparison: WB protocol allows 'size overrides' on variant names, like 5x5+5_shogi. (And users can select such variants in XBoard by using size-override rank, file and holdings controls in the New Variant dialog.) Normally the engine would consider each differently overridden version of a variant a separate variant, and list it as such in its variants feature. (E.g. Shokidoki lists shogi, 5x5+5_shogi (= mini-Shogi) 6x6+6_shogi (= Judkin's Shogi), 7x7+6_shogi (= Tori Shogi) and 8x8+6_shogi (Euro-Shogi). But I also did define a standard pseudo-variant 'size', where 10x10+0_size in the variants feature would imply that the engine will accept size overrides for board sizes up to 10x10 (but no holdings) on any of the reported variants. This is very similar in purpose to your concept of game variables, like having spin options for ranks and files with maximum 10. The equivalent of your castling option in this form of the protocol would be to allow a 'flex_' prefix on any variant name to indicate that variant uses flexible castling, (e.g. flex_capablanca) and define a standard pseudo-variant flex_castling to indicate that every supported variant also is offered in the 'flex' flavor. (Basically just a short-cut, to prevent very long variant lists.) GUIs understanding the flex_ thing could display / enable a checkbox 'flexible castling' in their New Variant dialog. This way of doing things looks more natural to me, because it puts the modifiers where they really belong: they basically create different variants, which deserve to have different names.

After having slept on it I realize there is a problem. General configurable variant engines will usually be configurable to play CwDA, but they will have been designed without the peculiarity of army selection in mind, and thus have no special features for it. The only way for such engines to play CwDA is by adding each flavor as a separate variant. If a GUI would not support this method, it would thus become incompatable with most engines that can play CwDA, as there will be very few dedicated CwDA engines. The partial naming I proposed in my previous posting requires engine support that they would not provide.

So it seems impossible to prevent variant proliferation in the engine's variant feature of WB protocol. But that shouldn't stop a GUI from keeping this proliferation out of its variant-selection dialog. So I want to propose a naming convention that would give special treatment to variant names of the form A~B(C) (e.g. FIDE~Nutters(CwDA) ). These should be considered as belonging to the "variant group" C, which should be enabled on par with primary variants. The GUI would collect lists of all A and all B occurring with that C in the engine's variants feature, and offer a separate selection method (e.g. a combobox) to select an A and a B.

This would give rise to rather long variant names, which would be clipped in the display of some existing GUIs; this is why I put the least-informative part at the end, rather than having something like CwDA:FIDE~Nutters. In addition, GUIs that do not support this convention would display a very long variant list, which they might also truncate. (This problem exists already with WinBoard + Sjaak II without the help of CwDA, and the latter provides a work-around for it through an engine-defined option defining the meaning of wild-card variant 'fairy'. In engine-engine tourneys selection through a WinBoard command-line parameter is always possible, though.) But newer GUIs would reduce the entire group to just a single item in their primary selection menu no matter how many flavors the engine reports, and provide additional methods for sub-selection of a flavor. (E.g. comboboxes behind the radio button that selects the item, or a separate dialog that pops up after the item gets selected through the primary dialog.) Engines don't have to know anything about this 'streamlining' of the variant selection.

P.S. due to construction of a new kitchen in my home I probably won't have much opportunity to think about these things or post here in the upcoming week.

I think this is a reasonable proposal.  For games with required parameters, I can provide some information about how the game variables are mapped into the xboard variant name.

It seems we will need two-letter piece names though.  One thing I do not want is for the notation of a piece to change depending on what the opposing army is.

I hope your kitchen renovation has gone well!

The problem with multi-letter ID is that they do not work in FEN when mixed with single letter. (And even if not mixed, it leads to FENs that are unreadable for humans). As an alterative to FEN there could be DEMN, which uses piece IDs whith a first-letter upper case and the optional following letters in lower case, and then lists the white and black pieces in separate sections concatenated by #. (Advantage is that that this also works for multi-player variants.) I was still not happy with the human redability, though.

In XBoard I finally adopted the solution of ' dressed letters', where each piece ID is a single letter (captital / lower case used to distinguish color in FEN), optionally followed by a punctuation sign, like quotes or an exclamation point. So L, L' , L`, L" and L! all indicate different pieces. This gives FENs that are quite readable. It can also be use in SAN, but there alternatively only the letter could be used, conventional coordinate disambiguation sloving the problem that G can both be Gold an Great General. (The WinBoard Alien Addition currently uses this method in Tenjiku Shogi.)

Note that there is not really any need for pieces of opposing armies to have different ID; it should always be clear whose turn it is (in SAN), or which color (in FEN). So both Bishop and Bede can be B without causing any problems, as they will never occur in the same army.

Note that there is not really any need for pieces of opposing armies to have different ID

Actually, you can't count on that.  A pawn may promote to any piece from either army.

XBoard I finally adopted the solution of  dressed letters

Yup.  I've already added support for this.  You made a good case why this was logical for shogi variants with lots of pieces.

The problem with multi-letter ID is that they do not work in FEN when mixed with single letter. (And even if not mixed, it leads to FENs that are unreadable for humans).

I use an underscore as a prefix to force recognition of two-letter notations.  It is not required if the first letter of the two-letter code does not represent a valid piece on its own, although you can use it anyway.  The underscore is never displayed to the user.

As for human readability, I don't really understand this concern.  They're not readable already.  Can you look at a chess FEN and visualize the board (like which pieces are on the same ranks or diagonals?)  I certainly can't.  To me FEN is just a dense encoding for computer interpretation.  In ChessV, you can just paste an FEN into a dialog and it will load it.

Actually, you can't count on that.  A pawn may promote to any piece from either army.

O sh*t, I hadn't thought of that. Probably because it cannot happen in Fairy-Max.  I still wouldn't be unhappy with the dressed-letter approach, though. Charging R and N could be R' and N', Bede could be B'. To me that doesn't look any worse than CN, RN and Bd. For the classical armies this would give: NBRQ, WFB'A, F'N'R'C, W'HSM. In almost any situation the punctuation would be irrelevant, as you would virtually never promote to Waffle, Fibnif, Woody Rook etc. So if we choose to use conventional disambiguation in SAN rather than punctuation, it would never virtually never need to disambiguate anything. And if the FEN is considered just an arbitrary string for computer consumption, it becomes irrelevant how it is done there anyway.

I use an underscore as a prefix to force recognition of two-letter notations.

I did something similar for the internal representation of start positions in my engine HaChu, using colon instead of underscore, but I wasn't very happy with the result. If auxilary symbols are used, you might as well just require all pieces to be separated by commas. (This is the notation that is also used in the TSA rule books for the various Shogi variants.) Forsyth notation was originally conceived for human reading, in newspapers. For computers it would be much more convenient to have some fixed-length format. You either don't care much about the length, (like in GUI-engine communication), or you care so much (in database applications) that FEN is hopelessly inefficient size-wise.

My problem has always been that XBoard would have to be changed in uncountable places to support even just 2-letter IDs. Which makes me inclined to consider options that objectively might not be the best. With the dressed letters the Id currently still fits in a byte, and only on reading or writing a mapping has to take place of the ASCII range 64-128 on the three other available 64-bit ranges. If I would manage to change all variables ever used to hold piece IDs to ints, I could use a similar strategy in storing mult-letter IDs e.g. as firstLetter + 256*secondLetter + ...

This night it occurred to me that for CwDA we could adopt the convention that L' means a piece that is normally in the opponent army. Then B can be Bishop for FIDE and Bede for Clobberers, and in the extremely unlikely case FIDE would want to promote to a Bede, this would become B'.

On vacation with only a phone so I can't type a long response.  I'm sorry, but I am not in favor of this idea.  ChessV expects pieces to have the same notation for both players (except for case.)  It would require a lot of changes to change that.  I don't mind changing things if there is a good reason but I don't like to complicate the system just to benefit one game.  I am doubtful this has any real advantage outside CwDA.  I think two letter notations would be best, but I understand that could be a radical change (needing to code in understanding of the prefix.)  I can live with heavy use of the suffixes.

I liked the idea because it creates complete freedom in the choice of id for any army, without having to worry about 'collisions' with other armies. Otherwise you would always get into trouble once the number of different armies gets large enough. It also looks a bit cumbersome to use unnecessary complex ids in a game between two given armies just because there exist other armies with which the ids would collide, while no actual collison occurs between any of the actually allowed pieces.

Oh, I certainly agree with those advantages.  I'm just concerned that this will require changes in tons of places.  Let me look and try to get a feel for just how much changing will be required...

Well, a work-around could be to define the FIDE and Clobberer Bede as different piece types, which just happen to have the same move. (And similar for Bishop.) As IDs would only occasionally clash, this means most of the time you would need 12 piece types for CwDA, instead of  10, which is probably well within the capability of ChessV. In Fairy-Max I often have to do that too, not because of the ID, but because it doesn't automatically flip the moves of black pieces. So pieces like Pawns always have to be defined as a white and a black variety. (And this posed the problem of allowing 'different' white and black pieces with the same ID.)

That is indeed a clever workaround, but it is a fairly ugly hack.  So I've started the painful process of having different piece -> notation mappings for each player.  Hopefully it won't be too bad.

I have hit my first road block however.  It seems that communication of promotion moves in CECP always uses the lower-case.  The example given in the standard documentation is "e7e8q".  Clearly, this is white's move, but the q is lower-case.  This far, I have always been sending in lower case.  How do we address this?  If I go forward with this change blindly, it will send promotion q for black but Q for white - which totally makes sense, but I'm afraid it will break engines that aren't expecting it.  An alternative, which seems ok (but please check my logic here) is to send the notation for the player which is moving, but forced to lower-case.  This shouldn't break anything, and it seems it will still work with what you are proposing.

This is indeed the way XBoard does it; for protocol moves it forces the promotion suffix to lower case, for SAN to upper case. (Also in drop moves, btw.) In these contexts case sensitivity would not provide essential info, as it is known who is on move. I don't know any variant where one can promote to pieces of the opponent's color.

For FEN XBoard strictly adheres to the convention that upper case is white and lower case is black, even when asymmetry of the armies would not make this strictly necessary because the type already implies the color.

I don't see how this interferes with white and black using different ID for the same piece type, though. If they would take the ID from the same table they would certainly have to do a case conversion for one of the colors, or either in SAN or CECP moves. That they now take it from a twice larger table without first masking away the color info doesn't change that.

Possible variant on that: use Q' etc. to indicate the pieces of black's starting army, and the plain letters for white's. That way, each symbol still means the same thing no matter which side it's on.

Or use English symbols QRBN for one color and German symbols DTLS for the other...

Ok, I've completed the change and everything seems to work.  The two sides can now have different notations for the same piece, so long as player 1's is upper-case and player 2's is lower-case.  When defining piece types when we aren't using this feature, the type notation is specified as it is for white and will automatically be changed to lower-case for black as it has always been so no existing code should break.  To specify notations when the players use different ones, just put a slash in between.  For example: "B'/b"

Here is a summary of what notations are now supported.  A notation can be a single letter.  It can be a single letter followed by ' or !.  It can also be two letters.  If the first letter of a two-letter notation conflicts with a single-letter notation, it must be prefixed by an underscore.  Use of an underscore is permitted even if there is no conflict.  The underscore is never displayed to the user.

Possible variant on that: use Q' etc. to indicate the pieces of black's starting army, and the plain letters for white's. That way, each symbol still means the same thing no matter which side it's on.

Thanks for the suggestions, but I don't think this is an improvement.  With H.G.'s approach, the apostrophe is only necessary in the rare case where a pawn promotes to a piece from the other player's army.  Also, his suggestion is backwards-compatible with Chess.  In CwDA, if both players are playing the FIDEs, everything should be identical to chess.  With this proposal, black's pieces would all have apostrophes.  Sure, you could further stipulate that the apostrophe is only used for pieces not found in the other army, but that requires additional logic for no obvious benefit.

Or use English symbols QRBN for one color and German symbols DTLS for the other

I don't think this solves any problem.  You could still have conflicts with other armies.  Besides, what's German for Bede?

Great! My kitchen is finally installed, nearly a week behind schedule due to several setbacks. (It was supposed to be finished while I was on holiday in Cyprus.) But my home is still a shambles, so it might take another week before things are back to normal here.

About the use of 'dressed letter' IDs in SAN (not really a protocol matter!): perhaps it is best to treat the punctuation just as an alternative form of disambiguation in the SAN parser. So that 'B' in the SAN move would match B, B', B", B!, ... A 'tie breaker' rule could solve cases where both a bare B and a punctuated B would match the input move, in favor of the bare B.

This would allow the user (or GUI designer) to generate the kind of SAN he likes best: either always writing puctuation, or never writing it, and rely on the usual coordinate disambiguation to indicate which piece was moved.

Congratulations on the new kitchen, and the Cyprus vacation.  That's certainly one thing I envy about Europeans - being able to pop off to so many awesome places without having to cross the Atlantic (which is a major PITA.)

Regarding SAN, for promotion, wouldn't you need to rely on presence or absense of the quote to determine which piece was being promoted to?  Note that I have no strong feelings about SAN.  ChessV doesn't do anything with it at present.

I was thinking.  If you're making a strong engine for cwda, I think I will join in and add cwda support to Quadrox.