An Interesting Back Game Position

[PersianLord]

Prelude

As some of you might remember, a few weeks ago I made a post about testing a new sterategy, i.e. playing deliberate backgames. After my first match against zbilbo, I got interested about a position in which I thought GNUBG evaluated it poorly. Afterwards, I shouted in FIBS hall that bots can't be trusted when analyzing backgames and that sparked a heated discussion. Among the dissident voices, that's of dorbel (Paul) was the loudest. I uploaded the position and also e-mailed the whole match in .mat for him to evaluate. It seems that after analyzing the position deeply, he was about to post the position here at fibsboard, but as he shouted recently, he was unable to do so. Then he kindly e-mailed me his lenghty analysis of the position and since I think it's of educational value, I post it here. I hope, as dorbel wished, other fellow fibsters will contribute to this topic.



An Interesting Back Game Position


      Some background to this position and the discussion of it may be of interest. A hardy perennial in the backgammon garden is the assertion that, “Bots (Gnu, Snowie et al) do not understand backgames”. The player making this assertion usually means that he does understand them and plays them better than the bot. On hearing this statement in a recent shout lobby, I challenged PersianLord, for it was he, to show me a position from a game where the bot would make a very bad play. He produced this position from a one point match that he had played against zbilbo.

He said that Gnu wanted to play 15/11, 6/3 and that the play was terrible. This led to a lively discussion with intelligent contributions from Zorba (a Gnu expert), inim (a computer expert), austin (who made some astute observations) and myself, after PL had posted the position on his blog. Nobody liked making the 3pt, but I felt that it was quite a reasonable play, if not absolutely best and volunteered to research the position. Here it is.



Double match Point. Black to play 4-3.


Let’s say straight away that this is really a post backgame position. Black has played his 1-2 backgame and hit a shot after White has born off four checkers. In the early and middle stages of a backgame being hit is usually irrelevant and often helpful. Here being hit isn’t helpful, usually reducing your chances of hitting next turn and always delaying the making of the points that Black badly needs. Black’s problem now is that he doesn’t have a prime or a strong home board to contain the White straggler. How should he best proceed? PL thought that it was clearly correct to build a prime in the outfield and then roll it home. This is a recognised technique, but it suffers from a serious flaw here, which is that Black already has four men in his home board, so he can’t for the moment build a six prime and will be hard pressed to assemble even a four or five prime until his back men have regained contact with the rest of his forces and/or he has recycled one man (or more) from his home board. Black will have to play more pragmatically for a while.

You have to recognise that Black’s play is going to be extremely difficult, while White’s is very easy, so that Black will need to make World Class plays to get near to the theoretical equity of the position. I have practised these a lot, so here are some things that I have learned. Their relative importance can change from move to move, so you’ll need good judgment to see what your priority should be when two aims collide.

(A) Avoid leaving a direct shot in the outfield if you can.

( Blocking large doubles is very effective.

(C) If you have to leave a shot, avoid plays that let him pick up two checkers.

(D) Don’t allow him complete freedom of movement. Hitting is often correct whenever he is in the outfield or ready to jump a prime.

(E) Making random points in the outfield more than six pips away from the straggler can pay dividends, as they create some numbers that loosen another blot.

Let’s see how that might lead us to find a good play here.

The first play that I thought of was 18/14, 18/15. This blocks 6-6, 4-4, 3-3 and 2-2 and of course doesn’t leave a direct shot. 5-5 leaves a double shot. White won’t be forced to open another blot with 6-1, 6-2, 5-2 or 5-1, because we haven’t left either of the anchors yet, but you can’t have everything and there is time for that later. I don’t see this play as an attempt to build a rolling prime, but it may become that later of course. This play came second in the Snowie rollout, with 51% wins.

That move covers four of our list, but it does leave White free to play all his roll next turn, so let’s think about hitting, 15-8*. This can’t be any more than a distraction, but it does give Black a chance to recycle his blot. What happens next is anybody’s guess. I think it will lead to a longer game with even harder plays than the other plans, precisely because a winning plan isn’t clear. It’s tough enough to pick plays when you are clear in your mind how you plan to win. If you are uncertain which plan to adopt or if the play requires you to switch between plans as the dice dictate, things can only get tougher. Sometimes you see a position like this in a chouette. Everybody in the team will have their own idea of how to conduct this, so it is often more effective to let the captain play his own game until the right strategy is clearer. Anyway, it did very well in Snowie rollouts, winning 50.3% of the games, but whether it’s a good plan for a human to seek more complication in a position that is already hard to play is a moot point.

How about 18/14, 6/3? PL and others were adamant that this “impure and ugly” play just couldn’t be right, but I’m not so sure. If Black isn’t going to build his rolling prime just yet, making a point inboard might well work. The plan after making the 3pt will be to keep hitting and break the anchors. White can dance of course and also enter awkwardly and be obliged to loosen up another blot. It may look messy, but White’s blot(s) will find it hard to survive in an outfield crowded with Black predators whether they are arranged in a prime or not. As the game moves on, Blacks home board can only improve, White’s can’t. This play topped the Snowie rollout, winning 51.5%

15/11, 6/3 looks too loose for me; too many shots, doesn’t block 6-6 and let’s White hit twice sometimes.

15/11, 14/11 has some merit. It does free 6-6, but 5-5, 4-4 and 3-3 are blocked and it does bring two more builders down for the home board. Well off the pace in the rollout though.

23/19, 17/14 is very creative. This was austin’s play and I like it a lot. It loosens a second checker when White can’t play a 1 with his straggler. It doesn’t always pay off though. If White can then make the 2pt it’s a big step forward. Still, the checker on the 19pt is good, generating extra return shots when White hits somewhere next turn. This was a very close contender in the rollout, with 50.8% wins. To me this clearly dominates its close relative 24/20, 17/14. I don’t believe that it can be right to leave the 24pt before the 23pt, as it makes White’s task much simpler.

I can’t see any other plays that appeal, either theoretically or because they rolled out well. You will notice that I used Snowie for the rollouts, using 2-ply standard and playing 1,000 games. This is because I think that Gnu just doesn’t handle this position at all well. Its rollouts assembled a reasonable list of plays, but when tested by rollout they only won between 28 and 33% of the time! Quite clearly (to me) it just butchers this position now and later. I don’t believe it to be a matter of how strong the playing engine is, rather more that there are just too many legal plays, 65 in this position, for it to assemble a strong list of candidate plays in the first place. If you set it to 2-ply for example, it will give all the plays a glance at 0-ply, ditch anything outside the set parameters at that level, then prune again at 1-ply. This may well leave behind a play that would be a contender if considered at the higher level. At 4-ply incidentally, an agonisingly slow process, it plumps heavily for 15/8*. It may even be right, but you can’t do 4-ply rollouts without risking dying first, so we can’t use Gnu to test that. 2-ply rollouts are feasible if you truncate them a bit, but their pathetic win rate shows that it must be playing Black very badly.

It will be interesting to hear what Gnu experts like Zorba and inim think of this.


So as far as Gnu is concerned, in this “contain a straggler after the bear-off” position, PersianLord’s point can be said to be made. However, this is a position in which human players outside the top stratum will struggle as well, so perhaps it will be more accurate to say that Gnu doesn’t have its usual edge here. Snowie on the other hand does, I think, play it better. It too suffers from a tendency to prune some pretty decent plays before they get into the candidate list, but the rollout results are believable. Snowie 3-ply evaluation plays 15/11, 14/11 incidentally. It also on evaluation overestimates its winning chances by a lot, but it always does this in similar positions and I always allow for that anyway. It’s a good tool, if flawed, but gets better with experience.

So, to sum up, four logical plays stand out. (a) 18/14, 18/15, (b) 23/19, 17/14, (c) 18/14, 6/3 and (d)15/8*. I like them in that order but I can’t say that one of these is clearly better than the rest. However, I don’t believe that anything else will do as well.

I would play any of these plays against any other play not in the list as a prop. If asked to take either side in a prop I would pick White, even though Black is a theoretical favourite. White’s side is very easy to play, Black’s is very tough as we have seen. On this evidence, I would play White against Gnu or any human playing Black for money. 

This is a great position to use for practice. Play it over and over, analysing after each game and watch your results improve as you learn to avoid the traps. Anybody got Jellyfish? I’d be interested to know how it does here. Anybody with BgBlitz might like to try that.

[stog]

excellent dorbel analysis thx both
i like austin's play
You shout: and dorbel's
You shout: stopping the big dubb
You shout: tho if i was playin backwoods id play 17 13 , 16 13
You shout:

[lewscannon]

Thanks, dorbel, that was an interesting read. (no, I'm not being snotty or sarcastic here)

[adrian]

Chapeau bas monsieur Dorrbel! Great analysis!

[stiefnu]

I ran this most interesting position through BGBlitz at 3-ply.  Of dorbel's four candidates, two figure in its top 10 favourites:
1.    0.366 mwp / -0.268       18-14, 6-3
2.    0.342 mwp / -0.315 (-0.024)    15-11, 14-11
3.    0.341 mwp / -0.318 (-0.025)    15-11, 11-8
4.    0.331 mwp / -0.338 (-0.035)    18-14, 23-20
5.    0.327 mwp / -0.346 (-0.039)    23-19, 17-14
6.    0.325 mwp / -0.350 (-0.041)    14-10, 10-7
7.    0.311 mwp / -0.379 (-0.055)    16-12, 15-12
8.    0.310 mwp / -0.380 (-0.056)    23-19, 6-3   
9.     0.306 mwp / -0.389 (-0.061)    14-10, 23-20
10.    0.301 mwp / -0.397 (-0.065)    24-20, 6-3

However, after a rollout, the order gets changed somewhat, a couple get dropped in favour of other moves and one of dorbel's picks gets elevated to Top of the Pops:
1.   23-19, 17-14
2.   14-10, 23-20
3.   15-11, 11-8
4.   23-19, 6-3
5.   23-19, 18-15
6.   14-10, 10-7
7.   23-19, 24-21
8.   18-14, 23-20
9.   15-11, 14-11
10.    23-19, 14-11

The wins & gammons differ quite markedly.  At 3-ply, BGBlitz has (for White) 69.4 wins, 47.6 G+BG & 5.5 BG.  After a roll out this changes to 58.3, 44.4 & 29.6 respectively.

I ought to add a health warning here - I am not that familiar with the ins and outs of analyses and rollouts.  So if someone would care to double-check the above, correcting it if necessary, I will not be in the least offended.

stiefnu

[dorbel]

Thank you stiefnu. Note that the position is double match point. Did you specify this in your rollout? Of course things may well be very different if gammons and/or the cube are still alive. it also looks as if Blitz, like snowie, considerably overestimates the winning chances initially.

[stiefnu]

Note that the position is double match point. Did you specify this in your rollout?

Yes, I set the match length to 1 and score at 0-0.  However, I do seem to have overlooked your suggested move 15/8* in my summary, having only just noticed that BGBlitz gives this as either 15-12, 12-8 or 15-11, 11-8 (depending on whether you enter the dice as 34 or 43) - duh.  Odd that it omits the conventional asterisk to denote the hit though.  Repeating the rollout now gives somewhat different results, showing 6-3, 24-20 in 2nd place.  This may be because the rolloout seems, to my untutored eye, rather truncated.  However, I am not familiar enough with BGBlitz to increase the number of games from what appears to be a standard 108.  Can someone else do this?

stiefnu

[FrankBerger]

However, I do seem to have overlooked your suggested move 15/8* in my summary, having only just noticed that BGBlitz gives this as either 15-12, 12-8 or 15-11, 11-8  - duh.  Odd that it omits the conventional asterisk to denote the hit though.

Well, I never found the asterisk necessary because it's redundant. Maybe I change it when I have a little spare time

Repeating the rollout now gives somewhat different results, showing 6-3, 24-20 in 2nd place.  This may be because the rolloout seems, to my untutored eye, rather truncated.  However, I am not familiar enough with BGBlitz to increase the number of games from what appears to be a standard 108.  Can someone else do this?

Rollouts with 108 at such a strange position give very varying results. If you have started the rollout from the best moves dialog box, the default settings are used. You can change them under "Setup/Rollout Default Parameter"

I rolled out the candidates with 2-ply (rest of the world counting) 384 games and variance reduction:

+12-11-10--9--8--7-------6--5--4--3--2--1-+   O: Green  55
|    X     X  X  X |   |          O  X  X |
|          X     X |   |          O  X  X |
|                  |   |          O       |
|                  |   |          O       |
|                  |   |          6       |
^|                  |BAR|                  |
|                  |   |                  |
|                  |   |                  |
|                  |   | X                |
|                  |   | X                |
|             X    | O | X        X       |
+13-14-15-16-17-18------19-20-21-22-23-24-+   X: Red  222

Position-ID: xAlkDTb8DwAABA    Match-ID: MAEgAAAAAAAA

Green - 0 Red - 0 in a 1 point match.
Green to move.

Rollout
=========
Number of Games:384

Move : 15-11, 11-8
       Wins    G+BG    BG   
Green  39.1    37.7    20.2 
Red    60.9     0.0     0.0 
Match Winning Probability Green: 0.391
95% Confidence Interval:         0.303 /  0.418

Move : 18-15, 18-14
       Wins    G+BG    BG   
Green  39.2    36.1    18.0 
Red    60.8     0.0     0.0 
Match Winning Probability Green: 0.392
95% Confidence Interval:         0.272 /  0.381

Move : 15-11, 14-11
       Wins    G+BG    BG   
Green  41.9    37.4    20.9 
Red    58.1     0.0     0.0 
Match Winning Probability Green: 0.419
95% Confidence Interval:         0.366 /  0.478

Move : 23-19, 17-14
       Wins    G+BG    BG   
Green  41.9    37.3    17.6 
Red    58.1     0.0     0.0 
Match Winning Probability Green: 0.419
95% Confidence Interval:         0.329 /  0.445

Move : 18-14, 6-3
       Wins    G+BG    BG   
Green  42.3    36.8    20.9 
Red    57.7     0.0     0.0 
Match Winning Probability Green: 0.423
95% Confidence Interval:         0.369 /  0.475

The results look reasonable to me, but lok at the large standard deviation!

To the starting point "bots can't be trusted when analyzing backgames"   I think on average the bots play pretty well. GnuBG has the problem that in very, very unusual positions it might make strange moves,  whereas BGBlitz is robust  even in unusual positions (at least this holds true for the old AI. I have no idea whether the AI from 2.6.0 behaves similar) and no experience with Snowie.

To conclude that bots can't be trusted in general in backgame analysis is IMHO simply wrong.

  but to conclude from

[playBunny]

Well, I never found the asterisk necessary because it's redundant. Maybe I change it when I have a little spare time

I think it's a good idea. Maybe it's just me but I sometimes find myself scanning a set of moves for the hits and that means looking for asterisks.