How do 'luck' calculations work..

[diane]

1. Rollout¹    13/11 13/10                  eq:+0,906
     Player  : 79,43% (G:33,22% B:1,06%)
     Opponent: 20,57% (G:2,93% B:0,25%)
     Confidence: ± 0,005 (+0,901<E<+0,911)
     Duration: 17 minutes 32 seconds

   2. Rollout¹    13/10 8/6                    eq:+0,895 (-0,011)
     Player  : 79,00% (G:33,07% B:1,07%)
     Opponent: 21,00% (G:2,93% B:0,21%)
     Confidence: ± 0,005 (+0,890<E<+0,900)
     Duration: 14 minutes 20 seconds

   3. Rollout¹    13/8                         eq:+0,894 (-0,012)
     Player  : 79,60% (G:31,59% B:1,07%)
     Opponent: 20,40% (G:2,74% B:0,22%)
     Confidence: ± 0,005 (+0,889<E<+0,899)
     Duration: 12 minutes 32 seconds

   4. Rollout¹    13/11 8/5                    eq:+0,885 (-0,021)
     Player  : 78,72% (G:33,03% B:1,08%)
     Opponent: 21,28% (G:2,98% B:0,23%)
     Confidence: ± 0,005 (+0,880<E<+0,890)
     Duration: 12 minutes 16 seconds

Ok, I have a feeling like I am picking up a can of worms and throwing them all over the floor...but here goes  

I have heard tell that a good player will appear more lucky, because they are 'creating luck', a concept I am wrestling with.

I spotted something in some of the match rollouts - and would like to see if I now understand this - or if I am making it too simple.

In the roll out above [and indeed all rollouts/evaluations], the best moves have the highest equity, with the least good moves coming out with negative results - judged against the best move.

If the equity increases with a good move...the player has a better chance of something good happening next roll - ie more rolls work well for them etc...

Under analysis - does this 'maximising equity per move' directly translate into a good luck rating in analysis...quite simply because the program is is using potential equity changes to measure how lucky a player is in a match?

Does anyone understand that - and am I completely missing the point?

[sixty_something]

it is late or early here and i am soon going to bed, but i can't resist giving you my perspective to see if others can clarify this for both of us .. i spent a lot of time with GNUbg during my hiatus from FIBS and other on-line activities .. during that time i asked this very same question

the answer i found was that "luck" in GNUbg is indeed a calculation .. the "luck" calculation is based on the change in equity from one position to the next regardless of whether the player chose the optimal or best move or not .. for a match, one can take the sum of all these "luck" calculations for consecutive positions to come up with the totals we see on the Match Analysis well within any round-off error

i very carefully did those calculations for every consecutive pair of positions in a few matches .. it was quite tedious .. as i recall, my notes were on the backs of a few envelopes .. so, this was very sophisticated stuff - lol .. i may have entered them into Excel, but i can't seem to find them at the moment .. the originals are somewhere in a box or buried under a stack of miscellaneous other papers in preparation for my move next month

regardless, my definition of a "luck" calculation based on those careful empirical observation is simply the change in equity from one position to the next .. GNUbg uses threshholds to define lucky, very lucky, unlucky, and very unlucky .. what we see are only very lucky in boldface and very unlucky in italics

for a match or game. "luck" is simply the sum of all "luck" calculations for all positions in the game .. the various ways GNUbg describes our luck is also defined with threshholds .. the exact values of those threshholds are not known by me

regarding the allegation that "a good player will appear more lucky, because they are 'creating luck'", is  related to "luck" calculations after the fact .. where one may literally see the "creation of luck" is a little more complicated .. you may glimpse it, IMO, but not actually find it by looking carefully at a series of Temperature Maps for each possible move .. what you will notice is that the best moves tend to be those that create the more intense shading in a temperature map regardless of the roll, i.e. the best moves optimize potential equity gain from one move to the next roll whatever the next roll may be

therein, IMO, is the "creation of luck" concept best illustrated

in conclusion i would add that "creation of luck" approaches equity in the future tense while "luck calculations" measure it in the present tense which when summed provide a past tense tabulation for an entire game or match

hope that helps .. remember it is late or early here still and my research notes are on the back of an envelope in an undisclosed location .. my brain may be in a similar undisclosed location as well

[blitzxz]

Under analysis - does this 'maximising equity per move' directly translate into a good luck rating in analysis...quite simply because the program is is using potential equity changes to measure how lucky a player is in a match?

Does anyone understand that - and am I completely missing the point?

No it does not. "Objective" (bot) luck doesn't depend on what the player moves. And everybody has zero luck in the long run even the world best players. (But in the long run really means in the EXTREMELY LONG run.) However, maximizing equity increaces chances to win and subjectively this may seem as good player is "making his luck to happen".

[boomslang]

I havent checked GnuBG's source code, but I am pretty sure 'luck' is calculated as follows:

For a specific roll, luck is the difference between the equity/MWC after playing the roll correctly and the weighed average of 21 equities/MWC's after playing all 21 possible rolls correctly.

Luck for a game/match for a specific player is the sum of luck for all rolls for the player during that game/match.

I have heard tell that a good player will appear more lucky, because they are 'creating luck', a concept I am wrestling with.

They are not creating luck, they move in such a way that few of their opponents rolls are good rolls and more of their own rolls are good rolls. This duplication and diversification is already incorporated in the equity of the position. The player might apppear more lucky, but it is just the result of maximizing equity, i.e. making the correct move.

[RickrInSF]

i think a good example of the appearance of creating luck can be seen with dorbel's last 5 3 move, many lower ranked players would have simply moved two men off the 13 and made two blocks, instead dorbel made it so the next roll, there are more "lucky" rolls

[sixty_something]

I havent checked GnuBG's source code, but I am pretty sure 'luck' is calculated as follows:

For a specific roll, luck is the difference between the equity/MWC after playing the roll correctly and the weighed average of 21 equities/MWC's after playing all 21 possible rolls correctly.

Luck for a game/match for a specific player is the sum of luck for all rolls for the player during that game/match.

that sounds as if it may be correct, boomslang .. i do recall small errors creeping into my calculations which i couldn't explain .. when summed for the match the errors on "luck" calculations for individual rolls tended to be well within an acceptable margin of error

if you ever have a chance while browsing through GNUbg code, i'd be interested in knowing that answer .. also, do you know what the threshhold settings are in match analysis for the various luck factor phrasings, i.e. "Go to bed!", "Go to Las Vegas immediately!", etc

[ah_clem]

I have heard tell that a good player will appear more lucky, because they are 'creating luck', a concept I am wrestling with.


I think the best way to understand this is that there are two notions of the term "luck" . People who understand probability (including blitzxz, boomslang, and the folks behind gnubg)  define the term in strictly mathematical terms - the luck for each roll is just the difference between the equity before the roll and the equity immediately afterward.  With this definition, nothing anyone can do over the board (other than manipulating the dice) affects luck.  Luck is orthogonal to  the checker play and cube decisions.


But most people don't understand probability, so their notion of "luck" is more nebulous.  In their world, if a player rolls something that allows him to do something good, that's a "lucky" roll. If the player rolls something that forces him to play something awkward, that's an "unlucky" roll.

The thing is,  good players position their checkers so that lots of rolls allow them to do something good and few rolls force them to do something awkward, thus many rolls will look "lucky".  Conversely, beginners will pile their checkers up on a few points where most rolls deteriorate their position, and when the inevitable happens they see it as "unlucky". So if you think about "luck" like most people do, the good player is more "lucky" because he gets more good rolls and fewer bad ones.  Of course, if you understand probability, you can see that it's not luck at work here, but skill at maximizing the number of good rolls and minimizing the number of bad ones.

For example, a beginner will think it's "unlucky" that he misses an indirect shot seven points away (1 in 6 or 17% chance), and then think it's "lucky" that his opponent hits a direct shot six points away ( 17/36 or  47% chance) But the player who understands that there's a huge difference between two seemingly very similar positions will play the percentages, and the player who doesn't understand the game will think it's luck.

Bottom line: good players will create positions where many rolls allow them to have good plays.  When the good rolls come up, many weaker players will think it's "lucky", but it ain't.  It's just playing the odds and maximizing equity.

[diane]

I have heard tell that a good player will appear more lucky, because they are 'creating luck', a concept I am wrestling with.


I think the best way to understand this is that there are two notions of the term "luck" .

For example, a beginner will think it's "unlucky" that he misses an indirect shot seven points away (1 in 6 or 17% chance), and then think it's "lucky" that his opponent hits a direct shot six points away ( 17/36 or  47% chance) But the player who understands that there's a huge difference between two seemingly very similar positions will play the percentages, and the player who doesn't understand the game will think it's luck.

Thanks clem - these two I am good with.  It is just the creating luck thing...which is essentially making the best moves so more rolls work for you next time...expressed mathematically, but in words  I am interested in.

Gnu gives an overall luck rating on a match - measured about the change of equity you would have got, if you had played the best move, for each move played.

In my head...if you play well and increase the number of rolls which 'work well' next go, then they will give higher equity change - and therefore rate as 'lucky'.

Does this accumulation of the positive equity equate to, say, bots appearing more lucky [on analysis] more often?

[ah_clem]

In my head...if you play well and increase the number of rolls which 'work well' next go, then they will give higher equity change - and therefore rate as 'lucky'.

Placing your checkers well will result in a higher  equity, but not a higher  equity change due to the roll. gnu calculates luck by looking at the equity before the roll and comparing it to the equity after the roll.  The pre-roll calculation is basically the average of the equity of all the possible rolls, so any equity advantage you gain from placing the checkers well will already be reflected in the pre-roll equity.  Since luck is just the change in equity, this increased equity won't be won't be reflected in the luck.

To cite a simple example, lets say  it's almost the end of the game and you have a choice of two moves - one allows you to win with 100% certainty  no matter what happens, and the other gives you opponent a non-zero chance.  For example:


GNU Backgammon  Position ID: AAAARQAAAAAAAA
                 Match ID   : cAklAAAAAAAA
+13-14-15-16-17-18------19-20-21-22-23-24-+     O: oin
|                  | O |                  | OOO 0 points
|                  |   |                  | OOO
|                  |   |                  | OOO
|                  |   |                  | OOO
|                  |   |                  | OO
v|                  |BAR|                  |     1 point match (Cube: 1)
|                  |   |                  | XX
|                  |   |                  | XX 
|                  |   |                  | XXX
|                  |   |                  | XXX Rolled 21
|                  |   |       X        X | XXX 0 points
+12-11-10--9--8--7-------6--5--4--3--2--1-+     X: You
Pip counts: O 25, X 5




If you play 4/1  it doesn't matter what the dice do, you've already won.  Compared to 4/2 1/off, it's a big increase in equity, yet since the dice are irrelevant, every roll will be marked as having neutral luck.  Your  advantage is already built into the pre-roll equity calculation.  Hopefully this is easy to see at 100% GWC, but it works the same way at every other GWC.

[boomslang]

Does this accumulation of the positive equity equate to, say, bots appearing more lucky [on analysis] more often?

I think if you look at the luck of bots (and good players) have during a match or game, then you will see that they are more often 'lucky' (meaning having a positive sum of MWC/equity) than 'unlucky' (meaning having a negative sum of MWC/equity).

Bots need less luck to win a match because they don't waste equity/MWC by making bad moves. This means that *if* they lose, it is because they were really unlucky. But since all players will have zero luck in the long run, this must be compensated by having a 'moderate' positive luck more than 50% of the matches/games.

I dont know if anyone can back this up with empirical data though. I might give it a go if I can find some spare time.

[ah_clem]

Does this accumulation of the positive equity equate to, say, bots appearing more lucky [on analysis] more often?

I am aware of no evidence that bots appear more lucky on analysis.  If you stick to the mathematical definition of luck as implemented by gnubg et. al. the bots should have neutral luck in the long run, just like everyone else.

If you find evidence that their luck is  different (outside the customary statistical confidence intervals, of course)  I'd be very interested in seeing it.

[RickrInSF]

vic shouting everytime he looses to a bot does NOT count as evidence

[dorbel]

I agree with boomslang. The total amount of luck between two sides must tend towards zero over time, because there is an equal total of positive and negative luck on every roll. However, this doesn't mean that there are an equal number of lucky and unlucky rolls on each turn. Good play will lead to more "lucky" than "unlucky" rolls in the course of a game, hence the better player will be slightly more likely to have positive luck at the end of each game and at the end of each match. The lesser player will balance this out with a greater amount of luck in fewer matches, thus wasting a lot of it.

Part of the problem is that the bot definition of luck is incomplete. There are several other different kinds of "luck", i.e. events over which we have no control, that affect the outcome of games but are not measured by the bot. However, good play can only ever have a positive effect in the long term, so forget about luck and play better! You'll get luckier.

[sixty_something]

i am inclined to agree that "luck" calculations in the long run will tend toward a zero sum for both bots and humans .. however, i think there may be a difference in the distribution of luck

for example, let's take a look at a "PerfectBot", call it PB, that plays only 5 point matches and plays at the highest possible skill setting .. after say a thousand matches, the distribution of "luck" in each match against a wide variety of opponents, would, IMO, look like a normal distribution for PB and be tending toward a zero sum

i would further suggest that the opponents "luck" distribution would look different .. exactly how different i am not sure .. would it be skewed? would it have a wider range of outliers? would it still tend toward a zero sum?

the only way to tell is by collecting such a dataset .. it would seem to me that the owner of the GammonBot's and BlunderBot's might be able to help us here .. for example, when a bot finishes a match, couldn't it simply output to a file a data record with a collection of match analysis values for collection and review? if so, i think we'd come a lot closer to understanding the answer to this very intriguing question

IMO, since a bot would tend toward making perfect moves which by definition include a careful analysis of how any "luck" in the next roll(s) might impact equity values

would said bot therefore tend toward optimizing "luck"?

i think that may be a distinction without a difference .. "equity" and "luck" are just two aspects of the goal seeking neural net seeking out a match win .. whether they are independent aspects is a very interesting question

finally, none of this directly explains the three sets of 66's by a bot that beat Tanika today or vic's lament that his "luck" in a match today was a personal worst at -208 .. for me, those two examples are interesting in that we humans tend to remember such examples of bad "luck" far more than good "luck" .. would a collection of data over time from bots games with humans recording and examining both "skill" and "luck" provide any additional insights? perhaps, IMO

while we might not see individual instances, as above, i think the data over time would confirm by tending toward zero sum "luck" that humans roll three sets of 66's to win as often as bots do and, when human "luck" hits +208 or bots hit -208, it is much easier to assign and remember as ones skill not luck

[ah_clem]

I agree with boomslang. The total amount of luck between two sides must tend towards zero over time, because there is an equal total of positive and negative luck on every roll. However, this doesn't mean that there are an equal number of lucky and unlucky rolls on each turn. Good play will lead to more "lucky" than "unlucky" rolls in the course of a game, hence the better player will be slightly more likely to have positive luck at the end of each game and at the end of each match. The lesser player will balance this out with a greater amount of luck in fewer matches, thus wasting a lot of it.

Yes.  Good players will have more rolls with positive luck because they arrange their checkers so that most rolls play well.   IOW, with good play most rolls are above average.  The thing is, with so many rolls above average, they're not very far above average so the roll will be only slightly lucky. Conversely, poor play makes for a very few lucky rolls, but when they happen they are way above the average roll (which is brought down by so many poorly playing rolls) and are marked as very lucky.

So with good checker play you should expect a lot of rolls that are marked as slightly lucky and a few that are marked as very unlucky.  For poor play, one expects the opposite: lots of slightly unlucky plays with a few very lucky ones.

I'm not sure that it follows that "the better player will be slightly more likely to have positive luck at the end of each game and at the end of each match."  Maybe I'm missing something....

[Zorba]

Maybe it helps to see backgammon as a game of american football (sorry for the real football fans, but yeah, this is what it's called  ). I'll ignore (back-)gammons and the cube. The middle line of the game is equity 0, or 50% GameWinningChances. Your scoring line is equity +1, or 100% GWC, Opponent's scoring line is -1 or 0% GWC.

The game starts with the ball in the middle: equity zero (or 50% GameWinningChances).  A lucky roll brings you closer to the goalline, as equity increases. An unlucky roll takes you farther away, decreasing equity. The good side of the middle line is positive equity, or >50% GWC; the bad side of the middle line is negative equity, or <50% GWC. The same goes for opponent's rolls, but with the opposite effect for you. So basically, luck comes in two forms for a player: good luck on his own roll, or bad luck on opponent's roll.

If this was all there was to backgammon, just luck and no skill, then it's just waiting until one side gets enough cumulative luck (including bad luck for opponent) to cross the goalline (i.e. win the game). Obviously both sides have equal chances, and the total amount of luck needed to win a game this way is half the playing field, in bg equity +1 (or 50% GWC).

Now, the skill factor basically means that you can make errors in the game, which will bring you farther away from the goalline than just the luck factor would have determined. The same is true for your opponent, but with the reverse effect for you: an error by opponent will bring you closer to your goalline, without needing luck.

An example: you win the opening roll 3-1. This is lucky. You go from the middle line to the next line, ten yards closer to the goalline, just on luck alone. Equity wise, say you jump from 0 to +0.1 and in GWC from 50% to 55%.

Now suppose you play the 3-1 wrong, 24/23 13/10. This is like a 0.1 error, or -5% GWC. It will set you back 10 yards, and you're back to the middleline.

You were lucky, but you haven't made any progress in the game, because of your blunder. Basically, you threw away some of the luck you got, by playing a bad move. If you do this often, and your opponent doesn't, it's easy to see that you will need more good luck to win a game, than your opponent does!

So, when a good player plays a bad player, the latter will make a lot more errors, bringing the better player closer to the goallline all the time, without him needing luck for that.

As it is however, single backgammon games tend to have a huge luck factor, meaning that the advantage a better player creates by the above process is typically quite small. Like, with equal luck for both sides, after a series of moves, the better player has moved 10 yards from the middle line, closer to the goalline. Still plenty of yards to go, and luck will be needed to cross that distance. But your opponent will need even more luck to win this game, as he's farther away from his goalline.

In equity terms: after a bunch of moves in a game, the better player can expect to have gained f.i. +0.2 equity due to opponent's errors being worse than his. Now the better player needs "only" 0.8 luck to win the game; the worse player needs 1.2 luck to win the game. 0.8 luck is more likely to happen than 1.2 luck. So yes, the better player gets lucky more often, but by smaller amounts; the worse players gets lucky less often, but by bigger amounts (as he needs them!). Multiply the two (frequency of luck * amount of luck) and the total luck both sides receive is still equal.

[diane]

These are fascinating worms..I mean replies 

This is uncovering more than I originally considered.

To clarify what I am attempting to understand...here are some more words, and sorry Zorba, I dont do American Handball 

Backgammon is a mix of luck and skill, the more skillful you are, the better your chances of winning - and that is the only aspect of the game you can influence.....or is it.....

If I roll Zorbas 3-1 opening as described above - and play the optimum move, I have an equity change about that roll.
If I play it correctly, I influence the potential equity change on the next roll positively....
If I play it wrongly, I have the opposite effect.

So, roll 1 has an imaginary equity gain of +3 [nice roll  ]

Played correctly - total so far +3
played incorrectly, still +3, cos that is irrespective of what I did with it.

The we split into two parallel universes 

Universe 1 - where I played correctly, so the next roll has more good things I can do with it, therefore better potential equity gain..

Roll 2 has an imaginary equity gain of +2 [it wasnt as nice an imaginary roll]

I play this correctly and get the +2 equity...total in the imaginary game so far = +5 luck factor...ooh arent I getting lucky 

Universe 2 - where I played the first roll incorrectly...the next roll has less good things I can do with it - therefore less potential equity gain..

Roll 2 here has +1 associated with it...I play this one incorrectly too - but since in a luck calculation, we are only looking at what I could have got...it is still +1

Total for game so far..+4 luck factor

And on this process goes.

Leaving aside that bad luck comes too, regardless of what you play, and what your opponent is doing with his own good or bad fortune...doesn't this show that by playing well, a backgammon player influences her own luck factor [as calculated by the analysis programs] for the better....leading to a potential statement that...

Increasing your skill level not only improves your chances of winning, but increases your overall game luck factor?

Now this wont ever overcome a -208 luck factor, but it should help to convince the ever increasing 'analysing crowd' that it is still worth improving play, because improved play improves luck ratings.

That is of course, if my reasoning isn't as full of holes as the England defence 

[blitzxz]

i think the data over time would confirm by tending toward zero sum "luck"

I've been thinking this too, but after about 35000 moves in my database my luck rate per move is +4 millipoints EMG. This means that if I would be able to play with -4 snowie error rate I should be able to challange perfect playing (zero lucky) bots even after 35000 moves. (If I'm understanding this correctly?) I'm starting to believe I must possess supernatural lucky aura and it will never go to zero. Any one else have long stats about their luck?

[Zorba]

If I roll Zorbas 3-1 opening as described above - and play the optimum move, I have an equity change about that roll.
If I play it correctly, I influence the potential equity change on the next roll positively....
If I play it wrongly, I have the opposite effect.

The first statement is correct; the second and third are wrong.

About one: In bot-speak, the roll changes your equity, in this case positive, because it's a lucky roll.

The bot determines that it's a lucky roll, by looking at what it thinks is the best play with the roll, determining the equity after playing that (8/5/6/5) and then comparing that to all the other scenarios possible, and the associated best plays with that. 8/5 6/5 gives you a better position than any of the other scenarios, so it's a lucky roll, far above average.

Now, you have just rolled, not moved yet. In these equity calculations, a move can only be correct (equity is what the bot already determined above) or an error (equity is less than what the bot had determined; the difference is your error). You cannot "gain" equity with a good move; all you can do is avoid losing equity by not making any errors.

So a bad move, lowers your equity right away. It doesn't per se influence anything on the next roll. You just start out with lower equity.

A good move, i.e. the correct move, gives you the equity the bot thinks you should have (i.e. higher than after any other move). It doesn't per se influence anything on the next roll either. You just start out with the equity you should have had.

[diane]

Any one else have long stats about their luck?

Yes...I should go to bed...

I have all the matches I have played saved - and analysed hundreds of them..I kept a spreadsheet for a long time and it was, frankly, depressing.

[socksey]

vic shouting everytime he looses to a bot does NOT count as evidence

Maybe..........maybe not!     Keep in mind vic's rating and keep in mind vic has been playing this game since he was a child.   '

socksey



"Why does your OB/Gyn leave the room when you get undressed if they are going to look there anyway?" - Anonymous

[diane]

You cannot "gain" equity with a good move; all you can do is avoid losing equity by not making any errors.

So a bad move, lowers your equity right away. It doesn't per se influence anything on the next roll. You just start out with lower equity.

A good move, i.e. the correct move, gives you the equity the bot thinks you should have (i.e. higher than after any other move). It doesn't per se influence anything on the next roll either. You just start out with the equity you should have had.

I didn't say you gain equity ...well, not intentionally    I am saying the equity is there on the move - and that is added into the overall luck factor as quoted by the bot at the end of the match. And I think how you play a move must influence what options you have on the next roll...dorbel seems to be following my warped mind...

Good play will lead to more "lucky" than "unlucky" rolls in the course of a game, hence the better player will be slightly more likely to have positive luck at the end of each game and at the end of each match.

So forget about luck and play better! You'll get luckier.

I am 'forgetting about luck'...what I am trying to build here is a persuasive argument to put to others obsessing about luck  In words that any one can understand 

The whole trend towards analysing matches and focussing in on how lucky or unlucky a player is detracts from what is really going on IMO. 

I know there *is* luck involved - and if two highly skilled bots play each other, the luck will decide it.

But since the majority of matches played by people are not in that scenario...the thought that you can make yourself more lucky by playing well is the inspiration needed to focus people in on playing well...I think

[rebcalale]

I didn't say you gain equity ...well, not intentionally    I am saying the equity is there on the move - and that is added into the overall luck factor as quoted by the bot at the end of the match. And I think how you play a move must influence what options you have on the next roll...dorbel seems to be following my warped mind...

I am 'forgetting about luck'...what I am trying to build here is a persuasive argument to put to others obsessing about luck  In words that any one can understand 

But since the majority of matches played by people are not in that scenario...the thought that you can make yourself more lucky by playing well is the inspiration needed to focus people in on playing well...I think

LUCK is easy to understand: u or anyone else should have the same chance to get the rolls u/or your opponent need to win.  Play to the odds and u should win more than u lose.  Of course it is not a straight line and there will be situations where luck seems to favor one or the other player regardless of how well u play.  However beware of game environments where the dice r not legit.  This is common on the Internet.   For example, on fibs the dice r simply not reasonable.  Why do I write this, easy just chart the rolls an u will see that what I mean.  Far too many once in a lifetime rolls and if u keep track u will see there r just too many repeated rolls.  Now does this mean fibs favors one or the other player?  Hard to prove but it does look like it is biased. Interesting that NO ONE, let me repeat that, NO ONE has ever offered concrete evidence that fibs dice r legit.  So until that happens be wary!

[diane]

LUCK is easy to understand: u or anyone else should have the same chance to get the rolls u/or your opponent need to win. 

NO ONE has ever offered concrete evidence that fibs dice r legit.  So until that happens be wary!

Point 1- this thread is proving that that is simply not the case...

Point 2 - NO ONE has ever offered concrete evidence that fibs dice arent legit

But that isn't the conversation here - please have that on the other thread.

[pck]

Any one else have long stats about their luck?

gumpi was kind enough to run 1100 of my fibs matches from the past 3 years through XG and my total luck came out almost exactly zero.

[sixty_something]

WAY

i just scrolled back and discovered quite a few interesting posts i missed over the weekend .. i have been busy getting an email based chess game going with my grandson .. luck is the great equalizer .. its complete absence in chess is one of the reasons i love backgammon so much more .. for example, i could never hope to beat Bobby Fischer at chess, but i can beat dorbel every now and then  

pck, or gumpi, does that summary of 1100 matches include a luck analysis for each game in some kind of summary form .. if so, i would love to see "luck" by match in a frequency distribution .. for a PerfectBot, i suspect that distribution would tend toward a very traditional Normal Distribution bell-shaped curve .. understaniding the standard deviation for that curve would also be most interesting .. for individual players, i suspect it may be slightly skewed toward unlucky for all the best moves we may miss .. if there is any correlation, diane, between making "our own luck" such skewing would prove it .. however, "making" is a misnomer as i doubt that ANY human player or bot would have a curve skewed toward luck over a large number of rolls, although blitzxz seems to suggest otherwise .. finally, individual "luck" distributions may even reveal variances in outliers, standard deviations, max and min values, and more for those of us who take bigger chances, for those who play a much more conservative game, and perhaps may even be correlated to individual ratings

sounds like good thesis material to me  

if necessary, i'd be glad to help extract some of those details for individual matches and stick them in Excel .. hopefully, it won't be necessary .. otherwise, i'll b simulating luck in no time  

[pck]

pck, or gumpi, does that summary of 1100 matches include a luck analysis for each game in some kind of summary form .. if so, i would love to see "luck" by match in a frequency distribution .. for a PerfectBot, i suspect that distribution would tend toward a very traditional Normal Distribution bell-shaped curve ..

XG doesn't provide, as far as I know, stats on luck variance. I agree that would be indeed be an interesting thing to have.

understanding the standard deviation for that curve would also be most interesting .. for individual players, i suspect it may be slightly skewed toward unlucky for all the best moves we may miss .. if there is any correlation, diane, between making "our own luck" such skewing would prove it .. however, "making" is a misnomer as i doubt that ANY human player or bot would have a curve skewed toward luck over a large number of rolls, ...

If any correlation between luck and skill were found, that would point either to dice manipulation or to a conceptually defective definition of "luck", since if you were able to influence your luck, the results of that ability couldn't properly be called "luck" anymore, but would instead be the outcome of a part of your skill. Conceptually, luck is what we are given without deserving it, while skill is what we make of what we have so received. All formal definitions of either luck or skill must maintain conceptual coherence with this basic idea, otherwise we will create statistical mirages prone to misinterpretation.

[Zorba]

In other words, overall luck will approach zero for any player, in the long run.

You can't create luck. You can "create" equity though, by not making (as many and as big) errors as your opponent... Strictly speaking, it's more like you can destroy equity with bad moves, and the idea of backgammon is to let your opponent do that, and not do it yourself!

BTW, XG offers some graphs on the luck distribution per game, weighted game, match or weighted match.

Mine doesn't have quite enough data to be smooth yet, but does show a kind of bell-shaped curve, however, there's a big dip slightly left of the middle: games (and even matches) with close to zero luck or very little luck for my opponent are relatively rare! That's probably because when both sides had roughly equal amounts of luck, the game will get to the end with both players still around the 50% winning chances mark, but volatility will increase as the end of the game comes in sight, and at some point either you get very lucky winning, or very unlucky losing, simply because the game has to end somewhere.

Another thing XG shows in the graphs is how many of the games you won with a certain amount of luck. I won all games in which I got lucky. I also won all games with zero luck, and I won about half the games where my opponent had a very small amount of luck. I lost all the games where my opponent had a more sizeable amount of luck.

This is all very much in line with what you'd expect, as on average I guess most of my opponents are at least 150 rating points weaker.

[sixty_something]

If any correlation between luck and skill were found, that would point either to dice manipulation or to a conceptually defective definition of "luck", since if you were able to influence your luck, the results of that ability couldn't properly be called "luck" anymore, but would instead be the outcome of a part of your skill. Conceptually, luck is what we are given without deserving it, while skill is what we make of what we have so received. All formal definitions of either luck or skill must maintain conceptual coherence with this basic idea, otherwise we will create statistical mirages prone to misinterpretation.

pck, i am NOT suggesting that we can "make" ourselves more lucky .. remember, i said "making our own luck" is probably a misnomer .. however, i am not convinced that we error prone humans might demonstrate in a frequency distribution of "luck" some statistically significant variance with a theoretically PerfectBot .. that variance, if it exists, might be deemed a result of less skillful play suggesting that 'luck" and "skill" are not independent variables .. since both are dependent on "equity", itself a construct, i would suggest that this is not a "defective definition", but rather simply the nature of the way in which we have come to think about and define both "luck" and "skill"

all that notwithstanding, that last sentence of yours is an awesome one, pck, statistically poetic even .. in the best of all possible worlds, it should be true without doubt .. but as much as i have come to appreciate "equity" as a construct, i am doubtful we will ever eliminate "statistical mirages prone to misinterpretation" .. on the other hand, we certainly don't need to be creating mirages unnecessarily

finally, if "making our own luck" is a mirage better suited for motivating us to play more skillfully and psychologically accepting bots' seemingly endless supply of good luck, then perhaps "wasting our potential" is a way to describe how less skillful play may be reflected in "luck" calculations and therefore "luck" distributions .. of course, whether it is or is just another mirage has yet to be determined, for me anyway

Zorba's post came in while i was writing this which suggests to me an encouraging possibility of "seeing" the "luck" curve for Zorba and comparing it with that of his opponents, whom he identifies as being about 150 points on average below his rating .. so, let Zorba be my theoretical PerfectBot and his opponents mere mortals with less skill (my words not his)

is there any difference between Zorba's "luck" curve and that of his opponents? whether there is or not, we haven't proved much, but it might make an interesting side bet

[pck]

In other words, overall luck will approach zero for any player, in the long run.

To be more precise, that is the most probable outcome. All statistical convergence is probability-governed itself and hence does not occur with strict necessity.

BTW, XG offers some graphs on the luck distribution per game, weighted game, match or weighted match.

Very nice. I don't have XG, will have to ask gumpi if he can screenshot me those pages.

Incidentally, the Chi-squared distribution of my fibsdice in those 1100 matches sucked eggs. My 60.000 rolls deviated from what can be expected from fair dice so much that there is only a 3% chance of doing even worse. The same was not true for my opponents' dice. So I guess I'm vic with a license now.

[pck]

pck, i am NOT suggesting that we can "make" ourselves more lucky .. remember, i said "making our own luck" is probably a misnomer .. however, i am not convinced that we error prone humans might demonstrate in a frequency distribution of "luck" some statistically significant variance with a theoretically PerfectBot .. that variance, if it exists, might be deemed a result of less skillful play suggesting that 'luck" and "skill" are not independent variables

But that is exactly the conceptual error I was trying to point out. If luck and skill are mathematically defined in a way such that they turn out to be correlated, then these definitions have lost touch with our common understanding of the concepts. In our common understanding, luck is undeserved, and skill is not. We say that you earn victories by skillful play and that you luck out when it is felt that the dice won it for you. That is the conceptual basis, the given, which the math must reflect. Formalisms not sharing logical form with their underlying concepts will likely create the above mentioned mirages by producing numbers which do not really say what they seem to be saying.

.. since both are dependent on "equity", itself a construct, i would suggest that this is not a "defective definition", but rather simply the nature of the way in which we have come to think about and define both "luck" and "skill"

They are indeed both equity-related. And that is precisely the origin of the illusion that skill may "create" luck: Luck is equity bestowed on you by pure chance - the dice rolls. There is no "deserve" there. Skill on the other hand is equity not wasted by making the best possible moves with the rolls you get. You screw that up, it's your own fault. Good luck creates equity for you, good skill retains it. Related concepts, but yet distinct.

all that notwithstanding, that last sentence of yours is an awesome one, pck, statistically poetic even ..
in the best of all possible worlds, it should be true without doubt .. but as much as i have come to appreciate "equity" as a construct, i am doubtful we will ever eliminate "statistical mirages prone to misinterpretation" .. on the other hand, we certainly don't need to be creating mirages unnecessarily

Yes, the mirages are the pitfalls of statistics, often much more so than the complications of the underlying math (which can be considerable too of course). I do think that concerning luck and skill we can have our cake and eat it too though. We are less fortunate with other concepts, such as that of probability itself, which remains conceptually elusive due to its recursive nature.

finally, if "making our own luck" is a mirage better suited for motivating us to play more skillfully and psychologically accepting bots' seemingly endless supply of good luck, then perhaps "wasting our potential" is a way to describe how less skillful play may be reflected in "luck" calculations and therefore "luck" distributions .. of course, whether it is or is just another mirage has yet to be determined, for me anyway

See above. I hope to have made it clearer than in the first posting.

[diane]

I will read all this later when I am not having a quick lunch break    but some words to be put in are..

Refer to the title of this...I have never suggested we can have any influence over Lady Luck by anything other than human sacrifice or dice manipulation 

BUT what I am asking about is the 'luck as caluclated by a bot' - which so much store is put by...waaaaah I lose this because my 'luck score' is -208, or similar....

Since that calculation is based on numbers...the numbers regarding equity calculations must factor in...and if you can influence the equity of each move by how you move, then you can influence how lucky...or unlucky you appear to be...

I too win the games where I am lucky, and lose the ones where I am not - hence this interest...surely there has to be a way for a generally unlucky person like myself to get around this unmovable object   

I too, would be interested in this bell curve...I am a scientist - nothing appeals to me more than a good bell curve 

[pck]

I will read all this later when I am not having a quick lunch break    but some words to be put in are..
[...]
Since that calculation is based on numbers...the numbers regarding equity calculations must factor in...and if you can influence the equity of each move by how you move, then you can influence how lucky...or unlucky you appear to be...

See my reply to sixty in #30, paragraphs one and two. This is the same confusion: What you can influence by making the best move is the retention of equity post-roll. This we call the excercise of skill. The equity change from pre-roll to post-roll (but pre-move) is called luck, over which we have no control. We tend to confound and/or identify skill and luck because they are both about equity changes. The problem is conceptual, it is not "in the numbers".

I too, would be interested in this bell curve...I am a scientist - nothing appeals to me more than a good bell curve

Math is a fine subject but it needs conceptual elucidation to go along with it. Hence, yay philosophy.

[Zorba]

Here's my bell curve. Should be noted though, that I don't really have enough games played yet, and that XG considers me a bit lucky so far, which you can see in this graph.

Also, I think XG uses "luck per move" for this graph, whereas for the puposes of this discussion you'd probably rather want the "total luck" over all moves.

If someone played opponents that are rated significantly higher on average, then the dip in the bell curve will be slightly to the right of the middle, instead of on the left. Also, you can expect to lose most games with close to zero luck or only very slight luck for you, instead of winning most of them as in my graph.

[diane]

How do I get that program?

[pck]

Here's my bell curve. Should be noted though, that I don't really have enough games played yet, and that XG considers me a bit lucky so far, which you can see in this graph.

Also, I think XG uses "luck per move" for this graph, whereas for the puposes of this discussion you'd probably rather want the "total luck" over all moves.

Interesting graph. Is the luck on the x-axis your absolute luck (per move), or is it the difference in luck between you and your opponent? The latter would make a lot more sense to me.

[pck]

Mine doesn't have quite enough data to be smooth yet, but does show a kind of bell-shaped curve, however, there's a big dip slightly left of the middle: games (and even matches) with close to zero luck or very little luck for my opponent are relatively rare! That's probably because when both sides had roughly equal amounts of luck, the game will get to the end with both players still around the 50% winning chances mark, but volatility will increase as the end of the game comes in sight, and at some point either you get very lucky winning, or very unlucky losing, simply because the game has to end somewhere.

If I understand this correctly, you're describing matches which "move away from the centre of the bell curve" at the last moment. Hence the dip.
But why isn't this counterbalanced by matches in which one player is ahead in luck until close to the end and then loses due to a big equity swing, thus moving the match *to* the centre of the bell curve?

Are matches like that rarer than the type you describe above? Are last minute swings less likely after one player has luck-wise dominated the match, compared to matches with more balanced luck for both?

[Zorba]

XG uses (your luck minus opponent's luck) for luck, like Snowie does (and unlike GnuBG does).

As to your other question, most matches "need" a lot of luck to be decided, since the skill difference alone won't be enough. Using GWC, say 20% can be gained in one game by the skill difference between a good player and a beginner. Then the best player still needs 30% luck to win, or the worst player 70%.

If that amount of luck hasn't been reached and the game is nearing the end (no contact), then barring any super blunders in the endgame, a lot of luck has to happen for either player.

The opposite is not true, if one player already has been very lucky, he's likely to have really high winning chances, like 80% or more. Then either he also gets the remaining amount of luck to win the game, or the other player gets extreme luck in the end. An amount of luck that brings the total luck back to around zero like you describe, could not finish the match (unless there was an extreme skill difference), as it would bring both players back to some wide margin around 50% GWC (say, 30-70%).

[sixty_something]

XG uses (your luck minus opponent's luck) for luck, like Snowie does (and unlike GnuBG does).

so, the graph of your "luck" in a previous post which is skewed toward the right, indicates your "lucK" exceeds that of your opponent.. is that correct, Zorba?

while you suggest your sample size is not large enough, such a skewing for a large enough sample size would support the observation that the better player minimizes his error rate, and in so doing optimizes his "luck" potential, i.e. creating the statistical allusion that the better player has better luck or makes his own luck .. if over time, Zorba's skew remains postitiive or even increases, as i expect it might, this may provide at least one counter example to pck's theoretical arguement that over time the "luck" differential will tend toward zero reflecting equal luck in the long run regarless of skill level -- is that a fair synopsis of your thesis, pck

an interesting question is how big would the sample have to be to be statistically significant?

finally, it is important to note that as previously suggested "makes his own luck" is at best a misnor and at worst misleading .. my phrasing above is the best i can come up with that i believe accurately states my thesis and that suggested by others .. in summary it is:

the better player minimizes his error rate
in so doing he optimizes his 'luck' potential
yielding the impression the better player is luckier
than less skilled opponents

although, vic vs. the bots is proof enough in shouts

[Zorba]

No, no and NO!

Minimizing your error rate does not "optimize luck potential". It maximizes equity, with the luck given, as explained in many ways in several articles in this thread earlier.

And as far as there might be illusions of better luck, then XG won't show them, as it uses mathematics like it should, not subjective perceptions or inconsistent, confused mathematical models.

pck's theoretical argument is correct and is actually pretty trivial from the definition of luck, for a statistician.

I'd suggest just accepting that most people's "intuitive" or "perceptive" concept of probabilities or statistics as in "observing luck" are notoriously bad, skewed, inconsistent, etc. and therefore, you need to use mathematics to get correct answers to your questions.

As far as your last statements:

the better player minimizes his error rate
in so doing he optimizes his 'luck' potential
yielding the impression the better player is luckier
than less skilled opponents

are concerned, this is correct apart from the second sentence, which is unclear anyhow. What does "optimizing 'luck' potential" even mean? It sounds like a confused way of simply saying "maximizing equity".

[sixty_something]

it all seems clear to me

OK, when words fail, resort to numbers .. i will provide a few examples of what i have observed and what i am trying to say .. while these will be incomplete at best, perhaps they will be helpful in developing a better understanding .. if not of my thesis, of the way "luck" is calculated .. i may be completely wrong and if so will readily admit it, but i think i can illustrate what i am trying to say .. however, it will take a while to create them .. so, don't hold your breath

meanwhile, 

[pck]

pck's theoretical argument is correct and is actually pretty trivial from the definition of luck, for a statistician.

Yes, it's mathematically trivial because there is hardly any math involved! I was mainly concerned with giving an explanation of how we arrive at the confused notion that "skill begets luck". What I wrote in #30 is not a theoretical argument (for it can neither be verified nor falsified), but an attempt at a conceptual elucidation by looking at how "luck" and "skill" are used in everyday language and how these terms are related to their mathematical counterparts.

I'd suggest just accepting that most people's "intuitive" or "perceptive" concept of probabilities or statistics as in "observing luck" are notoriously bad, skewed, inconsistent, etc. and therefore, you need to use mathematics to get correct answers to your questions.

I'd like to add that by setting down a formal definition for, say, "luck", we define what counts as "correct". There is no a priori concept of "the correct definition of luck" for us to discover. But in order for (mathematical) answers to make sense, there must be conceptual harmony between them and the questions they are answers to. And while using math is good for quantification and precision, its use alone does not guarantee the aforementioned harmony.

[pck]

XG uses (your luck minus opponent's luck) for luck, like Snowie does (and unlike GnuBG does).

As to your other question, most matches "need" a lot of luck to be decided, since the skill difference alone won't be enough. Using GWC, say 20% can be gained in one game by the skill difference between a good player and a beginner. Then the best player still needs 30% luck to win, or the worst player 70%.

If that amount of luck hasn't been reached and the game is nearing the end (no contact), then barring any super blunders in the endgame, a lot of luck has to happen for either player.

The opposite is not true, if one player already has been very lucky, he's likely to have really high winning chances, like 80% or more. Then either he also gets the remaining amount of luck to win the game, or the other player gets extreme luck in the end. An amount of luck that brings the total luck back to around zero like you describe, could not finish the match (unless there was an extreme skill difference), as it would bring both players back to some wide margin around 50% GWC (say, 30-70%).

Ok, you're referring to the fact that the initial equity of 50% for each player needs to climb to 100% for one of the players in order for the match to be finished. That much is clear. I also get that a swing as I described it would not bring the match to the centre of the curve since it cannot finish the match.

The dip reflects the fact that matches which have zero or little overall luck (per move) are rare. Close-to-zero luck means that winning the match was mostly due to skill.

Hence the "dipped bell shape" of the curve is a misleading expression. It consists rather of two bell shapes which are superimposed, with their high points at those good/bad luck values which are most common.

[socksey]

What does "optimizing 'luck' potential" even mean? It sounds like a confused way of simply saying "maximizing equity".

Maybe it's an English thing!   

socksey



"The one thing I regret, is that I don't have more regrets." – Winston Churchill from his deathbed

[blitzxz]

What does "optimizing 'luck' potential" even mean?

I can tell what this means to me. You clear your mind and concentrate complitely to the flow of the game. Just before you or your opponent or computer rolls you clearly call the rolls you want in your head. And they start coming just the way you like them. I already showed how this works in the forum match 4.   I'm sure this must be the reason why I have positive luck in gnu analyzes.

It's also important to call only good rolls for yourself and not bad ones for opponent because this could have negative effect on your aura.

[stog]

funny how if we 'tune in' and concentrate, we play better

[pck]

So here's a puzzle:

I let gnu play one-pointers against itself with both players set to the same level of skill. The outcome of these matches should be completely luck dependent.

But the stats say otherwise: The "Luck adjusted result" was never 0%, but varied between 3 and 15% in my tests.

The Luck adjusted result is calculated as 50% - the luck difference of the players, which should be their skill difference, which should be zero.

Any ideas what's wrong with this picture?

[boomslang]

Two reasons why it isn't zero that come to mind:

1) The neural nets are not 'perfect'. With perfect nets you would expect that the equity of the current position (after your opponent moved and prior to your roll) would be equal to the expected equity after playing your 21 possible rolls correctly.  In practice this is not the case.  If the nets would be perfect there would be no need to set a bot to 2 plies.
I think this is the main reason why the luck adjusted result (luck in general, actually) should be handled with care.  Maybe if you re-analyse the match at higher plies (for analysis AND luck -- I believe it is command "set analysis luckanalysis plies 2") the luck adjusted result will be closer to zero?


2) If you analysed the match at a different level as the playing level of the 2 players it might be that both players did not make the same error total during that match. The difference in errors doesnt cancel out then.


I am sure there are more causes though!

[blitzxz]

So here's a puzzle:

I let gnu play one-pointers against itself with both players set to the same level of skill. The outcome of these matches should be completely luck dependent.

But the stats say otherwise: The "Luck adjusted result" was never 0%, but varied between 3 and 15% in my tests.

The Luck adjusted result is calculated as 50% - the luck difference of the players, which should be their skill difference, which should be zero.

Any ideas what's wrong with this picture?

Luck adjusted results are notoriously inaccurate in single games or matches. It's actually worthless number to me. I once tried this same test playing, analyzing and analyzing luck with same ply and I think my record was over +50% luck adjusted result to other identical gnubg. I tried analyzing the same match (and luck also) with higher ply but it really didn't help, still the other bot was hugely more skillful. I can't really understand what is the cause of this. However in my long stats (fibs matches), luck adjusted results and unnormalized error rates seemed to converge.

[pck]

Luck adjusted results are notoriously inaccurate in single games or matches. It's actually worthless number to me. I once tried this same test playing, analyzing and analyzing luck with same ply and I think my record was over +50% luck adjusted result to other identical gnubg. I tried analyzing the same match (and luck also) with higher ply but it really didn't help, still the other bot was hugely more skillful. I can't really understand what is the cause of this. However in my long stats (fibs matches), luck adjusted results and unnormalized error rates seemed to converge.

Analyzing with higher ply changed nothing for me either. Now it seems to me that this means that gnu's luck stats in general can be nothing more than a crude indication, since they are what the Luck adjusted result is calculated from. If this is true then vic has a lot of apologizing to do.

If the inaccuracies of gnu's luck calculation go both ways and are not systematically biased, they may of course even out in the long run as an effect of the process of averaging. This would indeed make the long stats more reliable than the single match-based ones.

[pck]

1) The neural nets are not 'perfect'. With perfect nets you would expect that the equity of the current position (after your opponent moved and prior to your roll) would be equal to the expected equity after playing your 21 possible rolls correctly.  In practice this is not the case.  If the nets would be perfect there would be no need to set a bot to 2 plies.

I agree. What I had assumed was that non-perfect equity calculations should at least be internally consistent, that is, the numbers should add up. Obviously that is not the case. About the reason I still wonder. I have attached a gnu 1-pointer sgf file. What happens in the first two rolls is this:

Player "You" rolls a 45, gnu says that's an equity gain of +1.587% MWC. Player "gnubg" of course loses this amount of equity, as well as another 3.551% on its first roll, a 63. So gnubg's equity after rolling that 63 should be 50 - 1.587 - 3.551 = 44.862. But what it actually says is 45.77, a discrepancy of 0.908% after only one roll on each side. This is too large to be attributed to rounding errors. And if player gnubg's equity is calculated independently from previous equities, then why does gnu not adjust the -3.551 accordingly? Perhaps that would have other undesirable consequences I'm not seeing.

2) If you analysed the match at a different level as the playing level of the 2 players it might be that both players did not make the same error total during that match. The difference in errors doesnt cancel out then.

Good point. From a 3-ply point of view two 2-ply bot players will usually break symmetry with regard to their skill levels in a single match. Their errors should even out across a large number of matches though. Nevertheless this is very important for the consideration of what we call "skill". If we can attribute the term "skill" only to the whole of a bot's algorithm, that is, to its behaviour in all possible match situations, and if it could further happen that algorithm A1 beats A2 (in the long run), A2 beats A3, and A3 beats A1, then that would eliminate the possibility of attribution of skill for these three bots. Skill would only be a partial order on the set of all possible bot-algorithms. To prove that this is or isn't possible should be difficult.

[sixty_something]

pck's theoretical argument is correct and is actually pretty trivial from the definition of luck, for a statistician.

Yes, it's mathematically trivial because there is hardly any math involved! I was mainly concerned with giving an explanation of how we arrive at the confused notion that "skill begets luck".

What I had assumed was that non-perfect equity calculations should at least be internally consistent, that is, the numbers should add up. Obviously that is not the case. About the reason I still wonder.

is it safe to say we can drop the word "trivial" from your and Zorba's somewhat dismissive rebuttals of my suggestions?

i think you may be observing some of the same things i did when i first looked at "luck" calculatioins in detail ... like you, i "still wonder" about the reasons why what seems conceptually obvious simply does not add up when put to the test of striking a balance sheet for "luck" calculations

when observed values don't match theory and assumed concepts, perhaps it is time to revisit both with an open mind .. i am

[blitzxz]

Player "You" rolls a 45, gnu says that's an equity gain of +1.587% MWC. Player "gnubg" of course loses this amount of equity, as well as another 3.551% on its first roll, a 63. So gnubg's equity after rolling that 63 should be 50 - 1.587 - 3.551 = 44.862. But what it actually says is 45.77, a discrepancy of 0.908% after only one roll on each side. This is too large to be attributed to rounding errors. And if player gnubg's equity is calculated independently from previous equities, then why does gnu not adjust the -3.551 accordingly? Perhaps that would have other undesirable consequences I'm not seeing.

If I understand you correctly you are just adding up equity loses. It doesn't work that way. If you lose 1% first roll and 2% second it doesn't mean that now you have 47% chances to win. I haven't figured it out yet but I'm guessing some sort of multiplying is needed to add up equity loses and to come up with over all chances.

[sixty_something]

i'm with blitzxz on this one .. i've found using equity changes alone tends to yield expected additive values within a reasonalbe roundoff error .. what do the corresponding equity numbers look like?

[pck]

If I understand you correctly you are just adding up equity loses. It doesn't work that way. If you lose 1% first roll and 2% second it doesn't mean that now you have 47% chances to win. I haven't figured it out yet but I'm guessing some sort of multiplying is needed to add up equity loses and to come up with over all chances.

I'm not doing that. First I deduct the equity gain of the first player's initial roll from the second player's initial 50% (one player's gain is the other one's loss). From the result I deduct the equity (another loss due to a bad initial roll for player #2) from the second player's first roll and compare the total with what gnu says about the total equity of player #2 after its first roll.

If you load the attached sgf into gnu you will see it. I can't see how any multiplying would be involved here.

[pck]

is it safe to say we can drop the word "trivial" from your and Zorba's somewhat dismissive rebuttals of my suggestions?

My "mathematically trivial" was not intended to be dismissive of your remarks. It was intended to mitigate Zorba's remark which sounded as if what I tried to argue in #30 was mainly concerned with technicalities. It wasn't. But I explained that right afterwards in #41.

i think you may be observing some of the same things i did when i first looked at "luck" calculatioins in detail ... like you, i "still wonder" about the reasons why what seems conceptually obvious simply does not add up when put to the test of striking a balance sheet for "luck" calculations
when observed values don't match theory and assumed concepts, perhaps it is time to revisit both with an open mind .. i am

Couldn't agree more!

[sixty_something]

My "mathematically trivial" was not intended to be dismissive of your remarks.

no problemo, amigo

i suspect we can all get thin skinned when staking out a new and still controverial opinion - i know i can .. further, it never ceases to amaze me how easy it is to misinterpret even the most carefully crafted remarks in posts like thiese .. in shouts at FIBS it is even worse .. there a silly off hand comment or well intentioned insult amongst friends can too often lead to all out shout war and hurt feelings
so it goes in this zany little world of FIBSoids

[Zorba]

Yes, it's mathematically trivial because there is hardly any math involved! I was mainly concerned with giving an explanation of how we arrive at the confused notion that "skill begets luck". What I wrote in #30 is not a theoretical argument (for it can neither be verified nor falsified), but an attempt at a conceptual elucidation by looking at how "luck" and "skill" are used in everyday language and how these terms are related to their mathematical counterparts.

I simply mean the theoretical (mathematical) argument that luck will tend towards zero in the long run, regardless of a player's skill. That can be verified quite easily using mathematics.

There is no a priori concept of "the correct definition of luck" for us to discover.

No, but this is not about philosophy. Once you've settled for a definition, like the bots have and which has been explained here and elsewhere, there are correct and false conclusions you can draw from it.

Most everything that's being discussed here is described very well in these articles by Douglas Zare:

http://www.bkgm.com/articles/Zare/AMeasureOfLuck.html
http://www.bkgm.com/articles/Zare/HedgingTowardSkill.html

[Zorba]

Hence the "dipped bell shape" of the curve is a misleading expression. It consists rather of two bell shapes which are superimposed, with their high points at those good/bad luck values which are most common.

I don't know if it that's true, it seems very hard to find out what kind of distribution of net luck per move really underlies the graph.

[Zorba]

So here's a puzzle:

I let gnu play one-pointers against itself with both players set to the same level of skill. The outcome of these matches should be completely luck dependent.

But the stats say otherwise: The "Luck adjusted result" was never 0%, but varied between 3 and 15% in my tests.

The Luck adjusted result is calculated as 50% - the luck difference of the players, which should be their skill difference, which should be zero.

Any ideas what's wrong with this picture?

Yes

Just because GnuBG 0-ply plays both sides, does not mean it plays both sides equally well in individual games. Only in the long run will they show equal skill levels. So the outcome of a particular match is not completely luck dependent: one side may have given up much more equity in errors than the other side.

Furthermore, GnuBG's luck evaluations, just like its error evaluations, are not perfect, so this is another factor contributing to inaccuracies, especially noticeable in the short run.

BTW, for practical purposes: GnuBG defaults to using 0-ply for its luck calculations. Luck calculations can be considered more difficult than normal evaluations, as 21 different dice rolls and their best plays have to be considered, so it's no surprise that 0-ply luck analysis can give rather inaccurate results generally. Use GnuBG's command line and the command boomslang mentioned to increase the ply level of GnuBG's luck analysis.

Another interesting thing to consider here is that a n-ply luck analysis is closer to a (n+1)-ply error analysis than to a n-ply error analysis, due to the 21 different rolls that have to be analyzed. This is also true for the time it takes to do such an analysis: using 2-ply for luck analysis is about as slow as doing a 3-ply error analysis.

[Zorba]

I agree. What I had assumed was that non-perfect equity calculations should at least be internally consistent, that is, the numbers should add up. Obviously that is not the case. About the reason I still wonder.

It's similar to the fact that 1-ply evaluations are not consistent with 0-ply evaluations. Neural nets aren't necessariliy consistent among plies, in practice, they never are on the whole.

Nevertheless this is very important for the consideration of what we call "skill". If we can attribute the term "skill" only to the whole of a bot's algorithm, that is, to its behaviour in all possible match situations, and if it could further happen that algorithm A1 beats A2 (in the long run), A2 beats A3, and A3 beats A1, then that would eliminate the possibility of attribution of skill for these three bots. Skill would only be a partial order on the set of all possible bot-algorithms. To prove that this is or isn't possible should be difficult.

As long as both players (bot or not) in a bg match are imperfect, this partial ordering can definitely exist and very likely does exist. F.i. against GnuBG 0-ply, some players have developed strategies that with a high probability will lead to positions where the bot starts playing very poorly, leading to almost certain wins from that point. These same players couldn't do that to most expert human players. Yet, these expert human players, when not aware of these special "bot-killer" strategies, might themselves lose against GnuBG 0-ply on average.

What we're aiming for with skill assessments is a measure of error against a perfect bot. That's why apart from 0-ply or 2-ply evaluations, many players use 3- or 4-ply for evaluations and better yet, rollouts, to try and remove as much bias as possible from the bots numbers.

What's so interesting about luck evaluations (compared to error evaluations) is that they are, by definition, unbiased, so they can't favour certain game type or strategies more than any other, in the long run. This is not true for error evaluations, which can be (and actually are) biased, as the bots play certain game types much better than others.

[Zorba]

is it safe to say we can drop the word "trivial" from your and Zorba's somewhat dismissive rebuttals of my suggestions?

i think you may be observing some of the same things i did when i first looked at "luck" calculatioins in detail ... like you, i "still wonder" about the reasons why what seems conceptually obvious simply does not add up when put to the test of striking a balance sheet for "luck" calculations

when observed values don't match theory and assumed concepts, perhaps it is time to revisit both with an open mind .. i am

Sorry for being dismissive, but I'm just trying to be clear on this and avoid confusion. Once you settle for the definition of luck as it has been described by some bg theorists (f.i. Zare in his articles) and the calculation of luck as it has been used in many backgammon programs in the past and the present, then some statements are just false, as they are mathematically incorrect.

"Overall luck tends to zero in the long run", regardless of the moves made by any player, is one such result from using this definition of luck. So I just don't think trying to argue this point makes any sense; it's a mathematical result.

What you can argue about, is how it comes that some people feel "something's wrong" with this approach. Maybe you don't agree with the above definition of luck. But in that case, perhaps you should try to come up with an alternative definition of luck. In the process, you might start to appreciate the current definition of luck 

I'd like to mention Zare's article once more, he describes it much better than I can:
http://www.bkgm.com/articles/Zare/AMeasureOfLuck.html

I'm all for an open mind, but I feel pretty strongly that it's no use to draw the conclusion from a very small sample (such as my earlier posted luck curve) that "observed values don't match theory". Actually, pck earlier claimed that in his larger sample, luck got very close to zero. If you do very long sessions with bots playing each other, as many people have done, luck will also tend towards zero.

[pck]

Just because GnuBG 0-ply plays both sides, does not mean it plays both sides equally well in individual games. Only in the long run will they show equal skill levels. So the outcome of a particular match is not completely luck dependent: one side may have given up much more equity in errors than the other side.

There is one case where gnu (or any bot) should play both sides equally well: If analysis and player plys are set to equal levels, gnu should not find that any player has made any mistake. And indeed it doesn't. Both ERs are always 0. Hence all equity change in the match should be attributed to luck and the luck adjusted result should always be zero.

However, the phenomenon blitzxz and I described persists when ply_analysis = ply_players.

Furthermore, GnuBG's luck evaluations, just like its error evaluations, are not perfect, so this is another factor contributing to inaccuracies, especially noticeable in the short run.

I assume ply_analysis = ply_players. Analyses may not be perfect, but the hard question is, why can gnu not make the numbers consistent (as opposed to correct) within any of its n-ply worlds, however flawed they may be in an absolute sense. Why is it not possible, with both ERs = 0 and skill out of the picture, to add up all equity changes produced by the dice rolls and get a total change of +50% for the winner and -50% for the loser?

This question is reinforced by the Zare articles you linked to:

Final  −  Initial  =  Net Luck  +  Net Skill    in    http://www.bkgm.com/articles/Zare/HedgingTowardSkill.html

(A formula like this, which is based on the concept that what is not skill is precisely luck, was actually the reason I posed my question in the first place.)

BTW, for practical purposes: GnuBG defaults to using 0-ply for its luck calculations. Luck calculations can be considered more difficult than normal evaluations, as 21 different dice rolls and their best plays have to be considered, so it's no surprise that 0-ply luck analysis can give rather inaccurate results generally. Use GnuBG's command line and the command boomslang mentioned to increase the ply level of GnuBG's luck analysis.

With different methods/ply-levels for luck and move evaluations, it is clear that discrepancies can occur (see below).

Another interesting thing to consider here is that a n-ply luck analysis is closer to a (n+1)-ply error analysis than to a n-ply error analysis, due to the 21 different rolls that have to be analyzed. This is also true for the time it takes to do such an analysis: using 2-ply for luck analysis is about as slow as doing a 3-ply error analysis.

I ran a few tests. First I set luckanalysis and everything else to 2-ply (which took much longer than 3-ply analysis). The discrepancies still showed up but seemed to be lower than before (around 3-6% as opposed to 7-15%). But I could run only very few test matches because evaluation took so long (Intel Core 2 Duo 7300, 2.6 GHz, gnu set to use both cores). Next I tried luckanalysis at 1-ply and everything else at 2-ply (your n -> n+1 suggestion). In my first trial match, the LAR discrepancy was almost zero. But my celebratory mood subsided when in further trials it went back to 5-7%, and in one particular 1-point match I got an LAR as big as 23%.

So in the experiments the problem keeps showing up.

The conceptual question is still open. Luck is that which isn't skill. So why have seperate, independent luck and skill analyses? Why not have only a skill analysis (= best move analysis) and report luck as whatever equity changes are not due to skill? (Or vice versa.) Then the numbers would be consistent. Using two different evaluation modes will break formulas such as Zare's and create confusion. If there are any conceptual reasons for doing this, we have not begun to touch them yet.

[pck]

I don't know if it that's true, it seems very hard to find out what kind of distribution of net luck per move really underlies the graph.

Look familiar?

[pck]

What we're aiming for with skill assessments is a measure of error against a perfect bot. That's why apart from 0-ply or 2-ply evaluations, many players use 3- or 4-ply for evaluations and better yet, rollouts, to try and remove as much bias as possible from the bots numbers.

What's so interesting about luck evaluations (compared to error evaluations) is that they are, by definition, unbiased, so they can't favour certain game type or strategies more than any other, in the long run. This is not true for error evaluations, which can be (and actually are) biased, as the bots play certain game types much better than others.

This is a (clear, thanks for that) description of what happens, but not a justification for it. I was asking for the latter. The luck / skill evaluations as you describe them will necessarily break formulas such as Zare's, which express fundamental conceptions we have about luck and skill. So the numbers generated by gnu will conceptually be at odds with the understanding of luck and skill we start out with.

Of course it may be worth to pay that price if there are other insights to be gained from proceeding like this. But the discrepancies which the above methods introduce do not give us a measure, or get us closer to, an absolute measure of correctness (of best move or luck evals) - or if they do, we have no way of knowing it (rollouts do not guarantee greater accuracy, although it is of course fair to assume they do in most cases).

Still, I'd rather keep it simpler and have consistent n-ply worlds in which luck and skill add up. It may just be a matter of taste.

[pck]

No, but this is not about philosophy. Once you've settled for a definition, like the bots have and which has been explained here and elsewhere, there are correct and false conclusions you can draw from it.

And how do we settle on a definition? Surely not any definition will do. We need conceptual analysis before we can put our thoughts into a formula.
So it is most definitley about philosophy (= conceptual clarification). (But not only about philosophy.)

I simply mean the theoretical (mathematical) argument that luck will tend towards zero in the long run, regardless of a player's skill. That can be verified quite easily using mathematics.

Your statement above is correct, but it cannot contribute to the understanding of the phenomena of luck and skill.

We would never accept a definition of luck which has luck not tending towards zero in the long run. For such a definition would violate our common, pre-math understanding of luck. The formal definition builds on that understanding, not the other way round. So it does not come as a surprise that mathematical luck tends to zero - it could not be otherwise, since we would change the definition if it did not.
The formal proof that it does tend to zero is a verification of the conceptual validity of the definition. If it failed, we'd srcap our definition and start anew. We would not say "I have proved the most peculiar thing about luck - it doesn't tend to zero in the long run".
The reason we would say that is conceptual in nature, we do not (and cannot) appeal to any further formalisms here.

Most everything that's being discussed here is described very well in these articles by Douglas Zare:

http://www.bkgm.com/articles/Zare/AMeasureOfLuck.html
http://www.bkgm.com/articles/Zare/HedgingTowardSkill.html

These are good articles, but the reason they make sense is not that they define something, but that what they define is in harmony with our common understanding of luck and skill. If their definitions weren't, we would have no use for them, or at least wouldn't call them definitions of "luck" and/or "skill".

[pck]

What you can argue about, is how it comes that some people feel "something's wrong" with this approach. Maybe you don't agree with the above definition of luck. But in that case, perhaps you should try to come up with an alternative definition of luck. In the process, you might start to appreciate the current definition of luck  

Zorba recommending conceptual investigation. I never thought I'd see the day. :: D--(

I'm all for an open mind, but I feel pretty strongly that it's no use to draw the conclusion from a very small sample (such as my earlier posted luck curve) that "observed values don't match theory". Actually, pck earlier claimed that in his larger sample, luck got very close to zero. If you do very long sessions with bots playing each other, as many people have done, luck will also tend towards zero.

I already remarked in a previous posting that luck/skill discrepancies may well cancel each other out over a lot of matches if their noise goes both ways (for which I see no reason why it shouldn't). It is quite possible to arrive at a result which matches the theory by means not justified by the theory.

[pck]

Another interesting thing to consider here is that a n-ply luck analysis is closer to a (n+1)-ply error analysis than to a n-ply error analysis, due to the 21 different rolls that have to be analyzed.

I experimented a little more and found a counterexample. The attached match was played and analysed at 2-ply. Its luck analysis ply-level is 1. The luck adjusted result is -23%. If you set luck analysis to 2-ply and re-analyse, the LAR becomes -13%. So there are matches where it gets better instead of worse when you switch from n,n+1 to n,n.

[sixty_something]

Maybe it's an English thing!   

socksey, are pck and Zorba speaking English enough for you?

English is my first and only language and i can barely keep up with reading them, much less replying .. seriously guys, nice work at elaborating on your positions .. i really haven't had time to read it all yet, but look forward to doing so

meanwhile, onward through the fog, err 92% relative humidity and heat, i am off for a morning walk

[stiefnu]

I got a wee bit lost amongst those calculations but this clip reminds me there's always someone even unluckier than me:  http://www.youtube.com/watch?v=zH67LCz9GTY#

[socksey]

Indeed!     I can relate to that, stiefnu!   

socksey



Coffee, chocolate, men..........some things are just better when they're rich! - unknown

[diane]

It is amazing how many times some people can tell that same story and think you are still interested 

[pck]

I got a wee bit lost amongst those calculations but this clip reminds me there's always someone even unluckier than me: 

rofl

I admit defeat. This is so much better than any technical discussion could ever be.

[boomslang]

I think if you look at the luck of bots (and good players) have during a match or game, then you will see that they are more often 'lucky' (meaning having a positive sum of MWC/equity) than 'unlucky' (meaning having a negative sum of MWC/equity).
[...]
I dont know if anyone can back this up with empirical data though. I might give it a go if I can find some spare time.

... and so I did: GNUbg Expert played 31 times a 64pt match against GNUbg Beginner. In total 42000 moves were made in 763 games. Two histograms of the luck (in eq.) for each roll of both players are plotted in the first graph. Clearly, these are not normally distributed. The huge peaks at 0 are, amongst others, rolls when the player was on the bar against a closed board and rolls when already in a lost/won position.

In the second graph are two kernel density plots of the two players' total EMG luck in each game. Both distributions look bell shaped but are also not normally distributed (Lilliefors test p < 0.0002).

The better player (blue) had a positive luck 57% of the games. This differs significantly from 50% (p = 0.0001). This means that if you consider a bot 'lucky' when it has a positive luck, it not just appears lucky more often, it actually is.  However, it was really unlucky quite a few times: the distribution is skewed to the left.

The difference in luck per roll (last graph) shows the same bimodal pattern as  Zorba's histogram. This is because for two players, X and O, the following relation holds:

  endresult(X) = 50% - error(X) + luck(X) + error(O) - luck(O),

in other words

  net luck diff = endresult(X) - 50% + error(X) - error(O).

A net luck of zero will require that the total error of the loser equals the total error of the winner plus 50% (I am talking GWC here). When a good player plays against a weaker player (150pts weaker on average as in Zorba's example, or maybe 500 pts lower as in my simulated example) then apparently the nature of backgammon makes this very unlikely. If the better players makes about 10% error during a game and the weaker player 40%, then the net luck will have two peaks at 100%-50%+10%-40%=20% (for won games) or -80% (for lost games). The 10% and 40% are just guesses though.

[boomslang]

whoops, 2nd graph should've been this one...

[diane]

I know how to do this 'simply'...a list of rolls...the same list...one game I will use them to play the best moves possible...then use them in the worst way possible...and see what the numbers tell me on analysis...

I realise the list of numbers wont all get played in one game...but it should be possible to align them quite well...

[dorbel]

The better player (blue) had a positive luck 57% of the games.

This figure coincides with my own shorter studies of matches played between me and gbots on fibs. I have broken the study down into games rather than matches. When you do this you can see that the "luckier" player ALWAYS wins the game. However the better player, which in my study is at least occasionally me, gets the positive luck more often. Of course in matches the better player will also tend to win more points when he wins the game (and fewer when he loses), because of his better cube use. Jokers appear to be equally distributed.

Many congratulations to boomslang for his elegant experiment, even though much of his mathematical language is incomprehensible to this reader! I have believed what he now shows to be true for a long time.

[pck]

Many congratulations to boomslang for his elegant experiment, even though much of his mathematical language is incomprehensible to this reader! I have believed what he now shows to be true for a long time.

What exactly have you believed? That skill creates better luck, or that the definition of luck as used by the bots is flawed?

boomslang was careful enough to include a crucial "if":

"This means that if you consider a bot 'lucky' when it has a positive luck..."

"'lucky'" here refers to our intuitive concept of luck, whereas the luck in "positive luck" refers to the mathematical concept embodied by gnu's luck calculations.

[pck]

First of all a major vote of thanks to boomslang for taking the time to conduct and evaluate this experiment. Very nice work indeed.

In the second graph are two kernel density plots of the two players' total EMG luck in each game. Both distributions look bell shaped but are also not normally distributed (Lilliefors test p < 0.0002).

The better player (blue) had a positive luck 57% of the games. This differs significantly from 50% (p = 0.0001). This means that if you consider a bot 'lucky' when it has a positive luck, it not just appears lucky more often, it actually is.  However, it was really unlucky quite a few times: the distribution is skewed to the left.

Thanks as well for being careful enough to include the crucial "if" here. As I argued in previous postings, we cannot possibly accept the definition of luck as it shows itself through these numbers (barring statistical flukes which I rule out because of the large number of rolls in the experiment). The densities in the second graph should be symmetrical around zero instead of showing the biases they do. As you say, these biases are responsible for the asymmetry in the 3rd graph, since in generating graph 3 from graph 2 the relation

 net luck diff = endresult(X) - 50% + error(X) - error(O).

is enforced.

As I said in #62, with independent and different methods and/or ply-levels for luck and skill evaluations, it is clear that discrepancies can occur which break the above equation ("within gnu", before your evaluation). The luck/skill data gnu puts out (= the data used to produce graph 2) will then be inconsistent.

Contrarily to what I had surmised, gnu's luck-evaluation flaws show systematic bias towards the better player. They don't go both ways so as to cancel out over time. A remarkable result indeed.

A net luck of zero will require that the total error of the loser equals the total error of the winner plus 50% (I am talking GWC here). When a good player plays against a weaker player (150pts weaker on average as in Zorba's example, or maybe 500 pts lower as in my simulated example) then apparently the nature of backgammon makes this very unlikely.

So what do you think? Have you shown with this that total luck does not tend to zero in the long run if the skill difference between the players is large? Or have you shown that gnu's calculation of luck is defective? You're obviously aware of the problem or you wouldn't have included that "if" above.

[vegasvic]

what does all this have to do with playing backgammon ?

I never get any luck .... so STFU all of you



hahahahaaaaaaa !!

[dorbel]

Yes, As I have said before, the bot definition of "luck" is both flawed and incomplete.

[sixty_something]

thanks, boomslang .. that is one nice piece of work

as they say, a picture is worth a thousand words or in this case 42,000 moves .. i tried to say it before as simply and completely as i could only to be summarily shot down .. it appears your graph and comments support my perception (certainly not unique) perhaps even establishing a proof of concept .. so, pardon me while i repeat myself

the better player minimizes his error rate
in so doing he optimizes his 'luck' potential
yielding the impression the better player is luckier
than less skilled opponents

in answer to an earlier question, yes, 'luck' potential is simply another way of saying equity change, but change on a subsequent turn .. when a bot optimizes for equity on the current move, part of that optimization must directly include the careful analysis of the opponents next possible rolls and moves as well as the bot's next turn where equity based "luck" is calculated .. this look ahead analysis seeks an optimal position in subsequent moves .. this appears to effectively optimize the equity based "luck" calculation for the better player .. i think that may be what we are seeing in boomslang's graphs

by definition, the less skilled player will make decisions that yield suboptimal equity gain, thus the "luck" calculation of the less skilled player will be impacted negatively or be suboptimal .. this impact is directly seen in equity changes from move to move .. since the "luck" calculation is a calculation of equity change based on the roll, the concept that optimal versus suboptimal play may impact "luck" calculation seems almost transparent to me .. obviously, it is neither transparent of trivial

most importantly, i am not saying the better player is luckier .. i am saying that the "luck" calculation is NOT a perfect reflection of pure luck which theoretically would be the same for both players regardless of skill .. however, since the "luck" calculation is an equity based calculation only approximating the impact of pure luck, it is biased toward equity not probability .. since the better player optimizes equity, is it any surprise we see results biased toward the better player?

all i am saying is "the nature of backgammon", as boomslang says, and the nature of the "luck" calculation yield "the impression the better player is luckier" .. boomslang's graph number 4 seems to directly support this .. indeed it appears to may be more than an impression, but it is a slippery slope as we have seen to attempt to equate pure theoretical luck with "luck" calculations .. i think all of us who play bots significantly more powerful than we are have repeated first hand experience that bots just seem too damn lucky - don't we? i would contend that this perception is merely a reflection f what boomslang's experiment has shown

now, boomslang, would you consider conducting another experiment to test another aspect of of this theory?

i believe that the more moves (or games) analyzed the more the calculated "luck" differential will diverge between a better player and a less skilled player probably up to some limit .. while 42,000 moves may be a large enough sample to have reached such a limit, it may not be .. so, if it isn't too time consuming, how would that last graph look after say 100,000 moves or more? any idea where that limit may be or if there is a limit at all?

finally, has anyone yet directly addressed diane's original question and really defined how the "luck" calculation really works? now that we seem to have established that it works differently than we expected, it seems a good time to revisit that original question

[boomslang]

This figure coincides with my own shorter studies of matches played between me and gbots on fibs. I have broken the study down into games rather than matches.

In my example I used games aswell (last two graphs).

However the better player, which in my study is at least occasionally me, gets the positive luck more often.

Maybe also another thing is going on..? Being (very) lucky throughout a whole game means lots of trivial moves and therefore small errors? I think for players up to intermediate level this is the case.

The densities in the second graph should be symmetrical around zero instead of showing the biases they do.

Why do you think that? I think they should have an average of zero (because luck tends to zero in the long run), but they cannot be symmetrical because the better player will be more likely to be really unlucky than to be really lucky. (In other words, it has virtually no chance to be really lucky because its opponent gives away too many equity and it had won the game/match already. Remember, these graphs are from the situation expert vs newbie.)

So what do you think? Have you shown with this that total luck does not tend to zero in the long run if the skill difference between the players is large? Or have you shown that gnu's calculation of luck is defective? You're obviously aware of the problem or you wouldn't have included that "if" above.

Not sure what you mean... What I meant to explain with this quote is the source of the 'dip' in the graphs of Zorba and me.
I included the 'if' because of Diane's remark about bots appearing lucky. When a bg player simply looks at whether or not a bot had a positive total luck and draws his conclusion on that -- and I think a lot of players do that -- he will indeed see a lucky bot more often than an unlucky bot. It is however unfair to conclude that bots are lucky in the long run, because of the negative skewness of the distribution of luck during a game or match for the bot. And yes, that means that I think the asymmetry of the distributions are not artefacts caused by GNUbg's inconsistency in evaluating luck and skill.

[boomslang]

now, boomslang, would you consider conducting another experiment to test another aspect of of this theory?

i believe that the more moves (or games) analyzed the more the calculated "luck" differential will diverge between a better player and a less skilled player probably up to some limit .. while 42,000 moves may be a large enough sample to have reached such a limit, it may not be .. so, if it isn't too time consuming, how would that last graph look after say 100,000 moves or more? any idea where that limit may be or if there is a limit at all?

finally, has anyone yet directly addressed diane's original question and really defined how the "luck" calculation really works?

I dont expect the results to be much different. The width and location of the dip (if any) is based upon the difference in playing strengths and not on the number of rolls. Maybe match length also plays a role.
I might generate some 58000 rolls more if I get bored (prolly after July 11th, the day Holland plays Germany).


Perhaps Inim can shed some light on GNUbg's internals regarding Diane's original question.

[diane]

I included the 'if' because of Diane's remark about bots appearing lucky.

Just to be clear, I have no personal 'feeling' that bots are lucky....I only used them as an example because [excluding blunderbots], they will play the best move in all circumstances and therefore have the best chance of appearing to have a high luck factor [as calculated by the bot] if there is anything in all this.

[pck]

Why do you think that? I think they should have an average of zero (because luck tends to zero in the long run), but they cannot be symmetrical because the better player will be more likely to be really unlucky than to be really lucky. (In other words, it has virtually no chance to be really lucky because its opponent gives away too many equity and it had won the game/match already. Remember, these graphs are from the situation expert vs newbie.)

You're right. I was confused when I wrote that, mixing in my thoughts about my own experiments where both players had equal skill. Average of zero it is indeed.

Not sure what you mean... What I meant to explain with this quote is the source of the 'dip' in the graphs of Zorba and me.
I included the 'if' because of Diane's remark about bots appearing lucky. When a bg player simply looks at whether or not a bot had a positive total luck and draws his conclusion on that -- and I think a lot of players do that -- he will indeed see a lucky bot more often than an unlucky bot. It is however unfair to conclude that bots are lucky in the long run, because of the negative skewness of the distribution of luck during a game or match for the bot.

I misunderstood your remark. You said "This means that if you consider a bot 'lucky' when it has a positive luck, it not just appears lucky more often, it actually is." I didn't give due notice to the crucial "often", effectively reading "luckier" instead. ("More often" here meaning "in more games", "positive luck" meaning "more luck than opp in a particular game" (= "positive luck difference for the bot" when we talk gnu).)

And yes, that means that I think the asymmetry of the distributions are not artefacts caused by GNUbg's inconsistency in evaluating luck and skill.

I can safely return to that assumption now too, with the above confusion cleared up for me. It makes a lot more sense this way. Thanks again for the clarifications.

[pck]

all i am saying is "the nature of backgammon", as boomslang says, and the nature of the "luck" calculation yield "the impression the better player is luckier" .. boomslang's graph number 4 seems to directly support this .. indeed it appears to may be more than an impression, but it is a slippery slope as we have seen to attempt to equate pure theoretical luck with "luck" calculations .. i think all of us who play bots significantly more powerful than we are have repeated first hand experience that bots just seem too damn lucky - don't we? i would contend that this perception is merely a reflection f what boomslang's experiment has shown

I agree. As boomslang points out, the difference between "being luckier" and "being lucky more often" is very important and a potential source of confusion:

To have been lucky more often does not necessarily mean that one has been luckier.

One may have been slightly luckier than one's opp for 17 matches and hugely less lucky in the next 3. One's total luck may then well be zero or close to zero, even though one was luckier far more often. As boomslang's data shows, this is indeed what is likely to happen when the skill difference between the players is large.

The phenomenon can also be explained without looking at empirical data. http://www.bkgm.com/articles/Zare/AMeasureOfLuck.html has this:

3. Strong players are lucky more often than weak players.

Strong players still have to be lucky to win, just less so. Because they don't need to be as lucky, their good luck is spread out to win many matches and their bad luck is concentrated in a few losses. Someone who bears in and bears off efficiently doesn't roll larger numbers than someone who aims for the ace point. He/she just uses the pips received more efficiently. The same thing happens with luck.

A strong player throwing away .2 less equity than I do will still have to be lucky by +.8 to beat me (in a match). Of course, this will happen 60% of the time.


In every match I start out with a 50% chance of winning. To win, I need to generate another 50% of equity to get to 100%. Let's assume my opp plays consistently worse than I do, that is, he throws away a lot more equity than I do by making bad moves. Therefore, if I win the match, much or most of the 50% equity I needed (in order to get from my initial 50% to the final 100%) will have been generated by that differential in skill. The rest will come from a moderate to small amount of luck (I have "virtually no chance to be really lucky" as boomslang notes in #82). In the matches I lose, the skill differential will still be there, so in those, my opp will necessarily have to have had a lot more luck than me. (*)

Now let's look at a long run of many matches played between me and this opp. I know from the previous paragraph that there will be small amounts of luck for me in matches I win, and lots of bad luck in matches I lose. So for the luck total to even out to zero, the number of matches which I win must be larger than the number of matches I lose. Hence I will have the better luck in more matches as a consequence of my better skill.

The same is of course to be expected to happen when a human player plays a better skilled bot.

The empirical data which boomslang presented is consistent with this, hence it cannot be inferred from his data that the bots' luck calculations are flawed.

(*) I may win a match despite the fact that my opp is luckier than me. But for that to happen the skill differential needs to be huge (> 50%), as can be calculated from the equation 50% + luck_my - luck_opp + skill_opp - skill_my = 100% which characterizes the matches I win. (skill_my/opp = my/opp's equity *waste*). So the above argument holds for players whose skill diff. is 50% or less.

[sixty_something]

today's Writers Almanac celebrates the birthday of historian David McCullough who practices the habit of painstaking research into primary sources .. for example, his biography of Harry Truman, took him 10 years to research and write. Truman (1993) won the Pulitzer Prize.

Writing is thinking. To write well is to think clearly. That's why it's so hard.

that quote made me think of this thread  

[sixty_something]

OK, Zorba, i know you have been busy and probably a wee bit hungover watching Le Grand Orange** make the finals, but your voice is keenly missing after boomslang's excellent experiment .. for me, i am not convinced i am even close to understanding the whole concept yet .. i have been busy as well with prepping to move at the end of the month and playing chess with my grandson .. i have not even begun your suggested reading assignment, much less read this entire thread (it got a little out of control last week)

further, i am also not sure whether my simple four line statement is any better understood or acceptable by anyone including myself .. it was the best i could do at the time and i haven't had time to persue it any further

so, if perchance you find time before or after the FIFA Finals, i'd sure like to read your take on these new developments whatever they may be - pro, con, or fed up

of course, other voices are welcome too .. it seems we have come close to wrapping up this topic for now .. i suspect it will raise its head again here or in another topic .. if we are close to some kind of conclusion, let's each try to wrap it up with some kind of summary .. IMHO, i think this thread could provide very useful fodder for someone to publish some serious work helping us all and hopefully others to better understand the concepts of pure luck and calculated "luck" in backgammon

** Le Grand Orange was actually a redheaded rookie sensation named Rusty Staub with the original Houston Colt .45's (now Astros) way back in 1963 when he was only 19 and i 16 going on 17 .. without "the Google" that was as close as i could get to a nickname for the Dutch team

[dorbel]

The contributions of boomslang in particular, as well as pck and Zorba, have contributed to our understanding of what bots mean by "luck" and why superior players get the "luck" more often than not. It is important to understand that what we mean by "luck", usually winning a game with a joker, is something different and something that on average will be equal in the long run. Other than that, I am not sure how understanding what is going on will help us to play better! A good play is still a good play, same as it ever was.
However, I don't recall ever seeing this research anywhere else and kudos to these guys for adding something new to the game right here on fibs. Applause, applause.

[diane]

However, I don't recall ever seeing this research anywhere else and kudos to these guys for adding something new to the game right here on fibs. Applause, applause.

Yes, I had no idea this simple question would lead to 5 pages of interesting stuff [5 pages of whining about luck, maybe - given that it is a fibs specialist subject  ]

[sixty_something]

something i'd especially like to see are definitions of key terms and phrases .. for example,

• LUCK - pure luck such as that we see when rolling fair dice which we all agree is theoretically equal regardless of skill
• "luck" - the calculated expression of equity gain or loss for an individual player resulting directly from the roll of the dice in a given situation

are those definitions complete and acceptable?

can we concurr LUCK and "luck" are two independent entities with different properties?

can we further state that boomslang's work provides convincing evidence that in the long run a significantly better player tends to accumulate more "luck" than the less skilled player?

finally, has anyone confirmed exactly how "luck" is calculated?

the best explanation i recall of the "luck" calculation is the difference between the optimal equity for a given roll and the weighted average of the optimal equity for all possible rolls

are there other terms and phrases we need to define for the purpose of better understanding and communication?

[sixty_something]

in keeping with the definitions i suggested, i modified the quote below for improved and consistent readability (i hope)

It is important to understand that what we mean by "luck" LUCK, usually winning a game with a joker, is something different [than calculated "luck"] and something that on average will be equal in the long run. Other than that, I am not sure how understanding what is going on will help us to play better! A good play is still a good play, same as it ever was.

i agree completely, dorbel, that such understanding will not yield better play directly .. however, i do believe it leads to improved understanding of analysis tools such as GNUbg and Snowie and toward better understanding equity changes .. indirectly that may yield better analysis to us as individual human players over the board .. i would not dare suggest exactly how

The contributions of boomslang in particular, as well as pck and Zorba, have contributed to our understanding of what bots mean by "luck" and why superior players get the "luck" more often than not.
...
However, I don't recall ever seeing this research anywhere else and kudos to these guys for adding something new to the game right here on fibs. Applause, applause.

i agree wholeheartedly .. while some of what has been written may now need to be re-read and re-interpreted, the overall level of understanding of "luck" calculations, equity changes, the mathematics of LUCK, and analysis engines has benefited us all as readers

while i have contributed very little to the technical aspects of this thread, i am proud to have participated in driving the argument forward in the face of withering crossfire from Zorba and pck .. frankly, i am still watching my back and open to reconsidering my simply stated opinion which initiated much of the dialog with Zorba and pck leading to boomslang's excellent experiment

your observation, dorbel, that you have not seen this research presented before is further evidence (from a reliable source, IMO) that diane's simple little question could lead to a serious paper that could be presented and/or published in many forums including academic, computing, and gaming

finally, kudos also to stog and webrunner as well for making FIBSboard available as a community resource for we FIBSters where stuff like this can happen -- life in FIBSland is better for it

[pck]

something i'd especially like to see are definitions of key terms and phrases .. for example,

• LUCK - pure luck such as that we see when rolling fair dice which we all agree is theoretically equal regardless of skill
• "luck" - the calculated expression of equity gain or loss for an individual player resulting directly from the roll of the dice in a given situation

are those definitions complete and acceptable?

can we concurr LUCK and "luck" are two independent entities with different properties?

I'd like to understand what LUCK is supposed to be or refer to. What exactly is it that we're seeing when LUCK occurs? With LUCK, are you referring to the idea of randomness, while "luck" quantifies my chances of winning a bg match? Is LUCK a non-numerical concept while "luck" is numerical?

can we further state that boomslang's work provides convincing evidence that in the long run a significantly better player tends to accumulate more "luck" than the less skilled player?

I don't think we can. As boomslang says in #82 (last paragraph) and I tried to explain in more detail and without resorting to empirical data in #86, it is not more luck that the more skillful player gets, but instead he will be the luckier player in more games/matches.

[Zorba]

boomslang's experiment is very nice indeed! It confirms the findings of XG about my matches, but with much more data and a bigger skilll difference it shows it much better. The skew in the luck graphs for GnuBG beginner and GnuBG expert and how they compare is very interesting. I agree with boomslang's conclusions and replies, and with pck's later conclusions which are similar. It's all in agreement with theory as far as I can see, and overall luck tends to zero regardless of skill.

The definition of luck and the usage of the word keeps causing trouble, it seems. This is how the bot determines the luck value for a single roll (rolls have luck attached to them, not moves!):

1. For the position before the actual roll, go through all 21 dice permutations. For each of the rolls, analyze for the best move and use the (0-ply) equity of that. Sum them all up (weigh non-doubles twice) and divide by 36. Now you have the average equity over all the next rolls, and the (1-ply) equity of the position before the roll.

2. For the current roll, analyze for best move (already done above, actually), use the (0-ply) equity. Subtract the value found in [1]. Now you have the luck value for the current roll.

If you use the above process to determine all the luck values for all 21 possible rolls, you'll see that the various luck values cancel out over all rolls. This is a logical result from the process used!

In other words, for a certain position, all the luck values for the various rolls add up (weighted) to zero, regardless of inaccuracies in the bot's evaluations. Because of this, luck values over various positions and rolls as in a game or match, will in the long run also tend to zero, regardless of any errors by the players (the luck calculations don't even consider what moves are played; they just take the resulting position from whatever move and go from there to determine luck for the next roll).

Here's another article on the subject (this site is a real goldmine BTW): http://www.bkgm.com/rgb/rgb.cgi?view+869

It's interesting to note that these luck calculations are also used for variance reduction in rollouts. Instead of using the actual result from a simulated game (trial) in a rollout, the bot keeps track of the net luck per trial and adjusts the end result according to that. This greatly lowers the variance (standard error), and the better a bot's luck evaluations are, the more it will be reduced.

BTW, a bot programmer might wonder, why not just ask the bot directly for an equity of a position before the roll in step 1 above? In GnuBG speak: why not use the 0-ply equity of the position, then roll, use 0-ply again and subtract the previous equity? The reason is that an imperfect bot is not consistent between plies, and by rolling the dice, you move up one ply. You'd be comparing the equity of the actual roll on a 1-ply higher level than you had determined the equity of the position before the roll.

As a result, luck can be biased using this method, won't always add up to zero anymore over all 21 rolls, and not necesssarily tend to zero in the long run anymore. This is not what we want, hence the more time-consuming "1-ply" method described above.

[jools]

I know this is really old but I stumbled onto something interesting...

Yesterday I was feeling hard-done-by at my terrible luck, so I logged out and went for a bike ride.  I got about 3km before a car driver took me out. Now my left leg, right arm and, strangely, both thumbs are pretty banged up - so I'm recuperating and reading old threads on backgammon.

I have learned three things...

1. I don't need thumbs to play backgammon, although dressing and feeding myself are pretty tricky.
2. Suddenly, I'm pretty happy with my dice.
3. I have a much better grasp of what bad luck really is.

[DianeJames]

Thank you for sharing the information i have been looking for this information quite a long time ago.

[MultanTVHD]

i read all discussion of this topic and this topic clear my all issues Thank you.