Roll-outs: how to use them

[sixty_something]

first, let me make it clear i'm looking for the answer not suggesting i know it

our new problem series has introduced me anew to the subject of roll-outs in GNUbg .. when i first tried using them, i was underwhelmed at the repeated similarity of roll-outs and the original GNUbg evaluation .. thus, i abandoned using them completely .. like other aspects of GNUbg and the game in general, i've discovered i just didn't understand what was really going on

i kept hearing players i respect talk about them from time to time, so i'd try another roll-out and still get similar results

now, in the problem series we've had at least three examples where roll-outs suggest a different solution than the GNUbg evaluation

so, i'm looking for answers on how to use roll-outs .. anyone have any? .. i'll add more as i run across clues and examples


All I really need to know about how to live and what to do and how to be I learned in kindergarten. Remember the Dick-and-Jane books and the first word you learned--the biggest word of all--look.
-- Robert Fulghum US author & Unitarian clergyman (1937 - )

[don]

The main thing about rollouts is that, to be really useful, you need to go to system settings and make sure your analysis mode is set high enough to make them more effective than, say, a 2-ply or whatever analysis (supremo, grandmaster, worldclass, ....).  The higher these settings, the better the rollout decision and the longer it will take.

I'd like an opinion on the best possible settings, if there's a user-defined setting that is best.

That said, obviously you should select the minimum number of move options to analyze, for speed.

--
don

[rubreg]

Is there really a 'best' setting? Isn't it all down to accuracy vs. reasonable calculation time?

Anyway, what do I know, being new to gnubg and all... Just wanted to mention this link here which has helped me quite a bit.

[Hardy_whv]

Evaluations are often wrong by large emouts of equity. In complex situations I'd never trust an evaluation, unless the answer is relatively obvious, even for my eyes. How reliable evaluations are depends on how familiar gnubg is with this type of situation. gnubg's neural net was trained in many, many sessions playing against himself (or ist it herself?). Situations that rarely happend during this training are therefore less well understood by gnubg.

Just a nice example: Try to play against gnubg with this starting position:



I never lost against gnubg yet with this starting position. If you later analyse gnubg's moves and cube decisions thoroughly, you will se how gnubg blundered his way through the match.

Therfore: The more uncommon a position is to gnubg, the more likely gnubg will not find the correct answer by simple evaluations. Use a rollout!

And yes, I support rubreg's statement: There is no best setting, it's a question of CPU time and the importance of a trustful result. I usually start with a few trials (324, 648), look how significant the difference between the decisions is and prolong the rollout (for the best candidates) until I get a significant output. For Nack Ballards and Paul Weavers Rollout Project I conducted up to 31,104 trials for a single decision. Such a rollout takes one to two weeks 

My preferred (and most trusted) settings are:

• no truncation!
• don't stop depending on STD
• don't stop depending on j.s.d
• I'd recommend cubeful rollouts
• use variance reduction
• use quasi random dice
• strength of the cube decisons: always "world class" (=2-ply)

For checker play I rarely use "expert" (=0-ply) for "quick and dirty" rollouts, when the position is easy to understand for gnubg, "world class" for all other rollouts.

I choose the amount of trials depending on the required precision, often starting with only 324 or 648, later 1296. If the decision is still very close, I usuallsy accept, that the difference is too small to say. (For opening moves I sometime do more trials, 2592+).

Hope that helped 

Hardy 

[socksey]

For opening and second rolls, I have found this helpful:

http://www.bkgm.com/rgb/rgb.cgi?view+650

socksey



"I hate war as only a soldier who has lived it can, only as one who has seen its brutality, its futility, its stupidity." â€" Dwight D. Eisenhower

[blitzxz]

I would like to rollout some opening moves and try to figure out average number of moves in a game. How can I do this? Does anyone know?

[spielberg]

I would like to rollout some opening moves and try to figure out average number of moves in a game. How can I do this? Does anyone know?

You've misunderstood the term "rollouts" as it's being used here. In this case it describes a method of getting a different type of anlaysis in GnuBG, a truly excellent piece of software available for free here

To be honest I can't really see a use for the average number of moves in a game so haven't bothered to think about it.

[blitzxz]

To be honest I can't really see a use for the average number of moves in a game so haven't bothered to think about it.

Well, I'm trying to play openings that lead to longer games to give more chances for mistakes. Slotting and double split for example lead longer games then typical gnu openings. But I really like to some how find out the exact average number of moves in different openings. And average cube would also be nice to know. Does some other bg-program have these features?

[Hardy_whv]

Well, I'm trying to play openings that lead to longer games to give more chances for mistakes. Slotting and double split for example lead longer games then typical gnu openings. But I really like to some how find out the exact average number of moves in different openings. And average cube would also be nice to know. Does some other bg-program have these features?

The idea is a good one (and actually not new). Giving information which play leads to longer matches would be effective if you want to improve your winning chances against players that are weaker than yourself. You might be willing to sacrifice some little amount of equity just to manoeuvre the game into a longer session.

Unfortunately, this feature is presently noto supported by any bot.

Hardy 

[blitzxz]

Unfortunately, this feature is presently noto supported by any bot.

It turns out that you can actually work some data out of game and match lenght. Gnu has this feature "relational database" which tracks also the number of moves. (Although it is now in big change and doesn't work well and I've never managed to use it properly.) You can count your avarage number of moves (you can alternatively keep track of your match lenght yourself) and compare it to other players avarage number of moves. That data can be found from the Big Brother database (which tracked fibsters some years ago). Number of rolls info is in here: http://www.bkgm.com/rgb/rgb.cgi?view+523 .

I have tried this lately and with my opening strategy (slot or double split) I'm able to play a lot more moves. For example avarage for 5-pointer in Big Brother database is only 128 rolls but my avarage is been lately over 50 moves/rolls more. So about 40 % increase in moves. In 7-pointers the BB-avarage is only 173 but I haven't even played a match under 200 moves/rolls after I started tracking them. (I should of course compare match lenghts to my own games with different strategies. But I should play hundreds of matches to achieve any reliable results and that just takes too much time. This is enough for me.)

This increase in moves might not mean that the players need same amout more skill. I have feeling (not sure) that especially double split games tend to be also easier.