FIBS rating and BOTS

[ako]

Hi there all,

I'm new to this board but not to FIBS. I've been playing regularly for a few years now. My standard has improved since the beginning but only to a stage where I could still be described as a 'Casual Player'; sickly term that it is. I'd love to improve and am looking for guidance but that's not the point of this post.

Recently, as my rating has stabilised at around 1675 or thereabouts, I have started playing the Bots quite a lot; Gammonbot (versions 1-whatever) along with fibzilla and fibzombie. Initially I was just curious as to how I did and also happy to play guaranteed non-droppers and also enjoy fast games as well as witnessing supposedly near-perfect play. Sometimes I would win to my smug satisfaction but would mostly lose - and quite often fairly horrifically, which is to be expected. But then after a while I realised that my rating was falling rapidly and so I continued the exercise as a little experiment. I found that regularly playing Bots would decrease my rating to around 1600 or so and possibly lower. I would then start playing humans and my rating would gradually climb back to the 1675 level. I've repeated the exercise several times to prove the point to myself and the same thing happens every time (I've never let the Bots take me below the 1600 mark but maybe I should to help with the experiment).

Now, as I understand the FIBS rating system it should produce the same rating (evened out over many games) regardless of whether one is playing above your rating or below; you get more points for beating higher rated players but win less often and less points for playing lower rated players but win more often. And losing has inverse consequences.

So something is happening here and after wracking my brains I've come up with these five alternatives:

1. I've misunderstood the FIBS rating system
2. The Bots have more luck than human players (that old biscuit)
3. The rating system is flawed
4. I subconsciously play worse against the Bots (I'm sure I don't)
5. Or, the Bots are incorrectly rated due to droppers.

Does anyone have any thoughts on this?

cheers

Steve

[maria]

Now, as I understand the FIBS rating system it should produce the same rating (evened out over many games) regardless of whether one is playing above your rating or below; you get more points for beating higher rated players but win less often and less points for playing lower rated players but win more often. And losing has inverse consequences.


Does anyone have any thoughts on this?

cheers

Steve

Hi Steve.
Next time you are on FIBS, in the command line type:  help formula

You might also be interested in getting this FIBS Javascript Rating Calculator.

Hope this was of some help.

http://home.nordnet.fr/~fhochede/ratings.shtml

[ah_clem]

5. Or, the Bots are incorrectly rated due to droppers.

Bingo!  We have a winner!

People play the bots and have no compunction about dropping when they're losing.  Thus, the bots are rated lower than they should be.

If you want to play a bot, download gnubg and play off-line.  It won't affect your rating, you can drop when the match is hopelessly lost without increasing you're saved games count, and you can play in tutor mode if you desire.  Playing bots is good for practice, but there's not much reason to play them on FIBS.  Unless you want to, of course.

[diane]

Playing bots is good for practice, but there's not much reason to play them on FIBS.  Unless you want to, of course.

I play them on fibs when I want to be logged in for the chat etc, but want a fast game with a player who wont whine, criticise or drop me 

[dorbel]

I don't think that "bots are rated much lower than they should be because they get dropped a lot" is either true or the whole answer. Over time, the effect of getting dropped is eroded. If for example your rating is 1900 and you get dropped in a match where you should have won 5 points, then the next time that you play you will win slightly more (or lose slightly less) than you would have done if your rating had gone up to 1905. Of course they will never quite make up the lost points, but over the lifespan of a bot they will get close enough to their "correct" rating for the difference to be statistically insignificant.
The observation that Ako's rating falls when he plays bots but climbs when he plays humans may be coincidental, but it may also be that he really does play worse against bots. Bots play very fast and there is always a tendency to play at the pace of your opponent. Try slowing down to snail pace against the bot and your results should improve. Another useful tip is that bots rarely lose their market, so you can usually take their cubes unless they have just rolled a large joker!
Finally, all ratings, bots included, vary over a range of 200 points, sometimes more, so a drop of 60 points or so is not significant. However you can take advantage of this by only playing bots when they are at the top of their range. Bonehead, for example, is 2116 as I write, but it is sometimes as low as 1930. Obviously best to catch it today.

[vikingblood80]

So something is happening here and after wracking my brains I've come up with these five alternatives:

1. I've misunderstood the FIBS rating system
2. The Bots have more luck than human players (that old biscuit)
3. The rating system is flawed
4. I subconsciously play worse against the Bots (I'm sure I don't)
5. Or, the Bots are incorrectly rated due to droppers.

Does anyone have any thoughts on this?

cheers

Steve

6. many players you are playing might be "overrated"

some argument against this may be is that you were able to reproduce your results several times..anyway nice experiment 

[Tom]

Maybe we're overlooking the obvious?

You say when you play bots your rating goes down, and playing humans it goes up.

Well it also happens that when you lose your rating goes down and when you win your rating goes up.

You do usually lose when playing bots, I take it you also mostly win when playing humans.

As far as bots being more lucky, I would think it is more that higher rated players have better luck because they position their checkers to take the maximum advantage of more rolls. It has nothing to do with being a bot.

Tom

[pck]

...
Finally, all ratings, bots included, vary over a range of 200 points, sometimes more, so a drop of 60 points or so is not significant. However you can take advantage of this by only playing bots when they are at the top of their range. Bonehead, for example, is 2116 as I write, but it is sometimes as low as 1930. Obviously best to catch it today.

Please correct me if I misinterpret your statement, but this sounds to me suspiciously like one of the standard mistakes one is prone to make concerning probabilities - "I haven't rolled 6 in ages, so it's got to come up any second now". But, as they say, dice have no memory. Likewise for luck in general. A long streak of luck on the bot's side does not make its getting bad or worse luck in the next match more probable.

pck

[garp_02]

Call me stupid, but isn't it simply a case of the bots being better and you lose rating points. When you play humans (would like to know at what rating, relative to your own, they are) you do better?

[dorbel]

yes pck, obviously bonehead's standard of play and luck won't vary with its rating, but if you play it at the top of its range, you will win slightly more and lose slightly fewer points than if you play it at the bottom. It's not a big difference, but you may as well have it as not.

[pck]

Just as I thought, I misunderstood your statement. Thank you for clarifying.

pck

[inim]

The main reason why the bots are chronically underrated is because they play low experience players. Boomslang did evaluate the strength of 2ply and 0ply play some years ago, see here: http://www.mail-archive.com/bug-gnubg@gnu.org/msg03852.html

The approach he did was to whitelist the bots, so only experienced and fairly rated players could play them. In the real FIBS world, many underrated players use the bot to "punp" their accounts quickly to some fair rating for them. This costs the bot. If I am really a 1750 player and create a new 1500 account and pump it playing bots, the 250 points i gain in short term are taken from the bots.

So we have a situation where the bots are always underrated. This doesn't mean they can't be 2200 some time, as the natural random fluctuation in their interval of course happens just as well. But their median is underrated, and playing low experience players has the most impact here I think. Should dwarf the impact of e.g. bot droppers by far.

[ah_clem]

... many underrated players use the bot to "punp" their accounts quickly to some fair rating for them. This costs the bot. If I am really a 1750 player and create a new 1500 account and pump it playing bots, the 250 points i gain in short term are taken from the bots.

Are there really that many 1750 players who create new accounts?  Seems to me that it's more common for a new player to be over rated, i.e. they start out at 1500 when they're playing at the 1300 level. I would think that these players would at least balance out the "pumpers" if not outnumber them.

[sorrytigger]

many underrated players use the bot to "punp" their accounts quickly to some fair rating for them. This costs the bot. If I am really a 1750 player and create a new 1500 account and pump it playing bots, the 250 points i gain in short term are taken from the bots.

Without doing much research now or performing any experiments: I think that the individual experience is taken when determining the loss. For the bot there is no factor 5 just because his opponent has 0 experience. For the bot K=1. Otherwise he should profit from this rule as well and in the long run gain more rating, as he should win more often against any lower rated players.

Correct me if I'm wrong.

Andreas

[blitzxz]

I don't either think that droppers can lower bot ratings hugely. Most bots have about 50 dropped games or so at any given time and they play thousands games daily which should be enough to gain back the lost points in a way that dorbel stated.

However, I think that the rating system is seriously flawed when there is huge/big difference in ratings. 1675 rated player should win 28 % of 5-point matches against 2050 rated bot. In one-pointers underdog should win 39 %. Both these numbers are clear overestimates for underdog in my experience. In the long run, these kind of errors in ratings should correct themselves, but if the most of the players are playing generally only against players whose rating is close to their own, this can't ever happen.

It might be interesting to make some sort of simulation where a) players play randomly against each other and b) players play only against opponents rated close to them. So my theory is that in b case, which is fibs, the spread of ratings should be lot smaller and low end players should be wrongly rated against high end. I'm not actually at all sure is this correct. I might be complitely wrong here.

Here's some result with bot that played only agains low/high rated players. http://www.bkgm.com/rgb/rgb.cgi?view+524 To me this proves that at least in 1998 low rated players were overrated compared to high rated.