Saved games anomoly

[jonesyjt]

RepBot has me at 8 saved games, which is accurate.  However, when I am invited to play, it shows I have 27 saved games in the little 'Invite' window.  This may turn people off from playing me.  Can anybody help?

[jonesyjt]

After a few days, it seems to be tryng to correct itself.  It now shows 13 when I really have 6.   Oh well.....

[shanemac]

As I understand it, this is what is behind what you are seeing.

(1) 3DFibs list of saved games
The first time that you log on to FIBS each day, 3DFibs downloads a file called list.html from the FIBS server with details of all saved games on the system at that time.
(Even if you interrupt this process, and prevent the file being updated, it only affects the information that your copy of 3DFibs uses.  Other people will still have access to uptodate information about you.)

(2) Invitations
When you invite someone, or when they invite you, 3DFibs consults the list.html file on your computer and tells you how many saved games they have.
N.B. The number of saved games you see relates to them and not to you.

(3) Repbot info
(from http://repbot.fibsboard.com/)
"RepBot gathers its data from http://www.fibs.com/savedgames which includes games completed in the past 24 hours. RepBot actually intersects yesterday's data and the data from the day before, this is more accurate but still miscounts some matches either saved recently or resumed in the past 48 hours."

So, if everything is working correctly, both 3DFibs and Repbot have statistics that are  accurate enough to get a reasonable idea of a player's record.

:eureka: The Quest for total accuracy
The only up-to-the-minute list of saved games is on the FIBS server itself.
Ctrl+s will show you a list of your saved games at any time.

[skater]

No...there's a bug.

If you look at todays list.htm it contains (15) matches for me (skater):

skater ChrisW Flara Gateway ParlorBot UMCthx arkantos bonehead darkman daveyp donzbot dorbel midge parazitii phukysucky rcschalck

Of these only Flara Gateway arkantos darkman parazitii rcschalck (6) are truly saved games (show saved)

The rest of the list is every match I played and completed yesterday (9) matches.  None of these players have a saved game for me in their list.  For example: daveyp MarStem Sharon does not list me.

So when you invite me FIBS shows that I have 15 saved games and gives a cautionary frown  .  RepBot list me as having 9 saved games and my reputation is BAD.

[Patti]

skater, if you look at the descriptive text on the savedgames list (which is available at http://fibs.com/savedgames) you'll see that recently-completed matches are included in the list.  It's not a bug, but rather a misunderstanding of the data that are being displayed.

Most likely, 3DFibs isn't also passing along that information.  If so, then it is in violation of my policies.

[skater]

Hi Patti

So I probably wasn't clear...I wasn't necessarily pointing the finger at fibs per se because "show saved" gives the correct count.   But there's definitely a bug (and perhaps two) from the perspective of end users of the clients, 3dfibs and repbot.  So let me understand what 3dfibs should be doing?  They should look at the saved games list and try to separate which were completed games that day.  But how do they do this?  It looks to me that sometimes there's a reciprocal pointer in the opponents list and sometimes not.  For example, I played ChrisW and daveyp yesterday.  I'm listed in ChrisW's list but not in daveyp's. 

I have no idea how repbot came up with a count of 9.

Thanks.

[Patti]

In my opinion, what 3DFibs should be doing is passing along the savedgames number unaltered.  It should also be passing along the definition of the number and a link to the savedgames page as the source material.   I've had no end of confusion due to programs that parse the savedgames list and pass along information but not the definition of the data or a link to the source.  Most people just assume that the savedgames number is the number of dropped games the user has, and that's patently false.  (I threatened to pull the list a couple of months ago when someone was threatening to produce a client that used the numbers to set a threshold and then automatically gag and blind a user with too many games.  That's blatant misuse of the data.)

There is insufficient information in the savedgames list to know which games are completed vs. in-progress vs. saved.

The underlying mechanism is this:  FIBS keeps completed matches around for some period of time-- I think it's either 8 or 12 hours, but I'm not sure.  The savedgames job just walks through the directory of matches on disk at (I think) 2 a.m.  If the completed match is still on disk, it counts it.  If it's been purged, it doesn't.  I think the purge job works in alphabetical-order chunks, which would explain why a recently-completed game might be in one person's list but not the other's.

Repbot had a trick whereby it kept yesterday's savedgames list and reported the intersection of the two.  This ostensibly eliminated recently-completed matches, although in reality it just produced a differently-wrong number.  RepBot hasn't been around for a while, though.

[playBunny]

The savedgames job just walks through the directory of matches on disk at (I think) 2 a.m.  If the completed match is still on disk, it counts it.

Is the savedgames job a replaceable module? If so then maybe some helpful soul could write a brighter one that looks to see whether each match has ended.

[Patti]

The savedgames script is a quick hack that I whipped up a couple of years ago.  In theory I could replace it, but I'd rather not right now-- the current implementation is incredibly lightweight and places almost no load on the system, while opening and parsing thousand and thousands of files is much more intensive. 

Plus, I put the information out there with the intent of it being consumed by humans.  Unfortunately, it started getting sucked in by a variety of clients, bots, and god knows what else, and I have no way to know or control what's using it or how it's being used.  Changing the data definition underneath an unknown number of programs seems like a very bad idea.  The only way that I'd be willing to consider it would be one that completely breaks existing nonhuman consumers.

[playBunny]

Thousands and thousands of matches are played per day? Or does that lazy purge job need to be reminded what "purge" means?! 

Hmmm, I wasn't thinking that the output should change in the slightest. Same format just without the matches that have actually finished.

[playBunny]

The only way that I'd be willing to consider it would be one that completely breaks existing nonhuman consumers.

I hope you aren't thinking of breaking any bunnies! 

[Patti]

It looks like FIBS is averaging around 3100 matches completed per day, based on a sample of about two weeks.  However, there are many times more files on disk-- saved games are kept around for a couple of months before being purged.

It's not a matter of changing the format of the output, but rather the meaning of the data.  If code is written that assumes that the data mean X, and then you change the definition of the data so that they really mean Y, unpredictable things happen.  The only way I'd be willing to redefine the meaning of the data would be to make sure existing programs couldn't get to the information at all.   (I could probably produce two sets of numbers, but that would be extra special messy and confusing.)

Imagine writing an accounting program that got a number called DailyTotal-- a number which meant gross revenues for the day.  Later someone changed DailyTotal to mean net revenues for the day.  Unless it was also changed, the code that consumed the DailyTotal information would still think that it was gross revenues, and that would throw a serious monkeywrench into the system... but it would do it in a way that was completely nonobvious. All of a sudden the program would just be giving you wrong information, but you'd have no idea why it happened.

If you made DailyTotal disappear completely and instead made the consumers ask for DailyNetTotal, then it would be very obvious that the code would have to change-- when the program tried to get DailyTotal it would get an error, and that error would prevent the program from humming along and giving bad information.

Going back to savedgames, I honestly don't know what programs consume those data.  I can't notify the authors because I don't know who they are, and even if I could they couldn't force everyone to update their client code.  Therefore, changing the meaning of the data would be a very bad idea.

[skater]

Thanks for the clear explanation(s)...at least now I have a pretty clear idea of what's going on. 

[playBunny]

It looks like FIBS is averaging around 3100 matches completed per day

Wow, that's well over a million per year!

saved games are kept around for a couple of months before being purged.

Is that completed matches as well or just the ones that are truly saved?

It's not a matter of changing the format of the output, but rather the meaning of the data

I'm not sure I see that. As I understand it, the data is an undifferentiated list of matches. You said in another post that there was no way to tell from a single savedgames output whether a given match was completed or not. So any change in that data merely redues the number of such matches. The nature of the data remains exactly the same with the exception that the count of completed matches would be zero. It's still the same data because there could be completed matches, there just wouldn't be.

The difference for users would be that those who, in ignorance, made the entirely reasonable assumption that completed matches were not in the data, would be correct and not prone to discovering discrepancies. For a consumer program that did inter-day differencing, it would simply find no matches to discount.

Still, now that I understand the nature of the "thousands and thousands", I quite agree that having the savedgames job sort through that lot would be an inefficient solution.

[don]

rePYbot now reports on the intersection of two days' successive saved games lists, as, I believe, does RepBot.  For general info, this typically removes about 2000 games from the list.  Parlorbot's saved dives from about 120 to 50 or so.  It also means that a game has to be saved for two days to be in rePY's database.  And if you play the same person every day (or bot), it's likely to show up as saved, whether it actually is saved or not.

--
don