reputation improvement

[jonesyjt]

Diane,

It all sounds good - I am curious though - how will repbot know if you have ever played? I can see how it would check for a saved match -but didnt think there was a way to tell if you simply played and finished a match.

It probably slipped by, but I mentioned that a new simple database would also have to be developed consisting of ordered pairs of player names, and a date.

From the user's point of view, complaining or vouching would be the same. However, with the aide of another simple database, (using data fibs already collects)

This new database would come from data already collected as soon as a match starts.  This database along with the saved games database will give the required info needed for this task.  The new database only needs hold an ordered pair once, NOT every game ever played, so the order is at most n^2 plus the date field.  (That is if every member plays every other member.)  This is quite manageable.

If all comes to pass, there would have to be a transition period for a new repbot that works in this way especially because as of now, there is no database as mentioned.

Everyone,
Think on this a bit, and give some feedback.

jonesyjt  

[Patti]

OK, I feel very strongly about this topic.  RepBot has, without a doubt, made my life worse, and from where I sit I believe that it is a negative addition to FIBS.  RepBot is fundamentally flawed in its current implementation.  

I've been studying reputation systems fairly intensely over the last couple of weeks as part of my job, so this discussion is particularly timely.  (RepBot has served as a fine example of a badly-designed system, and many people in my company are now familiar with it.)

Ebay has an interesting and useful, if minimal, rating system only because they have complete control over the transaction.  You can't contribute to someone's reputation unless you have completed a transaction with them, and you can only contribute once.  It's possible to create fake accounts and transactions in order to boost someone's rating, but there's a monetary cost in doing so that makes it largely not worth your while.  Additionally, you have a limited window of time in which to express an opinion.

The fatal flaw in RepBot is that anyone can express an opinion on anyone, and it counts.  This sucks for me, because it means that I spend a lot of time cleaning up accounts from jerks who create multiple accounts just to slam someone or boost their rating.  I get about an equivalent number of complaints from people about jerks who do this, largely from morons who can't read read the help text to see that I'm not responsible for what happens on RepBot.

In order to make an effective reputation system, the system must be changed so that you can only express an opinion about someone with whom you have recently completed a match.  This data is readily available merely from watching the report output.  (It is not "technical (sic) very hard to do", as one author suggested, but is actually quite trivial to implement.)  One day's worth of data should be sufficient; if you can't muster an opinion within the first 24 hours, then you don't really need to do so.  Of course, if you play another match, that window opens for another 24 hours.

If someone misbehaves in a match that you've watched, then it's up to the wronged opponent to file a complaint if they choose to-- ganging up on the perpetrator shouldn't be allowed.

Someone questioned the validity of the savedgames list.  Actually, I'm being polite-- they called it a joke.  I personally believe that this list is highly accurate *as the list is defined*.  By definition, it includes matches that are currently in process, as well as those that have been played recently but have not yet expired from the system.  I'm not certain of the expiry window, but it's less than 24 hours and may be as short as six hours.  The list is updated nightly, and I have never once received a complaint about that not happening on schedule.  (If it has, please let me know.) [1]

The problem is that people have chosen to interpret the savedgames list as an incompleted matches list, and that was never its intent.  The caveats about the usage of the list are clearly stated at the top of the page.  On the other hand, even with these caveats, the list does a reasonably good job of identifying who has many incompleted matches and who has only a few.

On my more cynical days, when I've had half a dozen people grousing at me that so and so filed a bogus complaint against them, I've been tempted to just give the bot the boot.  Suffice it to say that I would be very strongly in favor of someone fixing this thing so that it stops being a pain in my ass.

-Patti

[1] There is a problem with FIBS that causes bogus matches to be saved sometimes when the server crashes.  I have no way of detecting these, but FIBS doesn't crash very often and even when it does not all matches wind up in this state.  I don't believe there's ever more than one per person, so this bug should have only a tiny impact on any individual's saved game count.

[Patti]

P.S.  I have match logs going back to late 2002, and possibly earlier.  Should the system be altered so that opinions are only accepted from people who have played a match together, it would take me about twenty lines of code to produce a list of all opponents who have played at least one match against each  other.  This list could be used to validate "good" complaints and vouches, and all others could be removed from the system.

This list would not be 100% accurate, since accounts are often deleted for inactivity and then the handle is assumed by another individual, but this effect should be so small as to be negligible.

P.P.S.  I would expect relatively strong pushback from the people who currently form cabals to gang up on people they don't like.

[Tomawaky]

Thanks Patti
That was to be said

[don]

...sounds like a pretty accurate presentation to me, Patti.  Some of your points have been made before, but your suggestions for implementation are new and excellent (and no surprise since you are the one who knows best what's available from FIBS for support).

Kutchie kutchie koo to you too!

[Patti]

P.^n S.  I just wrote that script; it took more time to run it than it did to code it.  There are right around 900K unique opponent pairings since November of 2002.  Should this change be implemented, I will provide a current list to whoever is responsible for purging the database of nonqualifying vouches and complaints.

Incidentally the most frequent pairing:

MonteCarlo magee : 3345

[socksey]

Seems we are progressing toward a superior system which makes me very happy.  

How about implementing the awarding of a win to droppees when winning is immanent?  So happens the player I was playing this evening when you shut down Fibs for a minute, Patti, didn't return to resume my 7 ptr win almost completed.    
 

socksey



"He is brilliant - to the top of his boots." - David Lloyd George

[Patti]

Can't do that without modifying the FIBS codebase itself, and that's not gonna happen.

[socksey]

Death to droppers!   :angry:  After torture!    

socksey



"Some people say they haven't yet found themselves. But the self is not something one finds; it is something one creates." - Thomas Szasz, Hungarian-born psychiatrist  

[jonesyjt]

Many thanks to Patti!

Patti and I had a good long discussion on these issues and she has been a wealth of knowledge for me.  This project is still in the planning phase but IS moving forward.  Thanks for everyones input and support.  

jonesyjt    

[diane]

This is all great to hear - I am glad a project which has some enthusiasm from Patti is in the offing - since as was said - she knows the system best - and also gets all the - erm 'feedback' that the rest of us miss.

I wasnt aware the creation of new accounts to slam someone was such a problem - since a new account has very little sway with repbot - but I guess a lot dont know that.

Now I understand how the played matches would be kept track of - and you are right - I missed that first time round - sorry - not very technical here  I am very pleased to see the system can also be 'back dated' - this should straighten out a lot of already bogus reps.

Many thanks to you jonesyjt for undertaking the improvement of repbot      From what I can see it looks like a large task - and I will wait patiently for the new era    

[donzaemon]

All we would really need to fix the current problems with repbot is to have a minor change to fibs.  If the oldboard command was changed or overloaded so that it could take 2 parameters ,  oldboard(player, opponent) ,  then it would be possible for an outside party to creat a bot,  based on gnu  that  evaluates the saved matches for a player.  this bot could then award dropper points based on the ammount of negative equity a person has for each saved match. It would be a completely automated way to expose droppers and would NOT be subject to manipulation by players.  The upside is HUGE on several fronts and all that is necessary is a small change to one of the functions (5min to 1hr work changing oldboard and a recompile)
I believe a bot that could operate like this would effectively eliminate the dropper problem on fibs not to mention the complaints etc. that Patti gets.

[MadMatt]

What about complaining about abusers you've never played with? I imagine a lot of people file a complaint because of inane shouting, verbal abuse, rudeness, hogging the bots, creating fake tourneys etc., not just because they were dropped in a match ...

Not that I play on FIBS that much, but I'd like to keep the ability to complain about people who insult my mother in shouts, even if I've never played with them ... (That hasn't really happened, I'm just trying to make a point).

[diane]

I thought about this too - there are times when I think a complaint is justified for some reason other than they actually dropped me.  BUT I think the overall gains from this new system will outweigh the small losses.  As no one will be able to vouch for - or complain against someone they have never played - there can be no artificially high reputations - and the reps will be more accurate - so the need for extra complaints is removed.  

And trying to understand donzaemons contribution   - do you mean change that command so it can be accessed by any interested third party. As I understand it - the oldboard can only be accessed by either player - so for a bot to work the command would need to be altered to allow the bot to check it?

[donzaemon]

Yeah, It would mean that a third party could access anyone else's oldboard. This, in itself, could have an impact on droppers because it would be easy for anyone to see who dropped in cases where an obvious lost game was left. More importantly, though, is the ability to create a robot that uses this to assign 'dropper equity' to players based on the accumulated negative equity they have in their saved matches.  This could easily identify droppers on the server. And a robot wouldn't have to be based solely on this ...  it would be possible to have it handle complaints as well, but the point is the power it would have to accurately identify droppers, and it would all be done automatically.

[Shades]

Death to droppers!   :angry:  After torture!    

socksey

... be tortured by you may be a rare pleassure.... hehe...    

[Patti]

I seem to have been unclear earlier.

Changes to the FIBS code itself just aren't going to happen.  Period.  Full stop.  I don't have access to the source code.

The problem with multiple accounts is that people create many accounts, throw matches between them in order to boost experience, and then gang up on the person they want to harass.  Victim comlains to me, and I spend time digging through logfiles, tracking down the accounts, deleting them all, and then recreating them with 1500/0.  It's a pain in the ass for me.

As for complaining about people who shout things you don't like, etc... My vote is "tough".  This particular feature of repbot is heavily abused.

[diane]

I seem to have been unclear earlier.

Changes to the FIBS code itself just aren't going to happen.  Period.  Full stop.  I don't have access to the source code.

NO - this was not unclear on any of the million times I heard it - but thoughts are never a bad thing    It is the general chit chat on the subject of repbot which has brought jonsey to a point where he can now write and implement something which will be agreeable to most (give or take the cabals of course ) Long live the exchange of ideas    

[socksey]

As for complaining about people who shout things you don't like, etc... My vote is "tough". This particular feature of repbot is heavily abused.

I totally agree with this.  I don't have to play with anyone who is rude if I don't want to.  I just don't accept their invitation.  I don't have to listen to someone's rudeness if I don't want to.  I gag them.  Someone may be rude to someone else and not rude to me.  But when someone I don't know invites me to play, I would like to know if they are a dropper because I simply hate investing time in a match that is not completed.

I can't tell you how many times I have helped some who are not droppers to regain a good reputation by vouching for them when I investigated and found there were no saved games, but a group of people had caused the bad rep because of some silly disagreement.  The worst of which were the ones who wouldn't vouch for someone who asked them to and threatened if they refused to vouch, to have his friends complain and ruin the rep of the poor individual they were trying to force into submission.

socksey



"We in this country, in this generation, are by destiny rather than choice the watchmen on the walls of world freedom. We ask therefore, that we may be worthy of our power and responsibility, that we may exercise our strength with wisdom and restraint, and that we may achieve in our time and for all time the ancient vision of peace on earth, goodwill toward men." - John F. Kennedy (from a speech that was never delivered, due to his death)

[Shades]

The worst of which were the ones who wouldn't vouch for someone who asked them to and threatened if they refused to vouch, to have his friends complain and ruin the rep of the poor individual they were trying to force into submission.

socksey

vry true socksey.... and we all know who they are... so bootom question seems to be : why not taking sanctions against them??... we know them all.... are they  intouchables ?

Most of us come to FIBS for love of gammon and are only looking for a good game and some relaxing fun....