on the possible improvements proposed on http://www.fibsboard.com/repbot/

[MagoWiz]

As an enthusiast user of Repbot, from its very first days, here's my opinion about some of the possible improvements suggested in the list:





• Friends as the only query?

- does not make sense if we assume the repbot rationale



• Longer savedgame memory. After some period of time FIBS erases saved matches. RepBot no longer marks complaints as having a real saved match. A nice feature would be for FIBS to save the mark for a longer period of time, or maybe for the life of the complaint.

- Repbot is much more than a synopsis of saved games of anyone: I like the fact that reputation reflecs other players' mood about some player, not only possible dropping


• Limit input by known robots, as this can be abused by their authors. OTOH, bots get abused themselves and it makes sense to allow them to voice their opinion about users, if only in a limited or weighted manner.

- sure to be done: complaints or vouches by robots should be inconsistent with reputation by their very nature


• Announcing to everyone, or registered users, the user themselves, or their complainers when someone with a sufficiently bad reputation logs in.

- not bad.


• Respond to "is a dropper" in the shouts to remind that user of RepBot's availability

- useless, as long as the previous gets implemented


• Domain restrictions so that users with multiple accounts can not vouch for themselves. This could take the form of some kind of partial restriction, or weighted restriction. In fact, domains themselves can have reputations

- no, no, no: we Italians already suffer the consequences of same-domain based assumptions on players. Though these assumptions may be useful to fight a few bad guys, they eventually end up in stressing hundreds of good guys.


• Alert-like functionality for match brokering, where a user could specify their preferences based on rating, idle-time etc... in addition to savedgames, and reputation, to have RepBot suggest opponents.

- why not...


• User preferences could include notification when someone complains, vouches or withdraws "about" you, logging the activity of people you have complained about or any user specified (logins, logouts, matches played).

- excellent


• Web interface for querying the data.

- nice


• Some kind of second order calculation where the reputation of your complainers/vouchers is taken into account.

- good. This could be something called "Self-Qualified Reputation"


• Square root of experience instead of experience would flatten the experience contribution out a bit.

- I don't think any real human is perceiving a difference between a linearized or an exponential measuring scale. It could be very useful, instead, to have a slightly more sensible qualitative labelling based on actual reputation and vouches/complaints count. Maybe Repbot could learn a few more words in addition to "GOOD" and "BAD", so to provide bettere info to queries. i.e.:

> if based on less that 10 vouches/complaints, Reputation could tell "NOT DEFINED", otherwise...

> if between -1000 and + 1000, Reputation could tell "UNCERTAIN"

> if bw +1000 and +9999, "FAIR"
> if bw +10000 and +49999, "GOOD"
> if bw +50000 and +999999, "VERY GOOD"
> if +100000 or over, "HIGHEST"

> if bw -1000 and -9999, "POOR"
> if bw -10000 and -49999, "BAD"
> if bw -50000 and -999999, "VERY BAD"
> if -100000 or lower, "LOWEST"

Or, anything similar.

[don]

The most obvious RepBot change is to simply ban known duplicate IDs.

[inim]

Thank you so much MagoWiz, this is the best compilation of relevant issues available. Now RepBot is back to a maintained mode, it is a valuable resource for me. Please understand I can't do it all at once, but over time many of your suggestions will be considered.

Note that RepBot is an open source project, so anybody is free to contribute code to implement any of the features listed below. The prefered format is a Unix DIFF against a stable SVN version or HEAD. Please contact me if you plan do such work, coordination of activity may be useful.

• Friends as the only query?

- does not make sense if we assume the repbot rationale

Agreed. But no single metric can really catch all aspects of what a reputation is, so several of them would allow finer grained decisions. However, too many will simply confuse.

The other contraint is computational effort. All calculations have to take place over a database and likely include expensive table joins. Currently a reputation is calculated in about 80 ms, but expensive closure calculations such as "friend" may kill the CPU and allow for denial of service attacks.

Whatever is meaningful and computionally feasible will be considered and supported in a future "askextended" (or whatever the name) command. Suggestions welcome.

It may look like this:

> tell repbot askextended foo
Classic: 123456
Friends: 7890
Bots: -12435

Etc. pp. But this is a major feature, so don't expect it tomorrow.

• Longer savedgame memory. After some period of time FIBS erases saved matches. RepBot no longer marks complaints as having a real saved match. A nice feature would be for FIBS to save the mark for a longer period of time, or maybe for the life of the complaint.

- Repbot is much more than a synopsis of saved games of anyone: I like the fact that reputation reflecs other players' mood about some player, not only possible dropping

Agreed. It is not the task of a consumer of savedgames data to purge it, the "sh**-in-sh**-out" principle applies. It's up to Patti to improve the quality of input data, which however seems to be quite a technical problem. If she manages, all clients will benefit equally.

• Limit input by known robots, as this can be abused by their authors. OTOH, bots get abused themselves and it makes sense to allow them to voice their opinion about users, if only in a limited or weighted manner.

- sure to be done: complaints or vouches by robots should be inconsistent with reputation by their very nature

Weights have been implemented in version 2.0.1. I'll experiment with concrete weights in the near future.

• Announcing to everyone, or registered users, the user themselves, or their complainers when someone with a sufficiently bad reputation logs in.

- not bad.

Hard to do. First of all I don't think bots should shout. Using message is pointless, as subscribed users may not be logged in. Using tell limited to the known list of logged in users would work, but is a major feature to implement.

In addition, new users who are abused most are typically not aware of droppers or Repbot, and thus unlikely to be subscribed for warnings. However, subscribing new users by default (opt-out model) is an unacceptable intrusion into their privacy.

Thus, this is a "between a rock and a hard place" problem I cannot offer an easy solution for.

• Respond to "is a dropper" in the shouts to remind that user of RepBot's availability

- useless, as long as the previous gets implemented

That opens a box of pandora. Patti will kill RepBot if it encourages trash shouts of any type. What would be a lot better is tighter RepBot integration into GUI clients, e.g. by offering a "What is your opinion about your last opponent" requester after all played matches. The more opinions are cast, and the more noobs are aware of RepBot, the better the reputation quality will become.

• Domain restrictions so that users with multiple accounts can not vouch for themselves. This could take the form of some kind of partial restriction, or weighted restriction. In fact, domains themselves can have reputations

- no, no, no: we Italians already suffer the consequences of same-domain based assumptions on players. Though these assumptions may be useful to fight a few bad guys, they eventually end up in stressing hundreds of good guys.

Currently RepBot doesn't store timestamp and IP along with opinions. Doing so would more or less turn it into a fully fledged CLIP database, which is out of scope. I will likely add timestamp and IP for internal use (e.g. purging after abuse), but I currently would not want to base user visble output on that.

Cleaning fibs users from sock puppets, bot driven players, and other types of abusers is Patti's job, and I have no intention to become more papal than the pope. If she deletes a user for any reason, RepBot's garbage collection will follow within a few days and delete that user's opinions automatically. That must be good enough.

• Alert-like functionality for match brokering, where a user could specify their preferences based on rating, idle-time etc... in addition to savedgames, and reputation, to have RepBot suggest opponents.

- why not...

RepBot is not intended to become an egg-laying angora rabbit. Creeping featurism has killed many a project, and so adding new features has to keep focused on reputation.

• User preferences could include notification when someone complains, vouches or withdraws "about" you, logging the activity of people you have complained about or any user specified (logins, logouts, matches played).

- excellent

Agreed, this is my highest priority feature to be added. Expect it within one of the next releases.

• Web interface for querying the data.

- nice

I always dreamt about a visualization of RepBot data as directed graphs. I would be supportive to any advanced visualization project. This probably would include methods to access the database in a more effective way, such as SOAP calls. Maintaining a HTML site or any other advanced visualization for RepBot data myself is out of scope for me, however. RepBot is a data collector, not a data presentation engine.

• Some kind of second order calculation where the reputation of your complainers/vouchers is taken into account.

- good. This could be something called "Self-Qualified Reputation"

Such a good formula would be most welcome. Please keep in mind however that the computational effort has to be reasonable given the existing database scheme. Any SQL wizards out their to pick up the challenge?

• Square root of experience instead of experience would flatten the experience contribution out a bit.

- I don't think any real human is perceiving a difference between a linearized or an exponential measuring scale. It could be very useful, instead, to have a slightly more sensible qualitative labelling based on actual reputation and vouches/complaints count. Maybe Repbot could learn a few more words in addition to "GOOD" and "BAD", so to provide bettere info to queries. i.e.:

> if based on less that 10 vouches/complaints, Reputation could tell "NOT DEFINED", otherwise...

> if between -1000 and + 1000, Reputation could tell "UNCERTAIN"

> if bw +1000 and +9999, "FAIR"
> if bw +10000 and +49999, "GOOD"
> if bw +50000 and +999999, "VERY GOOD"
> if +100000 or over, "HIGHEST"

> if bw -1000 and -9999, "POOR"
> if bw -10000 and -49999, "BAD"
> if bw -50000 and -999999, "VERY BAD"
> if -100000 or lower, "LOWEST"

Or, anything similar.

Yes the core idea is excellent, but several caveats.

First, the current output can not be changed because of the legion of unmaintained clients to RepBot who actually rely on the current String literally. Furthermore, any fixed assumption about the range of the RepBot reputations is unsuited as it will break on any formula change, or even new abuse patterns. E.g. I am considering to lower the capping of the max. voucher value, as the current 10K seems a bit high.

What WOULD be nice and feasible is to supply information about e.g. minimum, maximum and average rating in whatever form. E.g. "This reputation is 20% above the average reputation".

This would adapt with DB changes and still gives good guidance. I gotta think a bit about this, but feels like a very good idea. A show stopper may again be computational complexity, as we have to do all calculations over a database ...

As for the scaling linear vs. exponential, I tend to agree with MagoWiz: scaling is irrelevant as long as as users understand the semantic of a numerical rating, and values are compareable among each other. The other critical issue is the sign, ideally "GOOD" and "BAD" would be watershed by the neutral zero. Negative reputation is actually a criterion used by many clients and users to refuse invitations etc. It will be a lot easier to place ratings above or below zero once things like average etc. are considered. E.g. a "BAD" reputation could mean that you are in the lower quarter of all reputations. Values such as "UNCERTAIN" and "UNDEFINED" make sense as well, but they are not exactly easy to define either.

Another point is: analytical functions suchs as sqrt(), log(), or ln() are above the head of many users, they may obfuscate more than they benefit. Linear rating in terms of some tangible parameters such as experience is a lot easier to understand. Higher math, especially statistics, should be useful in designing a better formula. But it shouldn't be required to understand the results.

Thus, let's concentrate on how to make the numerical values easier to understand rather than to lose ourselves in higher math.[/list]