Fibsboard

[burper]

fibs won't always be accessible, so the interface should probably be something like:

-at registration: append the name to a "pending" file

-later (when the user tries to login and they are not known yet?), check the "approval info" file against the forum info file for approval based on the criteria. if the info for that name is there, it removes it and does whatever approval/rejection is needed.

- an info gathering script wakes up and runs periodically, reads the "pending" file and dumps info into the "approval info" file, if it succeeds in contacting fibs, and staying connected long enough to complete the read-transaction, otherwise it leaves the name in the pending file for a later try.

So, two scripts, decoupled. InfoGetter can get all info about a user that might be interesting. InfoChecker does what you describe.

if (ForumName in InfoFile) {
   if (InfoName.invalid) reject and exit
   if (InfoName.email = ForumName.email) approve and exit
   if (InfoName.ip = ForumName.email) approve and exit
   reject and exit
}

Can I assume some flavor of unix with crontab?
Is some version of "expect" available?

[webrunner]

I am using debian sarge, crontab is available, so is bash, perl, curl and other linux stuff.
I prefer the php scripting language tho.

[burper]

First things first: does the 2-script thing make sense to you?

Picking the best language depends on the task. PHP is probably best for the script that interfaces directly with whichever CMS you choose because it is probably written in PHP, true? That I will wait on until you indicate what the API would be. In either case, it would probably be better to write the guts of it in a separate script that you could call from either one with a wrapper of PHP using the system call. Easier to unit test too.

For the script that contacts fibs, expect is the ideal language, assuming it is readily available. Expect was written just for this kind of purpose. Since it would be driven from crontab and its' only I/O would be the filesystem, no interaction with the forum script is required. In addition, I have built up a pretty rich fibs library of routines in expect, so it would be pretty quick to complete that part. Do a "which expect" to see if you have it, and if so "expect -v" and let me know the version.
http://packages.debian.org/cgi-bin/search_...ble&release=all

If its' reviewing the code you are worried about (and you should be), you will be able to read and understand it easily.

btw, do you use the term "forum script" rather than cms because that is what most fibsters will understand, or is innovision really focused on forums and not general cm?

[burper]

fibs access script GetFibsUsersInfo.exp with supporting libraries FIBS.exp and TELNET.exp to be put in same directory. crontab entry should cd there before running. These versions of FIBS.exp and TELNET.exp are cut way down from my every day versions, so include only what is needed for this app.

To test from bash:

Code Select Expand


% cd <the dir with the code in it>
% printf "%s\n%s\n%s\n" webrunner burper nosuchuserprob > ./infile
% ./GetFibsUsersInfo.exp webrunner <your password> ./infile ./outfile
% cat ./outfile


FIBS.exp:

Code Select Expand


# generic fibs expect script stuff, meant to be 'source'd in

set ff_unsint           "\[\[:digit:\]\]+"
set ff_alphatoken       "\[\[:alpha:\]_\]+"
set ff_somename         "$ff_alphatoken"
set ff_timezonechars    "\[\[\:alpha\:\]\/\]+"
set ff_day              "(?:Monday|Tuesday|Wednesday|Thursday|Friday|Saturday|Sunday)"
set ff_month            "(?:January|February|March|April|May|June|July|August|September|October|November|December)"
set ff_date             "$ff_day\, $ff_month $ff_unsint $ff_unsint:$ff_unsint $ff_timezonechars"
set ff_hostnamefield    "\[\[:alnum:\]\.-\]+"
set ff_emailfield       "\[\[:alnum:\]\.-_@\]+"
set ff_restofline       "(\.+)\r"
set ff_rating           "\-?$ff_unsint\.$ff_unsint"
set ff_experience       "$ff_unsint"

################################################################################
### whois section, which operates the same in telnet and clip modes
################################################################################
# whois specific error strings:
set ERR_WHOIS_GIVENAME  " please give a name as an argument.\r"
set ERR_WHOIS_NOSUCH    "No information found on user ($ff_somename).\r"
# whois output:
set FIBS_WHOIS_HEADER   "Information about ($ff_somename):\r"
set FIBS_WHOIS_LAST     "  Last login:  ($ff_date) from ($ff_hostnamefield)\r"
# whois last login details:
set FIBS_WHOIS_SECONDS  "  Still logged in\. ($ff_unsint) seconds idle\.\r"
set FIBS_WHOIS_MINSEC   "  Still logged in\. ($ff_unsint):($ff_unsint) minutes idle\.\r"
set FIBS_WHOIS_LOGOUT   "  Last logout: ($ff_date)\r"
# whois play status:
set FIBS_WHOIS_STAT_NOT "  Not logged in right now\.\r"
set FIBS_WHOIS_STAT_NNN "  ($ff_somename) is not ready to play, not watching, not playing\.\r"
set FIBS_WHOIS_STAT_RNN "  ($ff_somename) is ready to play, not watching, not playing\.\r"
set FIBS_WHOIS_STAT_NWN "  ($ff_somename) is not ready to play, watching ($ff_somename), not playing\.\r"
set FIBS_WHOIS_STAT_RWN "  ($ff_somename) is ready to play, watching ($ff_somename), not playing\.\r"
set FIBS_WHOIS_STAT_NNP "  ($ff_somename) is playing with ($ff_somename)\.\r"
# whois away status
set FIBS_WHOIS_AWAY     "(?:\n  ($ff_somename) is away: ($ff_restofline))+"
# whois rating and experience line
set FIBS_WHOIS_RATING   "  Rating: ($ff_rating) Experience: ($ff_experience)\r"
# whois address line:
set FIBS_WHOIS_NOADDR   "No email address\."
set FIBS_WHOIS_ADDR     "((?:$FIBS_WHOIS_NOADDR|$ff_emailfield))\r"
#return values of FIBS_parse_whois_info:
set RET_OK              0
set RET_ERR_TIMEOUT     1
set RET_ERR_PARSING     2
set RET_WHOIS_GIVENAME  3
set RET_WHOIS_NOSUCH    4

# FIBS_parse_whois_info
#   after doing a "whois <username>, use this to parse expected return
#
# Note: applications handling async stuff too need to check for _HEADER and
#   errors first, in addition to whatever other async stuff they care about
#
#   Returns:
#       $RET_OK                 ok
#       $RET_ERR_TIMEOUT        timeout at any stage of the parsing
#       $RET_ERR_PARSING        parsing error
#       $RET_WHOIS_GIVENAME     ERR_WHOIS_GIVENAME found
#       $RET_WHOIS_NOSUCH       ERR_WHOIS_NOSUCH found
#
#   Outputs:
#       whois_name:     username
#       whois_exists:   1 if found, 0 otherwise
#       whois_date_in:  last login date
#       whois_date_out: last logout date, if not logged in, "" otherwise
#       whois_host:     hostname of last login
#       whois_idle:     idle time in seconds if logged in, 0 otherwise
#       whois_playing:  username of opponent if playing, "" otherwise
#       whois_watching: username of opponent if watching, "" otherwise
#       whois_away:     away message if away, "" otherwise
#       whois_rating:   rating
#       whois_exp:      experience
#       whois_addr:     address

#TODO: break this beast into pieces, but potentially usable ones:
proc FIBS_parse_whois_info {} {
       global FIBS_spawn_id
       global ff_alphatoken ff_somename ff_date ff_day ff_month ff_timezonechars ff_hostnamefield ff_unsint ff_rating ff_experience ff_restofline ff_emailfield
       global ERR_WHOIS_GIVENAME ERR_WHOIS_NOSUCH
       global FIBS_WHOIS_HEADER FIBS_WHOIS_LAST FIBS_WHOIS_SECONDS FIBS_WHOIS_MINSEC FIBS_WHOIS_LOGOUT FIBS_WHOIS_STAT_NOT FIBS_WHOIS_STAT_NNN FIBS_WHOIS_STAT_RNN FIBS_WHOIS_STAT_NWN FIBS_WHOIS_STAT_RWN FIBS_WHOIS_STAT_NNP FIBS_WHOIS_AWAY FIBS_WHOIS_RATING FIBS_WHOIS_NOADDR FIBS_WHOIS_ADDR
       global whois_name whois_exists whois_date_in whois_date_out whois_host whois_idle whois_playing whois_watching whois_away whois_rating whois_exp whois_addr
       global RET_OK RET_ERR_TIMEOUT RET_ERR_PARSING RET_WHOIS_GIVENAME RET_WHOIS_NOSUCH

       #set defaults
       set whois_name          "";
       set whois_exists        0;
       set whois_date_in       "";
       set whois_date_out      "";
       set whois_host          "";
       set whois_idle          0;
       set whois_playing       "";
       set whois_watching      "";
       set whois_away          "";
       set whois_rating        "1500.00";
       set whois_exp           0;
       set whois_addr          "$FIBS_WHOIS_NOADDR";

       #handle wether user exists or not, and if so, parse the name
       expect {
               -i $FIBS_spawn_id -re "$FIBS_WHOIS_HEADER" {
                       set whois_name          "$expect_out(1,string)";
                       set whois_exists        1;
                       }
               -i $FIBS_spawn_id -re "$ERR_WHOIS_GIVENAME" {
                       return $RET_WHOIS_GIVENAME;
                       }
               -i $FIBS_spawn_id -re "$ERR_WHOIS_NOSUCH" {
                       return $RET_WHOIS_NOSUCH;
                       }
               timeout {
                       return $RET_ERR_TIMEOUT;
                       }
       }

       # get last login
       expect {
               -i $FIBS_spawn_id -re "$FIBS_WHOIS_LAST" {
                       set whois_date_in       "$expect_out(1,string)";
                       set whois_host          "$expect_out(2,string)";
                       }
               timeout {
                       return $RET_ERR_TIMEOUT;
                       }
       }

       # get last login details:
       expect {
               -i $FIBS_spawn_id -re "$FIBS_WHOIS_SECONDS" {
                       set whois_idle          "$expect_out(1,string)";
                       }
               -i $FIBS_spawn_id -re "$FIBS_WHOIS_MINSEC" {
                       set whois_idle          [eval
                               ($expect_out(1,string) * 60) +
                               $expect_out(2,string)];
                       }
               -i $FIBS_spawn_id -re "$FIBS_WHOIS_LOGOUT" {
                       set whois_date_out      "$expect_out(1,string)";
                       }
               timeout {
                       return $RET_ERR_TIMEOUT;
                       }
       }

       expect {
               -i $FIBS_spawn_id -re "$FIBS_WHOIS_STAT_NOT" {
                       if { $whois_idle != 0 } {
                               return $RET_ERR_PARSING; }
                       }
               -i $FIBS_spawn_id -re "$FIBS_WHOIS_STAT_NNN" {
                       if { $whois_name != $expect_out(1,string) } {
                               return $RET_ERR_PARSING; }
                       }
               -i $FIBS_spawn_id -re "$FIBS_WHOIS_STAT_RNN" {
                       if { $whois_name != $expect_out(1,string) } {
                               return $RET_ERR_PARSING; }
                       }
               -i $FIBS_spawn_id -re "$FIBS_WHOIS_STAT_NWN" {
                       if { $whois_name != $expect_out(1,string) } {
                               return $RET_ERR_PARSING; }
                       set whois_watching      "$expect_out(2,string)";
                       }
               -i $FIBS_spawn_id -re "$FIBS_WHOIS_STAT_RWN" {
                       if { $whois_name != $expect_out(1,string) } {
                               return $RET_ERR_PARSING; }
                       set whois_watching      "$expect_out(2,string)";
                       }
               -i $FIBS_spawn_id -re "$FIBS_WHOIS_STAT_NNP" {
                       if { $whois_name != $expect_out(1,string) } {
                               return $RET_ERR_PARSING; }
                       set whois_playing       "$expect_out(2,string)";
                       }
               timeout {
                       return $RET_ERR_TIMEOUT;
                       }
       }

       expect {
               -i $FIBS_spawn_id -re "$FIBS_WHOIS_AWAY" {
                       if { $whois_name != $expect_out(1,string) } {
                               return $RET_ERR_PARSING; }
                       set whois_away          "$expect_out(2,string)";
                       exp_continue;
                       }
               -i $FIBS_spawn_id -re "$FIBS_WHOIS_RATING" {
                       set whois_rating        "$expect_out(1,string)";
                       set whois_exp           "$expect_out(2,string)";
                       }
               timeout {
                       return $RET_ERR_TIMEOUT;
                       }
       }

       expect {
               -i $FIBS_spawn_id -re "$FIBS_WHOIS_ADDR" {
                       set whois_addr          "$expect_out(1,string)";
                       }
               timeout {
                       return $RET_ERR_TIMEOUT;
                       }
       }

       return $RET_OK;
}

# send_fibs: requires FIBS_spawn_id to be set by app
proc send_fibs { msg } {
       global FIBS_spawn_id;
       send -i $FIBS_spawn_id "$msg\n";
}

# FIBS_logout
proc FIBS_logout {} {
       global FIBS_spawn_id
       send_fibs "bye"
}



TELNET.exp:

Code Select Expand


# FIBS TELNET protocol specific stuff here

set ft_loginorder       "$ff_unsint"
set fc_time             "$ff_unsint:$ff_unsint"

#simple login proc.
#   Returns:
#       0 - successfully authenticated
#       1 - login failed, but fibs is responding
#       2 - timeout, maybe fibs not responding
#       3 - eof, maybe connection problems
proc TELNET_login { username password } {
       global FIBS_spawn_id;
       set logtry 0;
       expect {
               -i $FIBS_spawn_id "\nlogin: " {
                       if { $logtry > 0 } {
                               send_user "login failed\n";
                               return 1;
                       }
                       send_user "login prompt detected\n";
                       send_fibs "$username";
                       set logtry 1;
                       exp_continue;
                       }
               -i $FIBS_spawn_id "\npassword: " {
                       send_user "password prompt detected\n";
                       send_fibs "$password";
                       exp_continue;
                       }
               -i $FIBS_spawn_id "\n\*\* User $username authenticated.\r" {
                       send_user "authentication detected\n";
                       return 0;
                       }
               eof {
                       send_user "eof\n"; return 3;
                       }
               timeout {
                       send_user "timeout\n"; return 2;
                       }
       }
}




GetFibsUsersInfo.exp:

Code Select Expand


#!/usr/bin/expect -f
#
# Usage:
#   GetFibsUsersInfo <username> <password> <infile> <outfile>
# where:
#   username:   login name to authenticate to FIBS with
#   password:   password to authenticate to FIBS with
#   infile:     a file containing 0 or more FIBS user names
#       file is read only for this script, and will not be moved
#       cooperating app should read "outfile" and operate on "infile"
#       yes, it is tempting to use mkfifo for these
#   outfile:    the output file to write the information to, with one user
#       record per line, each having 4 tab-separated fields:
#           username
#           exists
#           host
#           address
#
###############################################################################

source FIBS.exp
source TELNET.exp

#check and set arguments
if { [llength $argv] < 4 } {
   send_user "Usage: GetFibsUsersInfo <username> <password> <infile> <outfile>\n"
   exit -1
}
set username    [lindex $argv 0]
set password    [lindex $argv 1]
set infile      [lindex $argv 2]
set outfile     [lindex $argv 3]

#error check input file
if {[file exists $infile]==0} { send_user "$infile does not exist\n"; exit -1 }
if {[file readable $infile]==0} { send_user "$infile not readable\n"; exit -1 }

#get data from input file
set infilechannel [open $infile]
set infiledata [read $infilechannel]
if {[catch {close $infilechannel} err]} { send_user "reading $infile: $err" }

#parse fibs usernames from input data
set fibsnames [split $infiledata \n]
foreach n $fibsnames {
   if {$n!=""} {
       send_user "n:$n\n"
   }
}

#uncomment for debugging:
#log_file -noappend -a ./GetFibsUsersInfo.logfile
log_user 0

send_user "spawning telnet fibs.com 4321\n"
spawn telnet fibs.com 4321
set FIBS_spawn_id $spawn_id

# login: telnet quieter, whois works the same anyway
TELNET_login $username $password

# set these to default values:
set whois_name          "";
set whois_exists        "";
set whois_date_in       "";
set whois_date_out      "";
set whois_host          "";
set whois_idle          "";
set whois_playing       "";
set whois_watching      "";
set whois_away          "";
set whois_rating        "";
set whois_exp           "";
set whois_addr          "";

#open output file for writing
set outfilechannel [open $outfile w]

foreach n $fibsnames {

       if { $n == "" } { send_user "noname\n"; continue; }

       send_fibs "whois $n"

       set ret [FIBS_parse_whois_info];
       send_user "ret=$ret\n";

       if { $ret == 1 } { send_user "timeout\n"; return $ret; }
       if { $ret == 2 } { send_user "parsing\n"; return $ret; }
       if { $ret == 3 } { send_user "name unspecified\n"; return $ret; }

       if { $ret == 4 } {
               send_user "$n: no such user\n";
               puts $outfilechannel "$n\t0\tnohost\tnoaddr\n";
               continue;
       }

       #send_user "whois_name=$whois_name\n";
       #send_user "whois_exists=$whois_exists\n";
       #send_user "whois_date_in=$whois_date_in\n";
       #send_user "whois_date_out=$whois_date_out\n";
       #send_user "whois_host=$whois_host\n";
       #send_user "whois_idle=$whois_idle\n";
       #send_user "whois_playing=$whois_playing\n";
       #send_user "whois_watching=$whois_watching\n";
       #send_user "whois_away=$whois_away\n";
       #send_user "whois_rating=$whois_rating\n";
       #send_user "whois_exp=$whois_exp\n";
       #send_user "whois_addr=$whois_addr\n";
       #TODO: get more things, e.g. rep,savedgames...

       #add to output file containing succesfully retrieved data
       set outstr "$whois_name\t$whois_exists\t$whois_host\t$whois_addr"
       puts $outfilechannel "$outstr"

}

close $outfilechannel
FIBS_logout

exit 0


[webrunner]

Invision is mainly forum, but i am moving away from invision.

Also i am still playing with the thought of adding more general BG related stuff so that would indeed be more CMS then forum. Currently i am choosing between SMF and OCPortal.

Since i would also like to encourage new members for Fibs and fibsboard i am changing my mind about the fibs check-up. An all-in-one sign-up would be fantastic tho.

And... i could use your code to do other stuff, like checking Fibs rating and putting fibs-users with f.i. a high rating or a big experince in a better usergroup.
Also the number of saved games and repbot rep in the profile would be really cool.

This could be a database-update script using cron.

[burper]

yes, the code is pretty general purpose as written. maybe i will add the repbot stuff at some point.
have you done anything with geolocation, e.g. http://www.maxmind.com/app/ip-location ?
Maintaining a fibs user db with locations would help with fibs parties.

GL with resurrecting bg-world here. would the name remain fibsboard?
How would you control spam without a fibs-check?
Would fibs users get their own forum topic areas where they could avoid this spam and only view the more general areas if they so choose?
Honestly, I think the connection to fibs is what drives this place. If someone wants general bg stuff, with plenty of random noise thrown in, r.g.b. is already serving that purpose.

I think more integration with fibs is the better direction, not more general bg, but its' your toy. have fun.

[burper]

what do you mean by all-in-one signup? sounds like more fibs-specific to me, not more general.
do you mean new account registration here would cause new account registration on fibs?

some interesting possibilities there. recovering passwords for one. added legitimacy for some perhaps as well. adding profile info. stats...

existing users could use a separate page here to change their password on fibs and get the same features without losing the experience and rating they have now.

link in the fibsleagammon users after that

[webrunner]

I am not trying top ressurrect bg-world, but just add extra stuff to fibsboard. Fibsboard wil remain fibsboard mainly, just with some extra added to it wich might help new fibs users to play.

Even fibscheck will not prevent spam, only strict moderation will. FB has a interesting pagerank. If spammers want to post here theyu will take the registration for fibs hurdle too. That is why i will assign at least one more moderator. Also i will be checking  in more often (was once a day, will be 2 or 3 times a day).

[burper]

extra "stuff" sounds good. you need to define it a bit better. is it mainly for new users then, or more general? what is driving this for you? you need higher membership here? an altruistic desire to help strangers get started in the exciting world of backgammon?

the fibs check did help with the random spam, which is a different problem than keeping fibsters in line. this particular type of spamming is advertising-related and probably automated now in a way that can recognize and deal with a variety of typical forum software. the trick would be to customize your account registration in a way they won't bother to crack, i.e. a fibs-check.

if you are trying to get me to be a moderator here so i can help delete that stuff when i see it, no thanks. deleting and editting other fibsters posts sounds cool though.

[webrunner]

Sorry it it seemed that way but i wasn't trying to get you to be a moderator.

I will try to add captcha checking to prevent spambots from registering.

And you ask me for my motives. I just want every site that i have be a big success as possible. And yes , more members is of course better for fibsboard, fibs and of course for me. I do get some revenue from the sponsors but they hardly make up for the time i spend and traffic this site generates.

So mainly it is because i still like FB, as one of my oldest existing websites and Fibs as the first community i ever joined. Call me sentimental  

[burper]

So would you give me a moderatorship if I asked?
Who are the other candidates? I would assume it would have to be someone who cares enough to visit this place quite often and posts a lot. Assuming number of posts is a good measure of that, here is the list of high-posters that are not already mods:
-diane (its' a sin she is not a mod, and i think you would drive the short term success of this place way up because people would love to keep track of the daily mod-battle with socksey)
-burper (a shoe-in!)
-alef (seems to be long gone)
-adamosad (used to be fairly active until last March)
-PortWine (has become unregistered)
-MadMatt (gone? hasn't posted since March)
-vegas_vic (great candidate, you should ask him. i would guess not interested though, and actually ahead of MadMatt if you count posts made with "vegasvic")
-don (i'll second any nomination, but he seems to be unregistered now)

Other notables: Shades (not around since March), dorbel (his inter-nation tourney forum is a huge success), Hardy_whv (seems quite active and enthusiastic lately), lewscannon (if you could enlist the king, it would be a major coupe for FB!!!), NIHILIST (who knew he was an all-time high-poster? I would have to second any nomination here as well), resh_lakish (I would second any nomination here as well), tryout (not around since Jan), tucsonAZ (only interested in 3dfibs)

I think Hardy_whv is your best choice, so let me be the first to nominate him. Seconds? Other than that, diane and I seem to be the only ones to stick it out with you through thick and thin, and there is something to be said about that.

captcha sucks for two reasons:

1) people are willing to put up with typing in strings they see in an image once during registration, but for EVERY form submit (forum posting)? I think you will drive down posting here if you do that, but that is my opinion. consider how some forums have a quick-reply option. you need to drive ease-of-use up, not down.

2) if it is a general purpose solution used by many others, it will get cracked. hell, i can imagine a job where all you do is get images forwarded to you for deciphering all day long. the more popular it gets, the more likely cracked.

Don't misunderstand me, I wasn't questioning your motives to be critical. You were asking for ideas, so I wanted to understand the framework of the question. I don't want to suggest stuff just to have you shut it down. Otherwise, I would volunteer my AntiGagBot, or ShoutLogger etc... As it is, you don't comment directly on the ideas I have put forth so far, e.g. geolocation.

[webrunner]

Geolocation is a standard addon on one of the applications so that is indeed interesting and i will take that idea with me in the decision. I have had several offers for creating content (general BG content) so i guess that won't be a problem.

Moderation is not a fun job, and moderators must be as unbiased as possible. Furtheremore it would be someone with an impeccable reputation. Unfortunately a few of the members you suggested don't apply for that. Therefore i will not ask them to be a Moderator.

It is fun to know tho that the person that despises Fibsboard the most, is one of the top posters, Keep it up NIHI! For someone that never ever made one positive remark about FB you sure post a lot.

I will probably select extra moderators myself and will talk with Socksey about this first since she is part of the team too.

Burper, i really appreciate all the time and effort you are putting in to this. I hope we can make this more successful then last time we tried to "work" together
I am sure we will...

The fact is that all things will take quite a long time to create and evolve. I have come a long way since i created Fibsboard, owning communities and forums with over a million posts. Testing and planning is everything.

Captcha would anly be activated with registration, not with every post.

[burper]

You are certainly getting better at diplomacy and side-stepping issues.

It is interesting to here you confirm my implication that the vast majority of your audience isnot fit to moderate itself. I will be interested in seeing who you come up with.

Would you consider unbanning don, or is he only banned for a fixed period of time? I believe he probably drives up your page hits. Another idea: you should post the archives of the old secret forums. They are a gold mine!

Testing and planning are not everything. Time to market is just as important, but perhaps only with rev 1.

As far as working together, I might be up for some of that. Thus my questions. Let me ask another specific one: would you consider starting to archive http://fibs.com/savedgames/list.html on a daily basis and making it web accessible? You must have wget or an equivalent handy, and cron. All you would need to do is datestamp and store it and provide a link to it. This would be a trivial way to add value to fibs for the community. I'll post a cron script if you need it, but you probably don't.

So if you started doing this, my contribution would be scripts to operate on this data, creating further value. Perhaps it would make sense to convert it to sql so that web scripts could do the viewing. All that could be worked out later.

[webrunner]

If you could give me a reason why this archive would be interesting for the fibs community and an example of what you mean with "operate on this data, creating further value" i would defenately consider something like that.

I see you still like to take discussions to the edge, that's ok by me.  As for the hidden forum, i don't know what you are implying and why you think making them public is a gold mine but that will never happen.

[burper]

I explained the value in great detail in a recent post in the repbot forum (or some place like that).

Keeping it as simple as I can:
-assume you had started at the beginning of this month and were succesful in downloading every day
-assume socksey appeared on my list of savedgames each even-numbered day
-assume today is the 6th day of the month
-you would know socksey and I had started 3 separate matches on those days, and know we had completed the first 2

that first simple example is only the beginning. go read that thread:
http://www.fibsboard.com/index.php?showtopic=1303&hl=

[webrunner]

OK, there you go.

Please send me two things:
1. An SQL file that creates the MySql tables
2. a script that gets the list and puts it into these tables.

I will set up a cronjob for it and create the mysql database.
If you want to create pages with data (results) send them to postmaster@fibsboard.com in PHP so i can put them on the webserver.
Make configurables parameters in the script for database name, user and password please, that saves me a lot of work.

[burper]

great. i'll get to work on it.

assume you imply that storing and making accessible the html copies is out of the question?
i.e. todays' http://fibs.com/savedgames/list.html might become http://www.fibsboard.com/savedgames/2006.08.06.html ?

i hadn't been thinking about sql related to this project before, so give me a few days to get the schema reasonable. very hard to change later.

[webrunner]

great. i'll get to work on it.

assume you imply that storing and making accessible the html copies is out of the question?
i.e. todays' http://fibs.com/savedgames/list.html might become http://www.fibsboard.com/savedgames/2006.08.06.html ?

i hadn't been thinking about sql related to this project before, so give me a few days to get the schema reasonable. very hard to change later.

I prefer not to put the archive up like that since it will produce an insane amount of traffic by all the bots people will create

Take your time, i need it too

[burper]

I understand the bot concern. Still, it might be good to start collecting the raw data even if you don't make it web accessible. That way we have some real data to work with when we get started. The information only becomes valuable over time, but if we have the raw stuff, we can pay it forward.

I am doing this at this end, but the data might mean more coming from you.

Here is a script you could call from crontab:

Code Select Expand


WORKDIR=.../savedgames/archives
TODAY=`date +%Y.%m.%d`
TODAYFILE=${WORKDIR}/GET.${TODAY}.html
LOGFILE=${TODAYFILE}.log

moddate() {
   grep "Last-Modified:" $1 \
       |awk '{print $5$4$3;}' \
       |sed \
           -e s/Jan/.01./ \
           -e s/Feb/.02./ \
           -e s/Mar/.03./ \
           -e s/Apr/.04./ \
           -e s/May/.05./ \
           -e s/Jun/.06./ \
           -e s/Jul/.07./ \
           -e s/Aug/.08./ \
           -e s/Sep/.09./ \
           -e s/Oct/.10./ \
           -e s/Nov/.11./ \
           -e s/Dec/.12./
}

wget \
   --append-output="${LOGFILE}" \
   --verbose \
   --no-cache \
   --timestamping \
   --server-response \
   --output-document=${TODAYFILE} \
   http://www.fibs.com/savedgames/list.html

MODDATE=`moddate $LOGFILE`
MODFILE=${WORKDIR}/LastModified.${MODDATE}.html
LOGMODFILE=${MODFILE}.log
mv ${TODAYFILE} ${MODFILE}
mv ${LOGFILE} ${LOGMODFILE}

exit 0



[socksey]

Are we getting off-track here?  Is the update to Fibsboard still in progress?

socksey



"All you have to do is go down to the bottom of your swimming pool and hold your breath." - David Miller, US DOE spokesperson, on protecting yourself from nuclear radiation