Steve's Chain Gang for Processing Log Files

Steve Ivy has created and documented a Linux-based log-file processing system for use with Conversant sites.

The short version:

1. Conversant mails his log files to him over night
2. cron tells fetchmail to retrieve the messages with the logfile attachments
3. Procmail sorts the mail (he receives log files for more than one site)
4. A python script pulls the attachment out of the email
5. cron runs analog to process the log files

Nice job, Steve!

My own system is much simpler, but not so automated. (I'm not that worried about my stats most of the time, except on recent posts like last night's mostly ignore essay on software.) My regular mail client downloads the log-email with all my other email, then I select them and run a script to extract the log files to separate folders for each site. If I feel like looking at the stats for any of the sites I run Analog manually, though lately I haven't been bothering (the logs will be there later if I ever need to take a look).

[Top]