| View previous topic :: View next topic |
| Author |
Message |
Vermyndax
Apprentice
Joined: 13 Dec 2003
Posts: 208
|
 Posted: Thu Jan 22, 2004 2:34 am Post subject: Spamassassin statistics |
 |
|
Hey all... was just wondering if anyone had implemented any scripts or log utilities to generate a report about spamd (Spamassassin) activity. If so, what did you use? I tried sa-stats.pl but it returns all 0's for every stat and I'm tired of trying to figure out why (Googling didn't help... several people asked about it, no one answered).
So... any of you use anything to analyze this and make a nice report? |
|
| Back to top |
|
 |
Jaxom
Tux's lil' helper
Joined: 31 Jan 2003
Posts: 137
|
| Posted: Thu Jan 22, 2004 6:07 am Post subject: |
|
|
I'll chime in here and add a "yes, that would be great" 
I don't know of anything that will do it though, but I sure would like to see how much stuff SA is actually stopping. I know I can see it in MY mail, but that doesn't tell me anything about how much it's helping my users.
_________________
Undisputed Heavyweight Champion. If it's undisputed, WHAT'S ALL THE FIGHTING ABOUT?!?! -- George Carlin |
|
| Back to top |
|
|
Vermyndax
Apprentice
Joined: 13 Dec 2003
Posts: 208
|
| Posted: Thu Jan 22, 2004 1:01 pm Post subject: stats |
|
|
| It would be even better if it was a script that generated an html page as output so I can show it off to the bosses |
|
| Back to top |
|
|
Vermyndax
Apprentice
Joined: 13 Dec 2003
Posts: 208
|
| Posted: Tue Feb 10, 2004 1:19 pm Post subject: solution |
|
|
I did find a solution to this with a sa-stats.pl script. However, now I've switched to amavisd-new with SA and ClamAV, so it kills the sa-stats.pl's abilities to read the log file (since amavisd doesn't allow spamd to write to the log file anymore).
Anyone know of a quickie way to get amavisd to allow spamd to write to the syslog as usual? |
|
| Back to top |
|
|
Torg
n00b
Joined: 21 Oct 2002
Posts: 30
|
| Posted: Tue Feb 10, 2004 8:59 pm Post subject: |
|
|
I wrote my own in sed/awk/bash it's a mess, and someone else could prolly write cleaner code.... but in the effort of open source, here it is:
| Code: |
#!/bin/bash
# Todd's SpamAssassin Stats Script v0.27t
# Much help was gained from O'Reilly's Sed & Awk book, and the Advanced Bash Scripting Guide. VERY good reads for this sort of task.
# File Location Vars:
templog=/spare/logtemp/maillogtrimmed.log
quickstats=/home/httpd/htdocs/logs/quickstats.txt
sourcecode=/home/httpd/htdocs/logs/source.txt
scores=/home/httpd/htdocs/logs/scores.txt
# Main Code:
if [ -e $templog ]; then
rm $templog
fi
cat /var/log/mail.log | sed -e 's/ / /g' | grep "identified" > $templog
cat /var/log/mail.log | sed -e 's/ / /g' | grep "clean" >> $templog
if [ -e $quickstats ]; then
rm $quickstats
fi
echo "Quickstats for Torgbox generated at `date | awk '{print $4 " - " $1 ", " $2 " " $3 " " $6}'`" > $quickstats
echo >> $quickstats
echo "Today's filtered spam:" >> $quickstats
cat $templog | grep "`date | awk '{print $2 " " $3}'`" | grep -c "identified" >> $quickstats
echo >> $quickstats
echo "Today's unfiltered mail:" >> $quickstats
cat $templog | grep "`date | awk '{print $2 " " $3}'`" | grep -c "clean" >> $quickstats
echo >> $quickstats
cat $templog | grep "`date | awk '{print $2 " " $3}'`" |grep "clean" | awk '{print $8}' | sed -e "s/\// /g ; s/(//g; s/)//g" |awk '{print $1}' | grep "-" | sed -e "s/-//g" | awk '{if ($1 > 50) totalbr ++};END {printf ("%s""%d\n\n","Whitelisted mail so far today: ",totalbr)}' >> $quickstats
echo "Today's spams by username:" >> $quickstats
cat $templog | grep identified | grep "`date | awk '{print $2 " " $3}'`" | awk '{print $10}' | sed -e 's/:/ /g' | awk '{print $1}' | sort | uniq -c | sort -r >> $quickstats
echo >> $quickstats
cat $templog | grep "identified" | grep "`date | awk '{print $2 " " $3}'`" | awk '{ sum += $14 };{size = sum / 1048576}; END {printf ("%24s" "%.2f""%2s\n","Spam size so far today: ",size,"Mb")}' >> $quickstats
echo >> $quickstats
echo "Logfile total spams filtered:" >> $quickstats
cat $templog | grep -c "identified" >> $quickstats
echo >> $quickstats
echo "Logfile total unfiltered mail:" >> $quickstats
cat $templog | grep -c "clean" >> $quickstats
echo >> $quickstats
cat $templog | grep "clean" | awk '{print $8}' | sed -e "s/\// /g ; s/(//g; s/)//g" |awk '{print $1}' | grep "-" | sed -e "s/-//g" | awk '{if ($1 > 50) totalbr ++};END {printf ("%s""%d\n\n","Logfile total whitelisted mail: ",totalbr)}' >> $quickstats
echo "Logfile total spams by username:" >> $quickstats
cat $templog | grep identified | awk '{print $10}' | sed -e 's/:/ /g' | awk '{print $1}' | sort | uniq -c | sort -r >> $quickstats
echo >> $quickstats
cat $templog | grep "identified" | awk '{sum += $14};{size = sum / 1048576}; END {printf ("%17s""%.2f""%2s\n","Total spam size: ",size,"Mb")}' >> $quickstats
cat $templog | grep "identified" | awk '{print $8}' | sed -e "s/\// /g ; s/(//g; s/)//g" | awk '{print $1}'| awk '{sum += $1};{total ++};{avg = sum / total};END{printf("%20s""%.1f\n","Average spam score: ",avg)}' >> $quickstats
if [ -e $scores ]; then
rm $scores
fi
echo "Spams sorted by score:" > $scores
cat $templog | grep "identified" | awk '{print $8}' | sed -e 's/\/5.0)//g' | sed -e 's/(//g' | sort | uniq -c | sort -r >> $scores
echo >> $quickstats
echo >> $quickstats
echo "Percentages of spam and non-spam:" >> $quickstats
cat $templog | awk '{ print $6 }' | awk ' /^identified$/ {spam++ total++} /^clean$/ {notspam++ total++} {percentspam = spam / total * 100;percentnot = notspam / total * 100 };END {print percentspam "% SPAM \n" percentnot "% NON-SPAM \n" spam " Spams / " notspam " Non-spams / " total " Total Emails"}' >> $quickstats
if [ -e $sourcecode ]; then
rm $sourcecode
fi
cat /root/makesalog > $sourcecode
|
If you need help making heads or tails of it, just say so... I wrote the whole thing, so I know how it all works.
-Torg |
|
| Back to top |
|
|
Torg
n00b
Joined: 21 Oct 2002
Posts: 30
|
| Posted: Tue Feb 10, 2004 9:10 pm Post subject: |
|
|
output looks like this:
| Code: |
Quickstats for Torgbox generated at 14:27:14 - Tue, Feb 10 2004
Today's filtered spam:
0
Today's unfiltered mail:
0
Whitelisted mail so far today: 0
Today's spams by username:
Spam size so far today: 0.00Mb
Logfile total spams filtered:
4550
Logfile total unfiltered mail:
18225
Logfile total whitelisted mail: 1565
Logfile total spams by username:
4298 safrax
252 torg
Total spam size: 25.99Mb
Average spam score: 29.5
Percentages of spam and non-spam:
19.978% SPAM
80.022% NON-SPAM
4550 Spams / 18225 Non-spams / 22775 Total Emails
|
This is from a smaller install of this script.... I've got it running on a 124 user mailserver at work also. (Little more interesting to read, but has too many usernames in it for me to print it here =P)
Unfortunatly, there isn't much traffic on this example, but you'll get the idea.
-Torg |
|
| Back to top |
|
|
Skywacker
n00b
Joined: 23 Jan 2004
Posts: 72
Location: Missouri
|
| Posted: Fri Mar 19, 2004 8:56 pm Post subject: |
|
|
works great, ty
any of have ideas or know of a program that does the same with some kind of html output over time?
_Skywacker |
|
| Back to top |
|
|
deurk
Apprentice
Joined: 11 Mar 2003
Posts: 190
Location: Earth 1.3
|
| Posted: Thu Jun 02, 2005 9:47 am Post subject: |
|
|
I'll hop in here and bump it once again...
This script is nice but messy and no html output, anyone got new ideas?
Thanks! |
|
| Back to top |
|
|
FutureGuru
n00b
Joined: 10 May 2004
Posts: 29
Location: Tampere, Finland
|
| Posted: Thu Jun 02, 2005 10:19 am Post subject: |
|
|
Have you tried to implement options to the script? So that for example -html option generates new html file and puts all messages to it.
| Code: |
if [ $# == 0]
then
(copy-paste your current script)
else if [ $# == 1 ]
then
if [ $1 == '-html' ]
then
(create file and put all writings to it)
fi
else
echo "Usage: xxxxxxx"
fi
|
I know this is not correct form as I don't have change to check all that, but I hope that this example shows my point.
_________________
Heh, that aint gonna take no more than 15 minutes. - Me 24 hours ago |
|
| Back to top |
|
|
georwell
Guru
Joined: 25 Jun 2003
Posts: 430
Location: Uppsala, Sweden
|
|
| Back to top |
|
|
Dr_Stein
Guru
Joined: 21 Mar 2003
Posts: 303
Location: Mountain View, CA
|
| Posted: Sat Jun 04, 2005 4:30 am Post subject: |
|
|
No, that won't help - it requires mailscanner.
The OP said he's using amavisd-new
mailgraph might work for you..
http://mail.pbp.net/cgi-bin/mailgraph.cgi <-- my installation
It's pretty easy to get going. No ebuild for it, but it only consists of like 2 files anyway.  I also have it running on my mail gateway box, but HTTP isn't allowed to it from the outside. |
|
| Back to top |
|
|
alterself.com
Apprentice
Joined: 13 Nov 2003
Posts: 175
Location: Iowa
|
| Posted: Mon Jun 20, 2005 3:14 pm Post subject: |
|
|
| mailgraph and amavis-stats are rrdtool based nice boss pleasing html based pretty graphs web pages |
|
| Back to top |
|
|
|
Networking & Security
