OSEC

Neohapsis is currently accepting applications for employment. For more information, please visit our website www.neohapsis.com or email hr@neohapsis.com
 
Re: extracting IPs from mail log

From: Len Conrad (LConradGo2France.com)
Date: Sat Oct 02 2004 - 17:52:28 CDT


here's a more complete one, grabs IPs and Class C's:

This one, "dictionary", harvests both single IPs and Class C's without PTR
that have sent above a threshold of unknown recipient rejects
today. Adjust the awk "if" sub-statements up or down to be less/more
aggressive.

Run the mta_clients_dict.map right after your "reject unknown recipient"
check, so PTR-less attackers who have been repeatedly rejected for sending
to unknown recipients are permanently blocked if they have the dumbss luck
to try known recipients.

Use this harvesting script as a template for other "bad" behaviors ("too
many errors after ...", "too many RHS rejects", "too many discos for
exceeding hard error limit", "too frequent connections", etc).

/usr/local/bin/mta_clients_dict.sh:

#!/bin/sh

LOGFILE=/var/log/maillog

PFSYSLOGNAME=`/usr/sbin/postconf -h syslog_name`
IMGLABEL=`hostname | cut -d "." -f1`

MATCHLINE="... .. ..:..:.. $IMGLABEL $PFSYSLOGNAME\/smtpd"

cp /dev/null /var/tmp/mta_clients_dict_raw.txt

egrep -i "$MATCHLINE.*from unknown\[.*User unknown in relay recipient
table" $LOGFILE |\
   awk '{ print $10 }' |\
   cut -d "[" -f 2 |\
   cut -d "]" -f 1 |\
   sort -f \
> /var/tmp/mta_clients_dict_raw.txt

sort -f < /var/tmp/mta_clients_dict_raw.txt >
/var/tmp/mta_clients_dict_raw.txt.sort
mv /var/tmp/mta_clients_dict_raw.txt.sort /var/tmp/mta_clients_dict_raw.txt

#dict by ip

uniq -ci < /var/tmp/mta_clients_dict_raw.txt |\
   tr -s " " |\
   sed s/'^ '/""/g |\
   awk '{ if ( $1 > 5 ) print $2" 554 ACL mta_clients_dict_ip"}' \
> /var/tmp/mta_clients_dict_ip.map

#get the dict by classc
uniq -i < /var/tmp/mta_clients_dict_raw.txt |\
   cut -d "." -f 1-3 |\
   uniq -ic | \
   awk '{ if ( $1 > 5 ) print $2" 554 ACL mta_clients_dict_classc"}' \
> /var/tmp/mta_clients_dict_classc.map

cat /var/tmp/mta_clients_dict_ip.map /var/tmp/mta_clients_dict_classc.map
/etc/postfix/mta_clients_dict.map |\
   sort -f | uniq -i \
> /var/tmp/mta_clients_dict.map

cp /var/tmp/mta_clients_dict.map /etc/postfix/mta_clients_dict.map

/usr/sbin/postmap /etc/postfix/mta_clients_dict.map

exit 0

===================

As always, run a new filter in warn_if_reject mode so you can see how it
works. ime, it works very damn well:

mx1# less /var/tmp/spam-stats.rpt

       1 ACL recipientrecipient.domain
       3 SMTP invalid recipientrecipient.domain
       5 SMTP Exceeded Hard Error Limit after END-OF-MESSAGE
       5 ACL provider PTR and ccTLD sender domain
       6 ETRN Mail theft attempt
      10 ACL provider PTR and ccTLD HELO
      13 DNS no A/MX for recipient.domain
      19 ACL SAV: new verification in progress
      41 ACL helo_hostnames
      46 DNS timeout for MTA PTR hostname (forged sender.domain)
      54 ACL to_local_recipients unknown recipient
      66 SMTP Exceeded Hard Error Limit after RSET
      73 SMTP Invalid HELO hostname
     229 ACL from_senders_bw
     286 DNS nxdomain for MTA PTR hostname (forged sender.domain)
     396 SMTP Exceeded Hard Error Limit after CONNECT
     442 RBL dnsbl.njabl.org
     477 ACL unk PTR and ccTLD
     595 RBL blackhole.securitysage.com
     650 ACL MAIL FROM: bigISP forged
     653 ACL from_senders_imgfx
     724 SMTP unqualified HELO hostname
     875 RBL block.rhs.mailpolice.com
    1034 ACL SAV: unverifiable sender address
    1046 SMTP invalid sendersender.domain
    1100 SMTP unauthorized pipelining
    1361 ACL SAV: undeliverable sender address
    2015 SMTP Exceeded Hard Error Limit after MAIL
    2316 RBL list.dsbl.org
    2468 RBL bl.spamcop.net
    2469 ACL MAIL FROM: ccTLD from unknown PTR
    3709 ACL No PTR for big ISP HELO hostname
    5380 RBL sbl.spamhaus.org
    5460 ACL greylist initial reject
    6121 RBL cbl.abuseat.org
    6417 ACL mailpolice sender.domain
    6919 DNS no A/MX for sender.domain
    7209 ACL unauthorized relay
   12701 SMTP HELO hostname is IP
   14130 ACL too many SHEL rejects
   46503 SMTP Exceeded Hard Error Limit after DATA
   87680 ACL mta_clients_bw
  107875 ACL mta_clients_dict <<<<<<<<<<<<<<
  528401 SMTP Exceeded Hard Error Limit after RCPT
1313381 ACL to_relay_recipients unknown recipient
==================
2171364 TOTAL (for 16 hours Saturday)

... for .map file with these items:

mta_clients_dict.map ClassC's : 4908
mta_clients_dict.map IP's : 205306

... and growing by the hour.

My policy is "If you do not have a PTR, you do have one foot in my hell.
Wanna try for the second foot?"

Len

_____________________________________________________________________
http://IMGate.MEIway.com : free anti-spam gateway, runs on 1000's of sites