• Frame 1

Intrusion Detection System (IDS)

I started working on our IDS back in 2010 to help stop spammers and hackers from reaching our server. It's main software is Fail2ban which is a log file scanner that allows you to take actions, such as banning spammers, when some text is matched in a log. For those who want to setup a similar system on their own Linux based server I've provided my files with a brief description. It's not a HOWTO but there's plenty of resources for you to work it out yourself.

Resources

I presume you're already running the following types of services that Fail2ban can analyze:

  • Iptables to create firewall rules
  • Apache Web Server
  • MySQL database
  • Postfix Mail Server using spam databases like www.barracudacentral.org
  • Clam AV antivirus
  • PAM (Pluggable Authentication Modules for Linux)
  • E107 or Wordpress CMS (Content Management System)

These are the resources you'll need to get an IDS working that looks like this:

  • A Google Maps API key so you can draw the map
  • Fail2ban (On Fedora Linux you install it with: dnf install fail2ban)
  • Fail2sql so you can store the bans in a MySQL database and use a GOIP database to locate where the servers are
  • A copy of my IDS web files so you can see how I make the web page (Updated 21/06/2016)
  • A copy of my fail2ban configuration files so you can see how I've written my rules (Updated 21/06/2016)
  • A copy of my fail2sql files so you can see how I've configured it to use GEOIP database (Updated 21/06/2016)
  • You might also need a copy of my sql (Updated 21/10/2016)
  • Add this to your root cron jobs to regularly update the GEOIP database:
    @monthly /usr/local/fail2sql/fail2sql -u 1>&1 > /dev/null

Setup

Once you have fail2ban working you're going to want to create your own rules. Though fail2ban comes with lots of great sample scripts the real power comes from creating your own. First, to keep things simple, only create one rule in /etc/fail2ban/jail.conf.

/etc/fail2ban/jail.conf

For example, here's my rule for scanning the mail server's logs:

[sp-postfix]
enabled = true
action = iptables-multiport[name=sp-postfix, port="110,143,995,993,25,465,587,22,21,20", protocol=tcp]
filter = sp-postfix
logpath = /var/log/maillog
maxretry = 2
bantime = 864000
findtime = 7200

The "action =" bit means: Use /etc/fail2ban/action.d/iptables-multiport.conf actions to block ports 110,143,995,993,25,465,587,22,21,20 when a rule in the sp-postfix filter is matched.

The "bantime = 864000" means: Ban the server that was matched by the sp-postfix filter for 864000 seconds (10 days)

In /etc/fail2ban/jail.conf you can also put in a rule to exclude IPs from being banned by any of the filters such as your office IP. I exclude all NZ ip addresses and large companies like Google. Here's my rule:

ignoreip = 127.0.0.1 134.252.211.130 104.47.116.0/24 114.23.118.8 163.47.0.0/16 50.3.75.252 190.2.0.0/16 103.9.41.0/24 43.245.172.0/24 115.69.190.0/24 103.26.17.0/24 202.174.180.0/24 103.225.6.0/24 90.16.255.49 60.225.133.0/24 125.7.87.39 130.211.252.134 10.240.1.0/24 103.232.108.0/24 203.94.0.0/16 163.47.106.0/24 163.47.228.0/24 103.241.57.0/24 190.2.22.0/24 181.15.110.0/24 118.93.0.0 5.10.84.200/29 5.10.119.152/29 5.132.175.0/24 5.224.175.0/24 5.225.175.0/24 14.1.32.0/20 14.1.48.0/21 14.1.56.0/23 14.1.58.0/24 14.1.60.0/22 14.1.64.0/19 14.128.4.0/22 27.96.64.0/22 27.110.120.0/21 27.111.12.0/22 27.121.96.0/22 27.123.20.0/22 27.252.0.0/16 31.201.0.204/30 31.201.175.0/24 37.58.67.208/28 37.58.67.232/29 37.58.76.128/27 37.58.78.96/27 37.58.97.104/29 37.58.97.112/29 37.222.175.0/24 37.223.175.0/24 41.79.175.0/24 46.36.194.181/32 46.36.194.182/31 46.36.194.184/30 46.36.194.188/31 46.36.194.190/32 46.36.198.36/30 46.36.198.40/32 46.36.202.61/32 47.72.0.0/16 49.0.28.0/22 49.50.192.0/19 49.50.224.0/21 49.50.240.0/20 49.143.228.0/22 49.224.0.0/14 57.70.128.0/19 58.28.0.0/16 58.84.224.0/20 60.234.0.0/16 61.28.206.0/24 63.240.124.80/29 64.34.137.0/27 64.45.25.160/28 64.106.143.40/29 64.106.143.48/31 64.106.156.50/31 64.106.156.52/30 64.106.156.56/29 64.106.156.64/28 64.106.174.10/31 64.106.174.12/30 64.106.174.16/30 64.106.229.70/31 64.106.229.72/29 64.187.102.120/29 65.17.219.80/29 65.17.219.88/31 65.17.234.40/29 65.17.234.48/28 65.17.234.64/28 65.99.197.120/29 65.99.197.192/29 66.219.29.0/27 67.21.22.200/30 67.23.187.0/27 67.201.33.160/28 67.215.3.104/29 67.222.145.198/31 67.222.145.200/30 67.222.145.204/31 69.61.33.152/29 69.90.77.184/29 70.33.241.16/28 72.9.147.18/31 72.9.147.20/30 72.9.147.24/31 72.13.82.112/29 72.29.27.0/27 72.35.5.112/28 76.74.223.240/28 78.24.205.112/28 80.190.227.128/28 80.190.227.152/29 82.113.134.160/27 89.187.105.192/28 92.61.245.128/28 92.61.245.144/29 96.8.113.160/29 101.98.0.0/16 101.100.0.0/16 101.110.112.0/22 103.2.120.0/22 103.2.140.0/22 103.3.232.0/22 103.4.121.0/24 103.4.136.0/22 103.4.152.0/22 103.5.28.0/24 103.5.71.0/24 103.5.72.0/22 103.5.80.0/22 103.5.108.0/22 103.5.156.0/22 103.5.224.0/22 103.6.28.0/22 103.6.64.0/22 103.6.89.0/24 103.6.144.0/22 103.6.212.0/23 103.6.232.0/22 103.7.96.0/23 103.7.168.0/22 103.8.16.0/22 103.8.72.0/23 103.8.80.0/22 103.8.140.0/22 103.8.232.0/22 103.8.252.0/22 103.9.40.0/22 103.9.106.0/24 103.9.216.0/22 103.9.236.0/22 103.10.44.0/22 103.10.233.0/24 103.11.124.0/22 103.11.178.0/24 103.11.240.0/22 103.13.8.0/22 103.13.39.0/24 103.13.116.0/22 103.14.3.0/24 103.14.40.0/22 103.14.68.0/27 103.14.68.40/29 103.14.68.48/28 103.14.68.64/26 103.14.68.128/25 103.14.69.0/24 103.14.70.0/23 103.14.140.0/22 103.14.168.0/22 103.14.216.0/22 103.15.75.0/24 103.15.126.0/23 103.15.152.0/22 103.16.20.0/22 103.16.172.0/22 103.16.180.0/23 103.18.56.0/22 103.18.118.0/23 103.18.136.0/22 103.18.200.0/22 103.19.4.0/22 103.19.10.0/23 103.19.36.0/24 103.20.88.0/24 103.20.136.0/22 103.21.172.0/22 103.21.193.0/24 103.21.194.0/24 103.22.234.0/23 103.23.16.0/22 103.23.142.0/23 103.23.212.0/24 103.29.30.0/23 103.29.90.0/23 103.244.50.0/24 103.244.180.0/22 103.244.212.0/22 103.245.172.0/22 103.246.52.0/22 103.246.180.0/22 103.246.249.0/24 103.246.250.0/23 103.247.56.0/22 103.247.72.0/22 103.247.152.0/22 103.247.192.0/22 103.247.247.0/24 110.34.48.0/22 110.44.16.0/22 110.92.16.0/22 110.173.160.0/20 110.232.144.0/22 111.65.224.0/20 111.69.0.0/16 112.109.64.0/20 112.109.80.0/21 113.21.224.0/22 113.130.120.0/22 113.197.64.0/22 113.197.96.0/22 114.23.0.0/16 114.31.208.0/21 114.110.32.0/21 114.134.0.0/20 114.134.160.0/20 114.134.176.0/21 115.126.128.0/21 115.187.96.0/19 115.188.0.0/15 116.12.56.0/21 116.66.240.0/21 116.89.224.0/20 116.90.76.0/22 116.90.128.0/20 116.93.128.0/20 116.193.144.0/21 116.199.208.0/20 116.251.128.0/18 116.251.192.0/21 116.251.200.0/22 117.18.80.0/21 117.53.156.0/22 117.104.176.0/21 117.120.32.0/21 118.67.192.0/21 118.82.128.0/17 118.90.0.0/16 118.92.0.0/15 118.148.0.0/15 119.15.0.0/19 119.47.112.0/20 119.224.0.0/17 119.224.128.0/20 119.235.32.0/21 120.89.80.0/21 120.136.0.0/21 120.136.48.0/20 120.138.16.0/20 121.72.0.0/14 121.79.192.0/18 121.90.0.0/16 121.98.0.0/15 122.56.0.0/13 122.102.109.0/24 122.252.152.0/21 122.252.184.0/21 123.100.64.0/18 123.108.144.0/21 123.255.0.0/18 124.6.192.0/19 124.150.160.0/20 124.155.224.0/20 124.157.64.0/18 124.197.0.0/18 124.198.128.0/17 124.248.128.0/20 125.236.0.0/14 130.123.0.0/16 130.195.0.0/16 130.216.0.0/15 131.203.0.0/16 132.181.0.0/16 138.75.0.0/16 138.211.0.0/16 138.235.0.0/16 139.80.0.0/16 140.200.0.0/16 141.255.159.160/28 143.96.0.0/16 144.66.0.0/16 146.171.0.0/16 147.158.0.0/16 150.206.0.0/16 153.111.0.0/16 155.32.0.0/16 155.59.0.0/16 156.13.0.0/16 156.59.0.0/16 156.62.0.0/16 156.69.0.0/16 156.149.0.0/16 159.117.0.0/16 159.172.204.0/24 159.253.137.128/29 159.253.142.104/29 159.253.142.112/29 160.4.0.0/16 161.29.0.0/16 161.65.0.0/16 161.66.0.0/16 162.112.0.0/16 163.7.0.0/16 165.84.0.0/16 165.205.0.0/16 166.65.0.0/16 166.83.0.0/16 166.179.0.0/16 167.179.0.0/16 173.231.217.0/25 173.240.103.144/28 173.255.3.152/29 174.136.99.80/28 175.111.100.0/22 175.176.216.0/22 176.67.84.72/30 178.174.21.128/25 178.236.226.204/30 180.189.192.0/19 180.210.208.0/22 180.222.64.0/21 180.222.160.0/20 182.16.152.0/22 182.23.176.0/21 182.48.128.0/19 182.50.160.0/22 182.54.160.0/20 182.154.0.0/16 182.236.120.0/21 185.19.212.0/22 188.130.250.160/30 192.54.130.0/24 192.73.21.0/24 192.84.253.0/24 192.88.85.0/24 192.88.190.0/24 192.94.169.0/24 192.100.53.0/24 192.101.16.0/24 192.102.92.0/24 192.105.10.0/24 192.107.113.0/24 192.107.171.0/24 192.107.172.0/24 192.111.102.0/24 192.122.171.0/24 192.122.180.0/24 192.124.160.0/24 192.133.31.0/24 192.138.251.0/24 192.146.133.0/24 192.146.150.0/24 192.146.218.0/23 192.148.221.0/24 192.150.218.0/23 192.153.251.0/24 192.156.165.0/24 192.156.225.0/24 192.161.128.0/23 192.173.16.0/21 192.173.24.0/24 192.188.43.0/24 192.188.71.0/24 192.188.185.0/24 192.188.241.0/24 192.189.34.0/24 192.190.108.0/24 192.195.12.0/23 192.195.14.0/24 192.195.219.0/24 192.200.64.0/20 192.203.154.0/24 192.203.216.0/24 192.206.152.0/22 192.206.156.0/23 192.207.143.0/24 192.231.15.0/24 192.245.36.0/24 192.245.185.0/24 194.117.104.16/30 194.117.104.24/30 194.117.104.32/30 194.117.104.128/30 194.117.104.156/30 194.117.104.224/30 195.112.176.20/30 195.112.176.36/30 195.112.176.48/30 195.112.176.64/30 195.112.176.84/30 195.112.176.148/30 195.112.176.176/30 195.112.176.192/30 195.112.176.236/30 198.27.4.0/22 198.27.8.0/24 198.48.0.0/22 198.51.195.0/24 199.16.204.120/29 199.34.123.16/29 199.38.241.232/29 199.87.52.240/29 199.187.126.152/29 199.241.187.116/30 202.0.32.0/19 202.0.84.0/24 202.0.121.0/24 202.0.124.0/23 202.0.151.0/24 202.1.8.0/21 202.2.4.0/22 202.2.8.0/21 202.3.32.0/22 202.3.40.0/21 202.3.48.0/20 202.3.80.0/20 202.4.16.0/21 202.4.24.0/24 202.6.5.0/24 202.6.84.0/24 202.6.109.0/24 202.6.116.0/23 202.7.0.0/23 202.7.4.0/22 202.7.32.0/20 202.7.48.0/22 202.7.60.0/22 202.8.13.0/24 202.8.44.0/22 202.9.88.0/24 202.12.0.0/24 202.12.3.0/24 202.12.70.0/24 202.12.91.0/24 202.12.104.0/23 202.12.108.0/24 202.12.248.0/21 202.14.16.0/20 202.14.32.0/19 202.14.82.0/23 202.14.84.0/24 202.14.96.0/24 202.14.99.0/24 202.14.100.0/24 202.14.102.0/24 202.14.106.0/23 202.14.140.0/23 202.14.142.0/24 202.14.167.0/24 202.14.168.0/24 202.14.216.0/23 202.14.218.0/24 202.14.228.0/24 202.14.252.0/23 202.14.254.0/24 202.20.0.0/21 202.20.32.0/19 202.20.80.0/24 202.20.91.0/24 202.20.92.0/23 202.20.96.0/23 202.20.102.0/23 202.20.104.0/24 202.20.113.0/24 202.20.121.0/24 202.20.122.0/24 202.21.16.0/20 202.21.64.0/19 202.21.128.0/24 202.21.130.0/24 202.21.133.0/24 202.21.136.0/22 202.21.143.0/24 202.21.155.0/24 202.21.157.0/24 202.22.0.0/21 202.22.16.0/21 202.22.24.0/22 202.22.28.0/23 202.22.30.0/24 202.27.0.0/21 202.27.8.0/24 202.27.32.0/19 202.27.64.0/20 202.27.80.0/21 202.27.88.0/23 202.27.90.0/24 202.27.92.0/22 202.27.96.0/19 202.27.128.0/21 202.27.138.0/23 202.27.140.0/22 202.27.144.0/20 202.27.160.0/19 202.27.192.0/18 202.36.0.0/16 202.37.0.0/18 202.37.64.0/21 202.37.72.0/23 202.37.75.0/24 202.37.76.0/22 202.37.80.0/22 202.37.84.0/23 202.37.88.0/21 202.37.96.0/20 202.37.112.0/21 202.37.121.0/24 202.37.122.0/23 202.37.124.0/22 202.37.128.0/17 202.41.136.0/21 202.44.60.0/22 202.46.160.0/19 202.49.0.0/16 202.50.0.0/19 202.50.32.0/20 202.50.48.0/24 202.50.50.0/23 202.50.56.0/21 202.50.64.0/18 202.50.128.0/18 202.50.192.0/22 202.50.198.0/23 202.50.204.0/22 202.50.208.0/20 202.50.224.0/19 202.52.152.0/24 202.52.155.0/24 202.53.176.0/20 202.55.96.0/20 202.56.32.0/20 202.56.48.0/21 202.59.224.0/24 202.61.2.0/23 202.61.114.0/23 202.62.245.0/24 202.65.32.0/21 202.65.160.0/21 202.68.80.0/20 202.70.130.0/24 202.73.192.0/20 202.74.33.0/24 202.74.38.0/23 202.74.60.0/24 202.74.192.0/19 202.74.224.0/21 202.78.128.0/19 202.78.240.0/21 202.86.96.0/20 202.89.4.0/22 202.89.32.0/21 202.89.40.0/23 202.89.44.0/22 202.89.48.0/20 202.89.128.0/19 202.90.44.0/22 202.90.48.0/22 202.90.56.0/21 202.95.250.0/24 202.124.96.0/19 202.125.110.0/24 202.126.80.0/21 202.126.192.0/20 202.127.8.0/22 202.128.108.0/23 202.129.253.0/24 202.134.48.0/24 202.134.53.0/24 202.137.224.0/24 202.137.240.0/21 202.150.96.0/19 202.154.128.0/19 202.160.48.0/20 202.160.112.0/21 202.162.72.0/23 202.162.176.0/21 202.164.28.0/22 202.169.192.0/19 202.170.160.0/20 202.174.6.0/23 202.174.112.0/21 202.174.160.0/20 202.175.128.0/20 202.180.64.0/18 202.189.160.0/20 202.191.32.0/20 203.5.28.0/24 203.6.246.0/24 203.9.176.0/21 203.20.241.0/24 203.26.179.0/24 203.27.67.0/24 203.30.174.0/24 203.56.32.0/23 203.57.145.0/24 203.79.64.0/18 203.80.60.0/22 203.83.218.0/24 203.84.134.0/23 203.86.192.0/20 203.89.160.0/19 203.92.25.0/24 203.92.213.0/24 203.96.0.0/15 203.98.0.0/18 203.99.32.0/21 203.99.64.0/21 203.99.128.0/23 203.99.132.0/22 203.100.208.0/20 203.109.128.0/17 203.110.0.0/18 203.114.128.0/18 203.118.128.0/18 203.119.56.0/24 203.119.84.0/24 203.143.192.0/20 203.144.32.0/20 203.152.96.0/19 203.160.112.0/20 203.163.71.0/24 203.167.128.0/17 203.170.56.0/24 203.171.4.0/24 203.171.32.0/19 203.171.223.0/24 203.173.128.0/17 203.174.26.0/24 203.174.30.0/23 203.175.112.0/21 203.176.122.0/23 203.176.124.0/22 203.176.154.0/23 203.176.188.0/24 203.184.0.0/18 203.190.208.0/21 203.196.101.0/24 203.196.102.0/24 203.207.60.0/23 203.208.21.0/24 203.211.64.0/18 203.215.142.0/23 203.217.142.0/23 204.62.14.32/27 204.62.14.64/26 204.62.14.128/26 204.62.14.224/27 204.62.15.32/27 204.62.15.64/27 204.62.15.96/28 204.62.15.128/27 204.62.15.200/29 204.62.15.208/28 206.18.162.48/28 206.72.117.148/30 206.72.118.240/29 206.72.123.48/28 206.72.123.240/28 206.72.124.224/27 206.72.126.24/29 206.214.209.82/31 206.214.209.84/31 206.214.210.31/32 206.214.210.32/31 206.214.210.34/32 206.214.211.248/30 206.214.222.38/31 206.214.222.40/31 206.217.134.152/29 207.7.129.8/29 207.209.22.0/24 207.209.83.0/24 209.18.85.190/31 209.18.85.192/29 209.18.106.190/31 209.18.106.192/29 209.18.107.10/31 209.18.107.12/30 209.18.107.16/30 209.28.33.0/24 209.93.181.128/25 209.93.188.0/25 209.162.176.182/31 209.162.176.184/29 209.162.188.213/32 209.162.188.214/31 209.162.188.216/29 209.162.188.224/31 209.162.188.226/32 210.4.208.0/21 210.5.48.0/21 210.7.32.0/20 210.48.0.0/17 210.48.160.0/19 210.54.0.0/15 210.56.32.0/20 210.79.48.0/21 210.86.0.0/17 210.185.0.0/18 210.246.0.0/18 216.19.206.192/26 216.38.51.94/31 216.38.51.96/31 216.38.62.94/31 216.38.62.96/31 216.65.3.224/28 216.65.64.224/28 216.119.137.184/29 216.121.66.240/28 218.100.21.0/24 218.100.23.0/24 218.100.24.0/23 218.100.26.0/24 218.100.56.0/24 218.100.65.0/24 218.100.79.0/24 218.101.0.0/17 218.185.224.0/21 219.88.0.0/15 222.152.0.0/14 223.165.16.0/22 223.165.64.0/20 64.233.160.0/19 66.102.0.0/20 66.249.64.0/19 72.14.192.0/18 74.125.0.0/16 209.85.0.0/16 216.239.32.0/19 66.219.0.0/16 70.42.0.0/16 14.203.0.0/16 103.242.0.0/16 118.93.0.0/16 72.3.207.8

/etc/fail2ban/filter.d/sp-postfix.conf

Next you need to understand your sp-postix filter. Here's how I write my rules:

  1. I get a spam message and have a look at the source code and see that it came from smtp7.ymlpsrvr.com
  2. I then grep that name in my mail server's logs (/var/log/maillog) and see this line:
    Jul 21 16:00:57 vs4-c6 postfix/smtpd[15951]: EC74729A6: client=smtp7.ymlpsrvr.com[185.83.48.16]
  3. To block mail coming from any of ymlpsrvr.com's servers I just write a Regular expression in /etc/fail2ban/filter.d/sp-postfix.conf that looks like this:
[failregex = client_address\=\[\].*ymlpsv.com

What this means is:

  1. Scan every line of /var/log/maillog for the bit that says client=
  2. The \[\] bit after that is the remote server's IP address
  3. .* means keep search forward in the line
  4. If anywhere in the line contains the letters ymlpsl.com then this rule will enact the action in /etc/fail2ban/action.d/iptables-multiport.conf which is to use iptables to ban the server

There are also rules to gnore. I use these ones to make sure I never ban important servers like Google:

ignoreregex = client_address\=\[\].*googlebot
client_address\=\[\].*clear\.net\.nz
client_address\=\[\].*xtra\.co\.nz
client_address\=\[\].*inspire\.net\.nz
client_address\=\[\].*telecom\.co\.nz
client_address\=\[\].*slingshot\.co\.nz
client_address\=\[\].*nzcomms\.co\.nz
client_address\=\[\].*ucol\.ac\.nz
client_address\=\[\].*telstraclear\.net
client_address\=\[\].*sportmanawatu\.org\.nz
client_address\=\[\].*maxnet\.net\.nz
client_address\=\[\].*xnet\.co\.nz
client_address\=\[\].*actrix\.co\.nz
client_address\=\[\].*iconz\.net\.nz
client_address\=\[\].*vf\.net\.nz
client_address\=\[\].*callplus\.net\.nz

That's about it. If you have any questions, leave questions in the blog.