Apache-logger

General

The purpose of this tool is Detecting the Attacker from Logs using Machine Learning

This is something I have seen in many big websites specially with DDOS protection. Here is the scenario, what if you try to launch a DDOS attack on a website after some time you will see that the site is not reachable from your connection anymore. This is because of DDOS protection on that website. So, I thought why not try to make this thing.

Technologies Used

• Jenkins
• Machine Learning
• RHEL 8
• NodeJs
• Python

Scenario description

The tasks in a sample scenario will take place in the following order:

1. The Apache-logger will keep a log of the information about the clients hits or requests to the server.

2. The generated log with the client's data will be used to identify unusual patterns. For example, a client sending a request repeatedly. For this purpose, we can use here clustering to make clusters of different patterns of client request and to identify which cluster of client requests can cause some security and performance issues in the server.

3. If any unusual pattern is identified, Jenkins can be used to perform certain tasks. For example, run some command to block that IP commiting the action.

Get ready to start the ride then!

Demo

Note: Everything is done on RHEL 8 which is installed on Virtualbox

This is what my Jenkins looks like with all the tasks.

I have divided the task into multiple Jenkins jobs for Ease

Job1 -- Copy logs

This job will copy the logs of httpd from /var/log/httpd/access_log to different destination on system because we cannot disturb the original log file.

Job 2 -- send data from git

This job will pull the github repository and send it to the destination folder here the folder is at /home/anant/logger

Special mention to AMAN JAGDEV for helping me out with nodejs code. Here you can find his github repo

Job-3 create csv

This job is using Nodejs and sorting the log file into csv file using REGEX.

You will find the code snippet in the github repo.

Job-4 CSV label

Actually I forgot to label the columns in CSV file so I made this program Special thanks to RITIK AGARWAL for helping me out with this at the last hour.

Job-5 Find the attacker

This is where we are doing the machine learning magic

I would suggest to install python2 and pandas as well as sklearn to perform this step and use pip2 for installation.

Job-6 Reading attacker ip

Here we will read the attacker Ip address and send the data to my Email.

Note: Mail will be triggered in Post build Action.

Job-7 Saving-the-world

Here the system will block the malicious IP address

Useful commands

------------------------> To Block An IP Address <------------------------

#sudo iptables -A INPUT -s <IP Address> -j DROP

------------------------> To Unblock An IP Address <----------------------

#sudo iptables -D INPUT -s <IP Address> -j DROP

Testing the enviroment

To test the environment you can use [HULK DDos Tool for attacking] (https://allabouttesting.org/hulk-ddos-tool-complete-installation-usage-with-examples/#:~:text=HULK%20is%20a%20Denial%20of,network%20devices%20like%20a%20firewall.)

HULK is a Denial of Service (DoS**)** tool used to attack web servers by generating volumes of unique and obfuscated traffic. HULK's generated traffic also bypasses caching engines and hits the server's direct resource pool. This tool is used to test network devices like a firewall.

Note: Change your environment to python2

#python hulk.py <IP Address>

Python hulk.py http://192.168.0.110/

Mine worked with Python hulk.py http://192.168.0.110/