A simple log file watcher.
The application requires Node.js runtime environment.
In development environment, the version of Node.js is v11.10.0
.
You can download the code with:
$ git clone https://github.com/seokbeomKim/AbuseIPscanner
Then, install dependencies with npm
or yarn
.
Now we can compile typescript codes with
$ node_modules/typescript/bin/tsc
# Copy a configuration file
$ cp .config.json.example ~/.abuseipscanners
# Run the application
$ node main
There are few arguments available. You can check with node main -h
.
The example of configuration: .config.json.example
{
"files": [
{
"filepath": "./logs",
"rules": [
"apache_log",
"test_log"
]
}
],
"dbSite": [
{
"name": "abuseipdb",
"url": "https://www.abuseipdb.com/check/",
"cheerio": "#report-wrapper > div:nth-child(1) > div:nth-child(1) > div:nth-child(1) > h3:nth-child(1)",
"suspicious": "was found",
"method": "get"
},
{
"name": "ipvoid.com",
"url": "https://www.ipvoid.com/ip-blacklist-check/",
"cheerio": ".table-striped .label-success",
"suspicious": "blacklisted",
"method": "post",
"payload": "ip"
}
],
"rules": [
{
"name": "Apache access log",
"id": "apache_log",
"token": " ",
"index": "0",
"regex_match": "log_1",
"commands": [
"./test/echo.sh",
"./test/logger.sh"
]
},
{
"name": "Test access log",
"id": "test_log",
"token": "-",
"index": "2",
"regex_match": "_2",
"commands": []
}
]
}
Set a file (directory or regular file) path to watch.
{
"files": [
{
"filepath": "./logs",
"rules": [
"apache_log",
"test_log"
]
}
],
}
Set a rule to parse log file.
{
"rules": [
{
"name": "Apache access log",
"id": "apache_log",
"token": " ",
"index": "0",
"regex_match": "log_1",
"commands": [
"./test/echo.sh",
"./test/logger.sh"
]
},
{
"name": "Test access log",
"id": "test_log",
"token": "-",
"index": "2",
"regex_match": "_2",
"commands": []
}
]
}
For example, if the log file has a form as following -
200.206.200.179 - - [05/May/2019:00:58:11 +0900] "GET / HTTP/1.1" 302 -
200.206.200.179 - - [05/May/2019:00:58:11 +0900] "GET / HTTP/1.1" 302 -
103.73.157.162 - - [05/May/2019:01:28:20 +0900] "GET /xmlrpc.php HTTP/1.1" 404 1015
103.73.157.162 - - [05/May/2019:01:28:21 +0900] "HEAD /xmlrpc.php HTTP/1.1" 404 -
the line can be splitted with token (empty space)
.
In the case, we can find the ip address from splited items with index 0(zero)
.
-
id: unique rule id
-
token: token for line splitting
-
index: index of ip address from splitted line
-
regex_match: a regular expression to match filename. If you have a
files
rule as following:"files": [ { "filepath": "./logs", "rules": [ "apache_log", "test_log" ] } ],
there might be multiple files like
./logs/log_file_type_1
,./logs/log-file_type_2
, ... and so on. In that case, you can set a rule with regular expression of filename as:{ "name": "Test access log", "id": "test_log", "token": "-", "index": "2", "regex_match": "_type_2", "commands": [] }
-
commands: run to commands
When the application finds that the IP address has been reported to database sites, it executes the commands what you configured.
There are many sites to share suspicious IP address such as abuseipdb.com, ipvoid.com, ... and so on.
The application mainly use crawling method(with cheerio)
to determine if the IP address is suspicious.
It is not easy to figure out the exact rule for that, but if you want to add some sites, you can add it to configuration file.
"dbSite": [
{
"name": "abuseipdb",
"url": "https://www.abuseipdb.com/check/",
"cheerio": "#report-wrapper > div:nth-child(1) > div:nth-child(1) > div:nth-child(1) > h3:nth-child(1)",
"suspicious": "was found",
"method": "get"
},
{
"name": "ipvoid.com",
"url": "https://www.ipvoid.com/ip-blacklist-check/",
"cheerio": ".table-striped .label-success",
"suspicious": "blacklisted",
"method": "post",
"payload": "ip"
}
],
김석범(Sukbeom Kim), [email protected]
Licensed to Apache 2.0 License Copyright 2019 Sukbeom Kim