How to Extract IP Address from Log File

As system administrator, you will be required to keep an eye on the IP addresses accessing your website. The server logs are useful source for this kind of information. They contain IP address, datetime, request type, URL, server response and more for every request received by your server. In this article, we will look at how to extract IP address from log file for Apache and NGINX server.

How to Extract IP Address from Log File

The default location of Apache server file is

/var/log/http/access_log      [For RedHat based systems]
/var/log/apache2/access.log   [For Debian based systems]
/var/log/http-access.log      [For FreeBSD]

The default location for NGINX log file is

/var/log/nginx/access_log   [For RedHat based systems]
/var/log/nginx/access.log   [For Debian based systems]
/var/log/http-access.log    [For FreeBSD]

We will look at different use cases to extract IP addresses. In each of the following commands replace log file path with that of your server.

How to Get IP for each request

You can easily use awk command to print IP address for each request with the following command.

$ awk '{ print $1 }' /var/log/http/access_log

The first column of log file contains IP address. So we tell awk to simply print the first column for us.

How to Get Unique IP

The above command will list IP address for each request which will contain duplicate values. You can pass this output to uniq command to get a unique list of IP addresses accessing your website.

$ awk '{ print $1 }' /var/log/http/access_log | uniq

How to Get Specific IP address

If you are looking for a specific IP address (e.g. 192.34.45.46) in your log file, use grep command instead, as shown.

$ sudo grep "192.34.45.46" /var/log/http/access_log

How to Get most frequent IP addresses

If you need to find the top 10 most frequent IP address accessing your website, use the following awk command. It basically passes a list of all IP addresses to sort & uniq who count hits per IP and sort them in descending order of hits.

$ sudo awk '{ print $1}' /var/log/http/access_log | sort | uniq -c | sort -nr | head -n 10

If you want to regularly run these commands then it is advisable to create a shell script for it. Create a blank shell script with the following command.

$ sudo vi /home/extract_ip.sh

Add the following lines to it.

#!/bin/sh

echo "Unique IP Addresses accessing your site"
sudo awk '{ print $1 }' /var/log/http/access_log | uniq

echo "10 Most Frequent IP addresses accessing your site"
sudo awk '{ print $1}' /var/log/http/access_log | sort | uniq -c | sort -nr | head -n 10

Save and close the file.

Make Shell script executable.

$ sudo chmod +x extract_ip.sh

Run the shell script as shown with the following command.

$ sudo /home/extract_ip.sh

You may also create cronjob to run the above script regularly. Open crontab with the following command.

$ sudo crontab -e

Add the following line to run the above shell script everyday at 10.a.m and send the output to /home/ip_log.txt

0 10 * * * sudo /home/extract_ip.sh >>/home/log.txt 2>&1

Save and close the file.

So you can easily access all this information by simply opening /home/log.txt file

$ sudo cat /home/log.txt

In this article, we have learnt several ways to extract IP address from log files for Apache and NGINX server. We have also saved these commands to a shell script, and created a cronjob to automatically run this script everyday. You can customize them as per your requirement.

How to Extract IP Address from Log File