[SOLVED] Extract IP address after pattern is found in log file

Issue

I need to extract and then block a certain IP address (IPv4 only) within a log file using ipset command within a bash script. The logfile structure looks like this:

[Line 1] random text
[Line 2] random text
...
[Line 26] random text
Aug 31 13:40:14 [logs ] con_accept: [759] connection from 127.0.0.1:56860 is successfull on 0.0.0.0:25000
[Line 28] random text
...
[Line 38] random text
Aug 31 13:40:57 [logs ] _authrequest: [759] random text client connected and the sky is blue today more random text
[Line 40] random text

• Forgot to add that each line [Line 1] ... [Line X starts with a timestamp Aug 31 13:40:14 [logs ] con_accept: [759] connection from 127.0.0.1:56860 is successful on 0.0.0.0:25000
• The IP address that needs to be extracted and blocked will be displayed within the last 15 lines before the line that matches the pattern. In this case, can be found at Line 27.
• The script is first searching for sky is blue string
• When the string is found, it prints the last 15 lines before the matched string line and extracts the IP address that matches [759] which is the clientID
• To avoid false-positive and wrong blocks the [ID] must match, in this case [759] matches at Line 39 and Line 27. The clientID is not a predefined number so it will change. It can only be a number.
• IPv4 address 127.0.0.1 will be extracted and then blocked using ipset add blacklist $IP

So far, I'm either for grep or awk usage. Checking the last 15 lines after pattern has been found is easy, however, I'm trying to do extra checks and extract the correct IP address once the string is found and clientID match.

awk 'c-->0;$0~s{if(b)for(c=b+1;c>1;c--)print r[(NR-c+1)%b];print;c=a}b{r[NR%b]=$0}' b=15 s="sky is blue" logfile

grep -B 15 'sky is blue' logfile

Solution

You never additionally need grep (or sed) when you're using awk.

If you have tac on your system then:

$ cat tst.sh
#!/usr/bin/env bash

ip=$(
    tac "${@:--}" |
    awk -v b=15 -v s='sky is blue' '
        $0 ~ s {
            endNr = NR + b
            id = $2
        }
        ($6 == "con_accept:") && ($7 == id) {
            sub(/:.*/,"",$5)
            ip = $10
        }
        NR == endNr { exit }
        END {
            if ( ip == "" ) {
                exit 1
            }
            else {
                print ip
            }
        }
    '
) &&
echo ipset add blacklist "$ip"

$ ./tst.sh file
ipset add blacklist 127.0.0.1

otherwise less efficiently using GNU awk for arrays of arrays:

$ cat tst.sh
#!/usr/bin/env bash

ip=$(
    awk -v b=15 -v s='sky is blue' '
        { id = $7 }
        $6 == "con_accept:" {
            sub(/:.*/,"",$10)
            ips[NR][id] = $10
        }
        $0 ~ s {
            for (n=NR-1; n>(NR-b); n--) {
                if ( (n in ips) && (id in ips[n]) ) {
                    ip = ips[n][id]
                    exit
                }
            }
        }
        END {
            if ( ip == "" ) {
                exit 1
            }
            else {
                print ip
            }
        }
    ' "${@:--}"
) &&
echo ipset add blacklist "$ip"

$ ./tst.sh file
ipset add blacklist 127.0.0.1

Remove the echo when you're done testing and happy with the results.

The above assumes that the [Line N] strings at the start of each line in your question aren't really present. If they are it's an easy tweak to add 2 to each field number use.

Answered By - Ed Morton

Answer Checked By - David Goodson (WPSolving Volunteer)