Issue
I have two files, one is ips.yaml:
-
dedicatedip: 1.1.1.11
-
dedicatedip: 2.2.2.2
-
dedicatedip: ''
-
dedicatedip: 3.3.3.3
-
dedicatedip: 3.3.3.33
And this is my result.txt:
+----------+-----------------------+------------------+--------+-----------------------------+---------------+
| id | hostname | some other field | status | networks | plan |
+----------+-----------------------+------------------+--------+-----------------------------+---------------+
| 11111111 | some-hostname | some-value | Active | External_Networks=1.1.1.11 | not important |
| 11111111 | some another hostname | | Active | External_Networks=1.1.1.111 | not important |
| 11111111 | some.fqdn.com | | Active | External_Networks=1.1.1.112 | not important |
| 11111111 | fourth.hostname.com | | Active | External_Networks=1.1.1.1 | not important |
| 11111111 | the-other.com | IHaveSomething | Active | External_Networks=2.2.2.2 | not important |
| 11111111 | not.least.com | | Active | External_Networks=2.2.2.25 | not important |
| 11111111 | last.fqdn.com | The Last Value | Active | External_Networks=3.3.3.39 | not important |
+----------+-----------------------+------------------+--------+-----------------------------+---------------+
I want to remove each ips existed in ips.yaml from result.txt, so that the expected output would be:
+----------+-----------------------+------------------+--------+-----------------------------+---------------+
| id | hostname | some other field | status | networks | plan |
+----------+-----------------------+------------------+--------+-----------------------------+---------------+
| 11111111 | some another hostname | | Active | External_Networks=1.1.1.111 | not important |
| 11111111 | some.fqdn.com | | Active | External_Networks=1.1.1.112 | not important |
| 11111111 | fourth.hostname.com | | Active | External_Networks=1.1.1.1 | not important |
| 11111111 | not.least.com | | Active | External_Networks=2.2.2.25 | not important |
| 11111111 | last.fqdn.com | The Last Value | Active | External_Networks=3.3.3.39 | not important |
+----------+-----------------------+------------------+--------+-----------------------------+---------------+
This is my current bash:
while read -r yaml_ips
do
while read -r result_ips
do
if [ "$yaml_ips" == "$result_ips" ]
then
sed "/$yaml_ips/d" result.txt
fi
done < <(grep -Eo 'External_Networks=[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}' result.txt | awk -F '=' '{print $2}')
done < <(awk '/dedicatedip: [0-9]{1,3}.[0-9]{1,3}.[0-9]{1,3}.[0-9]{1,3}/ {print $2}' ips.yaml)
This is my current output:
+----------+-----------------------+------------------+--------+-----------------------------+---------------+
| id | hostname | some other field | status | networks | plan |
+----------+-----------------------+------------------+--------+-----------------------------+---------------+
+----------+-----------------------+------------------+--------+-----------------------------+---------------+
+----------+-----------------------+------------------+--------+-----------------------------+---------------+
| id | hostname | some other field | status | networks | plan |
+----------+-----------------------+------------------+--------+-----------------------------+---------------+
| 11111111 | some-hostname | some-value | Active | External_Networks=1.1.1.11 | not important |
| 11111111 | some another hostname | | Active | External_Networks=1.1.1.111 | not important |
| 11111111 | some.fqdn.com | | Active | External_Networks=1.1.1.112 | not important |
| 11111111 | fourth.hostname.com | | Active | External_Networks=1.1.1.1 | not important |
| 11111111 | last.fqdn.com | The Last Value | Active | External_Networks=3.3.3.39 | not important |
+----------+-----------------------+------------------+--------+-----------------------------+---------------+
Solution
With your shown samples only, please try following GNU awk code.
awk '
FNR==NR && /^ +/{
ips[$2]
next
}
(match($0,/External_Networks=(\S+)/,arr) && (arr[1] in ips)){
next
}
(/^+-/) || (/^\|/){
print
next
} ' FS=": " ips.yaml result.txt
Explanation: Adding detailed explanation for above code.
awk ' ##starting awk program from here.
FNR==NR && /^ +/{ ##Checking condition FNR==NR which will be TRUE when ips.yaml file is being read and checking condition if a line starts with 1 or more spaces then do following.
ips[$2] ##Creating array ips with index of $2 here.
next ##next will skip all further statements from here.
}
(match($0,/External_Networks=(\S+)/,arr) && (arr[1] in ips)){ ##Using match function to match External_Networks= followed by all 1 or more non-spaces and storing IPs values into array arr.
next ##next will skip all further statements from here.
}
(/^+-/) || (/^\|/){ ##Checking condition if line starts from +- OR | then do following.
print ##Print that line.
next ##next will skip all further statements from here.
} ' FS=": " ips.yaml result.txt ##Setting FS to colon space for file ips.yaml and then mentioning result.txt files.
Answered By - RavinderSingh13 Answer Checked By - Dawn Plyler (WPSolving Volunteer)
