[SOLVED] How to iterate through list of dictionaries and sort the output based on the key in Bash?

Issue

We are getting the output from an API call in the below list of dictionaries format,

[
{
        "Key": "/builder-deployer/test2/services/serviceA/ip",
        "Val": "10.1.2.1"
    },
    {
        "Key": "/builder-deployer/test2/services/serviceA/port",
        "Val": "2"
    },
    {
        "Key": "/builder-deployer/test2/services/serviceA/url",
        "Val": "serviceA.abc.com"
    },
    {
        "Key": "/builder-deployer/test2/services/serviceA/username",
        "Val": "jenkins"
    },
    {
        "Key": "/builder-deployer/test2/services/serviceB/ip",
        "Val": "10.1.2.2"
    },
    {
        "Key": "/builder-deployer/test2/services/serviceB/port",
        "Val": "3"
    },
    {
        "Key": "/builder-deployer/test2/services/serviceB/url",
        "Val": "serviceB.abc.com"
    },
    {
        "Key": "/builder-deployer/test2/services/serviceB/username",
        "Val": "jenkins"
    },
    {
        "Key": "/builder-deployer/test2/services/serviceC/ip",
        "Val": "10.1.2.2"
    },
    {
        "Key": "/builder-deployer/test2/services/serviceC/port",
        "Val": "4"
    },
    {
        "Key": "/builder-deployer/test2/services/serviceC/url",
        "Val": "serviceC.abc.com"
    },
    {
        "Key": "/builder-deployer/test2/services/serviceC/username",
        "Val": "jenkins"
    }
]

I have a requirement to fetch the unique username and IP combination in the form of username@ip from the list to install some dependencies. i.e. in the above list of dictionaries, we have 2 unique username@ip combinations. So I need to install packages on [email protected], [email protected] machines respectively.

I have the below code in a python script that does the same but most of our code is in Bash, we want to convert this also to Bash. Any help would be very helpful

ip=[]
for username in outputs:
    for ip in outputs:
        if username['Key'].split('/')[-2] == ip['Key'].split('/')[-2] and username['Key'].split('/')[-1] in ['username'] and ip['Key'].split('/')[-1] in ['ip']:
            ip.append(username['Val']+'@'+ip['Val'])
print(set(ip))

Solution

jq

$ FILE="file.json"
$ paste -d@ \
        <(jq -r '.[]|select(.Key|contains("/username")).Val' "$FILE") \
        <(jq -r '.[]|select(.Key|contains("/ip")).Val' "$FILE") | \
        sort -u

[email protected]
[email protected]    

awk

awk -F\" '/\/ip/{getline;ip=$4}/username/{getline;$0=$4"@"ip;print|"sort -u"}' "$FILE"

[email protected]
[email protected]

Answered By - ufopilot

Answer Checked By - Gilberto Lyons (WPSolving Admin)