[SOLVED] how to store values using nested loop in json file using shell script

Issue

I have entries in abc.env as follows-

connection_name_1=xyz
connection_name_2=gsh
schema_1_connection_name_1=dsd
schema_2_connection_name_1=rwf
schema_1_connection_name_2=dfs

I want to iterate based on connection_name and its associated schema for e.g.

connection_name_1 will have-
schema_1_connection_name_1
schema_2_connection_name_1
connection_name_2 will have-
schema_1_connection_name_2

and store it into a json file which looks like this -

 "connection_specific_details": [
    {
      "connection_name": "xyz",
      "schemas": [
        {
          "dbUser": "dsd"
        },
        {
         "dbUser": "rwf"
         }
      ]
    },
{
      "connection_name": "gsh",
      "schemas": [
        {
          "dbUser": "dfs"
        }
      ]
    }
  ]

Please comment if it is still unlcear, will try to elaborate in a different way but this is straight-forward json which I need to frame out of given values in env.

Solution

If you don't mind reading the file multiple times (and your output not being valid JSON):

1. Grep and read all connection name lines
2. Print the lines containing the connection name

grep '^connection_name_' abc.env | while read IFS== read -r connection value; do
  echo "$connection will have-"
  grep '^schema_' abc.env | grep "_$connection=" | cut -d= -f1
done > output.json # not valid JSON!

---

jq (https://jqlang.github.io/jq/) is a really useful tool to construct JSON documents. The overall logic of the loop does not change too much.

grep '^connection_name_' abc.env | while read IFS== read -r connection value; do
  grep '^schema_' abc.env \
    | grep "_$connection=" \
    | cut -d= -f2- \
    | jq -nR --arg connection_name "$value" '{
        $connection_name,
        schemas: [{dbUser:inputs}]
      }'
done | jq -s '{connection_specific_details: .}' > output.json

Answered By - knittl

Answer Checked By - Mary Flores (WPSolving Volunteer)