[SOLVED] Searching and replacing values recursively in the same time

Issue

I have a folder of Ansible playbooks, in which I want to grep specific YAML block and replace one of the Jinja2 variables with its value from the playbook folder.

For example:

grep -R -A 35 -w -h 'test_profiles:' .

test_profiles:
  - name: test_1
  source: '{{ service_name }}'
  

So I want to interpolate the service_name variable with its value

What I tried is to grep for the service_name but couldn't find a way to recursively replace the variable with its respective search result for each playbook folder

Folder structure:

• playbook 1
  • main.yml < contains service_name value
  • tasks
    • file.yml < contains test_profiles block which I need to replace the service_name within it
• playbook 2
  • main.yml
  • tasks
    • file.yml
• ... etc

main.yml from playbook 1 sample:

---
service_name: 'service 1'

main.yml from playbook 2 sample:

---
service_name: 'service 2'

file.yml from playbook 1 sample:

---
test_profiles:
  - name: test_1
  source: '{{ service_name }}'

other_header:
   key: value

file.yml from playbook 2 sample:

---
test_profiles:
  - name: test_2
  source: '{{ service_name }}'

another_header:
   another_key: value

Expected result:

test_profiles:
  - name: test_1
  source: 'service 1'
test_profiles:
  - name: test_2
  source: 'service 2'
test_profiles:
  - name: test_3
  source: 'service 3'

grep command I tried to get the service_name variable:

grep -R -oE 'service_name=([a-zA-Z]*-*[a-zA-Z]*)*|service_name:(\s*[a-zA-Z]*-*[a-zA-Z]*)*' .

But I couldn't find a way to combine both commands, to result in the expected result above.

Solution

This solution runs one awk per playbook. Parse the main.yml and store the value of service_name. Parse the second file and print only the lines block (starting with test_profiles: and ending with a line empty or with only white-space characters) with the variable replaced. You execute the script below from the root of your directory structure.

#!/bin/bash

for d in playbook*; do
    [[ -f "$d"/main.yml ]] && [[ -f "$d"/tasks/file.yml ]] || continue
    awk -F"'" 'FNR==NR && /^service_name:/{s=$2;next} {FS=OFS}
           /test_profiles:/ {f=1}
           f {gsub("{{ service_name }}",s)}
           f && (NF==0) {f=0}
           f' "$d"/main.yml "$d"/tasks/file.yml
done

Also if there is no value in the main.yml, this prints empty string for {{ service_name }}, if this is not a feature but a bug, we can change it. And any playbook not having both files at place, is being ignored.

---

Reference:

How to read two files with awk (NR==FNR)

gsub and the other GNU awk string functions

How to print using a flag a block between start and end pattern

Answered By - thanasisp