Saturday, November 13, 2021

[SOLVED] sed unable to parse multi-line string (Jenkins) parameter

Issue

I have app_name as a multi-line string parameter, I want to loop it through so I can create ingress objects for applications.

I have defined them as below:

app name as multi-line string parameter

Now, my shell script snippet for the Jenkins file is like below:

echo "${app_name}"
            for i in "${app_name}"; do
                kubectl delete ing ${i} -n${deployment_namespace}
                cp -apvf  /tmp/flexicache-ing.yaml /tmp/flexicache-ing-${i}.yaml
                pwd
                sed -i 's/#{deployment_ns}#/'"${deployment_namespace}"'/' /tmp/flexicache-ing-${i}.yaml
                sed -i 's/#{app_name}#/'"${app_name}"'/' /tmp/flexicache-ing-${i}.yaml
                sed -i 's/#{chart_name}#/'"${app_name}"'/' /tmp/flexicache-ing-${i}.yaml
                kubectl apply -f /tmp/flexicache-ing-${i}.yaml
            done

Output:

+ echo 'flexicache-data-processor
flexicache-incident-data-processor'
flexicache-data-processor
flexicache-incident-data-processor
++ tput bold
+ bold=''
++ tput sgr0
+ normal=''
+ for i in '"${app_name}"'
+ kubectl delete ing flexicache-data-processor flexicache-incident-data-processor -nsit1-blue
Error from server (NotFound): ingresses.extensions "flexicache-data-processor" not found
Error from server (NotFound): ingresses.extensions "flexicache-incident-data-processor" not found
+ cp -apvf /tmp/flexicache-ing.yaml /tmp/flexicache-ing-flexicache-data-processor flexicache-incident-data-processor.yaml
cp: target 'flexicache-incident-data-processor.yaml' is not a directory

I have tried it with an array as well but that seems to be not working as well.

Need to understand what I am doing wrong.

P.S: Changed the code to the below:

for i in ${app_name}; do
                kubectl delete ing ${i} -n${deployment_namespace}
                cp -apvf  /tmp/flexicache-ing.yaml /tmp/flexicache-ing-${i}.yaml
                pwd
                sed -i 's/#{deployment_ns}#/'"${deployment_namespace}"'/' /tmp/flexicache-ing-${i}.yaml
                sed -i 's/#{app_name}#/'"${app_name}"'/' /tmp/flexicache-ing-${i}.yaml
                sed -i 's/#{chart_name}#/'"${app_name}"'/' /tmp/flexicache-ing-${i}.yaml
                kubectl apply -f /tmp/flexicache-ing-${i}.yaml
            done

Output is still busted:

+ sed -i 's/#{app_name}#/flexicache-data-processor
flexicache-incident-data-processor/' /tmp/flexicache-ing-flexicache-data-processor.yaml
sed: -e expression #1, char 40: unterminated `s' command

EDIT 2: I did used code of <<< before bt still not woring. Code below:

            while IFS= read -r i; do
                kubectl delete ing ${i} -n${deployment_namespace}
                cp -apvf  /tmp/flexicache-ing.yaml /tmp/flexicache-ing-${i}.yaml
                pwd
                sed -i 's/#{deployment_ns}#/'"${deployment_namespace}"'/' /tmp/flexicache-ing-${i}.yaml
                sed -i 's/#{app_name}#/'"${app_name}"'/' /tmp/flexicache-ing-${i}.yaml
                sed -i 's/#{chart_name}#/'"${app_name}"'/' /tmp/flexicache-ing-${i}.yaml
                kubectl apply -f /tmp/flexicache-ing-${i}.yaml
            done <<< "${app_name}"

Output with failed sed...

+ sed -i 's/#{chart_name}#/flexicache-data-processor
flexicache-incident-data-processor/' /tmp/flexicache-ing-flexicache-data-processor.yaml
sed: -e expression #1, char 42: unterminated `s' command

EDIT: Tried the code below with no charm.....

[ec2-user@svcm268fa99bnp ~]$ echo ${app_name}
+ echo flexicache-data-processor flexicache-incident-data-processor
flexicache-data-processor flexicache-incident-data-processor
++ printf '\033]0;%s@%s:%s\007' ec2-user svcm268fa99bnp '~'
[ec2-user@svcm268fa99bnp ~]$ while IFS= read -r i; do cp -apvf /tmp/flexicache-ing.yaml /tmp/flexicache-ing-${i}.yaml; pwd; sed -i 's/#{app_name}#/'"${app_name}"'/' | tr '|' '\n' /tmp/flexicache-ing-${i}.yaml; done <<< "${app_name}"
+ IFS=
+ read -r i
+ cp -apvf /tmp/flexicache-ing.yaml /tmp/flexicache-ing-flexicache-data-processor.yaml
‘/tmp/flexicache-ing.yaml’ -> ‘/tmp/flexicache-ing-flexicache-data-processor.yaml’
+ pwd
/home/ec2-user
+ tr '|' '\n' /tmp/flexicache-ing-flexicache-data-processor.yaml
+ sed -i 's/#{app_name}#/flexicache-data-processor
flexicache-incident-data-processor/'
tr: extra operand ‘/tmp/flexicache-ing-flexicache-data-processor.yaml’
Try 'tr --help' for more information.
sed: -e expression #1, char 40: unterminated `s' command
+ IFS=
+ read -r i
+ cp -apvf /tmp/flexicache-ing.yaml /tmp/flexicache-ing-flexicache-incident-data-processor.yaml
‘/tmp/flexicache-ing.yaml’ -> ‘/tmp/flexicache-ing-flexicache-incident-data-processor.yaml’
+ pwd
/home/ec2-user
+ tr '|' '\n' /tmp/flexicache-ing-flexicache-incident-data-processor.yaml
+ sed -i 's/#{app_name}#/flexicache-data-processor
flexicache-incident-data-processor/'
tr: extra operand ‘/tmp/flexicache-ing-flexicache-incident-data-processor.yaml’
Try 'tr --help' for more information.
sed: -e expression #1, char 40: unterminated `s' command
+ IFS=
+ read -r i
++ printf '\033]0;%s@%s:%s\007' ec2-user svcm268fa99bnp '~'
[ec2-user@svcm268fa99bnp ~]$


Solution

Repeatedly running sed -i on the same file is an antipattern you want to avoid.

Quoting the input string makes the loop run exactly once, with the loop variable equal to the entire string. Perhaps a better arrangement would be a while read loop which by its nature reads one line at a time.

Inside the loop, you want to use the loop variable $i, not the whole string $app_name. This is what was wrong in your final attempts.

echo "$app_name"
while IFS= read -r i; do
    kubectl delete ing "$i" -n"$deployment_namespace"
    cp -apvf  /tmp/flexicache-ing.yaml /tmp/flexicache-ing-"$i".yaml
    pwd
    sed -e "s/#{deployment_ns}#/$deployment_namespace/" \
        -e "s/#{app_name}#/$i/" \
        -e "s/#{chart_name}#/$i/" /tmp/flexicache-ing.yaml \
            >/tmp/flexicache-ing-"$i".yaml
    kubectl apply -f /tmp/flexicache-ing-"$i".yaml
done <<<"$app_name"

The <<< here string syntax is Bash specific; if you can't use Bash, try echo "$app_name" | while IFS= read -r i ...

The cp command is probably superfluous; I left it in in case the options to cp do something crucial which needs to be preserved, even though we then overwrite the copied file with new contents.

You probably want to remove the pwd, and perhaps the echo before the loop.

The braces around the variable names do not do anything much useful here; I took them out and added proper quoting instead. I also simplified the quoting in the sed script; nothing in there seems to require single quotes.



Answered By - tripleee