Wednesday, April 6, 2022

[SOLVED] 'sed: no input files' when using sed -i in a loop

Issue

I checked some solutions for this in other questions, but they are not working with my case and I'm stuck so here we go. I have a csv file that I want to convert all to uppercase. It has to be with a loop and occupate 7 lines of code minimum. I have to run the script with this command:

./c_bash.sh student-mat.csv

So I tried this Script:

#!/bin/bash
declare -i c=0
while read -r line; do 
    if [ "$c" -gt '0' ]; then 
       sed -e 's/\(.*\)/\U\1/'      
    else
        echo "$line"
    fi
    ((c++))
done < student-mat.csv

I know that maybe there are a couple of unnecessary things on it, but I want to focus in the sed command because it looks like the problem here. That script shows this output:(first 5 lines):

school,sex,age,address,famsize,Pstatus,Medu,Fedu,Mjob,Fjob,reason,guardian,traveltime,studytime,failures,schoolsup,famsup,paid,activities,nursery,higher,internet,romantic,famrel,freetime,goout,Dalc,Walc,health,absences,G1,G2,G3
GP,F,17,U,GT3,T,1,1,AT_HOME,OTHER,COURSE,FATHER,1,2,0,NO,YES,NO,NO,NO,YES,YES,NO,5,3,3,1,1,3,4,5,5,6
GP,F,15,U,LE3,T,1,1,AT_HOME,OTHER,OTHER,MOTHER,1,2,3,YES,NO,YES,NO,YES,YES,YES,NO,4,3,2,2,3,3,10,7,8,10
GP,F,15,U,GT3,T,4,2,HEALTH,SERVICES,HOME,MOTHER,1,3,0,NO,YES,YES,YES,YES,YES,YES,YES,3,2,2,1,1,5,2,15,14,15
GP,F,16,U,GT3,T,3,3,OTHER,OTHER,HOME,FATHER,1,2,0,NO,YES,YES,NO,YES,YES,NO,NO,4,3,2,1,2,5,4,6,10,10
GP,M,16,U,LE3,T,4,3,SERVICES,OTHER,REPUTATION,MOTHER,1,2,0,NO,YES,YES,YES,YES,YES,YES,NO,5,4,2,1,2,5,10,15,15,15

Now that I see that it works, I want to apply that sed command permanently to the csv file, so I put -i after it:

#!/bin/bash
declare -i c=0
while read -r line; do 
    if [ "$c" -gt '0' ]; then 
       sed -i -e 's/\(.*\)/\U\1/'      
    else
        echo "$line"
    fi
    ((c++))
done < student-mat.csv

But the output instead of applying the changes, shows this:(first 5 lines)

school,sex,age,address,famsize,Pstatus,Medu,Fedu,Mjob,Fjob,reason,guardian,traveltime,studytime,failures,schoolsup,famsup,paid,activities,nursery,higher,internet,romantic,famrel,freetime,goout,Dalc,Walc,health,absences,G1,G2,G3
sed: no input files
sed: no input files
sed: no input files
sed: no input files
sed: no input files

So checking a lot of different solutions on the internet, I also tried to change single quoting to double quoting.

#!/bin/bash
declare -i c=0
while read -r line; do 
    if [ "$c" -gt '0' ]; then 
       sed -i -e "s/\(.*\)/\U\1/"       
    else
        echo "$line"
    fi
    ((c++))
done < student-mat.csv

But in this case, instead of applying the changes, it generate a file with 0 bytes. So no output when I do this:

cat student-mat.csv

My expected solution here is that, when I apply this script, it changes permanently all the data to uppercase. And after applying the script, it should show this with the command cat student-mat.csv: (first 5 lines)

school,sex,age,address,famsize,Pstatus,Medu,Fedu,Mjob,Fjob,reason,guardian,traveltime,studytime,failures,schoolsup,famsup,paid,activities,nursery,higher,internet,romantic,famrel,freetime,goout,Dalc,Walc,health,absences,G1,G2,G3
GP,F,17,U,GT3,T,1,1,AT_HOME,OTHER,COURSE,FATHER,1,2,0,NO,YES,NO,NO,NO,YES,YES,NO,5,3,3,1,1,3,4,5,5,6
GP,F,15,U,LE3,T,1,1,AT_HOME,OTHER,OTHER,MOTHER,1,2,3,YES,NO,YES,NO,YES,YES,YES,NO,4,3,2,2,3,3,10,7,8,10
GP,F,15,U,GT3,T,4,2,HEALTH,SERVICES,HOME,MOTHER,1,3,0,NO,YES,YES,YES,YES,YES,YES,YES,3,2,2,1,1,5,2,15,14,15
GP,F,16,U,GT3,T,3,3,OTHER,OTHER,HOME,FATHER,1,2,0,NO,YES,YES,NO,YES,YES,NO,NO,4,3,2,1,2,5,4,6,10,10
GP,M,16,U,LE3,T,4,3,SERVICES,OTHER,REPUTATION,MOTHER,1,2,0,NO,YES,YES,YES,YES,YES,YES,NO,5,4,2,1,2,5,10,15,15,15

Thank you for helping me.


Solution

Sed works on files, not on lines. Do not read lines, use sed on the file. Sed can exclude the first line by itself. See sed manual.

You want:

sed -i -e '2,$s/\(.*\)/\U\1/' student-mat.csv

You can do shorter with s/.*/\U&/.

Your code does not work as you think it does. Note that your code removes the second line from the output. Your code:

  • reads first line with read -r line
  • echo "$line" first line is printed
  • c++ is incremented
  • read -r line reads second line
  • then sed processes the rest of the file (from line 3 till the end) and prints them in upper case
  • then c++ is incremented
  • then read -r line fails, and the loop exits


Answered By - KamilCuk
Answer Checked By - Clifford M. (WPSolving Volunteer)
at