[SOLVED] Failed to avoid entering password multiple times using scp, and looping not working

Issue

I've written the below script that looping through names in a csv file and copy specific files to remote computer:

list_filepath=/home/bella/lists
destination_path=/home/pome/Documents/newfiles
NS_samples_source_path=/home/bella/final/acs/pa/1_ns
th_samples_source_path=/home/bella/final/acs/pa/4_th
ad_samples_source_path=/home/bella/final/new2018/ad

cat $list_filepath/new_list.csv | while read line
do

  if [[ $line == AD* ]]
     then
     echo $line
     ssh [email protected] mkdir -p $destination_path/$line
     cd $ad_samples_source_path/$line
     find $ad_samples_source_path/$line/ -regex '.*\.\(csv\|xml\|gz\)$' -exec scp {} [email protected]:$destination_path/$line \;

  elif [[ $line == NS* ]]
     then
     echo $line
     ssh [email protected] "mkdir -p $destination_path/$line"
     cd $NS_samples_source/$line
     find $NS_samples_source/$line/ -regex '.*\.\(csv\|xml\|gz\)$' -exec scp {} [email protected]:$destination_path/$line \;

  else
     echo $line
     ssh [email protected] "mkdir -p $destination_path/$line"
     cd $th_samples_source_path/$line
     find $th_samples_source_path/$line/ -regex '.*\.\(csv\|xml\|gz\)$' -exec scp {} [email protected]:$destination_path/$line \;

  fi

done

I have 2 main problems:

1. I have to enter multiple times the password for the remote, even though I'v set a password-less key as below:

   ssh-keygen
   ssh-copy-id -i [email protected]
   

   it still asks for the password for each file it copies.

2. The scripts ends after creating one directory with the copied files, it seems to fail to move to the next line in the csv file.

Ideas anybody how to address those issues?

Thanks!

Solution

Point 1 is because there is no SSH agent on your local host or the remote host is not configured to accept keys, and point 2 is because SSH will swallow the standard input that you were expecting to send to read. Relevant FAQ entries:

• I want to automate an ssh (or scp, or sftp) connection, but I don't know how to send the password....
• I'm reading a file line by line and running ssh or ffmpeg, only the first line gets processed!

Answered By - l0b0