Tuesday, October 4, 2022

[SOLVED] Bash FTP upload - events to log

Issue

I have bash script ftp upload file. In log file I have all events. How can I have in log file only several events, like: "Connected to $HOST" or "WARNING - failed ftp connection"; "Ok to send data"; "Transfer complete"; "10000 bytes sent in 0.00 secs (73.9282 MB/s)".

#!/bin/sh
echo "####################" >> $TESTLOG
echo "$(date +%Y%m%d_%H%M%S)" >> $TESTLOG

ftp -i -n -v <<SCRIPT >> ${TESTLOG} 2>&1
open $HOST
user $USER $PASSWD
bin
cd $DPATH
lcd $TFILE
mput *.txt
quit
SCRIPT

exit 0

####################
20210304_111125
Connected to $HOST.
331 Please specify the password.
230 Login successful.
Remote system type is UNIX.
Using binary mode to transfer files.
200 Switching to Binary mode.
250 Directory successfully changed.
Local directory now /home/pi/Data/data_files/tmp
local: 20210304_111125_ftp_10k.txt remote: 20210304_111125_ftp_10k.txt
200 PORT command successful. Consider using PASV.
150 Ok to send data.
226 Transfer complete.
10000 bytes sent in 0.00 secs (73.9282 MB/s)
221 Goodbye.

Solution

First, I'd like to say while this is a very common idiom, it has no error checking and I despise it.

Now that I got that off my chest...

ftp 2>&1 -inv << SCRIPT | grep -f patfile >> ${TESTLOG}

patfile is a list of patterns to keep. c.f. the grep manual page.

...to continue my rant, though...

What if someone changes the permissions on your $DPATH? The cd fails; ftp reports the failure to the console (your grep ignores it so it doesn't even get logged...), the script continues and puts the files in the wrong place. Full disk prevents files from being placed? Same cycle. A thousand things could go wrong, but ftp just blithely goes on, and doesn't even return an error on exit for most of them. Don't do this.

Just use scp. Yes, you have to set up something like ssh keys, but then the command is simple and checkable.

scp $TFILE/*.txt $HOST:$DPATH/ || echo "copy failed"

For a more elaborate response -

if scp $TFILE/*.txt $HOST:$DPATH/        # test the scp
then ssh "$HOST" "ls -ltr $DPATH/*.txt"  # show result if success
else echo "copy failed"                  # code here for if it fails...
     exit 1                              # as much code as you feel you need
fi

Otherwise, use a different language with an ftp module like Perl so you can check steps and handle failures.

(I wrote a ksh script that handled an ftp coprocess, fed it a command at a time, checked the results... it's doable, but I don't recommend it.)



Answered By - Paul Hodges
Answer Checked By - Mildred Charles (WPSolving Admin)
at