Issue
As the title says, I want to eliminate trailing newlines and store the result in variables (Bash).
The following does not work.
[xxx@ ~]$ var=`echo 'abc\ndef\n' | sed 's/\\n$//g'`
[xxx@ ~]$ echo $var
abc\ndef\n
The following works!
[xxx@ ~]$ var=`echo 'abc\ndef\n' | sed 's/\\\\n$//g'`
[xxx@ ~]$ echo $var
abc\ndef
I thought that only one backslash escape was needed for sed since it is enclosed in single quotes, but I guess my understanding is wrong.
Please tell me why the second command works correctly?
Solution
Regarding:
Please tell me why the second command works correctly?
This command produces the shown output:
$ echo 'abc\ndef\n' | sed 's/\\\\n$//g'
abc\ndef\n
As for why you get this unexpected output with old, deprecated backticks:
$ var=`echo 'abc\ndef\n' | sed 's/\\\\n$//g'`
$ echo "$var"
abc\ndef
which is different from this expected output with modern command substitution ($(...)
):
$ var=$(echo 'abc\ndef\n' | sed 's/\\\\n$//g')
$ echo "$var"
abc\ndef\n
it's because backslashes (\
) inside backticks are handled in a non-obvious manner as stated and demonstrated at the top of the related FAQ, https://mywiki.wooledge.org/BashFAQ/082, and that is one of the reasons to use $(...)
instead of backticks to execute commands.
So, if you want to remove a trailing \n
string using a pipe to sed
then do this:
$ var=$(echo 'abc\ndef\n' | sed 's/\\n$//g')
$ echo "$var"
abc\ndef
otherwise, if you really want to remove newlines as your Subject line says then read on:
echo 'abc\ndef\n'
produces this output:
$ echo 'abc\ndef\n'
abc\ndef\n
The only newline present in that output is the one at the end of the line that makes that a valid POSIX text file. The \n
s are literally the 2 characters \
and \n
.
You should use printf
instead of echo
for portability and consistency. In this case it would convert \n
strings to newlines:
$ printf 'abc\ndef\n'
abc
def
Now, to get RID of the newlines - sed by default reads input 1 line at a time so you can't remove a newline from a string that sed sees as there cannot be a newline within 1 line that's part of newline-separated input. You can do things with sed to make it read multiple lines but it's more robust and portable to use awk instead, e.g.:
to remove just the final newline:
$ printf 'abc\ndef\n' | awk '{printf "%s%s", sep, $0; sep=ORS}'
abc
def <--- no newline at the end
or to remove all newlines:
$ printf 'abc\ndef\n' | awk -v ORS= '1'
abcdef <--- no newline at the end
or to remove all except the final newline:
$ printf 'abc\ndef\n' | awk -v ORS= '1; END{print RS}'
abcdef <--- newline at the end
or similar depending on whether you want the string to end in a newline (and so be a valid POSIX text line/file) or not.
To execute a command, store it's result in a variable, and then print the contents of that variable, one way using command substitution would be:
$ var=$(printf 'abc\ndef\n' | awk -v ORS= '1; END{print RS}')
$ printf '%s\n' "$var"
abcdef
Note the use of $(...)
instead of the long deprecated backticks (https://mywiki.wooledge.org/BashFAQ/082), and the necessary quotes "..."
around the shell variable (see https://mywiki.wooledge.org/Quotes). Get into the habit of copy/pasting your code into http://shellcheck.net and it'll help you find and fix errors.
Answered By - Ed Morton Answer Checked By - David Goodson (WPSolving Volunteer)