[SOLVED] Running search and replace with sed via python subprocess results in unviewable characters

Issue

Given the file /tmp/hi with content: bali=${hi

and running the command on it sed -i -E 's/(^|[^.])hi/\1bi/g' /tmp/hi

results in the following content in bali=${bi as expected.

However, running the sed command inside python3.5 subprocess:

import subprocess
subprocess.run("sed -i -E 's/(^|[^.])hi/\1bi/g' /tmp/hi", shell=True)

results in the following content:

inspected the file in vi and it shows: bali=$^Abi

Why does it happen and how to achieve the same file content using python3.5 subprocess?

Solution

That's because the \1 is being interpreted by Python. You need to use raw string syntax (r"some \1 string with escape sequences") if you want to use escape sequences without having to escape them:

Python 3.5.3 (default, Jan 19 2017, 14:11:04)
[GCC 6.3.0 20170118] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> print("sed -i -E 's/(^|[^.])hi/\1bi/g' /tmp/hi")
sed -i -E 's/(^|[^.])hi/bi/g' /tmp/hi
>>> print(r"sed -i -E 's/(^|[^.])hi/\1bi/g' /tmp/hi")
sed -i -E 's/(^|[^.])hi/\1bi/g' /tmp/hi

Answered By - Cecile