[SOLVED] How could running in Travis CI affect behavior of $(shell) in a Makefile?

Issue

I'm experiencing behavior where a gmake $(shell) construct that works in my local environment fails under certain conditions when running within Travis CI invoked from github. When invoking the gmake construct

$(shell echo '{gen}' | sed -e "s#{gen}#$(GEN)#")

as part of a make action, it works fine both in my local Debian environment and inside CI. But when invoking it to set a Makefile variable:

SPECSRC = $(shell echo '{gen}' | sed -e "s#{gen}#$(GEN)#")

it still works locally, but fails when executed under Travis with the error message:

mf.test:11: *** unterminated call to function 'shell': missing ')'. Stop.

I've checked some obvious things - both are running /bin/bash in their respective environments, and there are very close (not identical) gmake / sed / bash versions in the two environments. Alternate sed-expression separators like '@' instead of '#' (see mf.test below) do not help.

If this were something happening in .travis.yml I'd think about metacharacters, but this is only invoked under Travis - the offending commands are executing entirely inside gmake through the $(shell) construct. Any thoughts on why this is happening, and how I could avoid it?

Trivial makefile 'mf.test' follows:

# Very stripped-down test Makefile

GEN = $(CURDIR)/gen

all: shell shellvar

shell:
        echo 'shell: trying inline shell-sed'
        echo $(shell echo '{gen}' | sed -e "s#{gen}#$(GEN)#")

SPECSRC = $(shell echo '{gen}' | sed -e "s#{gen}#$(GEN)#")
shellvar:
        echo 'shellvar: trying shell-sed through a variable'
        echo SPECSRC = $(SPECSRC)

I expected:

make -f mf.test

to result in something like (depending on path):

echo 'shell: trying inline shell-sed'
shell: trying inline shell-sed
echo /home/tree/git/opencl/github-Docs/gen
/home/tree/git/opencl/github-Docs/gen
echo 'shellvar: trying shell-sed through a variable'
shellvar: trying shell-sed through a variable
echo SPECSRC = /home/tree/git/opencl/github-Docs/gen
SPECSRC = /home/tree/git/opencl/github-Docs/gen

Run locally, this happens. Running under Travis CI (see link in my comment), I get

echo 'shell: trying inline shell-sed'
shell: trying inline shell-sed
echo /home/travis/build/KhronosGroup/OpenCL-Docs/gen
/home/travis/build/KhronosGroup/OpenCL-Docs/gen
mf.test:11: *** unterminated call to function 'shell': missing ')'.  Stop.

Solution

It's because there's a different version of GNU Make running on Travis. The formulation you have is not portable between different versions of GNU Make because the comment character # was not properly ignored inside variable/function references in some older versions of GNU Make.

If you want your makefile to be portable you have to "hide" the comment character in a variable, like this:

HASH := \#
SPECSRC = $(shell echo '{gen}' | sed -e "s$(HASH){gen}$(HASH)$(GEN)$(HASH)")

Note, it's (virtually) always a bad idea to use recursive assignment = with $(shell ...). Unless you have very unusual requirements you should always use simple assignment :=. This is not related to your problem however.

Answered By - MadScientist

Answer Checked By - Katrina (WPSolving Volunteer)