Issue
The following script works every time if I source ./script.sh but fails about 80% of the time if I execute it directly as ./script.sh.
#!/bin/bash
signing_input="Hi."
signature=$(echo -n "$signing_input" | openssl dgst -sha256 -sign private.pem)
echo -n "$signing_input" | openssl dgst -sha256 -verify public.pem -signature <(echo -n "$signature")
So if I "source" that or run line-by-line, I always get: Verified OK
... but if I run it in a script I usually but not always get: Error verifying data
I get the same results if I save the signature to a file first.
I generated the .pem files this way:
openssl ecparam -name secp256k1 -genkey -noout -out private.pem
openssl ec -in private.pem -pubout -out public.pem
... and it doesn't matter if I reuse them or regenerate them every time.
I'm on a Mac M1 running iTerm2 with zsh.
Solution
Is this really programming or development? I'm not sure, but I'll call it borderline unless the voting community decides otherwise.
That's binary data -- which sometimes includes 'null' (zero-valued) bytes. The ECDSA signature format for a curve group with order very close to 2^256 (which secp256k1 is) has 6 metadata bytes that are never zero, 0-2 padding bytes that are always zero when present and at least one is present 75% minus epsilon of the time, and almost always 64 bytes that except for two are indistinguishable from uniform random so at least one of them is zero 1-(1-1/256)^62 = 21.5463% of the time independent of the padding bytes -- thus 1-(1-.75)(1-.215463) = 80.3866% is the probability of at least one zero byte in such a signature.
And bash can't handle null bytes in a variable, so your $signature value is mangled and wrong, and verification fails. Modern versions of bash give a warning when this happens -- did you ignore it, or is yours ancient?
There is also a chance that the signature data ends in a newline byte (value 0x0A) (or even more than one) in which case command substitution $(...) removes it (or them) -- but only negligibly more than 1/256 ~ 0.4%, and given the chance of a null earlier, this adds only ~ 0.1% to the chance of mangling.
When you source the file it is executed by zsh NOT by bash (the shebang is ignored) and zsh DOES handle null bytes, so it mostly works -- but in the 0.4% case of a trailing newline zsh will also fail.
If you instead write to a file and read back, that will work in both/all shells always. Or if you encode to (and decode from) a form that doesn't use nulls (and ignores newlines), like base64:
signature=$(echo -n "$data" | openssl dgst -sha256 -sign private | openssl base64 -e)
echo -n "$data" | openssl dgst -sha256 -verify public -signature <(echo $signature | openssl base64 -d)
# base64 never contains chars that need quoting, and adding newline to it is fine
# (in fact the base64 -e command included a trailing newline which $(...) _removed_)
Hex is also okay, but can't be done by openssl alone, and I don't know which other program(s) I can rely on your system having.
Answered By - dave_thompson_085 Answer Checked By - David Marino (WPSolving Volunteer)