Issue
This question about using cURL with a username and password has suboptimal answers for me:
curl -u "user:pw" https://example.com
puts the pw in the process listcurl "https://user:[email protected]"
puts the pw in the process listcurl -u "user:$(cat ~/.passwd)" https://example.com
puts the pw in the process listcurl -u user https://example.com
prompts for the pwcurl --netrc-file ~/.netrc https://example.com
requires a file
#4 is secure, but I might run this command hundreds of times a day, so it's tedious. #5 is close to secure, but that file could be read by somebody with root access.
The cURL man page says (note the bold text):
-u/--user <user:password>
Specify the user name and password to use for server authentication. Overrides
-n/--netrc
and--netrc-optional
.If you just give the user name (without entering a colon) curl will prompt for a password.
If you use an SSPI-enabled curl binary and do NTLM authentication, you can force curl to pick up the user name and password from your environment by simply specifying a single colon with this option:
-u :
.
I've tried setting $USER
and $PASSWORD
(and $CURLOPT_PASSWORD
and others) in the environment, but cURL doesn't pick up either of them when invoked as curl -u : https://example.com
(nor does it work without the -u :
).
I'm not doing NTLM, so this doesn't work. Unless I'm missing something.
Is there a way to pass credentials to curl
solely through the environment?
(Workaround moved to an answer)
Solution
This bash
solution appears to best fit my needs. It's decently secure, portable, and fast.
#!/bin/bash
SRV="example.com"
URL="https://$SRV/path"
curl --netrc-file <(cat <<<"machine $SRV login $USER password $PASSWORD") "$URL"
This uses process substitution (<( command )
runs command
in a sub-shell to populate a file descriptor to be handed as a "file" to the parent command, which in this case is curl
). The process substitution contains a here-string (cat <<< text
, a variant of echo text
that won't put anything into your process list), creating a file descriptor for the netrc file in order to pass credentials to the remote web server.
The security afforded by process substitution is actually pretty sound: its file descriptor is not a temporary file and is unavailable from even other calls in the same shell instance, so this appears secure in this context; an adversary would have to dig through memory or launch a complicated attack to find its contents. Since the $PASSWORD
environment variable is also in memory, this should not increase the attack surface.
As long as you haven't used export PASSWORD
, a trick like ps ewwp $$
shouldn't reveal the password (as noted in this comment). It'd also be wise to use some less obvious variable name.
Here is a simplified insecure version of the above code that may help explain how it works:
#!/bin/sh
# INSECURE VERSION, DO NOT USE
SRV=example.com
URL="https://$SRV/path"
TMP=$(mktemp)
printf "machine %s login %s password %s\n" "$SRV" "$USER" "$PASSWORD" > "$TMP"
curl --netrc-file "$TMP" "$URL"
rm -f "$TMP"
This insecure version has lots of flaws, all of which are solved in the previous version:
- It stores the password in a file (though that file is only readable to you)
- It very briefly has the password in a command line
- The temporary file remains until after
curl
exits - Ctrl+c will quit without removing the temporary file
Some of that could be solved by:
#!/bin/sh
SRV=example.com
URL="https://$SRV/path"
TMP=$(mktemp /dev/shm/.XXXXX) # assumes /dev/shm is a ramdisk
trap "rm -f $TMP" 0 18
cat << EOF > "$TMP"
machine $SRV login $USER password $PASSWORD
EOF
(sleep 0.1; rm -f "$TMP") & # queue removing temp file in 0.1 seconds
curl --netrc-file "$TMP" "$URL"
I consider this version to be messy, suboptimal, and possibly less secure (though it is more portable). It also requires a version of sleep
that understands decimals (and 0.1 seconds may be too fast if the system is heavily loaded).
I had originally posted a workaround that included a perl
one-liner in my question, then (with help from Etan Reisner) I worked through a few better methods before settling on this here-string method, which is both lighter-weight (faster) and more portable.
At this point, it's elegant enough that I'd consider it the "answer" rather than an "ugly workaround," so I've migrated it to be this official answer. I've given @ghoti a +1 for his answer, which correctly states that cURL's command line program is incapable of doing what I want on its own, but I'm not "accepting" that answer because it doesn't help solve the issue.
Answered By - Adam Katz Answer Checked By - Willingham (WPSolving Volunteer)