Thursday, December 30, 2021

[SOLVED] Curl command from Python Subprocess to store output into variable

Issue

I am using the below python code to read data from a url. The curl command from unix works. But when i try to store the returned json in a python variable, it is always blank.

Any pointers ? I do see the output on the Spyder Console, but never in the variable.

    p =sp.Popen(["curl","-i","-X", "POST" ,"-H", "Content-Type:application/json" ,"-H", "Authorization:Basic NEg0VU9QR1BZODAWVI4N1dLUFpXRzp4SVpxUUkzbUFuVG9RUlJDcXBLWkdB","-d", '{ "grant_type": "client_credentials" }', "https://rridata.wikimapia.com/v1.0/oauth/token/"], stdout = sp.PIPE, shell=False)
#p =sp.check_output(['curl','-i','-X', 'POST' ,'-H', 'Content-Type:application/json' ,'-H', 'Authorization:Basic NEg0VU9QR1BZODATEpDc2oyNGRGa0c5SVpxUUkzbUFuVG9RUlJDcXBLWkdB','-d', '{ "grant_type": "client_credentials" }', 'https://rdata.wikimapia.com/v1.0/oauth/token/'])
out,err = p.communicate()
print out

EDIT: My environment details. I am on Windows 7 , executing the command from Anaconda Spyder IDE.


Solution

Since working on a secure connection behind firewall. Hence, i have to set the proxy before i make the call.

    os.environ['https_proxy']="https://iss-uk.corporate.pb.com:80"
p =sp.Popen(["curl","-i","-X", "POST" ,"-H", "Content-Type:application/json" ,"-H", "Authorization:Basic NEg0VU9QR1BZODAWVI4N1dLUFpXRzp4WG1HczbUFuVG9RUlJDcXBLWkdB","-d", '{ "grant_type": "client_credentials" }', "https://rridata.wikimapia.com/v1.0/oauth/token/"], stdout = sp.PIPE, shell=False)
#p =sp.check_output(['curl','-i','-X', 'POST' ,'-H', 'Content-Type:application/json' ,'-H', 'Authorization:Basic NEg0VU9QR1BZODATEpDc2oyNGRGa0c5SVpxUUkzbUFuVG9RUlJDcXBLWkdB','-d', '{ "grant_type": "client_credentials" }', 'https://rdata.wikimapia.com/v1.0/oauth/token/'])
out,err = p.communicate()
print ("out:",out,"err:",err)


Answered By - ForeverLearner
at