Issue
Running EC2 instance that has an IAM role attached that allows to copy/read files to/from an S3 bucket.
When logged into the EC2 instance (via ssh) I can perform all those tasks using aws s3 ... command. There are no credentials because it's using a role. Env does not have anything related to aws at all.
However, if I run a program (written in GO) that simply does:
exec.Command("bash", "-c", "aws s3 ls ....")
I get Partial credentials found in env, missing: AWS_SECRET_ACCESS_KEY
A bit confused here, shouldn't it just work since I'm running this process as the same user that I'm logged in as and that actually works as I've mentioned in the beginning? Why is it even looking for credentials?
Solution
OK, assuming you have a role (lets call it EC2S3AccessRole) that allows ec2 instance access to s3 bucket(s), you need to follow this doc: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instancedata-data-retrieval.html
Eg, in my case (golang), I would do something like this:
a. get token:
cmd := exec.Command("bash", "-c", `curl -X PUT http://169.254.169.254/latest/api/token -H "X-aws-ec2-metadata-token-ttl-seconds: 600"`)
b. get credentials using token from (a)
type IAMSecurityCreds struct {
Code string `json:"Code"`
LastUpdated time.Time `json:"LastUpdated"`
Type string `json:"Type"`
AccessKeyID string `json:"AccessKeyId"`
SecretAccessKey string `json:"SecretAccessKey"`
Token string `json:"Token"`
Expiration time.Time `json:"Expiration"`
}
...
cmd := exec.Command("bash", "-c", `curl -H "X-aws-ec2-metadata-token: `+token+`" -v http://169.254.169.254/latest/meta-data/iam/security-credentials/EC2S3AccessRole`)
...
json.Unmarshal into IAMSecurityCreds
c. once you have all three (token, key, secret), you can run something like this:
cmd := exec.Command("bash", "-c", `AWS_DEFAULT_REGION=us-east-1 AWS_ACCESS_KEY_ID=` + creds.AccessKeyID + ` AWS_SECRET_ACCESS_KEY=` + creds.SecretAccessKey + ` AWS_SESSION_TOKEN="` + creds.Token + `" aws s3 ........`)
That's it :)
Answered By - Pavel Answer Checked By - Timothy Miller (WPSolving Admin)
