I'm trying to generate an RSA public-private key pair. The public key will be uploaded to AWS CloudFront. I found a code sample here and changed two things:
bitsize is 2048. According to CloudFront documentation, this is the expected bit size.
Type for public key is PUBLIC KEY. CloudFront expects -----BEGIN PUBLIC KEY----- and -----END PUBLIC KEY----- lines.
Here's the final code:
package main
import (
"crypto/rand"
"crypto/rsa"
"crypto/x509"
"encoding/pem"
"io/ioutil"
)
func main() {
filename := "key"
bitSize := 2048
// Generate RSA key.
key, err := rsa.GenerateKey(rand.Reader, bitSize)
if err != nil {
panic(err)
}
// Extract public component.
pub := key.Public()
// Encode private key to PKCS#1 ASN.1 PEM.
keyPEM := pem.EncodeToMemory(
&pem.Block{
Type: "RSA PRIVATE KEY",
Bytes: x509.MarshalPKCS1PrivateKey(key),
},
)
// Encode public key to PKCS#1 ASN.1 PEM.
pubPEM := pem.EncodeToMemory(
&pem.Block{
Type: "PUBLIC KEY",
Bytes: x509.MarshalPKCS1PublicKey(pub.(*rsa.PublicKey)),
},
)
// Write private key to file.
if err := ioutil.WriteFile(filename+".rsa", keyPEM, 0700); err != nil {
panic(err)
}
// Write public key to file.
if err := ioutil.WriteFile(filename+".rsa.pub", pubPEM, 0755); err != nil {
panic(err)
}
}
When I upload the public key to CloudFront, I get the following error:
Your request contains empty/invalid/out of limits RSA Encoded Key
This code needs to run in a Lambda and rotate a secret in SecretsManager. Locally, I can run openssl genrsa and openssl rsa commands. The public key will then be accepted by CloudFront.
Why is the public key generated by the code not being accepted and how can the code be fixed?
If you are using Ruby SDK v3, you can generate cloudfront RSA key pair and upload the key by doing this
#!/usr/bin/env ruby
require 'rubygems'
require 'aws-sdk-cloudfront'
require 'openssl'
key = OpenSSL::PKey::RSA.new 2048
open 'private_key.pem', 'w' do |io| io.write key.to_pem end
open 'public_key.pem', 'w' do |io| io.write key.public_key.to_pem end
cloudfront_public_content = File.read('public_key.pem')
cloudfront_client = Aws::CloudFront::Client.new()
cloudfront_public_key_resp = cloudfront_client.create_public_key({
public_key_config: {
caller_reference: "test-key-ruby",
name: "test-ruby-key",
encoded_key: cloudfront_public_content,
comment: "string",
},
})
p cloudfront_public_key_resp.public_key.id
save the above codes as file test.rb
and just run
ruby test.rb
Related
I'm looking to integrade an S3 bucket with an API im developing, I'm running into this error wherever I go -
SignatureDoesNotMatch: The request signature we calculated does not match the signature you provided. Check your key and signing method.
status code: 403
I have done the following
Installed SDK & AWS CLI, and AWS configured
Double(triple) checked spelling of key & secret key & bucket permissions
Attempted with credentials document, .env, and even hard coding the values directly
Tested with AWS CLI (THIS WORKS), so I believe I can rule out permissions, keys, as a whole.
I'm testing by trying to list buckets, here is the code taken directly from the AWS documentation-
sess := session.Must(session.NewSessionWithOptions(session.Options{ <--- DEBUGGER SET HERE
SharedConfigState: session.SharedConfigEnable,
}))
svc := s3.New(sess)
result, err := svc.ListBuckets(nil)
if err != nil { exitErrorf("Unable to list buckets, %v", err) }
for _, b := range result.Buckets {
fmt.Printf("* %s created on %s\n", aws.StringValue(b.Name), aws.TimeValue(b.CreationDate))
}
Using debugger, I can see the sessions config files as the program runs, the issue is potentially here
config -
-> credentials
-> creds
-> v
-> Access Key = ""
-> Secret Access Key = ""
-> Token = ""
-> provider
->value
-> Access Key With Value
-> Secret Access Key With Value
-> Token With Value
I personally cannot find any documentation regarding "creds" / "v", and I don't know if this is causing the issue. As I mentioned, I can use the AWS CLI to upload into the bucket, and even when I hard code my access key etc in to the Go SDK I receive this error.
Thank you for any thoughts, greatly appreciated.
I just compiled your code and its executing OK ... one of the many ways to supply credentials to your binary is to populate these env vars
export AWS_ACCESS_KEY_ID=AKfoobarT2IJEAU4
export AWS_SECRET_ACCESS_KEY=oa6oT0Xxt4foobarbambazdWFCb
export AWS_REGION=us-west-2
that is all you need when using the env var approach ( your values are available using the aws console browser )
the big picture is to create a wrapper shell script ( bash ) which contains above three lines to populate the env vars to supply credentials then in same shell script execute the golang binary ( typically you compile the golang in some preliminary process ) ... in my case I store the values of my three env vars in encrypted files which the shell script decrypts just before it calls the above export commands
sometimes its helpful to drop kick and just use the aws command line equivalent commands to get yourself into the ballpark ... from a terminal run
aws s3 ls s3://cairo-mombasa-zaire --region us-west-2
which can also use those same env vars shown above
for completeness here is your code with boilerplate added ... this runs OK and lists out the buckets
package main
import (
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/aws/session"
"github.com/aws/aws-sdk-go/service/s3"
// "github.com/aws/aws-sdk-go/service/s3/s3manager"
"fmt"
"os"
)
func exitErrorf(msg string, args ...interface{}) {
fmt.Fprintf(os.Stderr, msg+"\n", args...)
os.Exit(1)
}
func main() {
region_env_var := "AWS_REGION"
curr_region := os.Getenv(region_env_var)
if curr_region == "" {
exitErrorf("ERROR - failed to get region from env var %v", region_env_var)
}
fmt.Println("here is region ", curr_region)
// Load session from shared config
sess := session.Must(session.NewSessionWithOptions(session.Options{
SharedConfigState: session.SharedConfigEnable,
}))
svc := s3.New(sess)
result, err := svc.ListBuckets(nil)
if err != nil { exitErrorf("Unable to list buckets, %v", err) }
for _, b := range result.Buckets {
fmt.Printf("* %s created on %s\n", aws.StringValue(b.Name), aws.TimeValue(b.CreationDate))
}
}
numBytes, err := downloader.Download(tempFile,
&s3.GetObjectInput{
Bucket: aws.String(bucket),
Key: aws.String(fileName),
},
)
In my case the bucket value was wrong, it is missing literal "/" at the end. Adding that fixes my problem.
Error i got - err: SignatureDoesNotMatch: The request signature we calculated does not match the signature you provided. Check your key and signing method.
status code: 403,
If anyone else happens to have this problem,
The issue was regarding environment variables much like Scott suggest above, however it was due to lacking
export AWS_SDK_LOAD_CONFIG="true"
If this environment variable is not present, then the Golang SDK will not look for a credentials file, along with this, I instantiated environment variables for both my keys which allowed the connection to succeed.
To recap
if you're attempting to use the shared credentials folder, you must use the above noted environment variable to enable it.
If you're using environment variables, you shouldn't be affected by this problem.
Whenever I try to generate a Presign URL using AWS API I get something like this:
https://{OBJECT_PATH}?X-Amz-Algorithm=AWS4-HMAC-SHA256\\u0026X-Amz-Credential=AKIAJMALZY6GKVGFVOCQ%2F20191003%2Fus-east-1%2Fs3%2Faws4_request\\u0026X-Amz-Date=20191003T111419Z\\u0026X-Amz-Expires=300\\u0026X-Amz-SignedHeaders=host\\u0026X-Amz-Signature=c72a66c249fe2dada4c3925388efc62cbc734c84e01ffbf4748a5f6be9a99026\
When I try to enter it, I get the following message:
<Error>
<Code>AuthorizationQueryParametersError</Code>
<Message>X-Amz-Algorithm only supports "AWS4-HMAC-SHA256"</Message
<RequestId>7236615A79C34A4B</RequestId>
<HostId>PHoX1otuX6apJrXRTOMM/8GvgiajrNPdElrnfSzAZCzpLzMG8NFu9RIJEiyEYHWAls91n882QNE=</HostId>
</Error>
I tried to replace \\u0026 with & because I saw on documentation (https://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-query-string-auth.html) that this is how presign URL should be structured, I get:
<Error>
<Code>SignatureDoesNotMatch</Code>
<Message>The request signature we calculated does not match the signature you provided. Check your key and signing method.</Message>
<AWSAccessKeyId>AKIAJMALZY6GKVGFVOCQ</AWSAccessKeyId>
<StringToSign>AWS4-HMAC-SHA256
20191003T112220Z
20191003/us-east-1/s3/aws4_request
a1d8f89ff0472249afb8f1320314eef950bcef423f05fe98420f53cc7f7d8e8f</StringToSign>
<SignatureProvided>a1b2b7080c67e6456cbd2ab678565e5f3857483378e69ca03e35cc83232b2844\</SignatureProvided>
<StringToSignBytes>{SEQUENCE_OF_BYTES}</StringToSignBytes>
<CanonicalRequest>GET/{OBJECT_KEY}?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAJMALZY6GKVGFVOCQ%2F20191003%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20191003T112220Z&X-Amz-Expires=300&X-Amz-SignedHeaders=host host:{BUCKET_NAME} host UNSIGNED-PAYLOAD</CanonicalRequest>
<CanonicalRequestBytes>{SEQUENCE_OF_BYTES}</CanonicalRequestBytes>
<RequestId>F3A0303ADD62D8FD</RequestId
<HostId>yU7FdXLoEO+j+KpZtQH1YjF6l3RnVugBi6rzCRJOgVYk7FEQqreobMuFdSiYtzIKUgl0Qr2GcAQ=</HostId>
</Error>
Finally, here is the code I use in order to generate these URLs:
var accessableURLs []string
for _, fileName := range fileNames {
objectKey := filepath.Join(fileDir, fileName)
req, _ := s3Obj.Client.GetObjectRequest(&s3.GetObjectInput{
Bucket: aws.String(s3Obj.Bucket),
Key: aws.String(objectKey),
})
urlStr, err := req.Presign(expireAfter)
if err != nil {
glog.Errorf("Failed during generating pre-signed S3 url, reason -%v ", err)
return nil, err
}
accessableURLs = append(accessableURLs, urlStr)
}
The following works with the v1 aws sdk. I'm not sure why your solution fails:
package main
import (
"fmt"
"time"
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/aws/session"
"github.com/aws/aws-sdk-go/service/s3"
)
func main() {
MyBucketName := "myAwesomeBucket"
MyKeyName := "/path/filename.txt"
days := 5
session, err := session.NewSession(&aws.Config{
Region: aws.String("us-east-1")},
)
s := s3.New(session)
r, _ := s.GetObjectRequest(&s3.GetObjectInput{
Bucket: aws.String(MyBucketName),
Key: aws.String(MyKeyName),
})
url, err := r.Presign(time.Duration(days) * 24 * time.Hour)
if err != nil {
fmt.Println("failure")
}
fmt.Println(url)
}
One thing to note is that the IAM creds used to sign the url must be valid for as long as the signed url is intended to be valid for. This ends up being an issue with ec2 machines that use temporary creds. A workaround is for the application to have a permanent IAM cred that it uses to sign the url with.
I'd like to upload files to my s3 bucket via the aws golang sdk. I have a web server listening to POST requests and I'm expecting to receive multiple files of any type.
Using the sdk, the s3 struct PutObjectInput expects Body to be of type io.ReadSeeker and I'm not sure how to extract the content from the files uploaded and in turn satisfy the io.ReadSeeker interface.
images := r.MultipartForm.File
for _, files := range images {
for _, f := range files {
# In my handler, I can loop over the files
# and see the content
fmt.Println(f.Header)
_, err = svc.PutObjectWithContext(ctx, &s3.PutObjectInput{
Bucket: aws.String("bucket"),
Key: aws.String("key"),
Body: FILE_CONTENT_HERE,
})
}
}
Use the FileHeader.Open method to get an io.ReadSeeker.
f, err := f.Open()
if err != nil {
// handle error
}
_, err = svc.PutObjectWithContext(ctx, &s3.PutObjectInput{
Bucket: aws.String("bucket"),
Key: aws.String("key"),
Body: f,
})
Open returns a File. This type satisfies the io.ReadSeeker interface.
Use the S3 Manager's Uploader.Upload method, http://docs.aws.amazon.com/sdk-for-go/api/service/s3/s3manager/#Uploader.Upload. We have an example at http://docs.aws.amazon.com/sdk-for-go/v1/developer-guide/s3-example-basic-bucket-operations.html#s3-examples-bucket-ops-upload-file-to-bucket.
I am working in Golang,now I am attempting to upload an image to AWS S3, but I get:
NoCredentialProviders: no valid providers in chain. Deprecated.
For verbose messaging see aws.Config.CredentialsChainVerboseErrors
My code is like this:
func firstFunction(){
//Connect to S3
AWSsession, err := ConnectAWS()
if err != nil {
fmt.Println("Error Connecting to AWS S3")
}
GetSingleMedia(AWSsession)
}
func ConnectAWS()(*session.Session, error){
//Create S3 Session
AWSsession, err := session.NewSession(&aws.Config{
Region: aws.String("us-west-2")},
)
if err != nil {
fmt.Println("Error AWS:", err.Error())
}
return AWSsession,err
}
func GetSingleMedia(...someparams,AWSsession *session.Session){
//o.Blob is correct, this is valid
data, err := ioutil.ReadAll(bytes.NewReader(o.Blob))
//Store: bytes.NewReader(o.Blob)
UploadImage(AWSsession,bytes.NewReader(o.Blob),bucket,"SomeID")
}
func UploadImage(AWSsession *session.Session,reader *bytes.Reader,bucket string, key string) (*s3manager.UploadOutput,error){
uploader := s3manager.NewUploader(AWSsession)
result, err := uploader.Upload(&s3manager.UploadInput{
Body : reader,
Bucket: aws.String(bucket),
Key : aws.String(key),
})
if err != nil {
fmt.Println("Error uploagin img: ",err.Error())
}
return result,err
}
Also, I have placed the creentials under /home/myuser/.aws/ there's a credential file, I don't get any error on creating the session, then, what could be the problem?
The error is triggered in UploadImage
EDIT:
Currently in the credentials file I have:
[default]
awsBucket = "someBucket"
awsAccessKey = "SOME_ACCESS_KEY"
awsSecretKey = "SOME_AWS_SECRET_KEY"
Sould I change any permission or something?
I would suggest you follow the guide here: http://docs.aws.amazon.com/cli/latest/userguide/cli-chap-getting-started.html
This command will ask you for access/secret key and write them in a correct format:
aws configure
It would appear you have a wrong format of credentials file. The correct format would be something like this:
[default]
aws_access_key_id = SOME_ACCESS_KEY
aws_secret_access_key = SOME_AWS_SECRET_KEY
I need to get public permanent (not signed) URL of a resource using golang and official aws go sdk. In Java AWS S3 SDK there's a method called getResourceUrl() what's the equivalent in go?
This is how you get presigned URLs using the go sdk:
func GetFileLink(key string) (string, error) {
svc := s3.New(some params)
params := &s3.GetObjectInput{
Bucket: aws.String(a bucket name),
Key: aws.String(key),
}
req, _ := svc.GetObjectRequest(params)
url, err := req.Presign(15 * time.Minute) // Set link expiration time
if err != nil {
global.Log("[AWS GET LINK]:", params, err)
}
return url, err
}
If what you want is just the URL of a public access object you can build the URL yourself:
https://<region>.amazonaws.com/<bucket-name>/<key>
Where <region> is something like us-east-2. So using go it will be something like:
url := "https://%s.amazonaws.com/%s/%s"
url = fmt.Sprintf(url, "us-east-2", "my-bucket-name", "some-file.txt")
Here is a list of all the available regions for S3.
Looks almost clean:
import "github.com/aws/aws-sdk-go/private/protocol/rest"
...
params := &s3.GetObjectInput{
Bucket: aws.String(a bucket name),
Key: aws.String(key),
}
req, _ := svc.GetObjectRequest(params)
rest.Build(req) // aws method to build URL in request object
url = req.HTTPRequest.URL.String() // complete URL to resource