How I know the signature of a signed url? - amazon-web-services

I have a lambda which generates a signed URL for users to upload files to s3 bucket. The code is in nodejs:
export const getSignedURL = async (): Promise<{ signedURL: string }> => {
try {
const s3 = new S3();
const params = {
Bucket: CONFIG.s3Bucket,
Key: `${CONFIG.s3PictureFolder}/${uuidv4()}`,
Expires: CONFIG.presignedURLExpires,
};
const signedURL = await s3.getSignedUrlPromise('putObject', params);
console.log(`generate signedURL url: ${signedURL}`);
return { signedURL };
} catch (err) {
console.error(err);
throw err;
}
};
I am able to get the url success. However, when I test it via curl:
curl -XPUT PRESIGNED_URL --data "my data"
I got this error:
<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>SignatureDoesNotMatch</Code><Message>The request signature we calculated does not match the signature you provided. Check your key and signing method.</Message><AWSAccessKeyId>XXXX</AWSAccessKeyId><StringToSign>
It seems that this URL requires an access key. Does this key mean AWS credential key which is issued by IAM?
This URL was generated by the lambda function. How do I know which key it uses? And I'd like to generate a public presigned url for users to upload. Is there a way to do that?

After come debugging, I found that I need to add Content-Type when generate the presigned URL like below:
const params = {
Bucket: CONFIG.s3Bucket,
Key: `${CONFIG.s3PictureFolder}/${uuidv4()}`,
Expires: CONFIG.presignedURLExpires,
ContentType: 'text/plain'
};

Related

Dart Signature V4 client Error: The request signature we calculated does not match the signature you provided

I am using the following code to connect AWS REST API using IAM authentication. It's working fine on Google Chrome but giving errors in Firefox.
import 'package:aws_signature_v4/aws_signature_v4.dart';
const signer = AWSSigV4Signer();
final scope = AWSCredentialScope(
region: 'ap-south-1',
service: AWSService.apiGatewayManagementApi,
);
final request = AWSHttpRequest(
method: AWSHttpMethod.post,
uri: Uri.https('<URL>', 'PATH'),
headers: const {
AWSHeaders.contentType: 'application/json;charset=utf-8',
},
body: json.encode(payload).codeUnits,
);
// Sign and send the HTTP request
final signedRequest = await signer.sign(
request,
credentialScope: scope,
);
final resp = await signedRequest.send();
Error:
{"message":"The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method. Consult the service documentation for details.
The Canonical String for this request should have been
'POST
<PATH>
content-type:application/json;charset=utf-8
host: <URL>
x-amz-content-sha256: <Key>
x-amz-date:20221017T065918Z
x-amz-user-agent:aws-sigv4-dart/0.2.2
content-type;host;x-amz-content-sha256;x-amz-date;x-amz-user-agent
<Key-1>
The String-to-Sign should have been
'AWS4-HMAC-SHA256
20221017T065918Z
20221017/ap-south-1/execute-api/aws4_request
<Key-2>
"
}
I am using docs from https://docs.amplify.aws/lib/project-setup/escape-hatch/q/platform/flutter/
I seen couple of question on similar issue with no answers.
Appreciate any help on this.

Uploading Base64 file to S3 signed URL

I need to upload an image to S3 using signed URL. I have the image in a base64 string. The below code runs without throwing any error, but at the end I see a text file with base64 content in the S3, not the binary image.
Can you please point out what I am missing?
Generate Signed URL (Lambda function JavaScript)
const signedUrlExpireSeconds = 60 * 100;
var url = s3.getSignedUrl("putObject", {
Bucket: process.env.ScreenshotBucket,
Key: s3Key,
ContentType: "image/jpeg",
ContentEncoding: "base64",
Expires: signedUrlExpireSeconds,
});
Upload to S3 (Java Code)
HttpRequest request = HttpRequest.newBuilder().PUT(HttpRequest.BodyPublishers.ofString(body))
.header("Content-Encoding", "base64").header("Content-Type", "image/jpeg").uri(URI.create(url)).build();
HttpResponse<String> response = httpClient.send(request, HttpResponse.BodyHandlers.ofString());
if (response.statusCode() != 200) {
throw new Exception(response.body());
}
I am not familiar with the AWS JavaScript SDK. But it seems that setting the 'Content-Type' metadata of the object (not the Content-Type of the putObject HTTP request) to 'image/jpeg' should do the trick.
Fixed it while just playing around with the combinations.
HttpRequest request = HttpRequest.newBuilder().PUT(HttpRequest.BodyPublishers.ofString(body))
Changed to
HttpRequest request = HttpRequest.newBuilder().PUT(HttpRequest.BodyPublishers.ofByteArray(body))

TimeoutError while generating AWS s3 Presigned URL

I've created a route that generates a pre-signed URL to upload an object to AWS S3. It was working initially but lately, it's returning a Timeout Error.
Here is my controller code:
async (req, res, next)=>{
const BUCKET = req.params.bucket;
const KEY = 'myKey_'+ uuid.v4();
const EXPIRATION = 60 * 60 * 1000;
let signedUrl;
try{
// Also have a configuration file ~.aws/.credentials
const s3 = new S3({
accessKeyId: process.env.AWS_ACCESS_KEY,
secretAccessKey: process.env.AWS_SECRET_KEY,
region: 'ap-south-1'
});
// creating a s3 presigner obj
const signer = new S3RequestPresigner({ ...s3.config });
// creating file upload request
const AWSUploadRequest = await createRequest(s3, new PutObjectCommand({
Bucket: BUCKET,
Key: KEY
}));
const expire = new Date(Date.now() + EXPIRATION);
// creating & formating presigned URL
signedUrl = formatUrl(await signer.presign(AWSUploadRequest, expire));
console.log(`Generated Signed URL: ${signedUrl}`);
}catch(err){
console.log(`Error creating presigned url ${err}`);
return next(
new ErrorResponse(
`Error while generating aws s3 presigned url`,
500
)
)}
res.status(200).json({
signedUrl
})
}
Here are my logs:
AWSUploadReq: {"method":"PUT","hostname":"s3.ap-south-1.amazonaws.com","query":{"x-id":"PutObject"},"headers":{"Content-Type":"application/octet-stream","user-agent":"aws-sdk-nodejs-v3-#aws-sdk/client-s3/1.0.0-rc.7 win32/v10.15.3"},"protocol":"https:","path":"/dammnn/myKey_file-c6f198d6-9e91-4892-8c88-8a49e15406c1"}
Error creating presigned url Error: TimeoutError
It's very vague as to why the request is getting timed out. Looking for some guidance on the same. Thanks.

Returning Dynamic Content-Type via AWS API Gateway and Lambda Function

I want to use an AWS API Gateway as a proxy for fetching files from an S3 bucket and returning them to the client. I'm using a Lambda function to talk to S3 and send the file to the client via the AWS API Gateway. I've rad that the best way to do this is to use a "Lambda proxy integration" so the entire request gets piped to Lambda without any modification. But if I do that then I can't setup an Integration Response for the resulting response from my Lambda function. So all the client gets is JSON.
It seems there should be a way for the API Gateway to take the JSON and transform the request to the proper response for the client but I can't seem to figure out how to make that happen. There are lots of examples that point to setting a content-type on the response from the API Gateway manually but I need to set the content-type header to whatever the file type is.
Also for images and binary formats my Lambda function is returning a base64 encoded string and the property isBase64Encoded set to true. When I go to the "Binary Support" section and specify something like image/* as a content type that should be returned as binary, it doesn't work. I only have success by setting the Binary Support content type to */* (aka everything) which won't work for non-binary content types.
What am I missing and why does this seem so difficult?
Turns out API Gateway isn't the problem. My Lambda function wasn't returning proper headers.
For handling binary responses I found you need to set Binary Support content type to */* (aka everything) and then have your Lambda function return the property isBase64Encoded set to true. Responses that are base64 encoded and indicated as such will be decoded and served as binary while other requests will be returned as is.
Here's a simple Gist for a Lambda function that takes a given path and reads the file from S3 and returns it via the API Gateway:
/**
* This is a simple AWS Lambda function that will look for a given file on S3 and return it
* passing along all of the headers of the S3 file. To make this available via a URL use
* API Gateway with an AWS Lambda Proxy Integration.
*
* Set the S3_REGION and S3_BUCKET global parameters in AWS Lambda
* Make sure the Lambda function is passed an object with `{ pathParameters : { proxy: 'path/to/file.jpg' } }` set
*/
var AWS = require('aws-sdk');
exports.handler = function( event, context, callback ) {
var region = process.env.S3_REGION;
var bucket = process.env.S3_BUCKET;
var key = decodeURI( event.pathParameters.proxy );
// Basic server response
/*
var response = {
statusCode: 200,
headers: {
'Content-Type': 'text/plain',
},
body: "Hello world!",
};
callback( null, response );
*/
// Fetch from S3
var s3 = new AWS.S3( Object.assign({ region: region }) );
return s3.makeUnauthenticatedRequest(
'getObject',
{ Bucket: bucket, Key: key },
function(err, data) {
if (err) {
return err;
}
var isBase64Encoded = false;
if ( data.ContentType.indexOf('image/') > -1 ) {
isBase64Encoded = true;
}
var encoding = '';
if ( isBase64Encoded ) {
encoding = 'base64'
}
var resp = {
statusCode: 200,
headers: {
'Content-Type': data.ContentType,
},
body: new Buffer(data.Body).toString(encoding),
isBase64Encoded: isBase64Encoded
};
callback(null, resp);
}
);
};
via https://gist.github.com/kingkool68/26aa7a3641a3851dc70ce7f44f589350

Invalid Date When Uploading to AWS S3

I'm trying to upload an image to S3 via putObject and a pre-signed URL.
Here is the URL that was provided when I generated the pre-signed URL:
https://<myS3Bucket>.s3.amazonaws.com/1ffd1c88-5661-48f9-a135-04bd569614dd.jpg?AWSAccessKeyId=<accessKey>&Expires=1458177431311&Signature=<signature>-amz-security-token=<token>
When I attempt to upload the file via a PUT, AWS responds with:
<?xml version="1.0" encoding="UTF-8"?>
<Error>
<Code>AccessDenied</Code>
<Message>Invalid date (should be seconds since epoch): 1458177431311</Message>
<RequestId>...</RequestId>
<HostId>...</HostId>
</Error>
Here is the curl version of the request I was using:
curl -X PUT -H "Cache-Control: no-cache" -H "Postman-Token: 78e46be3-8ecc- 4156-be3d-7e2f4688a127" -H "Content-Type: multipart/form-data; boundary=----WebKitFormBoundary7MA4YWxkTrZu0gW" -F "file=#[object Object]" "https://<myS3Bucket>.s3.amazonaws.com/1ffd1c88-5661-48f9-a135-04bd569614dd.jpg?AWSAccessKeyId=<accessKey>&Expires=1458177431311&Signature=<signature>-amz-security-token=<mySecurityToken>"
Since the timestamp is generated by AWS, it should be correct. I have tried changing it to include decimals and got the same error.
Could the problem be in the way I'm uploading the file in my request?
Update - Add code for generating the signed URL
The signed URL is being generated via the AWS Javascript SDK:
var AWS = require('aws-sdk')
var uuid = require('node-uuid')
var Promise = require('bluebird')
var s3 = new AWS.S3()
var params = {
Bucket: bucket, // bucket is stored as .env variable
Key: uuid.v4() + '.jpg' // file is always a jpg
}
return new Promise(function (resolve, reject) {
s3.getSignedUrl('putObject', params, function (err, url) {
if (err) {
reject(new Error(err))
}
var payload = { url: url }
resolve(payload)
})
})
My access key and secret key are loaded via environment variables as AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY.
Amazon S3 only supports 32 bit timestamps in signed urls:
2147483647 is the largest timestamp you can have.
I was creating timestamps 20 years in the future and it was breaking my signed URL's. Using a value less than 2,147,483,647 fixes the issue.
Hope this helps someone!
You just get bitten by bad documentation. Seems relate to this link
Amazon S3 invalid date when using expires in url_for
The integer are mean for " (to specify the number of seconds after the current time)".
Since you enter the time in epoch(0 = 1970-1-1), So it is current epoch time + your time, 46+46 years= 92 years. Appear to be crazy expiration period for s3.