My client are currently getting access to some objects with getSignedUrlPromise from the package aws-sdk. The request are done from the backend and the signed url is returned to the client, everything is fine.
I'm now trying to migrate from aws-sdk to #aws-sdk/client-s3. I'd like to keep to same structure, but i can't find such command in the documentation.
I'm pretty sure #aws-sdk/client-s3 is capable of returning a signed url
Are there any (non - hacky) ways to do it ?
EDIT: Relying on this, i should use #aws-sdk/s3-request-presigner on top of #aws-sdk/client-s3 to get presigned urls.
You can use #aws-sdk/s3-request-presigner. For example:
const { getSignedUrl } = require("#aws-sdk/s3-request-presigner");
const { S3Client, GetObjectCommand } = require("#aws-sdk/client-s3");
const clientParams = { region: "us-east-1" };
const getObjectParams = { Bucket: "mybucket", Key: "dogs/snoopy.png" };
const client = new S3Client(clientParams);
const command = new GetObjectCommand(getObjectParams);
const url = await getSignedUrl(client, command, { expiresIn: 3600 });
console.log(url);
Related
I've been working lately with AWS CDN + S3 bucket as origin.
When no lambda#edge function attached into the Origin Request it works fine but I need the lambda#edge fn for rerouting. Some of the js file's mimetype are not text/javascript instead text/html. Any idea?
Lambda#edge function is just a simple rerouting no special feature.
Here is the function:
'use strict';
exports.handler = (evt, context, cb) => {
const { request } = evt.Records[0].cf;
const uriParts = request.uri.split("/")[1];
const locales = ['en-US', 'ja', 'ms'];
if (!uriParts || !locales.includes(uriParts)) {
request.uri = '/en-US/index.html';
return cb(null, request)
}
request.uri = `/${uriParts}/index.html`;
console.log(`Request Uri: ${request.uri}`);
cb(null, request);
}
Sample response image
Setup:
S3 Bucket
en-US/
index.html
<bunch of js files from angular>
ja/
index.html
<bunch of js files from angular>
ms/
index.html
<bunch of js files from angular>
CDN Distribution + Origin is the S3
Solved. First of all, my bad for not understanding well the triggers of Lambda#Edge function. Refer to this article.
First try, I used origin request as s trigger and somehow came to understand that it wasn't I need for my case.
After some debug and reading, I used viewer request this time. Also, alter my Lambda#Edge function to this.
'use strict';
const path = require("path");
exports.handler = async (evt,context,cb) => {
const { request } = evt.Records[0].cf;
console.log(`Original Uri: ${request.uri}`);
const uriParts = request.uri.split("/");
const locale = uriParts.length > 1 ? uriParts[1] : "";
const locales = ["en-US", "ja", "ms"];
if (locale === "" || locale === "index.html") {
request.uri = "/en-US/index.html";
return cb(null, request);
}
if (!locales.includes(locale)) return cb(null, request);
const lastPartUrl = uriParts[uriParts.length - 1];
const fileExt = path.extname(lastPartUrl);
if (!fileExt) request.uri = `/${locale}/index.html`;
console.log(`New Uri: ${request.uri}`);
return cb(null, request);
};
Now every locale in our application is now working fine.
I am trying to generate a presignedurl to get an object from s3 inside a container via Fargate task. It has s3:GetObject and s3:PutObject permission. When I call s3.getSignedUrl(params) it only returns https://s3.ap-northeast-1.amazonaws.com/ instead of the full signed url here is the code that I am using inside the container
const getFileUrl = async (key, bucketName) => {
try {
const s3 = new aws.S3();
const url = await s3.getSignedUrl('getObject', { Bucket: bucketName, Key: key });
return url;
} catch (error) {
console.error(error);
return false;
}
}
I am scouring the documentation, and it only provides pseudo-code of the credentials for v3 (e.g. const client = new S3Client(clientParams)
How do I initialize an S3Client with the bucket and credentials to perform a getSignedUrl request? Any resources pointing me in the right direction would be most helpful. I've even searched YouTube, SO, etc and I can't find any specific info on v3. Even the documentation and examples doesn't provide the actual code to use credentials. Thanks!
As an aside, do I have to include the fake folder structure in the filename, or can I just use the actual filename? For example: bucket/folder1/folder2/uniqueFilename.zip or uniqueFilename.zip
Here's the code I have so far: (Keep in mind I was returning the wasabiObjKey to ensure I was getting the correct file name. I am. It's the client, GetObjectCommand, and getSignedUrl that I'm having issues with.
exports.getPresignedUrl = functions.https.onCall(async (data, ctx) => {
const wasabiObjKey = `${data.bucket_prefix ? `${data.bucket_prefix}/` : ''}${data.uid.replace(/-/g, '_').toLowerCase()}${data.variation ? `_${data.variation.replace(/\./g, '').toLowerCase()}` : ''}.zip`
const { S3Client, GetObjectCommand } = require('#aws-sdk/client-s3')
const s3 = new S3Client({
bucketEndpoint: functions.config().s3_bucket.name,
region: functions.config().s3_bucket.region,
credentials: {
secretAccessKey: functions.config().s3.secret,
accessKeyId: functions.config().s3.access_key
}
})
const command = new GetObjectCommand({
Bucket: functions.config().s3_bucket.name,
Key: wasabiObjKey,
})
const { getSignedUrl } = require("#aws-sdk/s3-request-presigner")
const url = getSignedUrl(s3, command, { expiresIn: 60 })
return wasabiObjKey
})
There are a credential chain that provide credential to your API calls from SDK
https://docs.aws.amazon.com/sdk-for-javascript/v2/developer-guide/setting-credentials-node.html
Loaded from AWS Identity and Access Management (IAM) roles for Amazon
EC2
Loaded from the shared credentials file (~/.aws/credentials)
Loaded from environment variables
Loaded from a JSON file on disk
Other credential-provider classes provided by the JavaScript SDK
You can embed the credential inside your source code but it's not the prefered way
new S3Client(configuration: S3ClientConfig): S3Client
Where S3ClientConfig contain a credentials property
https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/clients/client-s3/modules/credentials.html
const { S3Client,GetObjectCommand } = require("#aws-sdk/client-s3");
let client = new S3Client({
region:'ap-southeast-1',
credentials:{
accessKeyId:'',
secretAccessKey:''
}
});
(async () => {
const response = await client.send(new GetObjectCommand({Bucket:"BucketNameHere",Key:"ObjectNameHere"}));
console.log(response);
})();
Sample answer
'$metadata': {
httpStatusCode: 200,
requestId: undefined,
extendedRequestId: '7kwrFkEp3lEnLU+OtxjrgdmS6gQmvPdbnqqR7I8P/rdFrUPBkdKYPYykWivuHPXCF1IHgjCIbe8=',
cfId: undefined,
attempts: 1,
totalRetryDelay: 0
},
Here's a simple approach I use (in Deno) for testing (in case you don't want to go the signedUrl approach and just let the SDK do the heavy lifting for you):
import { config as env } from 'https://deno.land/x/dotenv/mod.ts' // https://github.com/pietvanzoen/deno-dotenv
import { S3Client, ListObjectsV2Command } from 'https://cdn.skypack.dev/#aws-sdk/client-s3' // https://github.com/aws/aws-sdk-js-v3
const {AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY} = env()
// https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/clients/client-s3/modules/credentials.html
const credentials = {
accessKeyId: AWS_ACCESS_KEY_ID,
secretAccessKey: AWS_SECRET_ACCESS_KEY,
}
// https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/clients/client-s3/interfaces/s3clientconfig.html
const config = {
region: 'ap-southeast-1',
credentials,
}
// https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/clients/client-s3/classes/s3client.html
const client = new S3Client(config)
export async function list() {
// https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/clients/client-s3/interfaces/listobjectsv2commandinput.html
const input = {
Bucket: 'BucketNameHere'
}
// https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/clients/client-s3/classes/command.html
const cmd = new ListObjectsV2Command(input)
// https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/clients/client-s3/classes/listobjectsv2command.html
return await client.send(cmd)
}
I am following the proposed solution by Trivikr for adding support for s3.getSignedUrl api which is not currently available in newer v3. I am trying to make a signed url for getting an object from bucket.
Just for convenience, the code is being added below:
const { S3, GetObjectCommand } = require("#aws-sdk/client-s3"); // 1.0.0-gamma.2 version
const { S3RequestPresigner } = require("#aws-sdk/s3-request-presigner"); // 0.1.0-preview.2 version
const { createRequest } = require("#aws-sdk/util-create-request"); // 0.1.0-preview.2 version
const { formatUrl } = require("#aws-sdk/util-format-url"); // 0.1.0-preview.1 //version
const fetch = require("node-fetch");
(async () => {
try {
const region = "us-east-1";
const Bucket = `SOME_BUCKET_NAME`;
const Key = `SOME_KEY_VALUE`;
const credentials = {
accessKeyId: ACCESS_KEY_HERE,
secretAccessKey: SECRET_KEY_HERE,
sessionToken: SESSION_TOKEN_HERE
};
const S3Client = new S3({ region, credentials, signatureVersion: 'v4' });
console.log('1'); // for quick debugging
const signer = new S3RequestPresigner({ ...S3Client.config });
console.log('2')
const request = await createRequest(
S3Client,
new GetObjectCommand({ Key, Bucket })
);
console.log('3');
let signedUrl = formatUrl(await signer.presign(request));
console.log(signedUrl);
let response = await fetch(signedUrl);
console.log("Response", response);
}catch(e) {
console.error(e);
}
I successfully create S3Client and signer but on creating request, I get the following error:
clientStack.concat(...).filter is not a function
Anything wrong I am doing?
Please also note that I am using webpack for bundling
Just add my example in TypeScript:
import { S3Client, GetObjectCommand, S3ClientConfig } from '#aws-sdk/client-s3';
import { getSignedUrl } from '#aws-sdk/s3-request-presigner';
const s3Configuration: S3ClientConfig = {
credentials: {
accessKeyId: '<ACCESS_KEY_ID>',
secretAccessKey: '<SECRET_ACCESS_KEY>'
},
region: '<REGION>',
};
const s3 = new S3Client(s3Configuration);
const command = new GetObjectCommand({Bucket: '<BUCKET>', Key: '<KEY>' });
const url = await getSignedUrl(s3, command, { expiresIn: 15 * 60 }); // expires in seconds
console.log('Presigned URL: ', url);
RESOLVED
I ended up successfully making the signed urls by installing the beta versions rather than preview (default) ones
I am using DigitalOcean for storing videos. I am using Firebase callable function to generate the signed URL with AWS SDK and sent it back to my app. When I use Firebase function to generate a signed URL then it gives me the Signature Does not match error. The Firebase function is as follows-
const functions = require('firebase-functions');
const admin = require('firebase-admin');
admin.initializeApp();
const AWS = require('aws-sdk');
exports.getS3SignedUrlUpload = functions.https.onCall((data, context) => {
const spacesEndpoint = new AWS.Endpoint('https://sgp1.digitaloceanspaces.com');
AWS.config.update({
endpoint: spacesEndpoint,
accessKeyId: 'mykey',
secretAccessKey: 'mySecretKey',
signatureVersion: 'v4',
region: "sgp1"
});
var s3 = new AWS.S3();
const s3Params = {
Bucket: 'myappvideo', //data.S3BucketName,
Key: 'video14.mp4',//data.key,
Expires: 60*60 // Expires in 10 minutes
};
var v = s3.getSignedUrl('putObject', s3Params);
console.log("PRESIGNED URL", v);
return v;
});
I accidentally use getObject instead of putObject. Now it is working fine for me. But I don't know why it works, as I don't have any networking concepts. The line where I did the change is:
var v = s3.getSignedUrl('putObject', s3Params);
to
var v = s3.getSignedUrl('getObject', s3Params);