I am using DigitalOcean for storing videos. I am using Firebase callable function to generate the signed URL with AWS SDK and sent it back to my app. When I use Firebase function to generate a signed URL then it gives me the Signature Does not match error. The Firebase function is as follows-
const functions = require('firebase-functions');
const admin = require('firebase-admin');
admin.initializeApp();
const AWS = require('aws-sdk');
exports.getS3SignedUrlUpload = functions.https.onCall((data, context) => {
const spacesEndpoint = new AWS.Endpoint('https://sgp1.digitaloceanspaces.com');
AWS.config.update({
endpoint: spacesEndpoint,
accessKeyId: 'mykey',
secretAccessKey: 'mySecretKey',
signatureVersion: 'v4',
region: "sgp1"
});
var s3 = new AWS.S3();
const s3Params = {
Bucket: 'myappvideo', //data.S3BucketName,
Key: 'video14.mp4',//data.key,
Expires: 60*60 // Expires in 10 minutes
};
var v = s3.getSignedUrl('putObject', s3Params);
console.log("PRESIGNED URL", v);
return v;
});
I accidentally use getObject instead of putObject. Now it is working fine for me. But I don't know why it works, as I don't have any networking concepts. The line where I did the change is:
var v = s3.getSignedUrl('putObject', s3Params);
to
var v = s3.getSignedUrl('getObject', s3Params);
Related
const S3 = require('aws-sdk/clients/s3');
const { getSignedUrl } = require("#aws-sdk/s3-request-presigner");
const { S3Client, GetObjectCommand } = require("#aws-sdk/client-s3");
dotenv.config();
const bucketName = process.env.AWS_BUCKET_NAME
const region = process.env.AWS_BUCKET_REGION
const accessKeyId = process.env.AWS_ACCESS_KEY
const secretAccessKey = process.env.AWS_SECRET_KEY
const s3 = new S3({
region,
accessKeyId,
secretAccessKey
})
router.get("/:id", async (req, res) => {
try {
const post = await Post.findById(req.params.id);
const getObjectParams = {
Bucket: bucketName,
Key: post.photo,
}
const command = new GetObjectCommand(getObjectParams);
const url = await getSignedUrl(s3, command, { expiresIn: 3600 });
post.imageUrl = url
res.status(200).json(post);
} catch (err) {
console.error('errorrr', err);
res.status(500).json(err);
}
});
Here is my code I've console logged post, getObjectParams, command and everything is there but when I console log url it's not logging and when I console.log errorrr it logs Cannot read properties of undefined (reading 'clone')
What is the issue here?
I think issue is with function getSignedUrl, but not sure what it is
My client are currently getting access to some objects with getSignedUrlPromise from the package aws-sdk. The request are done from the backend and the signed url is returned to the client, everything is fine.
I'm now trying to migrate from aws-sdk to #aws-sdk/client-s3. I'd like to keep to same structure, but i can't find such command in the documentation.
I'm pretty sure #aws-sdk/client-s3 is capable of returning a signed url
Are there any (non - hacky) ways to do it ?
EDIT: Relying on this, i should use #aws-sdk/s3-request-presigner on top of #aws-sdk/client-s3 to get presigned urls.
You can use #aws-sdk/s3-request-presigner. For example:
const { getSignedUrl } = require("#aws-sdk/s3-request-presigner");
const { S3Client, GetObjectCommand } = require("#aws-sdk/client-s3");
const clientParams = { region: "us-east-1" };
const getObjectParams = { Bucket: "mybucket", Key: "dogs/snoopy.png" };
const client = new S3Client(clientParams);
const command = new GetObjectCommand(getObjectParams);
const url = await getSignedUrl(client, command, { expiresIn: 3600 });
console.log(url);
I am scouring the documentation, and it only provides pseudo-code of the credentials for v3 (e.g. const client = new S3Client(clientParams)
How do I initialize an S3Client with the bucket and credentials to perform a getSignedUrl request? Any resources pointing me in the right direction would be most helpful. I've even searched YouTube, SO, etc and I can't find any specific info on v3. Even the documentation and examples doesn't provide the actual code to use credentials. Thanks!
As an aside, do I have to include the fake folder structure in the filename, or can I just use the actual filename? For example: bucket/folder1/folder2/uniqueFilename.zip or uniqueFilename.zip
Here's the code I have so far: (Keep in mind I was returning the wasabiObjKey to ensure I was getting the correct file name. I am. It's the client, GetObjectCommand, and getSignedUrl that I'm having issues with.
exports.getPresignedUrl = functions.https.onCall(async (data, ctx) => {
const wasabiObjKey = `${data.bucket_prefix ? `${data.bucket_prefix}/` : ''}${data.uid.replace(/-/g, '_').toLowerCase()}${data.variation ? `_${data.variation.replace(/\./g, '').toLowerCase()}` : ''}.zip`
const { S3Client, GetObjectCommand } = require('#aws-sdk/client-s3')
const s3 = new S3Client({
bucketEndpoint: functions.config().s3_bucket.name,
region: functions.config().s3_bucket.region,
credentials: {
secretAccessKey: functions.config().s3.secret,
accessKeyId: functions.config().s3.access_key
}
})
const command = new GetObjectCommand({
Bucket: functions.config().s3_bucket.name,
Key: wasabiObjKey,
})
const { getSignedUrl } = require("#aws-sdk/s3-request-presigner")
const url = getSignedUrl(s3, command, { expiresIn: 60 })
return wasabiObjKey
})
There are a credential chain that provide credential to your API calls from SDK
https://docs.aws.amazon.com/sdk-for-javascript/v2/developer-guide/setting-credentials-node.html
Loaded from AWS Identity and Access Management (IAM) roles for Amazon
EC2
Loaded from the shared credentials file (~/.aws/credentials)
Loaded from environment variables
Loaded from a JSON file on disk
Other credential-provider classes provided by the JavaScript SDK
You can embed the credential inside your source code but it's not the prefered way
new S3Client(configuration: S3ClientConfig): S3Client
Where S3ClientConfig contain a credentials property
https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/clients/client-s3/modules/credentials.html
const { S3Client,GetObjectCommand } = require("#aws-sdk/client-s3");
let client = new S3Client({
region:'ap-southeast-1',
credentials:{
accessKeyId:'',
secretAccessKey:''
}
});
(async () => {
const response = await client.send(new GetObjectCommand({Bucket:"BucketNameHere",Key:"ObjectNameHere"}));
console.log(response);
})();
Sample answer
'$metadata': {
httpStatusCode: 200,
requestId: undefined,
extendedRequestId: '7kwrFkEp3lEnLU+OtxjrgdmS6gQmvPdbnqqR7I8P/rdFrUPBkdKYPYykWivuHPXCF1IHgjCIbe8=',
cfId: undefined,
attempts: 1,
totalRetryDelay: 0
},
Here's a simple approach I use (in Deno) for testing (in case you don't want to go the signedUrl approach and just let the SDK do the heavy lifting for you):
import { config as env } from 'https://deno.land/x/dotenv/mod.ts' // https://github.com/pietvanzoen/deno-dotenv
import { S3Client, ListObjectsV2Command } from 'https://cdn.skypack.dev/#aws-sdk/client-s3' // https://github.com/aws/aws-sdk-js-v3
const {AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY} = env()
// https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/clients/client-s3/modules/credentials.html
const credentials = {
accessKeyId: AWS_ACCESS_KEY_ID,
secretAccessKey: AWS_SECRET_ACCESS_KEY,
}
// https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/clients/client-s3/interfaces/s3clientconfig.html
const config = {
region: 'ap-southeast-1',
credentials,
}
// https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/clients/client-s3/classes/s3client.html
const client = new S3Client(config)
export async function list() {
// https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/clients/client-s3/interfaces/listobjectsv2commandinput.html
const input = {
Bucket: 'BucketNameHere'
}
// https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/clients/client-s3/classes/command.html
const cmd = new ListObjectsV2Command(input)
// https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/clients/client-s3/classes/listobjectsv2command.html
return await client.send(cmd)
}
I'm testing #aws-sdk version 3 and I'm trying to load a profile from my configuration file (which works fine) and then assume a role. This is where I can't figure out how to do that. In aws-sdk version 2 I do like this
AWS.config.credentials = new AWS.TemporaryCredentials({
RoleArn: 'arn:aws:iam::XXX:role/XXX',
}, new AWS.SharedIniFileCredentials({ profile: 'myprofile' }));
}
How do I do the same thing using the new sdk? To just pass the profile name I do this
const { SSM } = require('#aws-sdk/client-ssm-node');
const ssm = new SSM({
profile: 'myprofile'
});
I have installed both #aws-sdk/credential-provider-node and #aws-sdk/credential-provider-ini but with no success in figuring out how to pass the credentials like I want, the documentation here https://www.npmjs.com/package/#aws-sdk/credential-provider-node doesn't tell me much. So, how do I do that?
It turns out I looked at the wrong nuget package. To assume a role this is one way to do it.
const { SSM } = require('#aws-sdk/client-ssm-node');
const { STSClient, AssumeRoleCommand } = require('#aws-sdk/client-sts-node');
const sTS = new STSClient({ profile: 'myProfile' });
const params = {
RoleArn: 'arn:aws:iam::XXX:role/XXX',
RoleSessionName: 'tempUser',
};
const assumeRoleCommand = new AssumeRoleCommand(params);
let assumedRole = null;
try {
assumedRole = await sTS.send(assumeRoleCommand);
} catch (error) {
console.log(error);
}
const ssm = new SSM({
profile: 'myProfile',
credentials: {
accessKeyId: assumedRole.Credentials.AccessKeyId,
secretAccessKey: assumedRole.Credentials.SecretAccessKey,
expiration: assumedRole.Credentials.Expiration,
sessionToken: assumedRole.Credentials.SessionToken,
},
});
I am following the proposed solution by Trivikr for adding support for s3.getSignedUrl api which is not currently available in newer v3. I am trying to make a signed url for getting an object from bucket.
Just for convenience, the code is being added below:
const { S3, GetObjectCommand } = require("#aws-sdk/client-s3"); // 1.0.0-gamma.2 version
const { S3RequestPresigner } = require("#aws-sdk/s3-request-presigner"); // 0.1.0-preview.2 version
const { createRequest } = require("#aws-sdk/util-create-request"); // 0.1.0-preview.2 version
const { formatUrl } = require("#aws-sdk/util-format-url"); // 0.1.0-preview.1 //version
const fetch = require("node-fetch");
(async () => {
try {
const region = "us-east-1";
const Bucket = `SOME_BUCKET_NAME`;
const Key = `SOME_KEY_VALUE`;
const credentials = {
accessKeyId: ACCESS_KEY_HERE,
secretAccessKey: SECRET_KEY_HERE,
sessionToken: SESSION_TOKEN_HERE
};
const S3Client = new S3({ region, credentials, signatureVersion: 'v4' });
console.log('1'); // for quick debugging
const signer = new S3RequestPresigner({ ...S3Client.config });
console.log('2')
const request = await createRequest(
S3Client,
new GetObjectCommand({ Key, Bucket })
);
console.log('3');
let signedUrl = formatUrl(await signer.presign(request));
console.log(signedUrl);
let response = await fetch(signedUrl);
console.log("Response", response);
}catch(e) {
console.error(e);
}
I successfully create S3Client and signer but on creating request, I get the following error:
clientStack.concat(...).filter is not a function
Anything wrong I am doing?
Please also note that I am using webpack for bundling
Just add my example in TypeScript:
import { S3Client, GetObjectCommand, S3ClientConfig } from '#aws-sdk/client-s3';
import { getSignedUrl } from '#aws-sdk/s3-request-presigner';
const s3Configuration: S3ClientConfig = {
credentials: {
accessKeyId: '<ACCESS_KEY_ID>',
secretAccessKey: '<SECRET_ACCESS_KEY>'
},
region: '<REGION>',
};
const s3 = new S3Client(s3Configuration);
const command = new GetObjectCommand({Bucket: '<BUCKET>', Key: '<KEY>' });
const url = await getSignedUrl(s3, command, { expiresIn: 15 * 60 }); // expires in seconds
console.log('Presigned URL: ', url);
RESOLVED
I ended up successfully making the signed urls by installing the beta versions rather than preview (default) ones