I am following the proposed solution by Trivikr for adding support for s3.getSignedUrl api which is not currently available in newer v3. I am trying to make a signed url for getting an object from bucket.
Just for convenience, the code is being added below:
const { S3, GetObjectCommand } = require("#aws-sdk/client-s3"); // 1.0.0-gamma.2 version
const { S3RequestPresigner } = require("#aws-sdk/s3-request-presigner"); // 0.1.0-preview.2 version
const { createRequest } = require("#aws-sdk/util-create-request"); // 0.1.0-preview.2 version
const { formatUrl } = require("#aws-sdk/util-format-url"); // 0.1.0-preview.1 //version
const fetch = require("node-fetch");
(async () => {
try {
const region = "us-east-1";
const Bucket = `SOME_BUCKET_NAME`;
const Key = `SOME_KEY_VALUE`;
const credentials = {
accessKeyId: ACCESS_KEY_HERE,
secretAccessKey: SECRET_KEY_HERE,
sessionToken: SESSION_TOKEN_HERE
};
const S3Client = new S3({ region, credentials, signatureVersion: 'v4' });
console.log('1'); // for quick debugging
const signer = new S3RequestPresigner({ ...S3Client.config });
console.log('2')
const request = await createRequest(
S3Client,
new GetObjectCommand({ Key, Bucket })
);
console.log('3');
let signedUrl = formatUrl(await signer.presign(request));
console.log(signedUrl);
let response = await fetch(signedUrl);
console.log("Response", response);
}catch(e) {
console.error(e);
}
I successfully create S3Client and signer but on creating request, I get the following error:
clientStack.concat(...).filter is not a function
Anything wrong I am doing?
Please also note that I am using webpack for bundling
Just add my example in TypeScript:
import { S3Client, GetObjectCommand, S3ClientConfig } from '#aws-sdk/client-s3';
import { getSignedUrl } from '#aws-sdk/s3-request-presigner';
const s3Configuration: S3ClientConfig = {
credentials: {
accessKeyId: '<ACCESS_KEY_ID>',
secretAccessKey: '<SECRET_ACCESS_KEY>'
},
region: '<REGION>',
};
const s3 = new S3Client(s3Configuration);
const command = new GetObjectCommand({Bucket: '<BUCKET>', Key: '<KEY>' });
const url = await getSignedUrl(s3, command, { expiresIn: 15 * 60 }); // expires in seconds
console.log('Presigned URL: ', url);
RESOLVED
I ended up successfully making the signed urls by installing the beta versions rather than preview (default) ones
Related
const S3 = require('aws-sdk/clients/s3');
const { getSignedUrl } = require("#aws-sdk/s3-request-presigner");
const { S3Client, GetObjectCommand } = require("#aws-sdk/client-s3");
dotenv.config();
const bucketName = process.env.AWS_BUCKET_NAME
const region = process.env.AWS_BUCKET_REGION
const accessKeyId = process.env.AWS_ACCESS_KEY
const secretAccessKey = process.env.AWS_SECRET_KEY
const s3 = new S3({
region,
accessKeyId,
secretAccessKey
})
router.get("/:id", async (req, res) => {
try {
const post = await Post.findById(req.params.id);
const getObjectParams = {
Bucket: bucketName,
Key: post.photo,
}
const command = new GetObjectCommand(getObjectParams);
const url = await getSignedUrl(s3, command, { expiresIn: 3600 });
post.imageUrl = url
res.status(200).json(post);
} catch (err) {
console.error('errorrr', err);
res.status(500).json(err);
}
});
Here is my code I've console logged post, getObjectParams, command and everything is there but when I console log url it's not logging and when I console.log errorrr it logs Cannot read properties of undefined (reading 'clone')
What is the issue here?
I think issue is with function getSignedUrl, but not sure what it is
I am trying to encrypt data using AWS Encryption SDK for javascript(react) in browser but getting this error : export 'encrypt' (imported as 'encrypt') was not found in '#aws-crypto/client-browser'
import { KmsKeyringBrowser, encrypt } from "#aws-crypto/client-browser";
const generatorKeyId = '*********************'
const keyIds = ['**************************']
const keyring = new KmsKeyringBrowser({generatorKeyId,keyIds})
const context = {
stage: 'demo',
purpose: 'simple demonstration app',
origin: 'us-west-2'
}
const encryptData = async (data) => {
const result = await encrypt(keyring,data, { encryptionContext: context })
console.log(result);
}
I have alos tried using buildEncrypt but it is not working. Below is the code :
import { KmsKeyringBrowser, encrypt } from "#aws-crypto/client-browser";
const generatorKeyId = '*********************'
const keyIds = ['**************************']
const keyring = new KmsKeyringBrowser({generatorKeyId,keyIds})
const context = {
stage: 'demo',
purpose: 'simple demonstration app',
origin: 'us-west-2'
}
const encryptData = async (data) => {
const result = await buildEncrypt({encrypt: (keyring,data, { encryptionContext: context })} )
console.log(result);
}
I am unable to find out what I was doing wrong. Any help would be apprecieated. Thankyou
I am scouring the documentation, and it only provides pseudo-code of the credentials for v3 (e.g. const client = new S3Client(clientParams)
How do I initialize an S3Client with the bucket and credentials to perform a getSignedUrl request? Any resources pointing me in the right direction would be most helpful. I've even searched YouTube, SO, etc and I can't find any specific info on v3. Even the documentation and examples doesn't provide the actual code to use credentials. Thanks!
As an aside, do I have to include the fake folder structure in the filename, or can I just use the actual filename? For example: bucket/folder1/folder2/uniqueFilename.zip or uniqueFilename.zip
Here's the code I have so far: (Keep in mind I was returning the wasabiObjKey to ensure I was getting the correct file name. I am. It's the client, GetObjectCommand, and getSignedUrl that I'm having issues with.
exports.getPresignedUrl = functions.https.onCall(async (data, ctx) => {
const wasabiObjKey = `${data.bucket_prefix ? `${data.bucket_prefix}/` : ''}${data.uid.replace(/-/g, '_').toLowerCase()}${data.variation ? `_${data.variation.replace(/\./g, '').toLowerCase()}` : ''}.zip`
const { S3Client, GetObjectCommand } = require('#aws-sdk/client-s3')
const s3 = new S3Client({
bucketEndpoint: functions.config().s3_bucket.name,
region: functions.config().s3_bucket.region,
credentials: {
secretAccessKey: functions.config().s3.secret,
accessKeyId: functions.config().s3.access_key
}
})
const command = new GetObjectCommand({
Bucket: functions.config().s3_bucket.name,
Key: wasabiObjKey,
})
const { getSignedUrl } = require("#aws-sdk/s3-request-presigner")
const url = getSignedUrl(s3, command, { expiresIn: 60 })
return wasabiObjKey
})
There are a credential chain that provide credential to your API calls from SDK
https://docs.aws.amazon.com/sdk-for-javascript/v2/developer-guide/setting-credentials-node.html
Loaded from AWS Identity and Access Management (IAM) roles for Amazon
EC2
Loaded from the shared credentials file (~/.aws/credentials)
Loaded from environment variables
Loaded from a JSON file on disk
Other credential-provider classes provided by the JavaScript SDK
You can embed the credential inside your source code but it's not the prefered way
new S3Client(configuration: S3ClientConfig): S3Client
Where S3ClientConfig contain a credentials property
https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/clients/client-s3/modules/credentials.html
const { S3Client,GetObjectCommand } = require("#aws-sdk/client-s3");
let client = new S3Client({
region:'ap-southeast-1',
credentials:{
accessKeyId:'',
secretAccessKey:''
}
});
(async () => {
const response = await client.send(new GetObjectCommand({Bucket:"BucketNameHere",Key:"ObjectNameHere"}));
console.log(response);
})();
Sample answer
'$metadata': {
httpStatusCode: 200,
requestId: undefined,
extendedRequestId: '7kwrFkEp3lEnLU+OtxjrgdmS6gQmvPdbnqqR7I8P/rdFrUPBkdKYPYykWivuHPXCF1IHgjCIbe8=',
cfId: undefined,
attempts: 1,
totalRetryDelay: 0
},
Here's a simple approach I use (in Deno) for testing (in case you don't want to go the signedUrl approach and just let the SDK do the heavy lifting for you):
import { config as env } from 'https://deno.land/x/dotenv/mod.ts' // https://github.com/pietvanzoen/deno-dotenv
import { S3Client, ListObjectsV2Command } from 'https://cdn.skypack.dev/#aws-sdk/client-s3' // https://github.com/aws/aws-sdk-js-v3
const {AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY} = env()
// https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/clients/client-s3/modules/credentials.html
const credentials = {
accessKeyId: AWS_ACCESS_KEY_ID,
secretAccessKey: AWS_SECRET_ACCESS_KEY,
}
// https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/clients/client-s3/interfaces/s3clientconfig.html
const config = {
region: 'ap-southeast-1',
credentials,
}
// https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/clients/client-s3/classes/s3client.html
const client = new S3Client(config)
export async function list() {
// https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/clients/client-s3/interfaces/listobjectsv2commandinput.html
const input = {
Bucket: 'BucketNameHere'
}
// https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/clients/client-s3/classes/command.html
const cmd = new ListObjectsV2Command(input)
// https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/clients/client-s3/classes/listobjectsv2command.html
return await client.send(cmd)
}
I am trying to disable users from my node JS backend using the AWS SDK v3. everything works normally except for the disable/enable user command.
I have tried everything I know but here is my code snippet:
const aws_creds = {
accessKeyId: process.env.ACCESS_KEY_ID,
secretAccessKey: process.env.SECRET_ACCESS_KEY,
};
const cognitoConfig = {
region: process.env.REGION,
credentials: aws_creds,
};
const cognito_v3 = new CognitoIdentityProviderClient(cognitoConfig);
Then in my route I am running this try / catch block
try {
if (status === "enable") {
const enableUserCommand = new AdminEnableUserCommand(userDetails);
const enableUserResults = await cognito_v3.send(enableUserCommand);
return res.status(200).json(enableUserResults);
}
const disableUserCommand = new AdminDisableUserCommand(userDetails);
const disableUserResults = await cognito_v3.send(disableUserCommand);
return res.status(200).json(disableUserResults);
} catch (err) {
console.log(err);
return res.status(400).json(err);
}
However, the err console log returns this:
TypeError: Cannot read property 'byteLength' of undefined
at Object.fromArrayBuffer (C:\Users\SER-01\Documents\ctr\simsim\simsim-backend\lambdas\simsim-auth\node_modules\#aws-sdk\util-buffer-from\dist\cjs\index.js:6:60)
at castSourceData (C:\Users\SER-01\Documents\ctr\simsim\simsim-backend\lambdas\simsim-auth\node_modules\#aws-sdk\hash-node\dist\cjs\index.js:29:31)
at Hash.update (C:\Users\SER-01\Documents\ctr\simsim\simsim-backend\lambdas\simsim-auth\node_modules\#aws-sdk\hash-node\dist\cjs\index.js:12:26)
at hmac (C:\Users\SER-01\Documents\ctr\simsim\simsim-backend\lambdas\simsim-auth\node_modules\#aws-sdk\signature-v4\dist\cjs\credentialDerivation.js:60:10)
at Object.getSigningKey (C:\Users\SER-01\Documents\ctr\simsim\simsim-backend\lambdas\simsim-auth\node_modules\#aws-sdk\signature-v4\dist\cjs\credentialDerivation.js:32:29)
at SignatureV4.getSigningKey (C:\Users\SER-01\Documents\ctr\simsim\simsim-backend\lambdas\simsim-auth\node_modules\#aws-sdk\signature-v4\dist\cjs\SignatureV4.js:139:39)
at SignatureV4.signRequest (C:\Users\SER-01\Documents\ctr\simsim\simsim-backend\lambdas\simsim-auth\node_modules\#aws-sdk\signature-v4\dist\cjs\SignatureV4.js:98:73)
at async C:\Users\SER-01\Documents\ctr\simsim\simsim-backend\lambdas\simsim-auth\node_modules\#aws-sdk\middleware-signing\dist\cjs\middleware.js:14:22
at async StandardRetryStrategy.retry (C:\Users\SER-01\Documents\ctr\simsim\simsim-backend\lambdas\simsim-auth\node_modules\#aws-sdk\middleware-retry\dist\cjs\defaultStrategy.js:56:46)
at async C:\Users\SER-01\Documents\ctr\simsim\simsim-backend\lambdas\simsim-auth\node_modules\#aws-sdk\middleware-logger\dist\cjs\loggerMiddleware.js:6:22
at async C:\Users\SER-01\Documents\ctr\simsim\simsim-backend\lambdas\simsim-auth\src\routes\auth.js:232:33 {
'$metadata': { attempts: 1, totalRetryDelay: 0 }
}
Any idea why?
I am using DigitalOcean for storing videos. I am using Firebase callable function to generate the signed URL with AWS SDK and sent it back to my app. When I use Firebase function to generate a signed URL then it gives me the Signature Does not match error. The Firebase function is as follows-
const functions = require('firebase-functions');
const admin = require('firebase-admin');
admin.initializeApp();
const AWS = require('aws-sdk');
exports.getS3SignedUrlUpload = functions.https.onCall((data, context) => {
const spacesEndpoint = new AWS.Endpoint('https://sgp1.digitaloceanspaces.com');
AWS.config.update({
endpoint: spacesEndpoint,
accessKeyId: 'mykey',
secretAccessKey: 'mySecretKey',
signatureVersion: 'v4',
region: "sgp1"
});
var s3 = new AWS.S3();
const s3Params = {
Bucket: 'myappvideo', //data.S3BucketName,
Key: 'video14.mp4',//data.key,
Expires: 60*60 // Expires in 10 minutes
};
var v = s3.getSignedUrl('putObject', s3Params);
console.log("PRESIGNED URL", v);
return v;
});
I accidentally use getObject instead of putObject. Now it is working fine for me. But I don't know why it works, as I don't have any networking concepts. The line where I did the change is:
var v = s3.getSignedUrl('putObject', s3Params);
to
var v = s3.getSignedUrl('getObject', s3Params);