AWS Elastic search service - IAM Role (Refresh session token - Node.js) - amazon-web-services

I'm using AWS Elastic search service and written a Node.js wrapper (runs within a ECS dockerized container) . IAM roles are used to create a singleton connection with Elastic search.
I'm trying to refresh my session token by checking AWS.config.credentials.needsRefresh() before every request - however it always returns true even after the session has expired. Obviously AWS is complaining with a 403 error. Any ideas will be greatly appreciated .
var AWS = require('aws-sdk');
var config = require('config');
var connectionClass = require('http-aws-es');
var elasticsearch = require('elasticsearch');
AWS.config.getCredentials(function() {
AWS.config.update({
credentials: new AWS.Credentials(AWS.config.credentials.accessKeyId,AWS.config.credentials.secretAccessKey,AWS.config.credentials.sessionToken),
region: 'us-east-1'
});
}
)
var client = new elasticsearch.Client({
host: `${config.get('elasticSearch.host')}`,
log: 'debug',
connectionClass: connectionClass,
amazonES: {
credentials: new AWS.EnvironmentCredentials('AWS')
}
});
module.exports = client;

Related

AWS credentials with Identity pool

How can I pass the Cognito user's idToken to Amplify.configure() , cause the users have to be authenticated, not as a guest user. So Instead of getting an unauthenticated identityId, How do I get an authenticated Id by setting in the aws amplify configure.
For sign-in and sign-up we are not using Amplify instead we are using Rest API
Currently I'm sending Object to configure as follows,
{
Auth: {
identityPoolId: const.identityPoolId,
IdentityId: 'us-east-1:xxxxxxxxxxxxxxxxxxxyyyyyyzzzzzz',
region: 'us-east-1'
},
Analytics: {
disabled: false,
autoSessionRecord: true,
AWSPinpoint: {
appId: PinpointId,
region: 'us-east-1',
bufferSize: 1000,
flushInterval: 5000,
flushSize: 100,
resendLimit: 3
}
}
}
};
Any help is highly appreciated. Thanks

How to setup AWS-SDK credentials in NextJS

I need to upload some files to S3 from a NextJs application. Since it is server side I am under the impression simply setting environment variables should work but it doesn't. I know there are other alternative like assigning a role to EC2 but I want to use accessKeyID and secretKey.
This is my next.config.js
module.exports = {
env: {
//..others
AWS_ACCESS_KEY_ID: process.env.AWS_ACCESS_KEY_ID
},
serverRuntimeConfig: {
//..others
AWS_SECRET_ACCESS_KEY: process.env.AWS_SECRET_ACCESS_KEY
}
}
This is my config/index.js
export default {
//...others
awsClientID: process.env. AWS_ACCESS_KEY_ID,
awsClientSecret: process.env.AWS_SECRET_ACCESS_KEY
}
This is how I use in my code:
import AWS from 'aws-sdk'
import config from '../config'
AWS.config.update({
accessKeyId: config.awsClientID,
secretAccessKey: config.awsClientSecret,
});
const S3 = new AWS.S3()
const params = {
Bucket: "bucketName",
Key: "some key",
Body: fileObject,
ContentType: fileObject.type,
ACL: 'public-read'
}
await S3.upload(params).promise()
I am getting this error:
Unhandled Rejection (CredentialsError): Missing credentials in config, if using AWS_CONFIG_FILE, set AWS_SDK_LOAD_CONFIG=1
If I hard code the credentials in code, it works fine.
How can I make it work correctly?
Looks like the Vercel docs are currently outdated (AWS SDK V2 instead of V3). You can pass the credentials object to the AWS service when you instantiate it. Use an environment variable that is not reserved by adding the name of your app to it for example.
.env.local
YOUR_APP_AWS_ACCESS_KEY_ID=[your key]
YOUR_APP_AWS_SECRET_ACCESS_KEY=[your secret]
Add these env variables to your Vercel deployment settings (or Netlify, etc) and pass them in when you start up your AWS service client.
import { S3Client } from '#aws-sdk/client-s3'
...
const s3 = new S3Client({
region: 'us-east-1',
credentials: {
accessKeyId: process.env.TRENDZY_AWS_ACCESS_KEY_ID ?? '',
secretAccessKey: process.env.TRENDZY_AWS_SECRET_ACCESS_KEY ?? '',
},
})
(note: undefined check so Typescript stays happy)
are you possibly hosting this app via vercel?
As per vercel docs, some env variables are reserved by vercel.
https://vercel.com/docs/concepts/projects/environment-variables#reserved-environment-variables
AWS_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY
Maybe that's the reason why it is not getting those env vars
I was able to workaround this by adding my custom env variables into .env.local and then calling for those variables
AWS.config.update({
'region': 'us-east-1',
'credentials': {
'accessKeyId': process.env.MY_AWS_ACCESS_KEY,
'secretAccessKey': process.env.MY_AWS_SECRET_KEY
}
});
As last step would need to add these into vercel UI
obviously not ideal solution and not recommended by AWS.
https://vercel.com/support/articles/how-can-i-use-aws-sdk-environment-variables-on-vercel
If I'm not mistaken, you want to make AWS_ACCESS_KEY_ID into a runtime variable as well. Currently, it is a build time variable, which won't be accessible in your node application.
// replace this
env: {
//..others
AWS_ACCESS_KEY_ID: process.env.AWS_ACCESS_KEY_ID
},
// with this
module.exports = {
serverRuntimeConfig: {
//..others
AWS_ACCESS_KEY_ID: process.env.AWS_ACCESS_KEY_ID
}
}
Reference: https://nextjs.org/docs/api-reference/next.config.js/environment-variables

Possible to set up multiple AWS configurations in rails?

Is there a way to set up multiple aws configurations in a rails
environment? For example:
Aws account #1
Aws.config.update({
region: aws_config[:region_1],
credentials: Aws::Credentials.new(aws_config[:access_key_id_1],
aws_config[:secret_access_key_1])
})
Aws account #2
Aws.config.update({
region: aws_config[:region_2],
credentials: Aws::Credentials.new(aws_config[:access_key_id_2],
aws_config[:secret_access_key_2])
})
I tried setting the second specification manually in the model, but it
doesn't login with the new credentials for some reason:
foo = Aws::S3::Client.new(
region: aws_config[:region_2],
access_key_id: aws_config[:access_key_id_2],
secret_access_key: aws_config[:secret_access_key_2]
)
bar = foo.list_objects_v2(bucket: aws_config[:bucket_2])

How to call AWS API Gateway Endpoint with Cognito Id (+configuration)?

I want to call an AWS API Gateway Endpoint that is protected with AWS_IAM using the generated JavaScript API SDK.
I have a Cognito UserPool and a Cognito Identity Pool. Both properly synced via ClientId.
I use this code to Sign in and get the Cognito Identity
AWS.config.region = 'us-east-1'; // Region
AWS.config.credentials = new AWS.CognitoIdentityCredentials({
IdentityPoolId: 'us-east-1:XXXXXXXXXXXXXXXXXXXXXXXX' // your identity pool id here
});
AWSCognito.config.region = 'us-east-1';
AWSCognito.config.credentials = new AWS.CognitoIdentityCredentials({
IdentityPoolId: 'us-east-1:XXXXXXXXXXXXXXXXXXXXXXXX' // your identity pool id here
});
var poolData = {
UserPoolId: 'us-east-1_XXXXXXXX',
ClientId: 'XXXXXXXXXXXXXXXXXXXXXXXX'
};
var userPool = new AWSCognito.CognitoIdentityServiceProvider.CognitoUserPool(poolData);
var authenticationData = {
Username: 'user',
Password: '12345678',
};
var authenticationDetails = new AWSCognito.CognitoIdentityServiceProvider.AuthenticationDetails(authenticationData);
var userData = {
Username: 'user',
Pool: userPool
};
var cognitoUser = new AWSCognito.CognitoIdentityServiceProvider.CognitoUser(userData);
cognitoUser.authenticateUser(authenticationDetails, {
onSuccess: function (result) {
console.log('access token + ' + result.getAccessToken().getJwtToken());
AWS.config.credentials = new AWS.CognitoIdentityCredentials({
IdentityPoolId: 'us-east-1:XXXXXXXXXXXXXXXXXXXX',
IdentityId: AWS.config.credentials.identityId,
Logins: {
'cognito-idp.us-east-1.amazonaws.com/us-east-1_XXXXXX': result.idToken.jwtToken
}
});
AWS.config.credentials.get(function (err) {
// now I'm using authenticated credentials
if(err)
{
console.log('error in autheticatig AWS'+err);
}
else
{
console.log(AWS.config.credentials.identityId);
}
});
},
onFailure: function (err) {
alert(err);
}
});
All this succeeds and I have an authorized Cognito Identity now.
Now I try to call the API Gateway Endpoint to execute the Lambda Function it points to.
var apigClient = apigClientFactory.newClient({
accessKey: AWS.config.credentials.accessKeyId, //'ACCESS_KEY',
secretKey: AWS.config.credentials.secretAccessKey, //'SECRET_KEY',
sessionToken: AWS.config.credentials.sessionToken, // 'SESSION_TOKEN', //OPTIONAL: If you are using temporary credentials you must include the session token
region: 'us-east-1' // OPTIONAL: The region where the API is deployed, by default this parameter is set to us-east-1
});
var params = {
// This is where any modeled request parameters should be added.
// The key is the parameter name, as it is defined in the API in API Gateway.
};
var body = {
// This is where you define the body of the request,
query: '{person {firstName lastName}}'
};
var additionalParams = {
// If there are any unmodeled query parameters or headers that must be
// sent with the request, add them here.
headers: {},
queryParams: {}
};
apigClient.graphqlPost(params, body, additionalParams)
.then(function (result) {
// Add success callback code here.
console.log(result);
}).catch(function (result) {
// Add error callback code here.
console.log(result);
});
But unfortunately this fails. The OPTIONS request succeeds with 200 but the POST then fails with 403.
I am pretty sure that there is no CORS problem here.
I am pretty sure the problem has to do with IAM Roles and AWS Resource Configurations.
My question is basically, can you please provide me with all the necessary AWS Resource Configurations and IAM Roles that are necessary for this to work please?
Resources I have are
API Gateway - with deployed API Endpoints
Lambda Function - called by the Endpoint
Cognito User Pool - with App synced to the Identity Pool
Cognito Identity Pool - with Authorized and Unauthorized Role mapped to it.
IAM Roles - for the Lambda Function and the Authorized and Unauthorized Role of the Cognito Identity Pool.
But I don't know how these Resources need to be configured properly to get this to work.
Thank you
What access permissions does the role of the Cognito Identity have? Make sure it has access to perform execute-api:Invoke on your API.
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"execute-api:Invoke"
],
"Resource": [
"arn:aws:execute-api:us-east-1:<account>:<rest-api>/*/POST/graphql"
]
}
]
}
You can get the exact resource ARN from the method settings page in the web console.
Even after following everything I was getting the same error. And the reason was I missed the "sessionToken" while initialising the apigClient.
var apigClient = apigClientFactory.newClient({
accessKey: AWS.config.credentials.accessKeyId, //'ACCESS_KEY',
secretKey: AWS.config.credentials.secretAccessKey, //'SECRET_KEY',
sessionToken: AWS.config.credentials.sessionToken, // 'SESSION_TOKEN', //OPTIONAL: If you are using temporary credentials you must include the session token
region: 'us-east-1' // OPTIONAL: The region where the API is deployed, by default this parameter is set to us-east-1 });
//OPTIONAL: If you are using temporary credentials you must include the session token -- is not really optional

S3 & node.js / Define new agent for aws.sdk

I use node.js and aws-sdk.
I want to define a new agent from a require aws-sdk.
var agent = new http.Agent()
agent.maxSockets = 20;
var s3 = new AWS.S3({accessKeyId: "acc",
secretAccessKey: "sec", region:"us-east-1", httpOptions: {agent: agent}});
For some reason, it gives me an Networking error when I use var download = s3t.getObject(s3Params);
What can be the reason? How I define new agent?
Edit : This is the error:
{ [NetworkingError: read ECONNRESET]
message: 'read ECONNRESET',
code: 'NetworkingError',
errno: 'ECONNRESET',
Below is how you define a new https agent in your AWS S3 client:
var s3 = new AWS.S3({
httpOptions: {
agent: new https.Agent(...)
}})
Or you can globally update the AWS client config:
AWS.config.update({
httpOptions: {
agent: new http.Agent(...)
}
})