Accessing two tables from different accounts within the same lambda function - amazon-web-services

Is it possible to access two tables within one lambda function while one of the tables is in the same account as the lambda function and the other is in another account?
I've seen articles on cross-account access delegation using IAM roles here and there. But I'm not sure how the code should reflect accessing a resource from another account. This is how I usually access some DynamoDb table:
const dynamodb = new AWS.DynamoDB();
const docClient = new AWS.DynamoDB.DocumentClient({ service: dynamodb });
docClient
.get({
TableName: 'SomeTable',
Key: { id }
});
Looking at the documentation, there's no mention of account ID in the constructor. So I'm not sure how I can have two connections at the same time, one pointing to one account and the other pointing to another account!?

Yes, it's possible to access 2 dynamoDb Tables (1 is in another AWS account) from same lambda function.
It's straight-forward to access the same Account's DynamoDB table.
dynamodb_client_of_same_account = boto3.client('dynamodb', region_name='us-east-1')
To access the Table of another AWS account, a new dynamoDb client needs to be created. You need to create cross account IAM Role and use STS to get temporary credentials. Follow the below steps to create cross account role.
AWS Account A has Lambda-----trying read/write on-----> AWS Account B with DynamoDB
Create an IAM Role in Account B to establish a Trust Relationship with Account A.
Go to IAM Role -------> Create Role ------> Select *Another AWS account *in the widget-----> Type AWS Account A Id and create the role
Don't forget to add DynamoDBAccess policy to this IAM Role.
Attach STS Assume Role policy to your AWS Lambda's Role in Account A
Create a new policy with below JSON and attach this policy to your Lambda Role.
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": "sts:AssumeRole",
"Resource": ""
}
]
}
Use the below code to create a new client to access DynamoDB table in another account.
roleARN = '*<ARN of the role created in Step-1>*'
client = boto3.client('sts')
response = client.assume_role(RoleArn=roleARN,
RoleSessionName='RoleSessionName',
DurationSeconds=900)
dynamodb_client_for_other_account = boto3.client('dynamodb', region_name='us-east-1',
aws_access_key_id=response['Credentials']['AccessKeyId'],
aws_secret_access_key=response['Credentials']['SecretAccessKey'],
aws_session_token = response['Credentials']['SessionToken'])

you could create a IAM user on the second account,
add DynamoDB permissions and get access/key secret for it
and then:
const dynamodb1 = new AWS.DynamoDB();
const dynamodb2 = new AWS.DynamoDB({
accessKeyId: 'x',
secretAccessKey: 'y',
region: 'z',
})
dynamodb1 will work with the role permissions from lambda
and dynamodb2 with the IAM user from the second account

Related

Using a multi-region S3 accesspoint with sagemaker (terraform)

I have a sagemaker model that will be deployed in different aws regions. This model will download models from an s3 bucket that is in region x. As long as the model is deployed in region x, the endpoint works.
However, when I deploy the sagemaker model from region y, it fails with the message.
Error: error creating SageMaker model: ValidationException: Could not
access model data at s3://mmmm/. Please ensure that the role
"arn:aws:iam::xxxx:role/dev-xxx-iam-role" exists and that its trust
relationship policy allows the action "sts:AssumeRole" for the service
principal "sagemaker.amazonaws.com". Also ensure that the role has
"s3:GetObject" permissions and that the object is located in region x.
My iam role permissions are as follows:
resource "aws_iam_policy_attachment" "sm_full_access_attach" {
name = "sm-full-access-attachment"
roles = [aws_iam_role.sagemaker_inferencer_iam_role.name]
policy_arn = "arn:aws:iam::aws:policy/AmazonSageMakerFullAccess"
}
resource "aws_iam_policy_attachment" "s3_full_access_attach" {
name = "s3-full-access-attachment"
roles = [aws_iam_role.sagemaker_inferencer_iam_role.name]
policy_arn = "arn:aws:iam::aws:policy/AmazonS3FullAccess"
}
with the assume role policy of:
data "aws_iam_policy_document" "sm_assume_role_policy" {
statement {
actions = ["sts:AssumeRole"]
principals {
type = "Service"
identifiers = ["sagemaker.amazonaws.com"]
}
}
}
The iam role is as follows:
resource "aws_iam_role" "sagemaker_inferencer_iam_role" {
name = "${var.app_environment}-inferencer-sm-${var.aws_region}-iam-role"
assume_role_policy = data.aws_iam_policy_document.sm_assume_role_policy.json
}
And the above works for same region buckets.
I can of course create have buckets in each region with replication rules from the original bucket. However, this is costly as the model files are huge. So I created a multi-region accesspoint for my original bucket with an alias zzz.mrap
However, when I specify the accesspoint alias in the aws_sagemaker_model resource as follows:
resource "aws_sagemaker_model" "sagemaker_multimodel" {
name = "${var.app_environment}-inferencer-sm-${var.aws_region}-model"
execution_role_arn = aws_iam_role.sagemaker_inferencer_iam_role.arn
primary_container {
image = local.multi_model_inferencer_container_name
mode = "MultiModel"
model_data_url = "s3://zzz.mrap/"
}
}
I get the following error:
Error: error creating SageMaker model: ValidationException: Could not
access model data at s3://zzzz.mrap/. Please ensure that the role
"arn:aws:iam::878435376106:role/dev-xxx-iam-role" exists and that its
trust relationship policy allows the action "sts:AssumeRole" for the
service principal "sagemaker.amazonaws.com". Also ensure that the role
has "s3:GetObject" permissions and that the object is located in
region x.
AWS says that I only have to replace the bucket name by the alias and that they support sagemaker, however this does not seem to be the case.
What am I doing wrong?

Why does S3 bucket created in terraform needs bucket policy to grant access to lambda

We use a combination of cloud formation and terraform where some common resources like DynamoDB, S3 are created using terraform and others like APIGateway are created using serverless and cloudformation. All resources are in the same AWS account
I have an S3 bucket in terraform
resource "aws_s3_bucket" "payment_bucket" {
bucket = "payment-bucket-${var.env_name}"
acl = "private"
tags = merge(
module.tags.base_tags,
{
"Name" = "payment-bucket-${var.env_name}"
}
)
lifecycle {
ignore_changes = [tags]
}
}
This creates a private bucket payment-bucket-dev in my AWS account when I run the tf-apply
We have an APIGateway in the same AWS account which is created using serverless and one of the lambda needs accesses to this bucket so I have created an IAM role for the lambda function to grant permission to access the bucket.
makePayment:
name: makePayment-${self:provider.stage}
handler: src/handler/makePayment.default
events:
- http:
path: /payment
method: post
private: true
cors: true
iamRoleStatementsName: ${self:service}-${self:provider.stage}-makePayment-role
iamRoleStatements:
- Effect: Allow
Action:
- s3:PutObject
Resource:
- arn:aws:s3:::#{AWS::Region}:#{AWS::AccountId}:payment-bucket-${self:provider.stage}/capture/batch/*
But when I run this lambda make-payment-dev , it throws an AccessDenied error unless I add bucket policy granting access to the lambda role
resource "aws_s3_bucket_policy" "payment_service_s3_bucket_policy" {
..
..
}
Why do I need to add S3 bucket policy when both s3 bucket and the lambda function and role are in the same account? Am I missing something?
Also, If I created the bucket using AWS::S3::Bucket as part of the cloud formation stack the Apigateway is in (we are using serverless), I don't need add bucket policy and it all works fine.
I think the problem is simply that the S3 bucket ARN is incorrect.
S3 bucket ARNs do not have account IDs or regions in them. Use arn:aws:s3:::mybucket/myprefix/*.
The answer depends on what AWS IAM role is applying the terraform plan because the AWS s3 bucket canned ACL rule: "private" restricts bucket access as: Owner gets FULL_CONTROL. No one else has access rights (default). per documentation: https://docs.aws.amazon.com/AmazonS3/latest/dev/acl-overview.html
You have to be relatively explicit at this point as to who can access the bucket. Often if I'm going with private ACL but want every other role in my AWS account to have access to the bucket I attach a bucket policy to the terraform aws_s3_bucket resource to first allow access to the bucket. Then I explicitly grant the lambda's role access to said bucket via another inline policy.
In your case it would look something like this:
// Allow access to the bucket
data "aws_iam_policy_document" "bucket_policy" {
statement {
sid = "S3 bucket policy for account access"
actions = [
"s3:ListBucket",
"s3:GetObject",
"s3:PutObject",
"s3:DeleteObject"
]
principals {
type = "AWS"
identifiers = [
"arn:aws:iam::{your_account_id_here}:root",
]
}
resources = [
"arn:aws:s3:::test_bucket_name",
"arn:aws:s3:::test_bucket_name/*",
]
condition {
test = "StringEquals"
variable = "aws:PrincipalArn"
values = ["arn:aws:iam::{your_account_id_here}:role/*"]
}
}
}
resource "aws_s3_bucket" "this" {
bucket = "test_bucket_name"
acl = "private"
policy = data.aws_iam_policy_document.bucket_policy.json
}
// Grant the lambda IAM role permissions to the bucket
data "aws_iam_policy_document" "grant_bucket_access" {
statement {
sid = "AccessToTheAppAuxFilesBucket"
actions = [
"s3:ListBucket",
"s3:GetObject",
"s3:PutObject",
"s3:DeleteObject"
]
resources = [
"arn:aws:s3:::test_bucket_name/*",
"arn:aws:s3:::test_bucket_name"
]
}
}
// Data call to pull the arn of the lambda's IAM Role
data "aws_iam_role" "cloudformation_provisioned_role" {
name = "the_name_of_the_lambdas_iam_role"
}
resource "aws_iam_role_policy" "iam_role_inline_policy" {
name = "s3_bucket_access"
role = data.aws_iam_role.cloudformation_provisioned_role.arn
policy = data.aws_iam_policy_document.grant_bucket_access.json
}
It's an open bug. acl and force_destroy aren't well imported with terraform import : https://github.com/hashicorp/terraform-provider-aws/issues/6193

How to get/generate aws quicksight secure dashboard url

I want to embed Quicksight dashboard to an application. I have gone through the AWS quicksight documents, I did not get where I will find secure signed dashboard url.
In order to generate Quicksight secure dashboard url, follow the below steps:
Step 1: Create a new Identity Pool. Go to https://console.aws.amazon.com/cognito/home?region=us-east-1 , click ‘Create new Identity Pool’
Give an appropriate name.
Go to the Authentication Providers section, select Cognito.
Give the User Pool ID(your User pool ID) and App Client ID (go to App Clients in userpool and copy id).
Click ‘Create Pool’. Then click ‘Allow’ to create roles of the identity pool in IAM.
Step 2: Assign Custom policy to the Identity Pool Role
Create a custom policy with the below JSON.
{
"Version": "2012-10-17",
"Statement": [
{
"Action": "quicksight:RegisterUser",
"Resource": "*",
"Effect": "Allow"
},
{
"Action": "quicksight:GetDashboardEmbedUrl",
"Resource": "*",
"Effect": "Allow"
},
{
"Action": "sts:AssumeRole",
"Resource": "*",
"Effect": "Allow"
}
]
}
Note: if you want to restrict the user to only one dashboard, replace the * with the dashboard ARN name in quicksight:GetDashboardEmbedUrl,
then goto the roles in IAM.
select the IAM role of the Identity pool and assign custom policy to the role.
Step 3: Configuration for generating the temporary IAM(STS) user
Login to your application with the user credentials.
For creating temporary IAM user, we use Cognito credentials.
When user logs in, Cognito generates 3 token IDs - IDToken, AccessToken, RefreshToken. These tokens will be sent to your application server.
For creating a temporary IAM user, we use Cognito Access Token and credentials will look like below.
AWS.config.region = 'us-east-1';
AWS.config.credentials = new AWS.CognitoIdentityCredentials({
IdentityPoolId:"Identity pool ID",
Logins: {
'cognito-idp.us-east-1.amazonaws.com/UserPoolID': AccessToken
}
});
For generating temporary IAM credentials, we call sts.assume role method with the below parameters.
var params = {
RoleArn: "Cognito Identity role arn",
RoleSessionName: "Session name"
};
sts.assumeRole(params, function (err, data) {
if (err) console.log( err, err.stack); // an error occurred
else {
console.log(data);
})
You can add additional parameters like duration (in seconds) for the user.
Now, we will get the AccessKeyId, SecretAccessKey and Session Token of the temporary user.
Step 4: Register the User in Quicksight
With the help of same Cognito credentials used in the Step 3, we will register the user in quicksight by using the quicksight.registerUser method with the below parameters
var params = {
AwsAccountId: “account id”,
Email: 'email',
IdentityType: 'IAM' ,
Namespace: 'default',
UserRole: ADMIN | AUTHOR | READER | RESTRICTED_AUTHOR | RESTRICTED_READER,
IamArn: 'Cognito Identity role arn',
SessionName: 'session name given in the assume role creation',
};
quicksight.registerUser(params, function (err, data1) {
if (err) console.log("err register user”); // an error occurred
else {
// console.log("Register User1”);
}
})
Now the user will be registered in quicksight.
Step5: Update AWS configuration with New credentials.
Below code shows how to configure the AWS.config() with new credentials generated Step 3.
AWS.config.update({
accessKeyId: AccessToken,
secretAccessKey: SecretAccessKey ,
sessionToken: SessionToken,
"region": Region
});
Step6: Generate the EmbedURL for Dashboards:
By using the credentials generated in Step 3, we will call the quicksight.getDashboardEmbedUrl with the below parameters
var params = {
AwsAccountId: "account ID",
DashboardId: "dashboard Id",
IdentityType: "IAM",
ResetDisabled: true,
SessionLifetimeInMinutes: between 15 to 600 minutes,
UndoRedoDisabled: True | False
}
quicksight.getDashboardEmbedUrl(params,
function (err, data) {
if (!err) {
console.log(data);
} else {
console.log(err);
}
});
Now, we will get the embed url for the dashboard.
Call the QuickSightEmbedding.embedDashboard from front end with the help of the above generated url.
The result will be the dashboard embedded in your application with filter controls.
this link will give you what you need from aws cli https://aws.amazon.com/blogs/big-data/embed-interactive-dashboards-in-your-application-with-amazon-quicksight/
this is the step 3 aws cli cmd to give you embeded URL ( i was able to excecute)
aws quicksight get-dashboard-embed-url --aws-account-id (your account ID) --dashboard-id (your dashgboard ID) --identity-type IAM
there are many other dependence to enable the embeded dashboard per aws dcouments. i have not able to successfully doen that. GL and let me know if you make it happen!
PHP implementation
(in addition to Siva Sumanth's answer)
https://gist.github.com/evgalak/d0d1adf099e2d7bff741c16a89bf30ba

Cognito Groups with IAM Permissions

What I want to implement:
I have a Cognito User-Pool and I have some Users and some Groups. I want that certain Users have access to API Gateway functions, some Users can access some functions and others have no access.
What I did:
I created three groups and assigned the Users to each of the groups. I gave each of the groups an IAM role and gave each roled spezific policies. The permission for the group for all users looks like this:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "VisualEditor0",
"Effect": "Allow",
"Action": "execute-api:*",
"Resource": "*"
}
]
}
I created Lambda functions and API Gateway Resources through the Serverless framework. I set the authorizer to a Cognito User-Pool authorizer.
(I tried a couple different things like using federated identities but that didnt seem to work as well)
What is my result:
All Users have full access to the API Gateway. The given permissions do not seem to make any difference to the access of each user.
Help:
What did I do wrong?
How can I achieve my goal?
The roles attached to a user pool group only come into picture when you generate credentials for the user using Cognito Federated Identity. Adding groups to a user pool
IAM roles and their permissions are tied to the temporary AWS
credentials that Amazon Cognito identity pools provide for
authenticated users. Users in a group are automatically assigned the
IAM role for the group when AWS credentials are provided by Amazon
Cognito Federated Identities using the Choose role from token option.
So basically
create an identity pool attached to your user pool.
change authorization for API gateway to IAM
after login to user pool, user id_token to generate the federated identity
use this identity (secret key + access key + token) for authorization with API gateway.
Now your roles should be honored. But mind you - you will be required to generate AWS SigV4 credentials on your own as for some reason this is not provided out of the box. I ended up using aws-sign-web for use in browser.
PS: your role seems to give blanket access to API gateway. you will need to fix that as well. e.g. sample role I used to limit access to one API endpoint
{
"Version": "2012-10-17",
"Statement": [
{
"Action": "execute-api:Invoke",
"Resource": [
"arn:aws:execute-api:us-east-2:<aws account id>:<API id>/*/*/acc/*"
],
"Effect": "Allow"
}
]
}
Sample code to generate federated identity
function getAccessToken(idToken, idenPoolId, userPool) {
let region = idenPoolId.split(":")[0];
let provider = "cognito-idp." + region + ".amazonaws.com/" + userPool;
let login = {};
login[provider] = idToken;
// Add the User's Id Token to the Cognito credentials login map.
let credentials = new AWS.CognitoIdentityCredentials({
IdentityPoolId: idenPoolId,
Logins: login
});
//call refresh method in order to authenticate user and get new temp credentials
credentials.get((error) => {
if (error) {
console.error(error);
//let response = {
// statusCode: 500,
// body: JSON.stringify(error)
//};
return null;
} else {
console.log('Successfully logged!');
console.log('AKI:'+ credentials.accessKeyId);
console.log('AKS:'+ credentials.secretAccessKey);
console.log('token:' + credentials.sessionToken);
let response = JSON.stringify({
'AKI': credentials.accessKeyId,
'AKS': credentials.secretAccessKey,
'token': credentials.sessionToken
});
return response;
}
});
}
I have a much better solution, and you don't need the IAM.
Simply save the pair of {username, serviceName} in a S3 or DB. So every time, you get the request for a service:
Check if the user is authorized (from Cognito).
Check if the user is authorized for the service (S3, MySQL, RDS, etc.).
Why I think it is better
Because adding/removing users from services, you don't need to login as an admin to IAM. And hopefully later on, you can create a dashboard for management.
Work Flow
UserA sends a request to your securityApi.
SecurityApi checks the token is authorized (user is valid or not).
If the UserA is valid, the securityApi, sends the username of the user (can get it from the payload of the token) and the service name to a DB, to see if the user has access to the user. For example for Mysql (use RDS for this):
SELECT username from ServiceX LIMIT 1 WHERE username = "xyz";
If the second or third steps passed the user is 1. valid user and 2. has the right to use the service. If the user is failed in step 2 or 3, the user is not authorized to use the service.

AWS quicksight login user programmatically without prompt

Our firm has built a custom admin portal and we want to provide our users a new link called reports which will link to aws quicksight dashboard but we don't want them to log in again in quicksight we cant the app (admin portal) to auto log them in with a specific role to see the dashboard. How can we achieve this?
Note: This answer is applicable only if you are using AWS Cognito
In order to generate Quicksight secure dashboard url, follow the below steps:
Step 1: Create a new Identity Pool. Go to https://console.aws.amazon.com/cognito/home?region=us-east-1 , click ‘Create new Identity Pool’
Give an appropriate name.
Go to the Authentication Providers section, select Cognito.
Give the User Pool ID(your User pool ID) and App Client ID (go to App Clients in userpool and copy id).
Click ‘Create Pool’. Then click ‘Allow’ to create roles of the identity pool in IAM.
Step 2: Assign Custom policy to the Identity Pool Role
Create a custom policy with the below JSON.
{
"Version": "2012-10-17",
"Statement": [
{
"Action": "quicksight:RegisterUser",
"Resource": "*",
"Effect": "Allow"
},
{
"Action": "quicksight:GetDashboardEmbedUrl",
"Resource": "*",
"Effect": "Allow"
},
{
"Action": "sts:AssumeRole",
"Resource": "*",
"Effect": "Allow"
}
]
}
Note: if you want to restrict the user to only one dashboard, replace the * with the dashboard ARN name in quicksight:GetDashboardEmbedUrl,
then goto the roles in IAM.
select the IAM role of the Identity pool and assign custom policy to the role.
Step 3: Configuration for generating the temporary IAM(STS) user
Login to your application with the user credentials.
For creating temporary IAM user, we use Cognito credentials.
When user logs in, Cognito generates 3 token IDs - IDToken, AccessToken, RefreshToken. These tokens will be sent to your application server.
For creating a temporary IAM user, we use Cognito Access Token and credentials will look like below.
AWS.config.region = 'us-east-1';
AWS.config.credentials = new AWS.CognitoIdentityCredentials({
IdentityPoolId:"Identity pool ID",
Logins: {
'cognito-idp.us-east-1.amazonaws.com/UserPoolID': AccessToken
}
});
For generating temporary IAM credentials, we call sts.assume role method with the below parameters.
var params = {
RoleArn: "Cognito Identity role arn",
RoleSessionName: "Session name"
};
sts.assumeRole(params, function (err, data) {
if (err) console.log( err, err.stack); // an error occurred
else {
console.log(data);
})
You can add additional parameters like duration (in seconds) for the user.
Now, we will get the AccessKeyId, SecretAccessKey and Session Token of the temporary user.
Step 4: Register the User in Quicksight
With the help of same Cognito credentials used in the Step 3, we will register the user in quicksight by using the quicksight.registerUser method with the below parameters
var params = {
AwsAccountId: “account id”,
Email: 'email',
IdentityType: 'IAM' ,
Namespace: 'default',
UserRole: ADMIN | AUTHOR | READER | RESTRICTED_AUTHOR | RESTRICTED_READER,
IamArn: 'Cognito Identity role arn',
SessionName: 'session name given in the assume role creation',
};
quicksight.registerUser(params, function (err, data1) {
if (err) console.log("err register user”); // an error occurred
else {
// console.log("Register User1”);
}
})
Now the user will be registered in quicksight.
Step5: Update AWS configuration with New credentials.
Below code shows how to configure the AWS.config() with new credentials generated Step 3.
AWS.config.update({
accessKeyId: AccessToken,
secretAccessKey: SecretAccessKey ,
sessionToken: SessionToken,
"region": Region
});
Step6: Generate the EmbedURL for Dashboards:
By using the credentials generated in Step 3, we will call the quicksight.getDashboardEmbedUrl with the below parameters
var params = {
AwsAccountId: "account ID",
DashboardId: "dashboard Id",
IdentityType: "IAM",
ResetDisabled: true,
SessionLifetimeInMinutes: between 15 to 600 minutes,
UndoRedoDisabled: True | False
}
quicksight.getDashboardEmbedUrl(params,
function (err, data) {
if (!err) {
console.log( data);
} else {
console.log(err);
}
}
);
Now, we will get the embed url for the dashboard.
Call the QuickSightEmbedding.embedDashboard from front end with the help of the above generated url.
The result will be the dashboard embedded in your application with filter controls.
AWS support single sign on.
I believe you can use that feature for the users.
please click here for more information