Have been reading aws guidelines for a while, but still cannot run a dynamoDB webservice. Already have a working codebase, just need to run a webservice and get access keys and endpoint url. The only button amazon shows is Create table, which I do not need as I create them from code.
First thing, the endpoints to access dynamoDB are regional and don't depend on the table name, you can find them here https://docs.aws.amazon.com/general/latest/gr/rande.html
Second thing, if you create tables and access them from the same application (running on AWS) you should use AWS roles and make sure you give the right permissions to the IAM policy associated with the role, if you're creating the tables from one specific service and accessing them from different services than you need to make sure every service has the right role and, again IAM policy associated to the role, finally if you're accessing them with different users you need to make sure those users have an associeted IAM policy that gives them access.
If you don't want to create a big amount of policies and you don't want to modified them when you create new resources you can use a prefix for the name of your tables such as app_a_... and in the policies give access to the right subset of resources using the same prefix for example
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "accessToAppAtables",
"Effect": "Allow",
"Action": "dynamodb:*",
"Resource": "arn:aws:dynamodb:<REGION>:<ACCOUNT_ID>:table/app_a_*"
}
]
}
Refer this document.
Related
In a traditional MySql Server situation, as the owner of a database, I create a User and from the database I grant certain access rights to the User object. An application can then (and only) access the database by supplying the password for the User.
I am confused and don't see a parallel when it comes to giving access to a DynamoDB table. From the DynamoDB Tables page, I can't find a means to grant permission for an IAM user to access a table. There is an Access Control tab, but that appears to be for Facebook/Google users.
I read about attaching policies but am confused further. How is access controlled if anyone can create a policy that can access all tables?
What am I missing? I just want to create a "login" for a Node application to access my DynamoDB table.
If anyone in your AWS account can create IAM policies you have a real security issue.
Only a few accounts should do that (Create IAM policies).
DynamoDB accesses work along with IAM user like you said, so, you need to do the following:
Create IAM groups to classify your IAM users, for example, DBAGroup for dbas, DEVGroup for developers and so on.
Create IAM policies to grant specific access to your DynamoDB tables for each group.
Apply the policies to the specific groups for granting accesses.
For login purposes, you need to develop a module that will verify the credentials with IAM service, so you need to execute IAM API calls. This module could be deployed within an EC2, could be a Javascript call to an API Gateway's endpoint along with a Lambda function, Etc.
What you need to do:
Create an account on IAM service that will be able to execute API calls to the IAM service for verifying credentials (Login and password).
This account should have only permissions for doing that (Verify user login and password).
Use the API credentials to be able to execute API calls.
If you don't want to create your own module for login purposes, take a look at Amazon Cognito
Amazon Cognito lets you add user sign-up/sign-in and access control to your web and mobile apps quickly and easily. Cognito scales to millions of users and supports sign-in with social identity providers such as Facebook, Google, and Amazon, and enterprise identity providers via SAML 2.0.
The last step is how your module execute API calls to IAM service? As you may know, we need API Credentials. So, using the logged user's credentials you will be able to execute API calls to read data from tables, execute CRUD operations, Etc.
To set specific permissions for certain tables as in SQL Server you must do this:
In Identity and Access Management (IAM) create a new security policy on the JSON tab:
Use the following JSON as an example to allow a full CRUD or remove the items within the "Action" section to allow only the desired items:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "ListAndDescribe",
"Effect": "Allow",
"Action": [
"dynamodb:List*",
"dynamodb:DescribeReservedCapacity*",
"dynamodb:DescribeLimits",
"dynamodb:DescribeTimeToLive"
],
"Resource": "*"
},
{
"Sid": "SpecificTable",
"Effect": "Allow",
"Action": [
"dynamodb:BatchGet*",
"dynamodb:DescribeStream",
"dynamodb:DescribeTable",
"dynamodb:Get*",
"dynamodb:Query",
"dynamodb:Scan",
"dynamodb:BatchWrite*",
"dynamodb:CreateTable",
"dynamodb:Delete*",
"dynamodb:Update*",
"dynamodb:PutItem"
],
"Resource": "arn:aws:dynamodb:*:*:table/MyTable"
}
]
}
Give the policy a name and save it.
After that, go to the Identity and Access Management (IAM) Users screen and create a new user as shown below.
Remember to set the field ** Access type ** as * Programmatic access *, it is not necessary to add the user to a group, click on "Atach existing policies directly" and add the policy previously created.
Finished! You already have everything you need to connect your application to Dynamodb.
I have been struggling to figure out how to communicate with the Amazon ES service from my EC2 instances.
The documentation clearly states that the Amazon ES service supports IAM User & Role based access policies. http://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/es-createupdatedomains.html#es-createdomain-configure-access-policies
However, when I have this access policy for my ES domain:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::123456789:role/my-ec2-role"
},
"Action": "es:*",
"Resource": "arn:aws:es:us-west-2:123456789:domain/myDomain/*"
}
]
}
I can't log into an ec2 instance and run a curl to hit my elasticsearch cluster.
Trying to do a simple curl of the _search API:
curl "http://search-myDomain.es.amazonaws.com/_search"
Produces an authentication error response:
{"Message":"User: anonymous is not authorized to perform: es:ESHttpGet on resource: arn:aws:es:us-west-2:123456789:domain/myDomain/_search"}
Just to be extra safe I put the AmazonESFullAccess Policy on my IAM Role, still doesn't work.
I must be missing something, because being able to programmatically interact with Elasticsearch from ec2 instances that use an IAM Role is essential to getting anything accomplished with the Amazon ES Service.
I also see this contradictory statement in the docs.
IAM-based Policy Example You create IAM-based access policies by
using the AWS IAM console rather than the Amazon ES console. For
information about creating IAM-based access policies, see the IAM
documentation.
That link to IAM documentation, is to the home page of IAM and contains exactly zero information about how to do it. Anyone got a solution for me?
When using IAM service with AWS, you must sign your requests. curl doesn't support signed requests (which consists of hashing the request and adding a parameter to the header of the request). You can use one of their SDK's that has the signing algorithm built in, and then submit that request.
See:
http://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/what-is-amazon-elasticsearch-service.html#signing-requests
You can find the SDKs for popular languages here:
http://aws.amazon.com/tools/
First, you said you can't login to an EC2 instance to curl the ES instance? You can't login? Or you can't curl it from EC2?
I have my Elasticsearch (Service) instance open to the world (with nothing on it) and am able to curl it just fine, without signing. I changed the access policy to test, but unfortunately it takes forever to come back up after changing it...
My policy looks like this:
{ "Version": "2012-10-17", "Statement": [
{
"Sid": "",
"Effect": "Allow",
"Principal": "*",
"Action": "es:*",
"Resource": "arn:aws:es:us-east-1:843348267853:domain/myDomain/*"
},
{
"Sid": "",
"Effect": "Allow",
"Principal": "*",
"Action": "es:*",
"Resource": "arn:aws:es:us-east-1:843348267853:domain/myDomain"
}
]
}
I realize this isn't exactly what you want, but start off with this (open to the world), curl from outside AWS and test it. Then restrict it, that way you're able to isolate the issues.
Also, I think you have an issue with the "Principal" in your access policy. You have your EC2 Role. I understand why you're doing that, but I think the Principal requires a USER, not a role.
See below:
http://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/es-createupdatedomains.html#es-createdomain-configure-access-policies
Principal
Specifies the AWS account or IAM user that is allowed or denied access
to a resource. Specifying a wildcard (*) enables anonymous access to
the domain, which is not recommended. If you do enable anonymous
access, we strongly recommend that you add an IP-based condition to
restrict which IP addresses can submit requests to the Amazon ES
domain.
EDIT 1
To be clear, you added the AmazonESFullAccess policy to the my-ec2-role? If you're going to use IAM access policies, I don't think you can have a resource based policy attached to it (which is what you're doing).
http://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_compare-resource-policies.html
For some AWS services, you can grant cross-account access to your
resources. To do this, you attach a policy directly to the resource
that you want to share, instead of using a role as a proxy. The
resource that you want to share must support resource-based policies.
Unlike a user-based policy, a resource-based policy specifies who (in
the form of a list of AWS account ID numbers) can access that
resource.
Possibly try removing the access policy altogether?
Why you don't create a proxy with elastic ip and allow your proxy to access your ES?
Basically exists three forms that you can limit access in your ES:
Allow everyone
White IP list
Signing the access key and secret key provided by AWS.
I'm using two forms, in my php apps I prefer to use proxy behind the connection to ES and in my nodejs app I prefer to sign my requests using the http-aws-es node module.
It's useful to create a proxy environment because my users needs to access the kibana interface to see some reports and it's possible because they have configured the proxy in their browsers =)
I must recommend to you close the access to your ES indexes, because it's pretty easy to delete them, curl -XDELETE https://your_es_address/index anyone can do it but you can say: "how the others users will get my ES address?" and I will answer you: "Security based in dimness isn't a real security"
My security access policy is basically something like it:
http://pastebin.com/EUKT1ekX
I encountered this issue recently and the root problem is that none of the Amazon SDKs yet support calling Elasticsearch operations like search, put, etc.
The only workaround at the moment is to execute requests directly against the endpoint using signed requests:
http://docs.aws.amazon.com/general/latest/gr/sigv4-signed-request-examples.html
The example here is for calling EC2, but it can be modified to instead call against Elasticsearch. Just modify the "service" value to "es". From there, you have to fill in values for
the endpoint (which is the full URL of your cluster including operation without request parameters)
the host (the part between https:// and your canonical URI like /_status
the canonical uri which is the URI after the first / inclusive (like /_status) but without the query string
the request parameters (everything after ? inclusive)
Note that I've only managed to get this working so far using AWS credentials as the assumption is that you pass in an access key and secret key to the various signing calls (access_key and secret_key in the example). It should be doable using IAM roles but you'll have to call into the security token service first to get temporary credentials that can be used to sign the request. Until you do that, be sure to edit your access policy on the Elasticsearch cluster to allow user creds (user/
you need to sign your request and unfortunately, it is no longer supported by the official elasticsearch library. Check this Github issue (https://github.com/elastic/elasticsearch-js/issues/1182#issuecomment-630641702)
They want to enforce their own cloud solution
I've already gone through the documentation and it closely mirrors my use-case except that I cannot use Google, Facebook or Amazon as my identity provider, but I already have an enterprise level OAuth 2.0 access token for authenticated users.
I understand that I could possibly use Enterprise Federated support from AWS STS to get temporary credentials and use them to further access the AWS resources but I fail to understand how can i configure the IAM Policy to use these credentials to achieve horizontal information hiding.
I have certain tables in DynamoDB in which I store the details of all the users of my application and my application supports multiple tenants so I want the users of one tenant to NOT being able to access data of other tenants. The IAM policy that I could configure is of the type:
"Condition": {
"ForAllValues:StringEquals": {
"dynamodb:LeadingKeys": ["${www.amazon.com:user_id}"]
}
}
Now my users are NOT logged in via Amazon ( or Google or Facebook ) and hence I cannot use the keys like "${www.amazon.com:user_id}" etc. Also my hash key for some tables are composite.
So my question is how to achieve multi-tenancy at the database level and be able to segregate or separate data per tenant i.e. to hide certain rows of my tables from the users who should not have access on it.
Is it possible to specify custom Policy Variables while defining the IAM policy and specify how to resolve those at runtime ? Or some other way be ?
My tables in Dynamo currently have composite hash keys, which are a combination of Tenant_ID and User_ID so can I specify some kind of rule in the IAM Policy so that I should be able to achieve horizontal information hiding ?
Please let me know if you need more information about my use case.
Regards,
Agraj
In order to enable fine-grained data access in DynamoDB, you must specify an IAM Policy Element Variable in the DynamoDB IAM policy.
A typical policy may look like this:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "FullAccessToUserItems",
"Effect": "Allow",
"Action": [
"dynamodb:*"
],
"Resource": [
"arn:aws:dynamodb:*:table/*"
],
"Condition": {
"ForAllValues:StringEquals": {
"dynamodb:LeadingKeys": [
"${cognito-identity.amazonaws.com:sub}"
]
}
}
}
]
}
Where ${cognito-identity.amazonaws.com:sub} is an IAM policy variable representing the user's sub in Cognito.
Unfortunately Amazon do not publish a list of available policy variables. What this does mean though is that your user management has to be managed through Amazon to enable fine grained security. You cannot define your own policy variables - they have to be pre-defined Amazon variables - hence fine grained security is only available where your users are managed in Amazon.
Additionally your DynamoDB partition key has to match the policy variable. For example you table partition key would have to be the Cognito sub.
If your OAuth token was received from Cognito you can simply post it to the Amazon Token Endpoint, which will give you back an id_token which contain the users Cognito sub.
I'm new to AWS. My client uses AWS to host his EC2 instances. Right now, we are trying to get me API access. Obviously, I need my authentication details to do this.
He set me up an IAM identity under his account, so I can login to the AWS web console and configure EC2 instances. I cannot, however, for the life of me, figure out where my API access keys are displayed. I don't have permissions to view 'My Account', which is where I imagine they'd be displayed.
So, what I'm asking, is how can he grant me API access through his account? How can I access the AWS API using my IAM identity?
Michael - sqlbot's answer is correct (+1), but not entirely complete given the comparatively recent but highly useful addition of Variables in AWS Access Control Policies:
Today we’re extending the AWS access policy language to include
support for variables. Policy variables make it easier to create
and manage general policies that include individualized access
control.
This enables implementation of an 'IAM Credentials Self Management' group policy, which would usually be assigned to the most basic IAM group like the common 'Users'.
Please note that the following solution still needs to be implemented by the AWS account owner (or an IAM user with permissions to manage IAM itself), but this needs to be done once only to enable credentials self management by other users going forward.
Official Solution
A respective example is included in the introductory blog post (and meanwhile has been available at Allow a user to manage his or her own security credentials in the IAM documentation too - Update: this example vanished again, presumably due to being applicable via custom solutions using the API only and thus confusing):
Variable substitution also simplifies allowing users to manage their
own credentials. If you have many users, you may find it impractical
to create individual policies that allow users to create and rotate
their own credentials. With variable substitution, this becomes
trivial to implement as a group policy. The following policy permits
any IAM user to perform any of the key and certificate related actions
on their own credentials. [emphasis mine]
{
"Version": "2012-10-17",
"Statement": [{
"Effect": "Allow",
"Action":["iam:*AccessKey*","iam:*SigningCertificate*"],
"Resource":["arn:aws:iam::123456789012:user/${aws:username}"]
}
]
}
The resource scope arn:aws:iam::123456789012:user/${aws:username} ensures that every user is effectively only granted access to his own credentials.
Please note that this solution still has usability flaws depending on how AWS resources are accessed by your users, i.e. via API, CLI, or the AWS Management Console (the latter requires additional permissions for example).
Also, the various * characters are a wildcard, so iam:*AccessKey* addresses all IAM actions containing AccessKey (see IAM Policy Elements Reference for details).
Extended Variation
Disclaimer: The correct configuration of IAM policies affecting IAM access in particular is obviously delicate, so please make your own judgement concerning the security impact of the following solution!
Here's a more explicit and slightly extended variation, which includes AWS Multi-Factor Authentication (MFA) device self management and a few usability enhancements to ease using the AWS Management Console:
{
"Version": "2012-10-17",
"Statement": [
{
"Action": [
"iam:CreateAccessKey",
"iam:DeactivateMFADevice",
"iam:DeleteAccessKey",
"iam:DeleteSigningCertificate",
"iam:EnableMFADevice",
"iam:GetLoginProfile",
"iam:GetUser",
"iam:ListAccessKeys",
"iam:ListGroupsForUser",
"iam:ListMFADevices",
"iam:ListSigningCertificates",
"iam:ListUsers",
"iam:ResyncMFADevice",
"iam:UpdateAccessKey",
"iam:UpdateLoginProfile",
"iam:UpdateSigningCertificate",
"iam:UploadSigningCertificate"
],
"Effect": "Allow",
"Resource": [
"arn:aws:iam::123456789012:user/${aws:username}"
]
},
{
"Action": [
"iam:CreateVirtualMFADevice",
"iam:DeleteVirtualMFADevice",
"iam:ListVirtualMFADevices"
],
"Effect": "Allow",
"Resource": "arn:aws:iam::123456789012:mfa/${aws:username}"
}
]
}
"You" can't, but:
In IAM, under Users, after he selects your user, he needs to click Security Credentials > Manage Access Keys, and then choose "Create Access Key" to create an API Key and its associated Secret, associated with your IAM user. On the next screen, there's a message:
Your access key has been created successfully.
This is the last time these User security credentials will be available for download.
You can manage and recreate these credentials any time.
Where "manage" means "deactivate or delete," and "recreate" means "start over with a new one." The IAM admin can subsequently see the keys, but not the associated secrets.
From that screen, and only from that screen, and only right then, is where the IAM admin can view the both key and the secret associated with the key or download them to a CSV file. Subsequently, one with appropriate privileges can see the keys for a user within IAM but you can never view the secret again after this one chance (and it would be pretty preposterous if you could).
So, your client needs to go into IAM, under the user he created for you, and create an API key/secret pair, save the key and secret, and forward that information to you via an appropriately-secure channel... if he created it but didn't save the associated secret, he should delete the key and create a new one associated with your username.
If you don't have your own AWS account, you should sign up for one so you can go into the console with full permissions as yourself and understand the flow... it might make more sense than my description.
I am relatively new to AWS, I own the primary AWS account, but need to create a "super-user" account that only has rights to create new users and can only add those users to a set predefined groups with their respective policies (eg. SES-Readonly and SES-FullAccess). I do not want that super-user to be able to create any other groups, nor should they be able to modify any policies applied to the groups. I also do not want this user to have access to the other AWS services (eg. EC2, S3 etc). Is this possible? If so, what would the policy look like?
I have read most of the IAM documentation, and looked at their examples, but I didn't find any examples that were similar to my use case :(
Thanks in advance!
Yes, you need to create an IAM user and then give it this iam policy.
{
"Statement": [
{
"Sid": "Stmt1375475989975",
"Action": [
"iam:AddUserToGroup",
"iam:CreateUser"
],
"Effect": "Allow",
"Resource": "arn:aws:iam::152997954706:user/AMISTACK-02-WEB-User-WYEMFOJZ4BDP"
}
]
}
arn:aws:iam::152997954706:user/AMISTACK-02-WEB-User-WYEMFOJZ4BDP is an example user ARN. You'll need to add in yours specific user's arn.
An easy way to create policy files is to use: http://awspolicygen.s3.amazonaws.com/policygen.html