Securing AWS Elasticsearch best practices - amazon-web-services

I currently working on a web project. We are trying to use elastic search. Our all web projects and other tools currently hosting on Amazon.
We just create an EC2 instance for Elastic search, Log Stash, and kibana. After that, we configured Elastic search. We're successfully using elastic search for our purposes.
But there is a problem way the communicating with elastic search and our Web APIs.
Our servers auto-scaling so we configured elastic search for IP security like below.
So we are currently suffocating because of auto-scaling. IPs changing time to time. How can we configure our Elastic Search for using from our Web API servers? What are the best practices for this situation?
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "",
"Effect": "Allow",
"Principal": {
"AWS": "*"
},
"Action": "es:*",
"Resource": "arn:aws:es:eu-west:1111111111:domain/xx-log/*",
"Condition": {
"IpAddress": {
"aws:SourceIp": [
"1.1.1.1",
"1.1.1.2",
"1.1.1.3",
...
]
}
}
}
]
}

Your best bet is to create an iam role, and then in your launch configuration set that role as the "IAM role". http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-roles-for-amazon-ec2.html
And then after use a policy that allows the AWS role and some specific IPs access to your elasticsearch domain.
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::ACCOUNTNUMBER:role/ROLENAME"
},
"Action": [
"es:ESHttpGet",
"es:ESHttpHead",
"es:ESHttpPost",
"es:ESHttpPut"
],
"Resource": "arn:aws:es:ca-central-1:ACCOUNTNUMBER:domain/ELASTICSEARCHDOMAINNAME/*"
},
{
"Sid": "",
"Effect": "Allow",
"Principal": {
"AWS": "*"
},
"Action": [
"es:ESHttpGet",
"es:ESHttpHead",
"es:ESHttpPost",
"es:ESHttpPut"
],
"Resource": "arn:aws:es:ca-central-1:ACCOUNTNUMBER:domain/ELASTICSEARCHDOMAINNAME/*",
"Condition": {
"IpAddress": {
"aws:SourceIp": "IPADDRESSHERE"
}
}
}
]
}

Related

Automating source IP of OpenSearch using Boto3/Lambda

Every time our IPs change, I have to keep updating this policy to access Kibana. I thought I could automate this, but is there any way I can delete an existing policy and create a new one on Lambda? I'm unable to find anything on Boto3 regarding this.
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": "*"
},
"Action": "es:*",
"Resource": "arn:aws:es:ap-south-1:xxxxxxxxxxx/*",
"Condition": {
"IpAddress": {
"aws:SourceIp": [
"xxxxxxxx",
"xxxxxxx",
"xxxxxxxxx",
"xxxxxxxxx"
]
}
}
}
]
}
In boto3 you can use update_elasticsearch_domain_config which has the option AccessPolicies. So basically you have to overwrite the entire policy. You can't just modify directly the IP addresses.

Enforce tag policy for all services in aws

I need a AWS IAM policy to enforce Tagging for all the services. (Not One By One).
Is that possible?
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "DenyCreateSecretWithNoProjectTag",
"Effect": "Deny",
"Action": "secretsmanager:CreateSecret",
"Resource": "*",
"Condition": {
"Null": {
"aws:RequestTag/Project": "true"
}
}
},
{
"Sid": "DenyRunInstanceWithNoProjectTag",
"Effect": "Deny",
"Action": "ec2:RunInstances",
"Resource": [
"arn:aws:ec2:*:*:instance/*",
"arn:aws:ec2:*:*:volume/*"
],
"Condition": {
"Null": {
"aws:RequestTag/Project": "true"
}
}
},
{
"Sid": "DenyCreateSecretWithNoCostCenterTag",
"Effect": "Deny",
"Action": "secretsmanager:CreateSecret",
"Resource": "*",
"Condition": {
"Null": {
"aws:RequestTag/CostCenter": "true"
}
}
},
{
"Sid": "DenyRunInstanceWithNoCostCenterTag",
"Effect": "Deny",
"Action": "ec2:RunInstances",
"Resource": [
"arn:aws:ec2:*:*:instance/*",
"arn:aws:ec2:*:*:volume/*"
],
"Condition": {
"Null": {
"aws:RequestTag/CostCenter": "true"
}
}
}
]
}
This is from AWS Documentation. I need all aws services together.
There is no way to enforce tagging for all possible resources up front. I recommend reading AWS white paper:
Tagging Best Practices - Amazon Web Services (AWS)
Even with AWS Organization and TagPolicies you can't fully enforce tags since:
Enforcement has no effect on resources that are created without tags.
Also with AWS Organization's TagPolicies only some resources are supported, not all resources available in AWS.
The white paper recommends using CloudFormation and Service Catalog to proactively tag resources.
If the account is part of an Organisation it can be enforced through tagging policies: https://docs.aws.amazon.com/organizations/latest/userguide/orgs_manage_policies_tag-policies.html.
If not you can use the AWS config rule of required tags: https://docs.aws.amazon.com/config/latest/developerguide/required-tags.html

How to manually setup AWS ElasticSearch and use kibana through browser?

I've been struggling on setting up AWS ES and use kibana. I was following Amazon Elasticsearch Service docs on AWS.
When I get to Step 2: Upload Data to an Amazon ES Domain for Indexing, I was failed on running curl -XPUT elasticsearch_domain_endpoint/movies/_doc/1 -d '{"director": "Burton, Tim", "genre": ["Comedy","Sci-Fi"], "year": 1996, "actor": ["Jack Nicholson","Pierce Brosnan","Sarah Jessica Parker"], "title": "Mars Attacks!"}' -H 'Content-Type: application/json' as the docs indicated, getting error of {"Message":"User: anonymous is not authorized to perform: es:ESHttpPut"}.
I've set the policy on ES as:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::my_id:user/my_iam_user"
},
"Action": "es:*",
"Resource": "arn:aws:es:us-west-2:my_id:domain/my-domain/*"
},
{
"Sid": "",
"Effect": "Allow",
"Principal": {
"AWS": "*"
},
"Action": "es:*",
"Resource": "arn:aws:es:us-west-2:my_id:domain/my-domain/*",
"Condition": {
"IpAddress": {
"aws:SourceIp": [my_ips]
}
}
}
]
}
I got the IPs above by calling ifconfig | grep "inet " | grep -v 127.0.0.1 from terminal, hitting checkip.amazonaws.com, and checking Developer Tools -> Network on chrome (those are 3 different IPs and I added them all).
I've also added following role on my IAM user:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "VisualEditor0",
"Effect": "Allow",
"Action": [
"es:DescribeReservedElasticsearchInstanceOfferings",
"es:DescribeReservedElasticsearchInstances",
"es:ListDomainNames",
"es:PurchaseReservedElasticsearchInstance",
"es:DeleteElasticsearchServiceRole",
"es:ListElasticsearchInstanceTypes",
"es:DescribeElasticsearchInstanceTypeLimits",
"es:ListElasticsearchVersions"
],
"Resource": "*"
},
{
"Sid": "VisualEditor1",
"Effect": "Allow",
"Action": "es:*",
"Resource": "arn:aws:es:us-west-2:my_id:domain/my-domain"
}
]
}
I've setup AWS CLI on my machine and I'm able to get the correct result through aws es describe-elasticsearch-domain --domain my-domain.
Still, I failed to call curl XPUT above and failed to access kibana for the same reason {"Message":"User: anonymous is not authorized to perform: es:ESHttpPut"}
Here's couples of article I read before I raised the question here:
https://aws.amazon.com/premiumsupport/knowledge-center/anonymous-not-authorized-elasticsearch/?nc1=h_ls
https://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/es-ac.html
https://aws.amazon.com/cn/blogs/database/set-access-control-for-amazon-elasticsearch-service/
Proper access policy for Amazon Elastic Search Cluster
https://aws.amazon.com/cn/blogs/security/how-to-control-access-to-your-amazon-elasticsearch-service-domain/
And still couldn't get it work.
Can anyone kindly guide me through the whole process of setting up the AWS ES manually, and being able to manipulate it through AWS CLI as well as kibana on browser? I would be really appreciated if it could be a detailed step-by-step guide instead of throwing aws docs. Thank you so much.
It turns out that the IP I used was incorrect. I should have called checkip.amazonaws.com while not under VPN, and the IP may change down the line. The policy on ES should looks like: Anther Answer
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::xxxxxxxxxxxx:root"
},
"Action": "es:*",
"Resource": "arn:aws:es:us-west-2:xxxxxxxxxxxx:domain/my-elasticsearch-domain/*"
},
{
"Sid": "",
"Effect": "Allow",
"Principal": {
"AWS": "*"
},
"Action": "es:*",
"Resource": "arn:aws:es:us-west-2:xxxxxxxxxxxx:domain/my-elasticsearch-domain/*",
"Condition": {
"IpAddress": {
"aws:SourceIp": [
"192.168.1.0",
"192.168.1.1"
]
}
}
}
]
}
*PS. Make sure you are calling checkip.amazonaws.com on the same browser (same user as well if you are using chrome)

Add multiple domain access policy to AWS Elasticsearch Service (Static IP and Lambda ARN)

After setting up AWS Elasticsearch, I installed Logstash and Kibana proxy on a static IP server, and added this domain access policy on ES and it's working fine:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "",
"Effect": "Allow",
"Principal": {
"AWS": "*"
},
"Action": "es:*",
"Resource": "arn:aws:es:ap-southeast-1:323137313233:domain/sg-es-logs/*",
"Condition": {
"IpAddress": {
"aws:SourceIp": [
"192.192.192.192"
]
}
}
}
]
}
Now I need to allow Lambda function to execute es:ESHttpDelete action on AWS ES, so I created the function with the existing role service-role/Elasticsearch then copied the relevent ARN from IAM Managment console to add it to AWS ES access policy, to come up with this:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": [
"arn:aws:iam:: 323137313233:role/service-role/Elasticsearch"
]
},
"Action": [
"es:*"
],
"Resource": "arn:aws:es:ap-southeast-1:323137313233:domain/sg-es-logs/*"
}
]
}
The problem is on ES I should either choose domain access policy for Static IP or ARN but not both. When I tried to merge them manually not by using the console it didn't work. I checked AWS documentation but they didn't mention if is that possible or not.
You can add multiple policy statements inside the Statement array in the JSON format of policy. So, your final policy would be something like:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "",
"Effect": "Allow",
"Principal": {
"AWS": "*"
},
"Action": "es:*",
"Resource": "arn:aws:es:ap-southeast-1:323137313233:domain/sg-es-logs/*",
"Condition": {
"IpAddress": {
"aws:SourceIp": [
"192.192.192.192"
]
}
}
},
{
"Effect": "Allow",
"Principal": {
"AWS": [
"arn:aws:iam:: 323137313233:role/service-role/Elasticsearch"
]
},
"Action": [
"es:*"
],
"Resource": "arn:aws:es:ap-southeast-1:323137313233:domain/sg-es-logs/*"
}
]
}

how to limit instance launch by instance type in AWS using IAM service

I am using the policy to limit RunIstances only to a specific instance types and a specific region. When I run the launch wizard or simulation under a test user I am getting "implicitly denied" error.
Here is the policy:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": "ec2:RunInstances",
"Resource": [
"arn:aws:ec2:us-east-1::instance/*"
],
"Condition": {
"StringEquals": {
"ec2:InstanceType": [
"t1.micro",
"m1.small"
]
}
}
},
{
"Effect": "Allow",
"Action": "ec2:RunInstances",
"Resource": [
"arn:aws:ec2:us-east-1::image/ami-*",
"arn:aws:ec2:us-east-1::subnet/*",
"arn:aws:ec2:us-east-1::network-interface/*",
"arn:aws:ec2:us-east-1::volume/*",
"arn:aws:ec2:us-east-1::key-pair/*",
"arn:aws:ec2:us-east-1::security-group/*"
]
},
{
"Effect": "Allow",
"Action": [
"ec2:Describe*",
"ec2:CreateSecurityGroup",
"ec2:DeleteSecurityGroup",
"ec2:AuthorizeSecurityGroupIngress",
"ec2:AuthorizeSecurityGroupEgress",
"ec2:CreateKeyPair"
],
"Resource": [
"*"
]
}
]
}
could somebody point to the issue?
In your resource blocks, insert an asterisk between the two ":" in the arn lines, to specify all accounts, or replace it with your account number.
"arn:aws:ec2:us-east-1:*:instance/*"
"arn:aws:ec2:us-east-1:*:image/ami-*",
"arn:aws:ec2:us-east-1:*:subnet/*",
"arn:aws:ec2:us-east-1:*:network-interface/*",
"arn:aws:ec2:us-east-1:*:volume/*",
"arn:aws:ec2:us-east-1:*:key-pair/*",
"arn:aws:ec2:us-east-1:*:security-group/*"
Please see the IAM policy generator tool. Your code does not look like the right syntax.
First, Allow all actions in EC2. Next, Deny specific actions in EC2.
Example to Allow all actions in EC2:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"NotAction": "ec2:*",
"Resource": "*"
}
]
}
Example to Deny creating resources in specific regions:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "DenyTheseActions",
"Effect": "Deny",
"Action": [
"ec2:RunInstances",
"ec2:StartInstances"
],
"Resource": "arn:aws:ec2:us-west-1",
"Resource": "arn:aws:ec2:us-east-1",
"Resource": "arn:aws:ec2:eu-west-1",
"Resource": "arn:aws:ec2:sa-east-1",
"Resource": "arn:aws:ec2:ap-northeast-1",
"Resource": "arn:aws:ec2:ap-southeast-1",
"Resource": "arn:aws:ec2:ap-southeast-2"
}
]
}
Currently the easier way to control access to AWS regions, EC2 and RDS Instances sizes and types would probably be using IAM policies with the Condition (Optional) policy element – lets you specify conditions for when a policy is in effect.
We'll setup an AWS IAM Policy using a Condition statement, which will allow full AWS EC2 and RDS services, strictly for the 3 more cost effective -> AWS regions which are:
us-east-1 (North Virginia, USA)
us-east-2 (Ohio, USA)
us-west-2 (Oregon, USA)
All other regions will not be allowed, this policy also sets up a conditional access control for starting both EC2 and RDS instances. Instead of specifying all of the possible type/class instances to run, we use Deny effect in the statement, which allows for a short and simple policy limiting by instances sizes (micro, small, medium and large). This has the further effect of preventing any other policies from overriding the block.
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "MultiServiceFullAccessCustom",
"Effect": "Allow",
"Action": [
"ec2:*",
"rds:*"
],
"Resource": [
"*"
],
"Condition": {
"StringEquals": {
"aws:RequestedRegion": [
"us-east-1",
"us-east-2",
"us-west-2"
]
}
}
},
{
"Sid": "Ec2RunInstanceCustomSize",
"Effect": "Deny",
"Action": "ec2:RunInstances",
"Resource": [
"arn:aws:ec2:*:*:instance/*"
],
"Condition": {
"ForAnyValue:StringNotLike": {
"ec2:InstanceType": [
"*.nano",
"*.micro",
"*.small",
"*.medium",
"*.large"
]
}
}
},
{
"Sid": "RdsFullAccessCustomSize",
"Effect": "Deny",
"Action": [
"rds:CreateDBInstance",
"rds:CreateDBCluster"
],
"Resource": [
"arn:aws:rds:*:*:db:*"
],
"Condition": {
"ForAnyValue:StringNotLike": {
"rds:DatabaseClass": [
"*.micro",
"*.small",
"*.medium",
"*.large"
]
}
}
}
]
}
What does this policy not protect us from?
This does not impose limits on the size of instances deployed though other services - most importantly, through an auto-scaling group or EKS.
We are not limiting in any way the total number of instances spun up by a given user. The normal limits help here, but there isn't a way to determine how many instances are currently running through just policy.
We're not capping costs, just adding some protections. If you want to set alerts for cost overruns or spikes, you need to look into the Billing controls.
terraform-aws-cost-billing-alarm
terraform-aws-cost-budget
terraform-aws-lambda-nuke
terraform-aws-lambda-scheduler-stop-start
We do not require any Multi Factor Authentication (MFA) for the role switch. The assumption is that users logging in are already using some form of second factor for authentication. This is mostly due to the increased complexity around using the CLI with MFA, and the fact that the role change is only to allow an override, not to enable an elevation to another security domain. You might want to add the feature; it is not difficult to add, and can bring reasonable protections.
We are only looking at instance startup, but you may consider who can shutdown or terminate instances, as that may result in downtime or data loss.
Reference links
https://blog.vizuri.com/limiting-allowed-aws-instance-type-with-iam-policy
https://blyx.com/2016/03/24/how-to-restrict-by-regions-and-instance-types-in-aws-with-iam/
https://aws.amazon.com/blogs/security/easier-way-to-control-access-to-aws-regions-using-iam-policies/
https://medium.com/faun/100-days-of-devops-day-10-restricting-user-to-launch-only-t2-instance-509aaaec5aa2
https://aws.amazon.com/blogs/security/back-to-school-understanding-the-iam-policy-grammar/