I am trying to make one of my AWS Rest API private. But after doing steps that AWS docs suggest, It's changing URL of that API.
Here's the steps that I tried:
Create VPC endpoint for API execution.
Change APIs endpoint type to Private
Add VPC endpoint id in VPC endpoint IDs.
Add resource policy to allow API execution from VPC.
Here's the Resource policy
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": "*",
"Action": "execute-api:Invoke",
"Resource": "my api arn"
},
{
"Effect": "Deny",
"Principal": "*",
"Action": "execute-api:Invoke",
"Resource": "My API ARN",
"Condition": {
"StringNotEquals": {
"aws:SourceVpce": "VPC endpoint id",
"aws:SourceVpc": "VPC id"
}
}
}
]
}```
These steps is making my API private/Invokable from VPC only, but I can't invoke that with same URL.
I must have to add VPC endpoint id in URL to call the API.
Old URL that I am using:
https://{restapi-id}.execute-api.{region}.amazonaws.com/{stage}
Here's how new URL looks like:
https://{rest-api-id}-{vpce-id}.execute-api.{region}.amazonaws.com/{stage}
Is there any way to make API private without changing URL?
These are the policies attached to Lambda function calling a REST API in API Gateway. I am trying to understand why we need to have two policies like this, e.g. can somebody explain in simple words what does the first policy do? Why we cannot limit everything with the second policy only? If we only need to allow POST to every path under a particular API what else is required? What is difference between "execute-api" resource and "apigateway"?
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"execute-api:Invoke"
],
"Resource": [
"arn:aws:execute-api:us-east-1:account-id:api-id/*/GET/pets"
]
}
}
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"apigateway:POST"
],
"Resource": [
"arn:aws:apigateway:us-east-1:account-id:api-id/*"
]
}
}
There are two types of permissions for API gateway:
For invoking an API. Your first statement allows invoking the API.
For creating and managing an API. Your second statement allows for creating child resources in the API.
You have two policies, because your resources are different for each statement. So your function can both manage the API and invoke it as well.
I have created the API Gateway with terraform and I am then attaching API's to it using the serverless framework.
I have created a resource policy based on this AWS tutorial (https://aws.amazon.com/premiumsupport/knowledge-center/api-gateway-resource-policy-access/) as I want to be able to use custom API Gateway domains but I do not want my API's accessible by anyone over the internet unless their IP address is in my whitelist.
Here is my rendered policy:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "allow",
"Effect": "Allow",
"Principal": "*",
"Resource": "arn:aws:execute-api:eu-west-1:*:/*/*/*"
},
{
"Sid": "ipwhitelist",
"Effect": "Deny",
"Principal": "*",
"Action": "execute-api:Invoke",
"Resource": "arn:aws:execute-api:eu-west-1:*:/*/*/*",
"Condition": {
"NotIpAddress": {
"aws:SourceIp": [
<<excluded>>
]
}
}
}
]
}
I have redeployed my API and now I am blocked regardless of whether my IP address is in the allowed list or not and according to the tutorial this should work.
I have also tested the policy by removing the entire deny section so it only allows all traffic and this is still resulting in my calls being blocked, when I delete the policy and the redeploy my serverless project it works again, so with that being said is there a reason why the allow policy would still block all IP addresses?
I am looking for ideas of where to look to find out why the white list is not working.
The answer to this is that I was missing a permission from my allow policy, the explicit allow is required to allow anything that is then excluded by the deny policy but it was missing any actions, I had to ensure the following was present in the terraform that generated the allow part of the policy:
actions = ["execute-api:Invoke"]
This is then translated into the following in the actual IAM policy:
"Action": "execute-api:Invoke"
I have setup the follow resource policy in api gateway to restrict access to a source IP (x is just a placeholder). When I manually hit the api endpoint from postman the policy correctly restricts access only to the cidr range I specified in the resource policy below.
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": "*",
"Action": "execute-api:Invoke",
"Resource": "arn:aws:execute-api:us-east-1:x:x/*/*/*”
},
{
"Effect": "Deny",
"Principal": "*",
"Action": "execute-api:Invoke",
"Resource": "arn:aws:execute-api:us-east-1:x:x/*/*/*”,
"Condition": {
"NotIpAddress": {
"aws:SourceIp": [
“x.x.x.x/32"
]
},
"StringNotEquals": {
"aws:sourceVpc": "vpc-x”
}
}
}
]
}
However, I have a lambda function which also calls the same https api gateway endpoint. This function essentially just passes test data into my api at hourly intervals. But, the lambda function is unable to hit the endpoint and gets a 403 forbidden error. I tried adding the sourceVpc to the resource policy, but this did not seem to work. I also tried adding the vpc cidr range too, but again this did not work.
Do you know what cidr I should add to the resource policy to allow my lambda to call my api endpoint too?
I added to the resource policy "aws:SourceIp" the NAT gateway ip of the subnets associated with my lambda function. This allowed my lambda function to invoke the API Gateway successfully.
I'm trying to use Elasticsearch for data storage for a Lambda function connected to Alexa Skills Kit. The Lambda works alright without Elasticsearch but ES provides much-needed fuzzy matching.
The only way I've been able to access it from Lambda is by enabling Elasticsearch global access but that's a really bad idea. I've also been able to access from my computer via open access policy or IP address policy. Is there a way to do read-only access via Lambda and read-write via IP?
On IAM I granted my Lambda role AmazonESReadOnlyAccess. On the ES side I tried this but it only worked for IP address:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": [
"arn:aws:iam::NUMBER:root",
"arn:aws:iam::NUMBER:role/lambda_basic_execution"
]
},
"Action": "es:*",
"Resource": "arn:aws:es:us-east-1:NUMBER:domain/NAME/*"
},
{
"Sid": "",
"Effect": "Allow",
"Principal": {
"AWS": "*"
},
"Action": "es:*",
"Resource": "arn:aws:es:us-east-1:NUMBER:domain/NAME/*",
"Condition": {
"IpAddress": {
"aws:SourceIp": "MY IP"
}
}
}
]
}
This forum post asks the same question but went unanswered.
The only way I know of to do this is to use a resource-based policy or an IAM-based policy on your ES domain. This would restrict access to a particular IAM user or role. However, to make this work you also need to sign your requests to ES using SigV4.
There are libraries that will do this signing for you, for example this one extends the popular Python requests library to sign ElasticSearch requests via SigV4. I believe similar libraries exist for other languages.
Now it's possible from your code with elasticsearch.js. Before you try it, you must install http-aws-es module.
const AWS = require('aws-sdk');
const httpAwsEs = require('http-aws-es');
const elasticsearch = require('elasticsearch');
const client = new elasticsearch.Client({
host: 'YOUR_ES_HOST',
connectionClass: httpAwsEs,
amazonES: {
region: 'YOUR_ES_REGION',
credentials: new AWS.EnvironmentCredentials('AWS')
}
});
// client.search({...})
Of course, before using it, configure access to elasticsearch domain:
For external (outside AWS) access to your Elasticsearch cluster, you want to create the cluster with an IP-based access policy. Something like the below:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "",
"Effect": "Allow",
"Principal": {
"AWS": "*"
},
"Action": "es:*",
"Condition": {
"IpAddress": {
"aws:SourceIp": [
"<<IP/CIDR>>"
]
}
},
"Resource": "arn:aws:es:<<REGION>>:<<ACCOUNTID>>:domain/<<DOMAIN_NAME>>/*"
}
]
}
For your Lambda function, create the role that the Lambda function will assume with the below policy snippet.
{
"Sid": "",
"Effect": "Allow",
"Action": [
"es:DescribeElasticsearchDomain",
"es:DescribeElasticsearchDomains",
"es:DescribeElasticsearchDomainConfig",
"es:ESHttpPost",
"es:ESHttpPut"
],
"Resource": [
"arn:aws:es:<<REGION>>:<<ACCOUNTID>>:domain/<<DOMAIN_NAME>>",
"arn:aws:es:<<REGION>>:<<ACCOUNTID>>:domain/<<DOMAIN_NAME>>/*"
]
},
{
"Sid": "",
"Effect": "Allow",
"Action": [
"es:ESHttpGet"
],
"Resource": [
"arn:aws:es:<<REGION>>:<<ACCOUNTID>>:domain/<<DOMAIN_NAME>>/_all/_settings",
"arn:aws:es:<<REGION>>:<<ACCOUNTID>>:domain/<<DOMAIN_NAME>>/_cluster/stats",
"arn:aws:es:<<REGION>>:<<ACCOUNTID>>:domain/<<DOMAIN_NAME>>/<<INDEX>>*/_mapping/<<TYPE>>",
"arn:aws:es:<<REGION>>:<<ACCOUNTID>>:domain/<<DOMAIN_NAME>>/_nodes",
"arn:aws:es:<<REGION>>:<<ACCOUNTID>>:domain/<<DOMAIN_NAME>>/_nodes/stats",
"arn:aws:es:<<REGION>>:<<ACCOUNTID>>:domain/<<DOMAIN_NAME>>/_nodes/*/stats",
"arn:aws:es:<<REGION>>:<<ACCOUNTID>>:domain/<<DOMAIN_NAME>>/_stats",
"arn:aws:es:<<REGION>>:<<ACCOUNTID>>:domain/<<DOMAIN_NAME>>/<<INDEX>>*/_stats"
]
}
I think you could more easily condense the above two policy statements into the following:
{
"Sid": "",
"Effect": "Allow",
"Action": [
"es:DescribeElasticsearchDomain",
"es:DescribeElasticsearchDomains",
"es:DescribeElasticsearchDomainConfig",
"es:ESHttpPost",
"es:ESHttpGet",
"es:ESHttpPut"
],
"Resource": [
"arn:aws:es:<<REGION>>:<<ACCOUNTID>>:domain/<<DOMAIN_NAME>>",
"arn:aws:es:<<REGION>>:<<ACCOUNTID>>:domain/<<DOMAIN_NAME>>/*"
]
}
I managed to piece the above together from the following sources:
https://aws.amazon.com/blogs/security/how-to-control-access-to-your-amazon-elasticsearch-service-domain/
How to access Kibana from Amazon elasticsearch service?
https://forums.aws.amazon.com/thread.jspa?threadID=217149
You need to go to the access policy of Lambda and provide the AWS ARN to connect
http://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/es-aws-integrations.html#es-aws-integrations-s3-lambda-es-authorizations
AWS Lambda runs on public EC2 instances. So simply adding a whitelist of IP addresses to the Elasticsearch access policy will not work. One way to do this will be to give the Lambda execution role appropriate permissions to the Elasticsearch domain. Make sure that the Lambda Execution role has permissions to the ES domain and the ES domain access policy has a statement which allows this Lambda Role ARN to do the appropriate actions. Once this is done all you would have to do is sign your request via SigV4 while accessing the ES endpoint Hope that helps!