I am trying to make one of my AWS Rest API private. But after doing steps that AWS docs suggest, It's changing URL of that API.
Here's the steps that I tried:
Create VPC endpoint for API execution.
Change APIs endpoint type to Private
Add VPC endpoint id in VPC endpoint IDs.
Add resource policy to allow API execution from VPC.
Here's the Resource policy
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": "*",
"Action": "execute-api:Invoke",
"Resource": "my api arn"
},
{
"Effect": "Deny",
"Principal": "*",
"Action": "execute-api:Invoke",
"Resource": "My API ARN",
"Condition": {
"StringNotEquals": {
"aws:SourceVpce": "VPC endpoint id",
"aws:SourceVpc": "VPC id"
}
}
}
]
}```
These steps is making my API private/Invokable from VPC only, but I can't invoke that with same URL.
I must have to add VPC endpoint id in URL to call the API.
Old URL that I am using:
https://{restapi-id}.execute-api.{region}.amazonaws.com/{stage}
Here's how new URL looks like:
https://{rest-api-id}-{vpce-id}.execute-api.{region}.amazonaws.com/{stage}
Is there any way to make API private without changing URL?
Related
I followed a tutorial to create a new domain in the elastic search service. I created a policy as follows,
{
"Version": "2012-10-17",
"Statement": [
{
"Action": [
"es:ESHttpDelete","es:ESHttpGet","es:ESHttpHead",
"es:ESHttpPost","es:ESHttpPut"
],
"Effect": "Allow",
"Resource": "*"
}
]
}
then i created a role for a lambda to access elastic service. later i plan to call elastic search from lambda. here is my role ARN
arn:aws:iam::566879691663:role/myRole
and then for elastic search domain , I assigned "public access" for network configuration. and for access policy, I selected "custom access policy" and added my above role. the access policy json looks like below
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": [
"arn:aws:iam::566879691663:role/myRole*"
]
},
"Action": [
"es:*"
],
"Resource": "arn:aws:es:us-east-1:566879691663:domain/mydomain/*"
}
]
}
once the domain is up and running, when I click on the kibana url generated, I get the following json response in the browser. how can i access this via browser ?
{"message : " user : anonymous is not authorized to perform this action..."}
also, to be able to access/upload programatically, using AWS4AUTH, which requires aws access and secret key, how to I generate those? do i need to create a user and assign the above policy to the user?
I have a IP white-listed Resource Policy on a AWS API Gateway like below. Now I have a Lambda that needs to call this API but is currently correctly disallowed.
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": "*",
"Action": "execute-api:Invoke",
"Resource": "..."
},
{
"Effect": "Deny",
"Principal": "*",
"Action": "execute-api:Invoke",
"Resource": "...",
"Condition": {
"NotIpAddress": {
"aws:SourceIp": [
"...",
"...",
]
}
}
}
]
}
The Lambda function is configured to access my VPC:
VpcConfig:
SubnetIds:
- "subA"
- "subB"
SecurityGroupIds:
- "sg1"
My question is two fold:
How should I change the API Resource Policy to allow access from this function? Replacing the white-listing policy with snippet below does not work. ("User: anonymous is not authorized to perform: execute-api:Invoke on resource: arn:aws:execute-api:... with an explicit deny")
{
"Effect": "Deny",
"Principal": "*",
"Action": "execute-api:Invoke",
"Resource": [
"arn:aws:execute-api:..."
],
"Condition" : {
"StringNotEquals": {
"aws:SourceVpc": "{{ID of VPC attached to Lambda}}"
}
}
}
Adding snippet above to the existing policy also breaks the existing white-listing policy (all access is denied with identical message as above). How do multiple policies next to each other compose? Do they? Is it an and or an or composition?
I don't think SourceVpc in the API Resource Policy is going to work, unless your API Gateway is a private VPC-only API.
Since your Lambda function is already running in a VPC, you need to simply add the IP addresses of your VPC NAT Gateways to the list of your API Gateway allowed IPs.
I have setup the follow resource policy in api gateway to restrict access to a source IP (x is just a placeholder). When I manually hit the api endpoint from postman the policy correctly restricts access only to the cidr range I specified in the resource policy below.
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": "*",
"Action": "execute-api:Invoke",
"Resource": "arn:aws:execute-api:us-east-1:x:x/*/*/*”
},
{
"Effect": "Deny",
"Principal": "*",
"Action": "execute-api:Invoke",
"Resource": "arn:aws:execute-api:us-east-1:x:x/*/*/*”,
"Condition": {
"NotIpAddress": {
"aws:SourceIp": [
“x.x.x.x/32"
]
},
"StringNotEquals": {
"aws:sourceVpc": "vpc-x”
}
}
}
]
}
However, I have a lambda function which also calls the same https api gateway endpoint. This function essentially just passes test data into my api at hourly intervals. But, the lambda function is unable to hit the endpoint and gets a 403 forbidden error. I tried adding the sourceVpc to the resource policy, but this did not seem to work. I also tried adding the vpc cidr range too, but again this did not work.
Do you know what cidr I should add to the resource policy to allow my lambda to call my api endpoint too?
I added to the resource policy "aws:SourceIp" the NAT gateway ip of the subnets associated with my lambda function. This allowed my lambda function to invoke the API Gateway successfully.
I have a API on AWS API gateway which calls LAMBDA function and I want to add IP whitelisting for that API so the allowed IPs only can have access to that API. How can I achieve this?
There is a really extended blog post about this which you can find here.
Bottom line, it comes to this:
Per method, select IAM Authorization method.
Create a new IAM policy that looks like the one below and attach it to the API Method.
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"execute-api:Invoke"
],
"Condition": {
"IpAddress": {
"aws:SourceIp": "xxx.xx.xx.xx/32"
}
},
"Resource": "arn:aws:execute-api:*:*:*"
}
]
}
You can add multiple SourceIps if needed.
I'm trying to use Elasticsearch for data storage for a Lambda function connected to Alexa Skills Kit. The Lambda works alright without Elasticsearch but ES provides much-needed fuzzy matching.
The only way I've been able to access it from Lambda is by enabling Elasticsearch global access but that's a really bad idea. I've also been able to access from my computer via open access policy or IP address policy. Is there a way to do read-only access via Lambda and read-write via IP?
On IAM I granted my Lambda role AmazonESReadOnlyAccess. On the ES side I tried this but it only worked for IP address:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": [
"arn:aws:iam::NUMBER:root",
"arn:aws:iam::NUMBER:role/lambda_basic_execution"
]
},
"Action": "es:*",
"Resource": "arn:aws:es:us-east-1:NUMBER:domain/NAME/*"
},
{
"Sid": "",
"Effect": "Allow",
"Principal": {
"AWS": "*"
},
"Action": "es:*",
"Resource": "arn:aws:es:us-east-1:NUMBER:domain/NAME/*",
"Condition": {
"IpAddress": {
"aws:SourceIp": "MY IP"
}
}
}
]
}
This forum post asks the same question but went unanswered.
The only way I know of to do this is to use a resource-based policy or an IAM-based policy on your ES domain. This would restrict access to a particular IAM user or role. However, to make this work you also need to sign your requests to ES using SigV4.
There are libraries that will do this signing for you, for example this one extends the popular Python requests library to sign ElasticSearch requests via SigV4. I believe similar libraries exist for other languages.
Now it's possible from your code with elasticsearch.js. Before you try it, you must install http-aws-es module.
const AWS = require('aws-sdk');
const httpAwsEs = require('http-aws-es');
const elasticsearch = require('elasticsearch');
const client = new elasticsearch.Client({
host: 'YOUR_ES_HOST',
connectionClass: httpAwsEs,
amazonES: {
region: 'YOUR_ES_REGION',
credentials: new AWS.EnvironmentCredentials('AWS')
}
});
// client.search({...})
Of course, before using it, configure access to elasticsearch domain:
For external (outside AWS) access to your Elasticsearch cluster, you want to create the cluster with an IP-based access policy. Something like the below:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "",
"Effect": "Allow",
"Principal": {
"AWS": "*"
},
"Action": "es:*",
"Condition": {
"IpAddress": {
"aws:SourceIp": [
"<<IP/CIDR>>"
]
}
},
"Resource": "arn:aws:es:<<REGION>>:<<ACCOUNTID>>:domain/<<DOMAIN_NAME>>/*"
}
]
}
For your Lambda function, create the role that the Lambda function will assume with the below policy snippet.
{
"Sid": "",
"Effect": "Allow",
"Action": [
"es:DescribeElasticsearchDomain",
"es:DescribeElasticsearchDomains",
"es:DescribeElasticsearchDomainConfig",
"es:ESHttpPost",
"es:ESHttpPut"
],
"Resource": [
"arn:aws:es:<<REGION>>:<<ACCOUNTID>>:domain/<<DOMAIN_NAME>>",
"arn:aws:es:<<REGION>>:<<ACCOUNTID>>:domain/<<DOMAIN_NAME>>/*"
]
},
{
"Sid": "",
"Effect": "Allow",
"Action": [
"es:ESHttpGet"
],
"Resource": [
"arn:aws:es:<<REGION>>:<<ACCOUNTID>>:domain/<<DOMAIN_NAME>>/_all/_settings",
"arn:aws:es:<<REGION>>:<<ACCOUNTID>>:domain/<<DOMAIN_NAME>>/_cluster/stats",
"arn:aws:es:<<REGION>>:<<ACCOUNTID>>:domain/<<DOMAIN_NAME>>/<<INDEX>>*/_mapping/<<TYPE>>",
"arn:aws:es:<<REGION>>:<<ACCOUNTID>>:domain/<<DOMAIN_NAME>>/_nodes",
"arn:aws:es:<<REGION>>:<<ACCOUNTID>>:domain/<<DOMAIN_NAME>>/_nodes/stats",
"arn:aws:es:<<REGION>>:<<ACCOUNTID>>:domain/<<DOMAIN_NAME>>/_nodes/*/stats",
"arn:aws:es:<<REGION>>:<<ACCOUNTID>>:domain/<<DOMAIN_NAME>>/_stats",
"arn:aws:es:<<REGION>>:<<ACCOUNTID>>:domain/<<DOMAIN_NAME>>/<<INDEX>>*/_stats"
]
}
I think you could more easily condense the above two policy statements into the following:
{
"Sid": "",
"Effect": "Allow",
"Action": [
"es:DescribeElasticsearchDomain",
"es:DescribeElasticsearchDomains",
"es:DescribeElasticsearchDomainConfig",
"es:ESHttpPost",
"es:ESHttpGet",
"es:ESHttpPut"
],
"Resource": [
"arn:aws:es:<<REGION>>:<<ACCOUNTID>>:domain/<<DOMAIN_NAME>>",
"arn:aws:es:<<REGION>>:<<ACCOUNTID>>:domain/<<DOMAIN_NAME>>/*"
]
}
I managed to piece the above together from the following sources:
https://aws.amazon.com/blogs/security/how-to-control-access-to-your-amazon-elasticsearch-service-domain/
How to access Kibana from Amazon elasticsearch service?
https://forums.aws.amazon.com/thread.jspa?threadID=217149
You need to go to the access policy of Lambda and provide the AWS ARN to connect
http://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/es-aws-integrations.html#es-aws-integrations-s3-lambda-es-authorizations
AWS Lambda runs on public EC2 instances. So simply adding a whitelist of IP addresses to the Elasticsearch access policy will not work. One way to do this will be to give the Lambda execution role appropriate permissions to the Elasticsearch domain. Make sure that the Lambda Execution role has permissions to the ES domain and the ES domain access policy has a statement which allows this Lambda Role ARN to do the appropriate actions. Once this is done all you would have to do is sign your request via SigV4 while accessing the ES endpoint Hope that helps!