This is driving me totally crazy
Im trying to call a api-gateway rest service from an angular app that i have restricted from API-GATEWAY with IAM access. So i need to call with IAM authentication. Im using temporary IAM credentials that i have already obtained
My call to the service fails saying there is no Access-Control-Allow-Origin header. When i try and call my service from postman i dont see the required header on the response. On my postman call I get a 403 status back since my authentication actually failed, but i still expected the header. If i remove the IAM authentication on the method it works, i get back the response string and the header im looking for.
What am i missing here? Surely even if my authentication failed i must still get back that header so that i can actually see the message that says your authentication failed.
Any help will be much appreciated
Thanks
From this link on the AWS forum it appears that there is an open issue related to exactly what im experiencing here... Not sure if what im looking for is currently possible
Did you allow your Cognito role to access to the API endpoints?
You have to use a policy similar to this one for the Cognito role:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"execute-api:invoke"
],
"Resource": [
"arn:aws:execute-api:<API_REGION>:<ACCOUNT_ID>:<API_ID>/*"
]
}
}
}
If you use IAM authentication for your endpoint and if you do not allow the Cognito role to access to your API, API Gateway will generate a 403 response. This response is not defined by your API definition because the request is refused by API Gateway before it could get to the point of been processed by your API. That is why your CORS configuration will not apply.
Please ensure you do not have AWS_IAM authentication enabled on your OPTIONS method, otherwise the browser will not be able to make the pre-flight request and your request will fail. You should still have AWS_IAM enabled on your other methods.
Related
I am trying to securely connect to my ES service from the serverless functions in Nextjs. The serverless functions have no fixed IP address or IP address range so I can't secure it that way. I have tried creating an IAM User and using the access key ID and secret access key to create the basic authorization header like this.
const auth = btoa(`${accessKey}:${accessSecret}`);
Then I used that as the Authorization header like this.
`Authorization: Basic ${auth}`
The request comes back with the following error.
{
"message": "Authorization header requires 'Credential' parameter. Authorization header requires 'Signature' parameter. Authorization header requires 'SignedHeaders' parameter. Authorization header requires existence of either a 'X-Amz-Date' or a 'Date' header. Authorization=Basic XXXXXXXXXXXXXXXXXXXXXXXXXX"
}
I have given the IAM user permission through the access policy.
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::XXXXXXXXX:user/elasticsearch"
},
"Action": "es:*",
"Resource": "arn:aws:es:us-east-2:XXXXXXXX:domain/team-up/*"
}
]
}
Is this the right way to secure the connection to the database? I have tried setting up a password on the database using the POST /_security/user/_password endpoint, but that fails too.
Amazon claims that basic authorization is supported.
https://aws.amazon.com/elasticsearch-service/faqs/
I didn't create a master user during the wizard setup of the aws elasticsearch domain. Once the domain is created there is no way to create a master user it seems. I deleted the elasticseach domain and recreated it with a master user.
I followed this guide.
https://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/fgac.html#fgac-walkthrough-basic
Hi I am trying to implement Custom authorization using api gateway and lambda. My current understanding is as follows. I have created simple GET method and deployed to Dev Enviroment. Create lambda authorize to return the IAM policy. I used python blue print api-gateway-authorizer-python. Below is the format of response we should get.
{
"Version": "2012-10-17",
"Statement": [
{
"Action": "execute-api:Invoke",
"Effect": "Deny",
"Resource": "arn:aws:execute-api:us-east-1:{ACCOUNTID}:{APIID}/ESTestInvoke-stage/GET/"
}
]
}
In the above IAM policy, Resource is ARN of my Api Dev stage. What is Action indicates? Also to test this now, How can I get token? I want to test it from postman? I am just confused here. I have my AWS account and authorization is nothing but my current account has access to this Dev stage? How internally it works? To store all the permissions do we need to maintain any other DB? Can someone help me to understand this? Any help would be appreciated. Thanks
To get a token you need an identity provider. Amazon Cognito is one of those (Google, Facebook works as well). To understand that policy you have to understand the chain of commands.
Suppose a client calls an API endpoint (GET /orders), this will trigger a service Lambda so the token can be verified. If the verification is successful, another Lambda (GetOrder a business Lambda this time) will be invoked by Api Gateway.
If your service Lambda (Lambda authorizer) will return a policy like this:
{
"principalId": "apigateway.amazonaws.com",
"policyDocument": {
"Version": "2012-10-17",
"Statement": [{
"Action": "execute-api:Invoke",
"Effect": "Allow",
"Resource": "arn:aws:execute-api:{REGION}:{ACCOUNT_ID}:{API_ID}/Prod/GET/"
}]
}
}
the API Gateway service (i.e. the principalId equals to apigateway.amazonaws.com) is allowed (i.e. Effect equals to Allow) to invoke (i.e. Action equals to execute-api:Invoke) the given API resource (e.g. Resource equals to arn:aws:execute-api:{REGION}:{ACCOUNT_ID}:{API_ID}/Prod/GET/).
In your case the ARN that you return is related to the tester of API Gateway, but it should point to your real function.
This article may help.
Ok, what are custom authorisers for API gateway: custom authorisers let you define your own authentication & authorisation logic.
How do you get the token: Thats part of your authentication and authorisation logic, If you are deploying your services on AWS, you can use AWS cognito. API gateway also supports cognito authorization.
https://docs.aws.amazon.com/apigateway/latest/developerguide/apigateway-integrate-with-cognito.html
Using postman to test? it's handy to use postman. I use it.
How does the permission internally work: You can use a token to authenticate a user. (If you are using a JWT token, you can also verify the user's claims).
Do you need an internal db? this is entirely depends on your use case. if your use case is simple as all users treated equal, you might not need a db. lets say some users can access some additional features, you may still not need a db (you can use claims). but if your application becomes complicated and you have to manage different access permissions, users, groups, etc, you may surely need a db.
I have an API Gateway api setup that I want to limit access to. I have a subdomain setup in AWS Route 53 that points to a CloudFront distribution where my app lives. This app makes a POST request to the API.
I have looked into adding a resource policy for my api based on the example 'AWS API Whitelist' but I can't seem to get the syntax correct, I constantly get errors.
I also tried creating an IAM user and locking down the API with AWS_IAM auth but then I need to create a signed request which seems like a lot of work that should be a lot easier via resource policies?
This is an example of the resource policy I tried to attach to my API:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::cloudfront:user/CloudFront Origin Access Identity {{CloudFrontID}}"
},
"Action": "execute-api:Invoke",
"Resource": [
"execute-api:/*/*/*"
]
}
]
}
This returns the following error:
Invalid policy document. Please check the policy syntax and ensure that Principals are valid.
"AWS": "arn:aws:iam::cloudfront:user/CloudFront Origin Access Identity {{CloudFrontID}}"
The problem with this concept is that this is a public HTTP request. Unless it's a signed request, AWS will not know about any IAM or ARN resources, it just knows it has a standard HTTP request. If you make the request with a curl -v command you will see the request parameters look something like this:
GET
/test/mappedcokerheaders
HTTP/2
Host: APIID.execute-api.REGION.amazonaws.com
User-Agent: curl/7.61.1
Accept: */*
It's possible you could filter the user Agent as I do see that condition defined here.
I would check all of the values that are coming in the request from cloudfront vs the request from your curl directly to the API by trapping the api gw request id in the response headers, and looking for those in your API Gateway Access Logs. You'll have to enable Access Logs though, and define what parameters you want logged, which you can see how to do here.
The problem is that OAI cannot be used in CustomOrigin. If you are not forwarding User-Agent to the API Gateway CustomOrigin, then the simplest approach for you is to add a resource policy in API Gateway which only allows aws:UserAgent: "Amazon CloudFront".
Be careful: User-Agent can very easily be spoofed. This approach is designed to only prevent "normal access" like a random bot on the web is trying to scrape your site.
The User-Agent header is guaranteed to be Amazon CloudFront. See the quote from Request and Response Behavior for Custom Origins.
Behavior If You Don't Configure CloudFront to Cache Based on Header Values: CloudFront replaces the value of this header field with Amazon CloudFront. If you want CloudFront to cache your content based on the device the user is using, see Configuring Caching Based on the Device Type.
Here is how the full resource policy looks like:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": "*",
"Action": "execute-api:Invoke",
"Resource": "arn:aws:execute-api:us-west-2:123456789012:abcdefghij/*/*/*",
"Condition": {
"StringEquals": {
"aws:UserAgent": "Amazon CloudFront"
}
}
}
]
}
Here is how to configure it in serverless.yml:
provider:
resourcePolicy:
- Effect: Allow
Principal: "*"
Action: execute-api:Invoke
Resource:
- execute-api:/*/*/*
Condition:
StringEquals:
aws:UserAgent: "Amazon CloudFront"
I have a subdomain setup in AWS Route 53 that points to a CloudFront
distribution where my app lives. This app makes a POST request to the
API.
What I understand is that you have a public service that can be called from the web browser ( https://your-service.com )
You want the service to respond only when the client's browser is at https://your-site.com. The service will not respond when the browser for example is on https://another-site.com
If that is the case,
you will need to read more about CORS
This will not prevent a random guy / web client to go to and call your service directly to https://your-service.com however. To protect the service from that, you need proper authentication system
For a Kinesis stream, I created a proxy API using AWS API Gateway. I added a custom authorizer using python Lambda for the proxy.
After publish of lambda function and deploy of API, I was able to successfully test the API using Gateway Test functionality. I could see the logs in cloudwatch which had detailed prints from custom auth lambda function. After successful authentication, API Gateway pushed the record to my Kinesis stream
However when I call the same API from Chrome Postman client, I get 500 Internal Server Error and response headers includes X-Cache → Error from cloudfront, x-amzn-ErrorType → AuthorizerConfigurationException
Lambda auth function returns the policy which allows execute request for my API. Policy Document returned is:
{
"policyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Action": "execute-api:Invoke",
"Resource": [
"arn:aws:execute-api:us-east-1:1234567:myapiId/staging/POST/*"
],
"Effect": "Allow"
}
]
},
"principalId": "Foo"
}
Why does the request fail from Chrome or curl but the same API test works fine from API Gateway?
AuthorizerConfigurationException is usually an indication that API Gateway failed to call your authorizer due a permissions error.
Please either make sure you've properly configured your function to be invoked by API Gateway. An easy to reset this is by removing and re-adding the function to your authorizer. The console will then prompt you to add the necessary permissions.
I was facing the same error, in my case a nodejs function, I was adding one context key as array.
{
policyDocument: {
Version: '2012-10-17',
Statement: [{
Action: 'execute-api:Invoke',
Effect: effect,
Resource: `${arn.split('/').slice(0, 2).join('/')}/*`,
}],
},
context: {
roles: ['admin']
}
As doc says:
You can access the stringKey, numberKey, or booleanKey value (for example, "value", "1", or "true") of the context map in a mapping template by calling $context.authorizer.stringKey, $context.authorizer.numberKey, or $context.authorizer.booleanKey, respectively. The returned values are all stringified. Notice that you cannot set a JSON object or array as a valid value of any key in the context map.
Remove the role key and it's working.
Figured out what was causing the issue. From python lambda function, I was returning a json string instance. Instead it should be json object.
Its strange that the same lambda function did not error when I tested the API from API Gateway "test" feature. But when the API was called from internet (curl or chrome) it failed.
#return policy_string ... this is incorrect.
return json.loads(policy_string)
In my case i was not returning a properly formatted IAM policy document. My Authorizer function was doing wrong assumptions on how to get some parameters from request, and default result was not proper policy (this was my specific case). I managed to debug it using CloudWatch log service, with traditional logging instructions coming from my function code.
I have been struggling to figure out how to communicate with the Amazon ES service from my EC2 instances.
The documentation clearly states that the Amazon ES service supports IAM User & Role based access policies. http://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/es-createupdatedomains.html#es-createdomain-configure-access-policies
However, when I have this access policy for my ES domain:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::123456789:role/my-ec2-role"
},
"Action": "es:*",
"Resource": "arn:aws:es:us-west-2:123456789:domain/myDomain/*"
}
]
}
I can't log into an ec2 instance and run a curl to hit my elasticsearch cluster.
Trying to do a simple curl of the _search API:
curl "http://search-myDomain.es.amazonaws.com/_search"
Produces an authentication error response:
{"Message":"User: anonymous is not authorized to perform: es:ESHttpGet on resource: arn:aws:es:us-west-2:123456789:domain/myDomain/_search"}
Just to be extra safe I put the AmazonESFullAccess Policy on my IAM Role, still doesn't work.
I must be missing something, because being able to programmatically interact with Elasticsearch from ec2 instances that use an IAM Role is essential to getting anything accomplished with the Amazon ES Service.
I also see this contradictory statement in the docs.
IAM-based Policy Example You create IAM-based access policies by
using the AWS IAM console rather than the Amazon ES console. For
information about creating IAM-based access policies, see the IAM
documentation.
That link to IAM documentation, is to the home page of IAM and contains exactly zero information about how to do it. Anyone got a solution for me?
When using IAM service with AWS, you must sign your requests. curl doesn't support signed requests (which consists of hashing the request and adding a parameter to the header of the request). You can use one of their SDK's that has the signing algorithm built in, and then submit that request.
See:
http://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/what-is-amazon-elasticsearch-service.html#signing-requests
You can find the SDKs for popular languages here:
http://aws.amazon.com/tools/
First, you said you can't login to an EC2 instance to curl the ES instance? You can't login? Or you can't curl it from EC2?
I have my Elasticsearch (Service) instance open to the world (with nothing on it) and am able to curl it just fine, without signing. I changed the access policy to test, but unfortunately it takes forever to come back up after changing it...
My policy looks like this:
{ "Version": "2012-10-17", "Statement": [
{
"Sid": "",
"Effect": "Allow",
"Principal": "*",
"Action": "es:*",
"Resource": "arn:aws:es:us-east-1:843348267853:domain/myDomain/*"
},
{
"Sid": "",
"Effect": "Allow",
"Principal": "*",
"Action": "es:*",
"Resource": "arn:aws:es:us-east-1:843348267853:domain/myDomain"
}
]
}
I realize this isn't exactly what you want, but start off with this (open to the world), curl from outside AWS and test it. Then restrict it, that way you're able to isolate the issues.
Also, I think you have an issue with the "Principal" in your access policy. You have your EC2 Role. I understand why you're doing that, but I think the Principal requires a USER, not a role.
See below:
http://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/es-createupdatedomains.html#es-createdomain-configure-access-policies
Principal
Specifies the AWS account or IAM user that is allowed or denied access
to a resource. Specifying a wildcard (*) enables anonymous access to
the domain, which is not recommended. If you do enable anonymous
access, we strongly recommend that you add an IP-based condition to
restrict which IP addresses can submit requests to the Amazon ES
domain.
EDIT 1
To be clear, you added the AmazonESFullAccess policy to the my-ec2-role? If you're going to use IAM access policies, I don't think you can have a resource based policy attached to it (which is what you're doing).
http://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_compare-resource-policies.html
For some AWS services, you can grant cross-account access to your
resources. To do this, you attach a policy directly to the resource
that you want to share, instead of using a role as a proxy. The
resource that you want to share must support resource-based policies.
Unlike a user-based policy, a resource-based policy specifies who (in
the form of a list of AWS account ID numbers) can access that
resource.
Possibly try removing the access policy altogether?
Why you don't create a proxy with elastic ip and allow your proxy to access your ES?
Basically exists three forms that you can limit access in your ES:
Allow everyone
White IP list
Signing the access key and secret key provided by AWS.
I'm using two forms, in my php apps I prefer to use proxy behind the connection to ES and in my nodejs app I prefer to sign my requests using the http-aws-es node module.
It's useful to create a proxy environment because my users needs to access the kibana interface to see some reports and it's possible because they have configured the proxy in their browsers =)
I must recommend to you close the access to your ES indexes, because it's pretty easy to delete them, curl -XDELETE https://your_es_address/index anyone can do it but you can say: "how the others users will get my ES address?" and I will answer you: "Security based in dimness isn't a real security"
My security access policy is basically something like it:
http://pastebin.com/EUKT1ekX
I encountered this issue recently and the root problem is that none of the Amazon SDKs yet support calling Elasticsearch operations like search, put, etc.
The only workaround at the moment is to execute requests directly against the endpoint using signed requests:
http://docs.aws.amazon.com/general/latest/gr/sigv4-signed-request-examples.html
The example here is for calling EC2, but it can be modified to instead call against Elasticsearch. Just modify the "service" value to "es". From there, you have to fill in values for
the endpoint (which is the full URL of your cluster including operation without request parameters)
the host (the part between https:// and your canonical URI like /_status
the canonical uri which is the URI after the first / inclusive (like /_status) but without the query string
the request parameters (everything after ? inclusive)
Note that I've only managed to get this working so far using AWS credentials as the assumption is that you pass in an access key and secret key to the various signing calls (access_key and secret_key in the example). It should be doable using IAM roles but you'll have to call into the security token service first to get temporary credentials that can be used to sign the request. Until you do that, be sure to edit your access policy on the Elasticsearch cluster to allow user creds (user/
you need to sign your request and unfortunately, it is no longer supported by the official elasticsearch library. Check this Github issue (https://github.com/elastic/elasticsearch-js/issues/1182#issuecomment-630641702)
They want to enforce their own cloud solution