I've created SNS topic
I've created API Gateway endpoint that invokes Lambda function
I've created topic HTTPS subscription that points to API Gateway endpoint
Problem: everything works fine when AUTH=none, but when i enabled AUTH=AWS_IAM, neither subscription nor messages are delivered to my lambda. They also wont show up in Lambda OR Gateway cloudwatch logs as it's usually the case with authentication errors.
Questions:
What's the identity delivered by HTTPS endpoint to AWS_IAM so it doesn't allows it ( my first thought was to relay SNS posters token but it doesn't seem be the case )
I couldn't find any way to associate HTTPS endpoint with any identity, is there a way?
There are lots of information about delivering SNS to SQS or Gateway to SNS, but couldn't find any information about achieving what i try to do.
Is there any method to debug AWS_IAM authentication problems? Documentation i've seen advices to "check priviliges" which is something i've been doing for many hours but i have no more ideas.
I'd be glad to hear any ideas from you, thanks.
As you may have seen in the docs, SNS can only do Basic/Digest Auth http://docs.aws.amazon.com/sns/latest/dg/SendMessageToHttp.html
There is a section in the docs about verifying the validity of the message but that is code you'd have to write yourself or lift from one of the SNS SDKs on the backend. There really isn't any way to get SNS to sign the request with AWS SigV4, unfortunately.
Why don't you let the Lambda function subscribe directly to the SNS topic (without going through API Gateway)?
That should be straightforward: https://docs.aws.amazon.com/sns/latest/dg/sns-lambda.html
Here is the complete link which will help you in solving your authentication problem. https://aws.amazon.com/premiumsupport/knowledge-center/iam-authentication-api-gateway/
If it's an "Check privileges" issue, then your IAM user doesn't have any sufficient access to the resources to make any changes.
Related
I need to create an API gateway to consume messages from an SNS. I see that there are some questions on the same, like this, which hasn't been answered directly.
The main issue I face is : How to make the API gateway subscribe to the SNS?
SNS supports HTTP/HTTPS endpoint subscriptions. So I don't see a reason why you could not subscribe API gateway https endpoint to SNS this way.
As Marcin has mentioned, it can be done. Mentioning the steps I used:
Create SNS, and a lambda
Whatever the language is, make sure to print the event in the log. (For JavaScript, use console.log(event), and for Python use print(event) etc)
Create an API Gateway (REST API), select a "New API", and create it.
In the API create a POST method, select lambda, and make sure to use it as a proxy.
Copy the trigger HTTPS link
Add a HTTPS subscription in the SNS, and use the trigger link here.
The confirmation would be pending now.
Go to the lambda, inside it's monitoring section, go to cloud watch logs.
Inside the logs, you will find the event object printed. Inside it, look for SubscribeURL, along which the URL would be mentioned.
Copy this URL, and go back to the SNS. Select your subscription and use the "Confirm Subscription" option, and paste this link there.
The status should come as "Confirmed" now.
I'm attempting to subscribe an SNS topic to a HTTPS endpoint I own. I'm reading the docs on how to process incoming messages from SNS and how the subscription confirmation needs to be done. I see two methods of confirmation:
https://docs.aws.amazon.com/sns/latest/dg/sns-http-https-endpoint-as-subscriber.html - Using the subscribeURL. We can perform a HTTP get request on the "SubscribeURL" attribute value and that would confirm the subscription.
Calling the ConfirmSubscription API - We pass the SNS Topic ARN and the token received when SNS sends a confirm subscription message to the SNS endpoint.
I'm trying to understand what's the difference between the two methods. The most obvious one to me was this - The choice of using the API will require AWS credentials since the request needs to be signed. But seems like the same call will succeed with just the HTTP GET request?
What's the best practice out there (if any) and/or which method is the one being followed by other folks using AWS/SNS?
There isn't a difference -- these two alternatives are in fact the same thing.
The SubscribeURL attribute is a pre-constructed (by the service) link to the ConfirmSubscription action on the SNS API endpoint.
The API accepts GET or POST. No signature is required in this case.
This call requires an AWS signature only when the AuthenticateOnUnsubscribe flag is set to "true".
https://docs.aws.amazon.com/sns/latest/api/API_ConfirmSubscription.html
Before SNS will talk to an endpoint, you need to prove that you control that endpoint. So your options are to write some code that can do it automatically (most of the SDKs support this) or to capture the token, and by returning it via the API call prove that you control it.
This is a one-time procedure, so you do not need to deploy any AWS credentials to your API endpoint - you can do it from a different system.
We generally build the confirmation handler into the application.
There is instructions on how to integrate between API Gateway and SNS here but this is a bit of a toy example.
I want to know how I can subscribe to a topic via API Gateway -> SNS Integration.
And to this purpose what I'm looking is general documentation on doing this as I assume it is possible. If you can ListTopics (which the example indicates) surely you can do other things...
Edit: So I now know that when I do an integration into SNS it sends the following to SNS: https://sns.eu-west-1.amazonaws.com/?Action=CreateTopic
So that's a good start as that is how to call SNS to create a topic.
So now the question is how to I parameterise this?
I have also figured out that I can do from post to the SNS endpoint and thus include my parameters. However, the I get a signature not present error...
What would be the most efficient way to EMIT a POST requests (a webhook precisely) from AWS on a daily basis ?
Right away, I started by simply creating a cloudWatch rule with an event schedule CRON that would trigger an SNS publication "every day at 18h", then I created an SNS topic "AlertMyWebhook" with all POST endpoints as subscribers of the topic.
But.. SNS requires me to CONFIRM subscription of each endpoints... which I can't by definition, since the https endpoint is not mine (webhook = HOOK into someone ELSE'S WEBapp).
So now I am starting to think crazy stuff like having a dynamoDB table to store all webhooks endpoint URL, coupled with a lambda function to read the table, take each https endpoints, and send a POST request to them...
Frankly speaking: that doesn't make any sense to me.
Is there any way to avoid SNS confirmation ?
If not, how on earth would you do to "trigger a POST every day at 18h" without creating a monolithic-like architecture ?
AWS SNS and Lambda functions are integrated with each other so you can subscribe a Lambda function to your topic. When a message is posted to that topic the subscribed Lambda function is invoked with the Payload(published message).
Using this Payload as input for the Lambda function trigger the POST requests for the endpoints. A good way to do is make all the HTTPS POST endpoints as Environment variables in Lambda. So there is no code change in the Lambda function whenever a new POST endpoint need to be added as the Subscription endpoints. For more of How to integrate AWS SNS and Lambda look here.
Invoking Lambda functions using Amazon SNS notifications
The sample NodeJS code to invoke the POST request
AWS Lambda HTTP POST Request
I would like to send SNS notifications to a Lambda function from one account to another exposing the lambda through and API gateway. But I'm having an Issue. I first tried with the API Gateway in the same account of the SNS topic and everything worked like a charm but when i try with a topic from other account all I got is a Bad Request Error. Do I need to configure any permission?
You need to enable permissions for the TOPIC in another account.
You can find an example in the AWS documentation, section
Allowing Any AWS Resource to Publish to a Topic