I'm attempting to subscribe an SNS topic to a HTTPS endpoint I own. I'm reading the docs on how to process incoming messages from SNS and how the subscription confirmation needs to be done. I see two methods of confirmation:
https://docs.aws.amazon.com/sns/latest/dg/sns-http-https-endpoint-as-subscriber.html - Using the subscribeURL. We can perform a HTTP get request on the "SubscribeURL" attribute value and that would confirm the subscription.
Calling the ConfirmSubscription API - We pass the SNS Topic ARN and the token received when SNS sends a confirm subscription message to the SNS endpoint.
I'm trying to understand what's the difference between the two methods. The most obvious one to me was this - The choice of using the API will require AWS credentials since the request needs to be signed. But seems like the same call will succeed with just the HTTP GET request?
What's the best practice out there (if any) and/or which method is the one being followed by other folks using AWS/SNS?
There isn't a difference -- these two alternatives are in fact the same thing.
The SubscribeURL attribute is a pre-constructed (by the service) link to the ConfirmSubscription action on the SNS API endpoint.
The API accepts GET or POST. No signature is required in this case.
This call requires an AWS signature only when the AuthenticateOnUnsubscribe flag is set to "true".
https://docs.aws.amazon.com/sns/latest/api/API_ConfirmSubscription.html
Before SNS will talk to an endpoint, you need to prove that you control that endpoint. So your options are to write some code that can do it automatically (most of the SDKs support this) or to capture the token, and by returning it via the API call prove that you control it.
This is a one-time procedure, so you do not need to deploy any AWS credentials to your API endpoint - you can do it from a different system.
We generally build the confirmation handler into the application.
Related
I need to create an API gateway to consume messages from an SNS. I see that there are some questions on the same, like this, which hasn't been answered directly.
The main issue I face is : How to make the API gateway subscribe to the SNS?
SNS supports HTTP/HTTPS endpoint subscriptions. So I don't see a reason why you could not subscribe API gateway https endpoint to SNS this way.
As Marcin has mentioned, it can be done. Mentioning the steps I used:
Create SNS, and a lambda
Whatever the language is, make sure to print the event in the log. (For JavaScript, use console.log(event), and for Python use print(event) etc)
Create an API Gateway (REST API), select a "New API", and create it.
In the API create a POST method, select lambda, and make sure to use it as a proxy.
Copy the trigger HTTPS link
Add a HTTPS subscription in the SNS, and use the trigger link here.
The confirmation would be pending now.
Go to the lambda, inside it's monitoring section, go to cloud watch logs.
Inside the logs, you will find the event object printed. Inside it, look for SubscribeURL, along which the URL would be mentioned.
Copy this URL, and go back to the SNS. Select your subscription and use the "Confirm Subscription" option, and paste this link there.
The status should come as "Confirmed" now.
I have tried to do some R&D but i couldn't find anything useful the only thing that i found is lambda functions is the only way. I want to write a simple application that execute when the Simple Queue Service receive any message , but i couldn't find a way to do that till now, since i don't want to use lambda. for example if i receive some message on Simple Queue Service and while receiving each messages i can trigger a event that is not lambda but instead of that any HTTP-request.
I think your choices are:
use lambda (which you said you didn't want to use, but its probably the best solution)
use your own app running on ec2 or even on premise to consume the message and invoke the http endpoint
use SNS instead of SQS for message delivery - SNS supports http endpoints.
You can use Amazon SNS to send notification messages to one or more
HTTP or HTTPS endpoints. When you subscribe an endpoint to a topic,
you can publish a notification to the topic and Amazon SNS sends an
HTTP POST request delivering the contents of the notification to the
subscribed endpoint. When you subscribe the endpoint, you select
whether Amazon SNS uses HTTP or HTTPS to send the POST request to the
endpoint.
from here: https://docs.aws.amazon.com/sns/latest/dg/sns-http-https-endpoint-as-subscriber.html
A short answer to your question is No, (until today)
Let me tell you the sineros i faced.In general Queue triggering lambda is widely used and for that u have to make sure about proper concurrency (minimum 5) in place and also database I/O if u are performing and any DB calls . But I've a scenario where we cannot use "lambda as a triggering service" as our DB is onprem "ORACLE" so the choices are .
Push to "SNS" and make http "what ever applies"(to a container we have custom Kubernetes routed through NLB ).Also make sure you push a batch of messages as it might make more http noise.
2.Poll the queue and perform the operations.
SQS triggering => Lambda and lambda invoking state machine (step functions)
I have a requirement where I need to send some inbound http query parameters from api A by publishing it to SNS and other api B is subscribed to sns where it should get the query params.
Is there any way I can acheive it?
Note: I have a fixed message structure when I publish to SNS which I cannot change, since I have other endpoints who are already subscribing to SNS, depend on it.
As Michael mentioned, we can use lambda. but I do not want to use another service provided by aws. I want to acheive the same using SNS only. I heard about SNS message attributes, but not sure whether we can acheive the same using it.
Thanks.
SNS doesn't support customizing the HTTP transaction or otherwise modifying the message being delivered... but SNS does support subscriptions that target a Lambda function.
This means you can write a Lambda function that parses the SNS event payload... and then, instead of SNS contacting the target system over HTTPS, your Lambda function makes the HTTPS request to the target endpoint, customized as required. If the HTTPS request fails, or the endpoint returns an error, the Lambda function should throw an exception, so that Lambda can know to retry.
https://docs.aws.amazon.com/lambda/latest/dg/invoking-lambda-function.html#supported-event-source-sns
I've created SNS topic
I've created API Gateway endpoint that invokes Lambda function
I've created topic HTTPS subscription that points to API Gateway endpoint
Problem: everything works fine when AUTH=none, but when i enabled AUTH=AWS_IAM, neither subscription nor messages are delivered to my lambda. They also wont show up in Lambda OR Gateway cloudwatch logs as it's usually the case with authentication errors.
Questions:
What's the identity delivered by HTTPS endpoint to AWS_IAM so it doesn't allows it ( my first thought was to relay SNS posters token but it doesn't seem be the case )
I couldn't find any way to associate HTTPS endpoint with any identity, is there a way?
There are lots of information about delivering SNS to SQS or Gateway to SNS, but couldn't find any information about achieving what i try to do.
Is there any method to debug AWS_IAM authentication problems? Documentation i've seen advices to "check priviliges" which is something i've been doing for many hours but i have no more ideas.
I'd be glad to hear any ideas from you, thanks.
As you may have seen in the docs, SNS can only do Basic/Digest Auth http://docs.aws.amazon.com/sns/latest/dg/SendMessageToHttp.html
There is a section in the docs about verifying the validity of the message but that is code you'd have to write yourself or lift from one of the SNS SDKs on the backend. There really isn't any way to get SNS to sign the request with AWS SigV4, unfortunately.
Why don't you let the Lambda function subscribe directly to the SNS topic (without going through API Gateway)?
That should be straightforward: https://docs.aws.amazon.com/sns/latest/dg/sns-lambda.html
Here is the complete link which will help you in solving your authentication problem. https://aws.amazon.com/premiumsupport/knowledge-center/iam-authentication-api-gateway/
If it's an "Check privileges" issue, then your IAM user doesn't have any sufficient access to the resources to make any changes.
I have a static website (a simple landing page) hosted on AWS S3, with Cloudfront enabled in front of it.
I would like to add a public contact form request.
I found some solutions but I would like to know which one is the best.
1/ Front-end JS send a message to an SNS topic and subscribe my email to it (but it means that I need to share an AccessKey in public)
2/ Front-end JS send a message to SNS, trigger a Lambda function which process and send email via SES.
3/ Front-end JS send a POST request to a public API Gateway, trigger a Lambda function which process and send email via SES
4/ Others?
Thank you for your help.
3rd option is the best and easiest one to implement. Also as #Mark said in comments, you don't need to keep your access keys in client side in this case.
3rd for sure. You could even use API Gateway's AWS Service Proxy and call SNS/SES API directly, without the need of using AWS Lambda.
This tutorial covers this use case.