Github Webhooks secret with AWS API Gateway - amazon-web-services

I am trying to have a Github Webhook launch an AWS Lambda I have.
The best way I can figure out how to do that is to use AWS API Gateway, the issue is security.
Github Webhooks will only send a secret with the POST call.
I can't find any way to have AWS API Gateway to verify this signature.
Or where I can add this functionality.
I assume I can write an AWS Lambda Authorizer.
But this is a lot of code in different places, starting to see the need for serverless framework.
Any easier setup within AWS I do not know about for this?

Came here because I was trying to integrate a Github webhook with AWS lambda and ran into the same problem as the OP. At the time of writing I believe the best solution is to include verification code in the main lambda, as others have suggested.
On the AWS Computer Blog from September 2017:
Enhanced request authorizer Lambda functions receive an event object that is similar to proxy integrations. It contains all of the information about a request, excluding the body.
Source: Using Enhanced Request Authorizers in Amazon API Gateway (amazon.com)
You can't perform HMAC as recommended by Github, because AWS authorizer lambdas don't give you access to the body of the HTTP request, which you need in order to compare digests.
This is a shame, because HMAC seems to be a pretty standard way of securing an endpoint that responds to a webhook. See for example this blog post, Webhooks do’s and dont’s: what we learned after integrating +100 APIs (restful.io). Twitter and Stripe do something similar:
Securing webhooks (twitter.com)
Checking Webhook Signatures(stripe.com)
To make the approach described above work, if you're using API Gateway you'll need to make sure that the header that contains the hash signature is forwarded as part of the event argument to the lambda. To do that, follow these instructions: How do I pass custom headers through Amazon API Gateway to an AWS Lambda function using custom Lambda integration for further processing? (amazon.com)

I couldn't find a way to do this with API Gateway. I validated within the LAMBDA using (Python).
High level overview : Calculate HMAC signature with GITHUB_SECRET then compare to the signature passed from Github.
You can obviously simplify, intentionally verbose for readability. There may be better ways, but I couldn't find one.
Make sure your Webhook is configured for application/json. Hopefully this helps someone else.
import logging
import json
import hmac
import hashlib
import re
from urllib.parse import unquote
logger = logging.getLogger(__name__)
logger.setLevel(logging.DEBUG)
GITHUB_SECRET = 'SECRET FROM GITHUB CONSOLE'
def lambda_handler(event, context):
logger.info("Lambda execution starting up...")
incoming_signature = re.sub(r'^sha1=', '', event['headers']['X-Hub-Signature'])
incoming_payload = unquote(re.sub(r'^payload=', '', event['body']))
calculated_signature = calculate_signature(GITHUB_SECRET, incoming_payload.encode('utf-8'))
if incoming_signature != calculated_signature:
logger.error('Unauthorized attempt')
return {
'statusCode': 403,
'body': json.dumps('Forbidden')
}
logger.info('Request successfully authorized')
# do stuff in Lambda
return {
'statusCode': 200,
'body': json.dumps(f'Work in progress')
}
def calculate_signature(github_signature, githhub_payload):
signature_bytes = bytes(github_signature, 'utf-8')
digest = hmac.new(key=signature_bytes, msg=githhub_payload, digestmod=hashlib.sha1)
signature = digest.hexdigest()
return signature

Related

AWs lambda - enable events in slack fails

I created an AWS Lambda function with the following default (Hello world example) code.
def lambda_handler(event, context):
# TODO implement
return {
'statusCode': 200,
'body': json.dumps('Hello from Lambda!')
}
I also added an API gateway as a trigger and when tested with curl, I can successfully retrieve the 'Hello from Lambda Message'
curl https://xxxxxxxxxxxxx.execute-api.us-east-1.amazonaws.com/default/test
"Hello from Lambda!"
Then, as I want to build a slackbot, I enable events and tested the api-gateway url in slack
slack app event
I'm failing to understand why there is no challenge nor token being sent in slack post to the api gateway URL
Besides the fact that I should be able to catch the challenge and return it to complete de verification process, there nothing being sent from slack challenge POST message (I should at least see the code:200 and the "Hello from Lambda!" message in the body anyway)
Following this article, I was expecting to see something like this, but mine has an empty response
Slack Enable events from article
Any clues? This is a new workspace and a new slack app, so it is possible that I forgot to set it correctly.
Thanks
I was having the same problem as you. I followed this example and got it to work: https://medium.com/analytics-vidhya/create-and-distribute-a-slack-bot-with-python-and-aws-in-1-hour-41c4a6c0f99d
The issue with my API was that initially I created an API in API Gateway of type REST. When I made an API of type HTTP, everything worked.

How to integrate Appsync api in android

I have given an amazon api to integrate. But I have no idea how to use this api
mutation createTrail {
createTruckTrailer(input:{truckId: "077",trailer:["AB22D"]}){
result
}
}
The snippet you've been given is an example of a GraphQL query, which can be sent to a GraphQL-enabled endpoint, such as the one you've been given which is hosted using AWS AppSync. A GraphQL query is structured differently than a REST api call; think of it as a structured way of making REST calls. Requests are sent to the ApiUrl you pasted above in a POST HTTP request, with a request body that looks like:
{
"query": "mutation createTrail {\n createTruckTrailer(input:{truckId: \"077\",trailer:[\"AB22D\"]}){\n result }\n }",
"operationName": "createTrail",
"variables": {}
}
However additional headers are necessary to authenticate with IAM. The response from the server will include the result as JSON. I highly recommend spending just a few minutes to become a little more familiar with GraphQL here: https://graphql.org/learn/
To make calls to this AppSync endpont from Android, take a look at the Amplify android client, which explains how to get started here (Skip step 4): https://aws-amplify.github.io/docs/android/start, and then setup your client to authenticate with IAM to your backend here: https://aws-amplify.github.io/docs/android/api#iam
Once your amplify client is fully set up, you can make the GraphQL Mutation shown above by following the steps here: https://aws-amplify.github.io/docs/android/api#import-sdk-and-config
Use retrofit with rxJava or coroutines.

Use ApiGateway Authorizer to Validate Github Payload Signature (X-Hub-Signature)

I am currently working on a simple api to receive Github event payloads, and I want to validate that they are coming from the correct source. With this I am working to use the hmac signature in the requests header (generated by github using a secret provided by me). To validate the signature, the ApiGateway authorizer requires the signature (X-Hub-Signature), the secret used to generate the signature, and the body of the message. As far as I can tell, Api Gateway does not allow you to pass the body to an ApiGateway Authorizer. Does anyone know a way around this that does not require additional proxy lambdas and s3?
*Note: The requester is the Github Webhook service (not able to add body to header)
Basic ApiGateway Auth Docs:
https://docs.aws.amazon.com/apigateway/latest/developerguide/apigateway-use-lambda-authorizer.html
Here is how you do it.
Pass your content to Authorization header of your incoming request, It will get delivered to your custom Authorizer.
Grab the contents of the from the below attribute,
event.authorizationToken
where event is one of the parameters (1st) passed to lambda,
I currently encrypt and add all the info to that header and gets delivered to the Custom Authorizer lambda.
You can also access additional parameters as below in your custom Authorizer lambda:
var headers = event.headers;
var queryStringParameters = event.queryStringParameters;
var pathParameters = event.pathParameters;
var stageVariables = event.stageVariables;
var requestContext = event.requestContext;
Hope it helps.

SurveyMonkey - Create Webhook with AWS API Gateway as Subscription Url

I am trying to Integrate SurveyMonkey with my backend.
This is the flow:
(1) We create a webhook(https://developer.surveymonkey.com/api/v3/#webhooks) using POST method from PHP, with Subscription url as an AWS API Gateway.
(2) When user completes a survey, SurveyMonkey invokes the subscription Url (API Gateway), which in turn calls a Lambda function to update the database.
The problem is when i try to give a API Gateway url as the subscription Url, SurveyMonkey returns a Bad Request response. If i use a general .php page, the webhook is getting created successfully.
In the conversation with SurveyMonkey Support Team, they say, the API Gateway should return a 200 response for HEAD request. I am not sure how to set this up in AWS.
Can anyone who has implemented / integrated, API Gateway with SurveyMonkey please guide me on how to solve this issue?
Here is the conversation with SurveyMonkey Support Team:
Thanks for getting back in touch. I just spoke with my colleagues on
the development team and they noted that there shouldn't be an issue
in using a lamda function for your subscription_URL. As long as it
accepts HEAD and POST requests and returns a 200. I hope this helps
you out; of course, please let me know if I can assist further.
Thanks for getting back in touch. I wanted to confirm that the lambda
function needs to return a 200 to a HEAD request in order for us to
enable the webhook properly. This isn't done after the webhook is
cerated, but as the webhook is created. It still seems like the
lambda function isn't responding, or responding correctly, as it is
created, so it will continue to fail. I hope this helps clarify;
please let me know if I can assist further.
One way or another, you need to get the API Gateway to respond to HEAD requests. There are a lot of ways to do this. You could add a HEAD method to the resource of your URL. You can do that in the API Gateway console under the resource section when you're editing you api. You can send the HEAD requests to a lambda function and have that function return a 200 status code for appropriate HEAD requests. You can set up a mock return in the api integration request, but you'll need to set up the correct integration details. You can also just set ANY lambda proxy integration and deal with everything on the lambda side.
Once you have something setup you should be able to test with curl:
curl -I http://yourUrl.example.com

Missing Authentication Token while accessing API Gateway?

I am trying to call a Lambda Function through AWS API Gateway.
When I mention Authentication type NONE it works fine but API become public and anyone with url can access my API.
To make API call secure, I am using Authentication type AWS_IAM and
also attached AmazonAPIGatewayInvokeFullAccess policy to my user but getting this error:
{ message: "Missing Authentication Token"}
I don't know what I am missing here.
I've lost some time for a silly reason:
When you create a stage, the link displayed does not contain the resource part of the URL:
API URL:
https://1111.execute-api.us-east-1.amazonaws.com/dev
API + RESOURCE URL
https://1111.execute-api.us-east-1.amazonaws.com/dev/get-list
The /get-list was missing
And of course, you need to check that the method configuration looks like this:
I think you are directly trying to access API link, this won't work because API is secured using IAM role and you must provide AWS authentication i.e Access key and Secret key.
Use the Postman Chrome extension to test your API:
http://docs.aws.amazon.com/apigateway/latest/developerguide/how-to-use-postman-to-call-api.html
I just had the same issue and it seems it also shows this message if the resource cannot be found.
In my case I had updated the API, but forgotten to redeploy. The issue was resolved after deploying the updated API to my stage.
Make sure you are clicking on the specific Resource first in the Stages tree, as that will populate a URL with the full path to the resource (rather than just the root path):
For other causes, see http://www.awslessons.com/2017/aws-api-gateway-missing-authentication-token/
Looks like (as of April 2019) AWS API Gateway throws this exception for a variety of reasons - mostly when you are hitting an endpoint that API Gateway is not able to reach, either because it is not deployed, or also in cases where that particular HTTP method is not supported.
I wish the gateway sends more appropriate error codes like HTTP 405 Method not supported or HTTP 404 not found, instead of a generic HTTP 403 Forbidden.
Found this in the docs:
If the AWS_IAM authorization were used, you would sign the request using the Signature Version 4 protocols.
Signing request with Signature Version 4
You can also generate an SDK for your API.
How to generate an SDK for an API in API Gateway
Once you've generated the SDK for the platform of your choice, step 6 mentions that if you're using AWS credentials, the request to the API will be signed:
To initialize the API Gateway-generated SDK with AWS credentials, use code similar to the following. If you use AWS credentials, all requests to the API will be signed. This means you must set the appropriate CORS Accept headers for each request:
var apigClient = apigClientFactory.newClient({
accessKey: 'ACCESS_KEY',
secretKey: 'SECRET_KEY',
});
I try all the above, if you did all steps in the above answers, and you not solve the problem, then:
on the left menu, hit the "Resources"
in the right to "Resources", hit the api method that you want to test, like "POST/GET etc)
hit the "ACTION" list (it's above to the API method in step 2
select "DEPLOY API" (please do it, even you already deploy yours api)
in "deployment stage" select "prod" or what ever you write in yours previous deploy (it will override yours previous deploy
hit deploy
I thing that because of, when I create the "METHOD REQUEST" (see step 2 how to go to this menu) , in "Authorization" I select "AWS_IAM"
after testing api, in aws test option, I try it in "postman"
then I understand the in "METHOD REQUEST" , in "Authorization", I should select "none"
I change it to none, but I thing the AWS, need to deploy it again, as I explain
Make sure you create Resource and then create method inside it. That was the issue for me. Thanks
In my case I missed adding '/' forward slash at the end of api.
Such a silly mistake.
https://le9dq5l9.execute-api.eu-west-1.amazonaws.com/v1/putdoctorinfo/
If you enable AWS_IAM authentication you must sign your request with AWS credentials using AWS Signature Version 4.
Note: signing into the AWS console does not automatically sign your browser's requests to your API.
sometimes this message shown when you are calling a wrong api
check your api endpoint
In my case it was quite a stupid thing.
I've get used that new entities are created using POST and it was failing with "Missing Authentication Token". I've missed that for some reason it was defined as PUT which is working fine.
This error mostly come when you call wrong api end point.
Check your api end point that you are calling and verify this on api gateway.
If you are using an API with endpoint of type PRIVATE, be sure of:
You are invoking the API from within your AWS account (example: from an EC2 instance created in your account)
Put necessary credential (access and secret keys) in the EC2 instance in route ~/.aws/credentials (this route is for linux instances) If IAM user use MFA aws_session_token value will be required too.
Use vpce (vpc endpoint) based URL. Example: curl https://vpce-0c0471b7test-jkznizi5.execute-api.us-east-1.vpce.amazonaws.com/dev/api/v1/status
Your EC2 instance have a security group than allow outbound traffic to another security group owned by the vpce like:
Your vpce security group allow inbound traffic from another security group (previous sg from ec2 instance) owned by the EC2 instance like:
See: https://docs.aws.amazon.com/apigateway/latest/developerguide/apigateway-private-apis.html
I had the same problem which I solved the following way:
GET Method test
https://54wtstq8d2.execute-api.ap-southeast-2.amazonaws.com/dev/echo/hello
Authorization tab ->
• select type(AWS signature)
• Add AccessKey and SecretKey
You must be using Invoke Url to trigger Lambda Function from your browser or POSTMAN... Instead, use the API end point which will be listed in:
select yourLambdaFuntion >> Configuration >> Triggers.
There you can see API end point.
For the record, if you wouldn't be using credentials, this error also shows when you are setting the request validator in your POST/PUT method to "validate body, query string parameters and HEADERS", or the other option "validate query string parameters and HEADERS"....in that case it will look for the credentials on the header and reject the request. To sum it up, if you don't intend to send credentials and want to keep it open you should not set that option in request validator(set it to either NONE or to validate body)
I had the same issue, and fixed it by removing the /dev/ and just put: https://1111.execute-api.us-east-1.amazonaws.com/get-list
I had same issue today because I was using GET instead of POST. Fixed the issues by changing method to POST in postman.
First of all, check whether the API you created in the lamda function is registered with your AWS project or not. For that, go to the API gateway in your AWS console. If it is not registered, register it. This is the main cause of this issue.
You can even see in your aws.export.js file, that there are paths corresponding to your API ['/items'].
Your API must be present there, otherwise it will not append the security token to requests. Just register it in your project cloud-logic in your console for this.
If it's there, then use the above mentioned solution
http://docs.aws.amazon.com/apigateway/latest/developerguide/how-to-use-postman-to-call-api.html
To contribute:
I had a similar error because my return response did not contain the 'body' like this:
return {
'statusCode': 200,
'body': "must contain the body tag if you replace it won't work"
}
If you set up an IAM role for your server that has the AmazonAPIGatewayInvokeFullAccess permission, you still need to pass headers on each request. You can do this in python with the aws-requests-auth library like so:
import requests
from aws_requests_auth.boto_utils import BotoAWSRequestsAuth
auth = BotoAWSRequestsAuth(
aws_host="API_ID.execute-api.us-east-1.amazonaws.com",
aws_region="us-east-1",
aws_service="execute-api"
)
response = requests.get("https://API_ID.execute-api.us-east-1.amazonaws.com/STAGE/RESOURCE", auth=auth)
Well for anyone still having the problem and I really feel very dumb after realizing this, but I passed in the url of /items the default one while adding API. But I kept calling the endpoint with /api. Special thanks to Carlos Alberto Schneider, as I realized my problem after reading your post.
According to my experience, please check the following steps:
On API gateway side, make sure you add the correct path and publish the resource at the stage you want. For some url pattern like path parameter(/user/{user_id}) need more attention to have a check.
Make sure you configure the correct options method for this resource, because sometimes it is the CORS that cause this problem.
On Lambda side, make sure you specify the correct handler name as the entrypoint.
Please always check cloudwatch logs of your lambda that can help u identify the problems on your lambda side.
In my case I was trying to do an UPDATE type request but in my AWS SAM template I had a PATCH type request:
Resources:
LambdaFunction:
Type: AWS::Serverless::Function # More info about Function Resource:
# https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#awsserverlessfunction
Properties:
CodeUri: src/
Handler: app.lambda_handler
Runtime: python3.8
MemorySize: 128
Events:
UpdateItem:
Type: Api
Properties:
Path: /my-endpoint
Method: patch
Things to check:
1. In resources, check for Authorization and API Key if these are required.
2. Redeploy the API, new changes might not reflected.
3. Get the url from stages, directly by clicking on the actions like GET, POST, These will contain the full path urls.
I had the same issue even if my access was regional and not private, no authorization on my method neither API. It's turned out that I was calling the root endpoint "https://azerty.execute-api.eu-west-3.amazonaws.com/dev/", which in local return me a json but raised an error. Be sure to call a proper endpoint, such as "https://azerty.execute-api.eu-west-3.amazonaws.com/dev/hello"