I want to have a lambda calling a Sagemaker instance in another region. If both are in the same region, everything works fine. If they are not, I get the following error:
The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method. Consult the service documentation for details.
The Canonical String for this request should have been
'POST
/endpoints/foo-endpoint/invocations
host:runtime.sagemaker.us-east-1.amazonaws.com
x-amz-date:20180406T082536Z
host;x-amz-date
1234567890foobarfoobarfoobarboofoobarfoobarfoobarfoobarfoobarfoo'
The String-to-Sign should have been
'AWS4-HMAC-SHA256
20180406T082536Z
20180406/us-east-1/sagemaker/aws4_request
987654321abcdeffoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarf'
I use aws-requests-auth (0.4.1) with boto3 (1.5.15 - updating to 1.7.1 didn't change anything, changelog) like this:
import requests
from aws_requests_auth.aws_auth import AWSRequestsAuth
auth = AWSRequestsAuth(aws_access_key=config['AWS']['ACCESS_KEY'],
aws_secret_access_key=(
config['AWS']['SECRET_ACCESS_KEY']),
aws_host=config['AWS']['HOST'],
aws_region=config['AWS']['REGION'],
aws_service=config['AWS']['SERVICE'])
payload = {'foo': 'bar'}
response = requests.post(post_url,
data=json.dumps(payload),
headers={'content-type': 'application/json'},
auth=auth)
printing auth only gives <aws_requests_auth.aws_auth.AWSRequestsAuth object at 0x7f9d00c98390>.
Is there a way to print the "Canonical String" mentioned in the error message?
(Any other ideas how to fix this are appreciated as well)
A work-around for the asked question:
req = requests.request('POST', 'http://httpbin.org/get')
req.body = b''
req.method = ''
print(auth.get_aws_request_headers(req,
aws_access_key=auth.aws_access_key,
aws_secret_access_key=auth.aws_secret_access_key,
aws_token=auth.aws_token))
The problem is not solved, though. And now I wonder what the first argument of auth.get_aws_request_headers is.
Related
There are a few examples for the way to pre-sign the URL of an S3 request, but I couldn't find any working example to pre-sign other services in AWS.
I'm trying to write an item to DynamoDB using the Python SDK botos. The SDK included the option to generate the pre-signed URL here. I'm trying to make it work and I'm getting a URL, but the URL is responding with 404 and the Item is not appearing in the DynamoDB table.
import json
ddb_client = boto3.client('dynamodb')
response = ddb_client.put_item(
TableName='mutes',
Item={
'email': {'S':'g#g.c'},
'until': {'N': '123'}
}
)
print("PutItem succeeded:")
print(json.dumps(response, indent=4))
This code is working directly. But when I try to presign it:
ddb_client = boto3.client('dynamodb')
params = {
'TableName':'mutes',
'Item':
{
'email': {'S':'g#g.c'},
'until' : {'N': '1234'}
}
}
response = ddb_client.generate_presigned_url('put_item', Params = params)
and check the URL:
import requests
r = requests.post(response)
r
I'm getting: Response [404]
Any hint on how to get it working? I checked the IAM permissions, and they are giving full access to DynamoDB.
Please note that you can sign a request to DynamoDB using python, as you can see here: https://docs.aws.amazon.com/general/latest/gr/sigv4-signed-request-examples.html#sig-v4-examples-post . But for some reasons, the implementation in the boto3 library doesn't do that. Using the boto3 library is much easier than the code above, as I don't need to provide the credentials for the function.
You send an empty post request. You should add the data to the request:
import requests
r = requests.post(response, data = params)
I think you are having this issue, that's why you are recieving a 404.
They recommend using Cognito for authentication instead of IAM for this cases.
I have a Lambda Function that it is accessible by an API Gateway. I can handle all POST and GET submitted requests to API endpoint (https://XXXXXXX.execute-api.us-east-1.amazonaws.com/default/myapi) inside my Lambda, but I need to use some segments at end of my URL when I am using PUT requests.
My Python code to call the API is here and it is working correctly:
import requests
import json
url = 'https://XXXXXXX.execute-api.us-east-1.amazonaws.com/default/myapi'
token = "my token"
data = {
"first_name": "Reza",
"birthday": "1986-09-12"
}
headers = {"Content-Type" : "application/json", "x-api-key":"MY_API_KEY"}
response = requests.put(url, data=json.dumps(data), headers=headers)
print(response.text)
But if I add users segment to end of the URL like this:
url = 'https://XXXXXXX.execute-api.us-east-1.amazonaws.com/default/myapi/users'
it will show this error:
{"message":"Missing Authentication Token"}
I need to add some static segments like users to return the list of all users and some dynamic segments like users/USER_ID (when USER_ID is a dynamic number) to return the information for a special user.
can you please guide me how I can use segmented URL in my AWS API Gateway?
The term you are using segmented URL might have caused your confusion. It is called path parameters with AWS. There is more than one way to do it. ANY+ integration is the easiest to handle.
Integrate with ANY+ integration to your lambda and you are good to go. All the path parameters will be delivered to your lambda.
http://www.1strategy.com/blog/2017/06/06/how-to-use-amazon-api-gateway-proxy/
Additional path parameter documentation,
https://docs.aws.amazon.com/apigateway/latest/developerguide/integrating-api-with-aws-services-lambda.html#api-as-lambda-proxy-expose-get-method-with-path-parameters-to-call-lambda-function
Good luck.
def sendEmail(to,apNumber,paperType,zipedFile):
sg = sendgrid.SendGridAPIClient(apikey=os.environ.get("API-KEY"))
to_email = mail.Email( "to#email.com")
from_email = mail.Email( "from#email.com" )
subject = 'This is a test email'
content = mail.Content('text/plain', 'Example message.')
message = mail.Mail(from_email, subject, to_email, content)
response = sg.client.mail.send.post(request_body = message.get())
return response
Setup Environment Variables.
To Set up environment Variable follow the below 3 steps
echo "export SENDGRID_API_KEY='YOUR_API_KEY'" > sendgrid.env
echo "sendgrid.env" >> .gitignore
source ./sendgrid.env
import os
from sendgrid import SendGridAPIClient
from sendgrid.helpers.mail import Mail
message = Mail(
from_email='...#gmail.com',
to_emails='....#gmail.com',
subject='Sending with Twilio SendGrid is Fun',
html_content='<strong>and easy to do anywhere, even with Python</strong>')
try:
**sg = sendgrid.SendGridAPIClient(api_key=os.environ.get('SENDGRID_API_KEY'))**
response = sg.send(message)
print(response.status_code)
print(response.body)
print(response.headers)
except Exception as e:
print(e)
I got the same error "UnauthorizedError: HTTP Error 401: Unauthorized"
I refactored the code from
sg = sendgrid.SendGridAPIClient(api_key=os.environ.get('SENDGRID_API_KEY'))
to
sg=SendGridAPIClient('......IeltIytmFYeQ0aSOt2UBYvv2E6Xh...')
Then it started working.
I change my API key settings to Full access in SendGrid portal.
I was getting "HTTP/1.1 401 Unauthorized" error in PHP. See the screenshot.
I solved it by removing getenv() from the API key.
Replace:
$apiKey = getenv("SendGrid_API_Key");
$sendgrid = new \SendGrid($apiKey);
with:
$apiKey = ("SendGrid_API_Key");
$sendgrid = new \SendGrid($apiKey);
I created a new key and that solved my issue. Exact issue for this is still unknown.
As it was functional before.
The error for me was because I used the wrong key , actually, one needs to use the key that is created the first time when one create the Api key , which one cannot edit again. So one have to click "Create Api key" button and use this key
I'm trying to make a simple Python Lambda that makes snapshots of our Elasticsearch database. This is done through Elasticsearch's REST API using simple HTTP requests.
However, for AWS, I have to sign these requests. I have a feeling it can be achieved through boto3's low-level clients probably with generate_presigned_url, but I cannot for the life of me figure out how to invoke this function correctly. For example, what are the valid ClientMethods? I've tried ESHttpGet but to no avail.
Can anyone point me in the right direction?
Edit: Apparently this workaround has been broken by Elastic.
I struggled for a while to do a similar thing. Currently the boto3 library doesn't support making signed es requests, though since I raised an issue with them it's become a feature request.
Here's what I've done in the meantime using DavidMuller's library mentioned above and boto3 to get my STS session credentials:
import boto3
from aws_requests_auth.aws_auth import AWSRequestsAuth
from elasticsearch import Elasticsearch, RequestsHttpConnection
session = boto3.session.Session()
credentials = session.get_credentials().get_frozen_credentials()
es_host = 'search-my-es-domain.eu-west-1.es.amazonaws.com'
awsauth = AWSRequestsAuth(
aws_access_key=credentials.access_key,
aws_secret_access_key=credentials.secret_key,
aws_token=credentials.token,
aws_host=es_host,
aws_region=session.region_name,
aws_service='es'
)
# use the requests connection_class and pass in our custom auth class
es = Elasticsearch(
hosts=[{'host': es_host, 'port': 443}],
http_auth=awsauth,
use_ssl=True,
verify_certs=True,
connection_class=RequestsHttpConnection
)
print(es.info())
Hope this saves somebody some time.
There are several Python extensions to the requests library that will perform the SigV4 signing for you. I have used this one and it works well.
While other answers are perfectly fine, I wanted to eliminate the use of external packages. Obviously, botocore itself has all the required functionality to sign requests it was just a matter of looking at the source code. This is what I ended up with for sending AWS API requests directly (things are hardcoded for the demonstration purposes):
import boto3
import botocore.credentials
from botocore.awsrequest import AWSRequest
from botocore.endpoint import URLLib3Session
from botocore.auth import SigV4Auth
params = '{"name": "hello"}'
headers = {
'Host': 'ram.ap-southeast-2.amazonaws.com',
}
request = AWSRequest(method="POST", url="https://ram.ap-southeast-2.amazonaws.com/createresourceshare", data=params, headers=headers)
SigV4Auth(boto3.Session().get_credentials(), "ram", "ap-southeast-2").add_auth(request)
session = URLLib3Session()
r = session.send(request.prepare())
I recently published requests-aws-sign, which provides AWS V4 request signing for the Python requests library.
If you look at this code you will see how you can use Botocore to generate the V4 request signing.
why not just use requests?
import requests
headers = {'Content-Type': 'application/json',}
data = '{"director": "Burton, Tim", "genre": ["Comedy","Sci-Fi","R-rated"],"profit" : 98 , "year": 1996, "actor": ["Jack Nicholson","PierceBrosnan","Sarah Jessica Parker"], "title": "Mars Attacks!"}'
response = requests.post('https://search-your-awsendpoint.us-west-2.es.amazonaws.com/yourindex/_yourdoc/', headers=headers, data=data)
this worked for me
I want to use the provided slack web API https://api.slack.com/web#basics to get some messages out of a channel. I looked at https://api.slack.com/methods/channels.history & used the Request API to call the service.
payload = {'token': 'XXXXXXXXXXXX', 'channel': '#scanbot' , 'count' : '10'}
r = requests.get('https://slack.com/api/channels.history', params=payload)
print r.status_code
print r.text
But i am getting the error:
200
{"ok":false,"error":"channel_not_found"}
I am pretty sure the channel exists and I am providing correct API key. Can someone point me at the correct directions please?
You need to pass the channel ID as the argument to the channels.history endpoint.
The channel IDs may be fetched by checking the channels.list endpoint.
See for example the source of the Slacker package.
The JSON-formatted response may be parsed with:
import json
data = json.loads(r.text)
print data