I have deployed a Tensorflow-Model in SageMaker Studio following this tutorial:
https://aws.amazon.com/de/blogs/machine-learning/deploy-trained-keras-or-tensorflow-models-using-amazon-sagemaker/
The Model needs a Multidimensional Array as input. Invoking it from the Notebook itself is working:
import numpy as np
import json
data = np.load("testValues.npy")
pred=predictor.predict(data)
But I wasnt able to invoke it from a boto 3 client using this code:
import json
import boto3
import numpy as np
import io
client = boto3.client('runtime.sagemaker')
datain = np.load("testValues.npy")
data=datain.tolist();
response = client.invoke_endpoint(EndpointName=endpoint_name, Body=json.dumps(data))
response_body = response['Body']
print(response_body.read())
This throws the Error:
An error occurred (ModelError) when calling the InvokeEndpoint operation: Received client error (415) from model with message "{"error": "Unsupported Media Type: Unknown"}".
I guess the reason is the json Media Type but i have no clue how to get it back in shape.
I tried this:https://github.com/aws/amazon-sagemaker-examples/issues/644 but it doesnt seem to change anything
This fixed it for me:
The Content Type was missing.
import json
import boto3
import numpy as np
import io
client = boto3.client('runtime.sagemaker',aws_access_key_id=..., aws_secret_access_key=...,region_name=...)
endpoint_name = '...'
data = np.load("testValues.npy")
payload = json.dumps(data.tolist())
response = client.invoke_endpoint(EndpointName=endpoint_name,
ContentType='application/json',
Body=payload)
result = json.loads(response['Body'].read().decode())
res = result['predictions']
print("test")
Related
I'm trying to verify if the public access block of my bucket mypublicbucketname is checked or not through Lambda function. For testing, I create a bucket and I have unchecked the public access block. So, I did this Lambda:
import sys
from pip._internal import main
main(['install', '-I', '-q', 'boto3', '--target', '/tmp/', '--no-cache-dir', '--disable-pip-version-check'])
sys.path.insert(0,'/tmp/')
import json
import boto3
import botocore
def lambda_handler(event, context):
# TODO implement
print(boto3.__version__)
print(botocore.__version__)
client = boto3.client('s3')
response = client.get_public_access_block(Bucket='mypublicbucketname')
print("response:>>",response)
I updated the latest version of boto3 and botocore.
1.16.40 #for boto3
1.19.40 #for botocore
Even if I uploaded them and the function seems correct I got this exception:
[ERROR] ClientError: An error occurred (NoSuchPublicAccessBlockConfiguration) when calling the GetPublicAccessBlock operation: The public access block configuration was not found
Someone can explain me why I have this error ?
For futur users. If you got the same problem with get_public_access_block(). Use this solution:
try:
response = client.get_public_access_block(Bucket='mypublicbucketname')
except botocore.exceptions.ClientError as e:
if e.response['Error']['Code'] == 'NoSuchPublicAccessBlockConfiguration':
print('No Public Access')
else:
print("unexpected error: %s" % (e.response))
for put_public_access_block, it works fine.
I am using AWS CodeStar (Lambda + API Gateway) to build my serverless API. My lambda function works well in the Lambda console but strangely throws this error when I run the code on AWS CodeStar:
"message": "Internal server error"
Kindly help me with this issue.
import json
import os
import bz2
import pprint
import hashlib
import sqlite3
import re
from collections import namedtuple
from gzip import GzipFile
from io import BytesIO
from botocore.vendored import requests
import logging
logger = logging.getLogger()
logger.setLevel(logging.DEBUG)
def handler(event, context):
logger.info('## ENVIRONMENT VARIABLES')
logger.info(os.environ)
logger.info('## EVENT')
logger.info(event)
n = get_package_list()
n1 = str(n)
dat = {"total_pack":n1}
return {'statusCode': 200,
'headers': {'Content-Type': 'application/json'},
'body': json.dumps(dat)
}
def get_package_list():
url = "http://amazonlinux.us-east-2.amazonaws.com/2/core/2.0/x86_64/c60ceaf6dfa3bc10e730c9e803b51543250c8a12bb009af00e527a598394cd5e/repodata/primary.sqlite.gz"
db_filename = "dbfile"
resp = requests.get(url, stream=True)
remote_data = resp.raw.read()
cached_fh = BytesIO(remote_data)
compressed_fh = GzipFile(fileobj=cached_fh)
with open(os.path.join('/tmp',db_filename), "wb") as local_fh:
local_fh.write(compressed_fh.read())
package_obj_list = []
db = sqlite3.connect(os.path.join('/tmp',db_filename))
c = db.cursor()
c.execute('SELECT name FROM packages')
for package in c.fetchall():
package_obj_list.append(package)
no_of_packages = len(package_obj_list)
return no_of_packages
Expected Result: should return an Integer (no_of_packages).
I'm trying to write an AWS Lambda service using Python 2.7 that will generate an In-Memory CSV file and email it as an attachment. I feel like I'm close with this script based on what I've learned but I'm not quite there.
# Import smtplib for the actual sending function
import smtplib
import sys
import csv
import cStringIO
from os.path import basename
from email.mime.text import MIMEText
from email.mime.multipart import MIMEMultipart
from email.mime.application import MIMEApplication
# Import the email modules we'll need
server = smtplib.SMTP('smtp.postmarkapp.com', 587)
server.starttls()
server.login('.....','.....')
list = []
row1 = ["One","Two","Three"]
list.append(row1)
msg = MIMEMultipart()
msg['To'] = "daniel#mydomain.com"
msg['From'] = "noreply#mydomain.com"
msg['Subject'] = "DG Test subject"
msg.attach(MIMEText("Test Message"))
csv_buffer = cStringIO.StringIO()
writer = csv.writer(csv_buffer, lineterminator='\n')
writer.writerow(["1","2","3"])
for row in list:
writer.writerow(row)
print(csv_buffer.getvalue())
msg.attach(csv_buffer)
try:
response = server.sendmail(msg['From'], ["daniel#mydomain.com"],msg.as_string())
server.quit()
except AttributeError as error:
print(error)
else:
print(response)
This gives me the following error:
1,2,3
One,Two,Three
'cStringIO.StringO' object has no attribute 'get_content_maintype'
Basically it comes down to not being sure how to use the csv_buffer object. Assuming I just need to add that attribute to the object somehow but I'm not quite sure how. If I try to add any additional arguments to the .attach() line, it complains that I have too many arguments.
Thanks!
I figured it out, thanks to stitching together a few SO posts.
import cStringIO
import csv
csv_buffer = cStringIO.StringIO()
writer = csv.writer(csv_buffer, delimiter=',', quoting=csv.QUOTE_ALL)
writer.writerow(["1","2","3"])
for row in list:
writer.writerow(row)
print(csv_buffer.getvalue())
# new lines
csv_file = MIMEText(csv_buffer.getvalue())
attachment = csv_file.add_header('Content-Disposition', 'attachment', filename="csv_file.csv")
msg.attach(csv_file)
Using Python, I followed and when it came to Test it, the following error popped up:
{
"errorMessage": "module initialization error"
}
What could I have done wrong?
from __future__ import print_function
import os
from datetime import datetime
from urllib2 import urlopen
SITE = os.environ['site'] # URL of the site to check, stored in the site environment variable, e.g. https://aws.amazon.com
EXPECTED = os.environ['expected'] # String expected to be on the page, stored in the expected environment variable, e.g. Amazon
def validate(res):
'''Return False to trigger the canary
Currently this simply checks whether the EXPECTED string is present.
However, you could modify this to perform any number of arbitrary
checks on the contents of SITE.
'''
return EXPECTED in res
def lambda_handler(event, context):
print('Checking {} at {}...'.format(SITE, event['time']))
try:
if not validate(urlopen(SITE).read()):
raise Exception('Validation failed')
except:
print('Check failed!')
raise
else:
print('Check passed!')
return event['time']
finally:
print('Check complete at {}'.format(str(datetime.now())))
You don't need any environment variables. Just keep it simple
from __future__ import print_function
import os
from datetime import datetime
from urllib2 import urlopen
def lambda_handler(event, context):
url = 'https://www.google.com' # change it with your own
print('Checking {} at {}...'.format(url, datetime.utcnow()))
html = urlopen(url).read()
# do some processing
return html
Here is another simple example.
from __future__ import print_function
def lambda_handler(event, context):
first = event.get('first', 0)
second = event.get('second', 0)
sum = first + second
return sum
Here is a sample event which will be used to invoke this lambda. you can configure event from Lambda web interface. (or google it)
{
"first": 10,
"second": 23
}
In my case, I missed adding the logging_config.ini to the lambda function.
I guess you would face similar error when the lambda function doesn't find the referenced file or package.
Thanks to the new cloud9 IDE integration, I was able to create one on the fly.
I am new to google python api client.I am learning from https://developers.google.com/api-client-library/python/start/get_started.I want to make an api which converts python object into JSON data and sends to a servlet.
The python code of file api.py:
import os
import urllib2
import httplib2
import json
import requests
from apiclient.discovery import build
from oauth2client.client import flow_from_clientsecrets
from oauth2client.tools import run_flow
from oauth2client.file import Storage
from oauth2client import tools
api_version='1'
_file_="D:\API"
CLIENT_SECRETS = os.path.join(os.path.dirname(_file_))
flow=flow_from_clientsecrets(CLIENT_SECRETS,
scope=[
'https://www.googleapis.com/auth/devstorage.full_control',
],
http = httplib2.Http()
auth_http = credentials.authorize(http)
service=build('SendNotif',api_version,http=http)
req = urllib2.Request('http://example/notify')
req.add_header('Content-Type', 'application/json')
data={"message":"Hello User you are notified"}
data_json = json.dumps(data)
response = urllib2.urlopen(req, json.dumps(data))
The error shown is:
D:\API>python api.py
File "api.py", line 25
auth_http = credentials.authorize(http)
^
SyntaxError: invalid syntax
please do help in correcting me..
thanks in advance....
You're missing a closing parenthesis for this line:
flow=flow_from_clientsecrets(CLIENT_SECRETS,