I'm trying to get a file from an S3 bucket as part of the Setup Thread Group for a JMeter test, and I have the following Groovy script in a JSR223 Sampler:
import java.io.IOException
import java.io.InputStream
import java.util.Properties
import com.amazonaws.services.s3.AmazonS3
import com.amazonaws.services.s3.AmazonS3ClientBuilder
import com.amazonaws.services.s3.model.GetObjectRequest
import com.amazonaws.services.s3.model.S3Object
import com.amazonaws.auth.BasicAWSCredentials
import com.amazonaws.auth.AWSStaticCredentialsProvider
import com.amazonaws.services.s3.S3CredentialsProviderChain
def awsAccessKey = "my_key"
def awsSecretAccessKey = "my_secret_key"
System.setProperty("aws.accessKeyId", awsAccessKey)
System.setProperty("aws.secretKey", awsSecretAccessKey)
def awsCreds = new BasicAWSCredentials(awsAccessKey, awsSecretAccessKey)
def s3Client = AmazonS3ClientBuilder.standard()
.withRegion("us-west-2")
.build()
def s3Object = s3Client.getObject(new GetObjectRequest("bucket-name", "filename"))
def is = s3Object.getObjectContent()
The code throws the following, both when I use .withCredentials(awsCreds) and when I set the system properties:
java.lang.NoClassDefFoundError: Could not initialize class com.amazonaws.services.s3.S3CredentialsProviderChain
I've also converted the same code to the following Beanshell Sampler:
import java.io.IOException;
import java.io.InputStream;
import java.util.Properties;
import com.amazonaws.services.s3.AmazonS3;
import com.amazonaws.services.s3.AmazonS3ClientBuilder;
import com.amazonaws.services.s3.model.GetObjectRequest;
import com.amazonaws.services.s3.model.S3Object;
import com.amazonaws.auth.BasicAWSCredentials;
String awsAccessKey = "my_key";
String awsSecretAccessKey = "my_secret_key";
System.setProperty("aws.accessKeyId", awsAccessKey);
System.setProperty("aws.secretKey", awsSecretAccessKey);
BasicAWSCredentials awsCreds = new BasicAWSCredentials(awsAccessKey, awsSecretAccessKey);
AmazonS3 s3Client = AmazonS3ClientBuilder.standard().withRegion("us-west-2").build();
S3Object s3Object = s3Client.getObject(new GetObjectRequest("bucket-name", "filename"));
InputStream is = s3Object.getObjectContent();
However, this results in the following:
org.apache.jorphan.util.JMeterException: Error invoking bsh method: eval
Sourced file: inline evaluation of: ``import java.io.IOException; import java.io.InputStream; import java.util.Propert . . . '' : Typed variable declaration : Method Invocation AmazonS3ClientBuilder.standard
When I put this same code into a test Java class, it runs with no issues. I have aws-java-sdk-1.11.160.jar and aws-java-sdk-core-1.11.160.jar in my jmeter/lib, so I don't believe it's a missing dependency. Am I missing something else?
Try to set credentials explicitly, as you build your client instance.
Like,
...
BasicAWSCredentials awsCreds = new BasicAWSCredentials(awsAccessKey, awsSecretAccessKey);
AWSStaticCredentialsProvider awsCredsProvider = new AWSStaticCredentialsProvider(awsCreds);
AmazonS3 s3Client = AmazonS3ClientBuilder.standard().withRegion("us-west-2").withCredentials(awsCredsProvider).build();
For the groovy way, for class class com.amazonaws.services.s3.S3CredentialsProviderChain, you need aws-java-sdk-s3-1.11.161.jar in your path.
https://mvnrepository.com/artifact/com.amazonaws/aws-java-sdk-s3
The beanshell error message isn't helpful, it is just saying there is an error in your code somewhere but doesn't really say where. I guess you could turn on additional debugging to find out, but I'd stick with groovy.
In aws sdk java it stated:
To use the SDK, add the full path to the lib and third-party directories to the dependencies in your build file, and add them to your java CLASSPATH to run your code.
You need to include the three specified Jackson libraries as well. That will correct the issue.
Related
I have deployed a Tensorflow-Model in SageMaker Studio following this tutorial:
https://aws.amazon.com/de/blogs/machine-learning/deploy-trained-keras-or-tensorflow-models-using-amazon-sagemaker/
The Model needs a Multidimensional Array as input. Invoking it from the Notebook itself is working:
import numpy as np
import json
data = np.load("testValues.npy")
pred=predictor.predict(data)
But I wasnt able to invoke it from a boto 3 client using this code:
import json
import boto3
import numpy as np
import io
client = boto3.client('runtime.sagemaker')
datain = np.load("testValues.npy")
data=datain.tolist();
response = client.invoke_endpoint(EndpointName=endpoint_name, Body=json.dumps(data))
response_body = response['Body']
print(response_body.read())
This throws the Error:
An error occurred (ModelError) when calling the InvokeEndpoint operation: Received client error (415) from model with message "{"error": "Unsupported Media Type: Unknown"}".
I guess the reason is the json Media Type but i have no clue how to get it back in shape.
I tried this:https://github.com/aws/amazon-sagemaker-examples/issues/644 but it doesnt seem to change anything
This fixed it for me:
The Content Type was missing.
import json
import boto3
import numpy as np
import io
client = boto3.client('runtime.sagemaker',aws_access_key_id=..., aws_secret_access_key=...,region_name=...)
endpoint_name = '...'
data = np.load("testValues.npy")
payload = json.dumps(data.tolist())
response = client.invoke_endpoint(EndpointName=endpoint_name,
ContentType='application/json',
Body=payload)
result = json.loads(response['Body'].read().decode())
res = result['predictions']
print("test")
I have been thinking of this, is there anyway to put my credential into
settings.py.
My generator.py is as following
import os
import sys
import collections
import boto
from boto.s3.key import Key
from flask import Flask, render_template, url_for, abort, request
from flask.ext.frozen import Freezer
from werkzeug import cached_property
from werkzeug.contrib.atom import AtomFeed
import markdown
import yaml
from string import strip
...
class Blog(object):
def __init__(self, app, root_dir='',file_ext=None):
self.root_dir = root_dir
self.file_ext = file_ext if file_ext is not None else app.config['POSTS_FILE_EXTENSION']
self._app =app #_ for private and internal
self._cache = SortedDict(key = lambda p: p.date, reverse= True)
self._initialize_cache()
...
app = Flask(__name__)
app.config.from_envvar('SETTINGS_FILE') #configuration file
blog = Blog(app, root_dir= 'posts')
freezer = Freezer(app)
DOMAIN = 'cuboid.example.net'
AWS_ACCESS_KEY_ID = 'ABCDEFGHIJK'
AWS_SECRET_ACCESS_KEY = 'HUIHUGERUGHIUHSIUHH'
currently I cant put the credential in it as the boto does not settings.py Can you tell me if there is a way to work AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY into settings.py?
Create a file called settings.py, place your keys inside of it, load it with import settings and then reference them as settings.AWS_ACCESS_KEY_ID.
However, from a security perspective you should really use IAM Roles instead. They automatically provide temporary keys to each EC2 instance.
http://docs.aws.amazon.com/IAM/latest/UserGuide/roles-usingrole-ec2instance.html
I am new to google python api client.I am learning from https://developers.google.com/api-client-library/python/start/get_started.I want to make an api which converts python object into JSON data and sends to a servlet.
The python code of file api.py:
import os
import urllib2
import httplib2
import json
import requests
from apiclient.discovery import build
from oauth2client.client import flow_from_clientsecrets
from oauth2client.tools import run_flow
from oauth2client.file import Storage
from oauth2client import tools
api_version='1'
_file_="D:\API"
CLIENT_SECRETS = os.path.join(os.path.dirname(_file_))
flow=flow_from_clientsecrets(CLIENT_SECRETS,
scope=[
'https://www.googleapis.com/auth/devstorage.full_control',
],
http = httplib2.Http()
auth_http = credentials.authorize(http)
service=build('SendNotif',api_version,http=http)
req = urllib2.Request('http://example/notify')
req.add_header('Content-Type', 'application/json')
data={"message":"Hello User you are notified"}
data_json = json.dumps(data)
response = urllib2.urlopen(req, json.dumps(data))
The error shown is:
D:\API>python api.py
File "api.py", line 25
auth_http = credentials.authorize(http)
^
SyntaxError: invalid syntax
please do help in correcting me..
thanks in advance....
You're missing a closing parenthesis for this line:
flow=flow_from_clientsecrets(CLIENT_SECRETS,
I am trying to generate SAML metadata for Service provider,I have created simple servlet and trying to generate metadata in a methoad doGet()
this is the code I have tried below
import java.io.IOException;
import java.io.PrintWriter;
import java.net.HttpRetryException;
import javax.servlet.ServletException;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import javax.xml.namespace.QName;
import org.opensaml.Configuration;
import org.opensaml.saml2.metadata.EntityDescriptor;
import org.opensaml.xml.XMLObjectBuilderFactory;
public class MetadataProviderServlet extends HttpServlet{
protected void doGet(HttpServletRequest request,HttpServletResponse response) throws ServletException,IOException{
PrintWriter writer = response.getWriter();
response.setContentType("text/html");
writer.println("Hello world");
String entityId = "www.sampleEntityId.com";
XMLObjectBuilderFactory builderFactory = Configuration.getBuilderFactory();
EntityDescriptor descriptor = (EntityDescriptor)(builderFactory.getBuilder(EntityDescriptor.DEFAULT_ELEMENT_NAME).buildObject(EntityDescriptor.DEFAULT_ELEMENT_NAME));
descriptor.setEntityID(entityId);
}
}
But when I run the web application using one application server I got the NUllpointer Exception in particularly creation the object Entity Descriptor.Can Anyone please figure out what went wrong?
You need to initialise the OpenSAML library first. OpenSAML is initialised using the bootstrap method.
DefaultBootstrap.bootstrap();
I write about this on my blog here
I'm working on a simple script to send emails via the GMail API, and an old script I found to access their SMTP interface wasn't working.
So I used the following script from their quickstart page to start first with reading:
#! /usr/bin/env python
#
import httplib2
from apiclient.discovery import build
from oauth2client.client import flow_from_clientsecrets
from oauth2client.file import Storage
from oauth2client.tools import run
CLIENT_SECRET = '.client.json'
OAUTH_SCOPE = 'https://www.googleapis.com/auth/gmail.readonly'
STORAGE = Storage('gmail.storage')
flow = flow_from_clientsecrets(CLIENT_SECRET, scope=OAUTH_SCOPE)
http = httplib2.Http()
credentials = STORAGE.get()
if credentials is None or credentials.invalid:
credentials = run(flow, STORAGE, http=http)
http = credentials.authorize(http)
gmail_service = build('gmail', 'v1', http=http)
threads = gmail_service.users().threads().list(userId='me').execute()
if threads['threads']:
for thread in threads['threads']:
print 'Thread ID: %s' % (thread['id'])
Running this gives a NotImplementedError as shown in this question.
So I imported and called run_flow instead of run, as I did not install gflags to continue. However, I get the following error:
TypeError: run_flow() takes at least 3 arguments (3 given)
I understand from the linked question that argparse should help. I could add the call to parser that that question uses, but I would have no idea what arguments to pass on the command line.
Anyone successfully implemented something with this who could give some help?
You don't need to pass extra arguments to the command line when using run_flow python.
import argparse
...
from oauth2client import tools
...
from oauth2client.tools import run_flow
...
parser = argparse.ArgumentParser(parents=[tools.argparser])
flags = parser.parse_args()
....
credentials = run_flow(flow, STORAGE, flags, http=http)
Then you can run
python quickstart.py