Swagger validation sensitivity when using boto - amazon-web-services
So I can use aws cli to import an API using its swagger yml file and it works after a bit of sensitivity, but then the same file fails when I try to do it using boto3
The code looks like this
client.import_rest_api(
failOnWarnings=False,
parameters={},
body="file://C:/somewhereinmyfilesystem/myvalidswagger.yml")
all the code is on the same line, I just broke it over to show on here.
Any ideas what's wrong with the command? The file works fine direct from the cli, and the error I get back when I run the python file is consistent.
Is "Invalid Swagger 2.0 input," the new "Object reference not set to an instance of an object"? Thanks for any help :-)
This method should be taking bytes or file as the input of the body.
In you put file://C:/somewhereinmyfilesystem/myvalidswagger.yml as the body, it will send file://C:/somewhereinmyfilesystem/myvalidswagger.yml directly to API Gateway.
You can load the content in the file into a local variable, and send as bytes or you can send a file object.
Sending file directly:
client.import_rest_api(
failOnWarnings=False,
parameters={},
body=open("C:/somewhereinmyfilesystem/myvalidswagger.yml", r))
Load into memory, then send:
with open('C:/somewhereinmyfilesystem/myvalidswagger.yml', 'r') as content_file:
content = content_file.read()
client.import_rest_api(
failOnWarnings=False,
parameters={},
body=content)
Related
Kinesis put records not returned in response from get records request
I have a Scala app using the aws-java-sdk-kinesis to issue a series of putRecord requests to a local kinesis stream.
The response returned after each putRecord request indicates its successfully putting the records into the stream.
The scala code making the putRecordRquest:
def putRecord(kinesisClient: AmazonKinesis, value: Array[Byte], streamName: String): Try[PutRecordResult] = Try {
val putRecordRequest = new PutRecordRequest()
putRecordRequest.setStreamName(streamName)
putRecordRequest.setData(ByteBuffer.wrap(value))
putRecordRequest.setPartitionKey("integrationKey")
kinesisClient.putRecord(putRecordRequest)
}
To confirm this I have a small python app that basically consumes from the stream (initialStreamPosition: LATEST). And prints the records it finds by iterating through the shard-iterators. But unexpectedly however it returns an empty set of records for each obtained shardIterator.
Trying this using the aws cli tool, I do however get records returned for the same shardIterator. I am confused? How can that be?
Running the python consumer (with LATEST), returns:
Shard-iterators: ['AAAAAAAAAAH9AUYVAkOcqkYNhtibrC9l68FcAQKbWfBMyNGko1ypHvXlPEuQe97Ixb67xu4CKzTFFGoLVoo8KMy+Zpd+gpr9Mn4wS+PoX0VxTItLZXxalmEfufOqnFbz2PV5h+Wg5V41tST0c4X0LYRpoPmEnnKwwtqwnD0/VW3h0/zxs7Jq+YJmDvh7XYLf91H/FscDzFGiFk6aNAVjyp+FNB3WHY0d']
Records: []
If doing the "same" with the aws cli tool however I get:
> aws kinesis get-records --shard-iterator AAAAAAAAAAH9AUYVAkOcqkYNhtibrC9l68FcAQKbWfBMyNGko1ypHvXlPEuQe97Ixb67xu4CKzTFFGoLVoo8KMy+Zpd+gpr9Mn4wS+PoX0VxTItLZXxalmEfufOqnFbz2PV5h+Wg5V41tST0c4X0LYRpoPmEnnKwwtqwnD0/VW3h0/zxs7Jq+YJmDvh7XYLf91H/FscDzFGiFk6aNAVjyp+FNB3WHY0d --endpoint-url http://localhost:4567
Returns:
{"Records":[{"SequenceNumber":"49625122979782922897342908653629584879579547704307482626","ApproximateArrivalTimestamp":1640263797.328,"Data":{"type":"Buffer","data":[123,34,116,105,109,101,115,116,97,109,112,34,58,49,54,52,48,50,54,51,55,57,55,44,34,100,116,109,34,58,49,54,52,48,50,54,51,55,57,55,44,34,101,34,58,34,101,34,44,34,116,114,97,99,107,101,114,95,118,101,114,115,105,111,110,34,58,34,118,101,114,115,105,111,110,34,44,34,117,114,108,34,58,34,104,116,116,112,115,58,47,47,116,101,115,116,46,99,111,109,34,44,34,104,99,99,34,58,102,97,108,115,101,44,34,115,99,34,58,49,44,34,99,111,110,116,101,120,116,34,58,123,34,101,116,34,58,34,101,116,34,44,34,100,101,118,34,58,34,100,101,118,34,44,34,100,119,101,108,108,34,58,49,44,34,111,105,100,34,58,49,44,34,119,105,100,34,58,49,44,34,115,116,97,116,101,34,58,123,34,108,99,34,58,123,34,99,111,100,101,34,58,34,115,111,109,101,45,99,111,100,101,34,44,34,105,100,34,58,34,115,111,109,101,45,105,100,34,125,125,125,44,34,121,117,105,100,34,58,34,102,53,101,52,57,53,98,102,45,100,98,102,100,45,52,102,53,102,45,56,99,56,98,45,53,97,56,98,50,56,57,98,52,48,49,97,34,125]},"PartitionKey":"integrationKey"},{"SequenceNumber":"49625122979782922897342908653630793805399163707871723522","ApproximateArrivalTimestamp":1640263817.338,"Data":{"type":"Buffer","data":[123,34,116,105,109,101,115,116,97,109,112,34,58,49,54,52,48,50,54,51,56,49,55,44,34,100,116,109,34,58,49,54,52,48,50,54,51,56,49,55,44,34,101,34,58,34,101,34,44,34,116,114,97,99,107,101,114,95,118,101,114,115,105,111,110,34,58,34,118,101,114,115,105,111,110,34,44,34,117,114,108,34,58,34,104,116,116,112,115,58,47,47,116,101,115,116,46,99,111,109,34,44,34,104,99,99,34,58,102,97,108,115,101,44,34,115,99,34,58,49,44,34,99,111,110,116,101,120,116,34,58,123,34,101,116,34,58,34,101,116,34,44,34,100,101,118,34,58,34,100,101,118,34,44,34,100,119,101,108,108,34,58,49,44,34,111,105,100,34,58,49,44,34,119,105,100,34,58,49,44,34,115,116,97,116,101,34,58,123,34,108,99,34,58,123,34,99,111,100,101,34,58,34,115,111,109,101,45,99,111,100,101,34,44,34,105,100,34,58,34,115,111,109,101,45,105,100,34,125,125,125,44,34,121,117,105,100,34,58,34,102,53,101,52,57,53,98,102,45,100,98,102,100,45,52,102,53,102,45,56,99,56,98,45,53,97,56,98,50,56,57,98,52,48,49,97,34,125]},"PartitionKey":"integrationKey"},{"SequenceNumber":"49625122979782922897342908653632002731218779711435964418","ApproximateArrivalTimestamp":1640263837.347,"Data":{"type":"Buffer","data":[123,34,116,105,109,101,115,116,97,109,112,34,58,49,54,52,48,50,54,51,56,51,55,44,34,100,116,109,34,58,49,54,52,48,50,54,51,56,51,55,44,34,101,34,58,34,101,34,44,34,116,114,97,99,107,101,114,95,118,101,114,115,105,111,110,34,58,34,118,101,114,115,105,111,110,34,44,34,117,114,108,34,58,34,104,116,116,112,115,58,47,47,116,101,115,116,46,99,111,109,34,44,34,104,99,99,34,58,102,97,108,115,101,44,34,115,99,34,58,49,44,34,99,111,110,116,101,120,116,34,58,123,34,101,116,34,58,34,101,116,34,44,34,100,101,118,34,58,34,100,101,118,34,44,34,100,119,101,108,108,34,58,49,44,34,111,105,100,34,58,49,44,34,119,105,100,34,58,49,44,34,115,116,97,116,101,34,58,123,34,108,99,34,58,123,34,99,111,100,101,34,58,34,115,111,109,101,45,99,111,100,101,34,44,34,105,100,34,58,34,115,111,109,101,45,1pre05,100,34,125,125,125,44,34,121,117,105,100,34,58,34,102,53,101,52,57,53,98,102,45,100,98,102,100,45,52,102,53,102,45,56,99,56,98,45,53,97,56,98,50,56,57,98,52,48,49,97,34,125]},"PartitionKey":"integrationKey"}],"NextShardIterator":"AAAAAAAAAAE+9W/bI4CsDfzvJGN3elplafFFBw81/cVB0RjojS39hpSglW0ptfsxrO6dCWKEJWu1f9BxY7OZJS9uUYyLn+dvozRNzKGofpHxmGD+/1WT0MVYMv8tkp8sdLdDNuVaq9iF6aBKma+e+iD079WfXzW92j9OF4DqIOCWFIBWG2sl8wn98figG4x74p4JuZ6Q5AgkE41GT2Ii2J6SkqBI1wzM","MillisBehindLatest":0}
The actual python consumer I have used in many other settings to introspec other kinesis streams we have and its working as expected. But for some reason here its not working.
Does anyone have a clue what might be going on here?
So I was finally able to identify the issue, and perhaps it will be useful for someone else with similar problem. In my setup, I am using a local kinesis stream (kinesalite) which doesn't support CBOR. You have to disable this explicitly otherwise I was seeing the following error when trying to deserialize the received record. Unable to unmarshall response (We expected a VALUE token but got: START_OBJECT). Response Code: 200, Response Text: OK In my case, setting the environment variable: AWS_CBOR_DISABLE=1 did the trick
Making An HTTP PUT through BrightScript to AWS S3 Bucket with pre-signed url
I've set up an AWS api which obtainins a pre-signed URL for uploading to an AWS S3 bucket. The pre-signed url has a format like https://s3.amazonaws.com/mahbukkit/background4.png?AWSAccessKeyId=someaccesskeyQ&Expires=1513287500&x-amz-security-token=somereallylongtokenvalue where backgournd4.png would be the file I'm uploading. I can successfully use this URL through Postman By: configuring it as a PUT call, setting the body to Binary so I can select the file, setting the header to Content-Type: image/png HOWEVER, I'm trying to make this call using BrightScript running on a BrightSign player. I'm pretty sure I'm supposed to be using the roURTransfer object and PutFromFile function described in this doucmentation: http://docs.brightsign.biz/display/DOC/roUrlTransfer Unfortunately, I can't find any good working examples showing how to do this. Could anyone who has experience with BrightScript help me out? I'd really appreciate it.
you are on the right track.
i would do
sub main()
tr = createObject("roUrlTransfer")
headers = {}
headers.addreplace("Content-Type","image/png")
tr.AddHeaders(headers)
info = {}
info.method = "PUT"
info.request_body_file = <fileName>
if tr.AsyncMethod(info)
print "File put Started"
else
print "File put did not start"
end if
delay(100000)
end sub()
note i have used two different methods to populate the two associative arrays. you need to use the addreplace method (rather then the shortcut of .) when the key contains special characters like '-'
this script should work , though i don't have a unit on hand to do a syntax check.
also you should set up a message port etc and Listen to the event that is generated to confirm if the put was successful and/or what the response code is.
note when you read responses from url events. if the response code from the server is anything other then 200 the BrightSign will trash the response body and you can not read it. This is not helpful as services like dropbox like to do a 400 response with more info on what was wrong (bad API key etc) in the body. so in that case you are left in the dark doing trial and error to figure out what was wrong.
good luck, sorry i didn't see this question sooner.
boto3 publish message SNS
I think i am trying to do something really simple but I am doing something wrong in some point or missing something. Goal: Send a message from python 2.7 using the boto3 (aws sdk library) I have the application Key, registrationId, etc...and my code is response = client.publish( TargetArn=platform_endpoint['EndpointArn'], Message="Hi there" ) Then i receive the messageId from aws, and the notification popup in the terminal movil, but always empty, without any text, "Hi there" in this case. I tried to use the aws SNS console and works, the phone receive the notification with text normally. I also tried to send a JSON, but same result, and if the easiest goal is failing...better fix this to go for JSON :p Any suggestion is welcome
Finally the solution was use json.dumps to convert the JSON to string response = client.publish( TargetArn=platform_endpoint['EndpointArn'], Message=json.dumps(jsonObj), MessageStructure='json' )
How to read file from WSO2 ESB without using poll interval
I am able to read the file from specified READFILEURI path if I can specify the poll interval. My current requirement is to read the file from given path only when we trigger the proxy service.It should not poll the file automatically.When I click on "try this service" and send the request then only the file should read from the read file path and do processing. TO test this I have removed the "transport.vfs.pollInterval" attribute from my proxy configuration and tested it.But file is not getting read from the path once I trigger the proxy request. Could some one help me how to achieve this.
I guess it should work if you use the file connector. https://docs.wso2.com/display/ESBCONNECTORS/Working+with+the+File+Connector You should be able to create a simple proxy that get's triggered via http/https and the uses the above mentioned file connector read operation to read the file. Unfortunately I cannot give you an exact example because we're still on an older esb version where this connector isn't available. Hope that helps. Regards Martin
Amazon S3 Client setReadLimit
While uploading file to S3 , we are getting this random error msg for one single case "If the request involves an input stream, the maximum stream buffer size can be configured via request.getRequestClientOptions().setReadLimit(int)" source being : https://github.com/aws/aws-sdk-java/blob/master/aws-java-sdk-s3/src/main/java/com/amazonaws/services/s3/AmazonS3Client.java As per AWS SDK for Java 1.8.10 We can set maximum stream buffer size to be configured per request via request.getRequestClientOptions().setReadLimit(int) We are using com.amazonaws.services.s3.AmazonS3 object to upload data. Can anyone suggest how we can set ReadLimit() via com.amazonaws.services.s3.AmazonS3 https://aws.amazon.com/releasenotes/0167195602185387
It sounds like you're uploading data from an InputStream, but some sort of transient error is interrupting the upload. The SDK isn't able to retry the request because InputStreams are mark/resetable by default. The error message is trying to give guidance on buffer sizes, but for large data, you probably don't want to load it all into memory anyway. If you're able to upload from a File source, then you shouldn't see this error again. Because Files are resettable, the SDK is able to retry your request if it encounters an error during the first attempt.
A little bit necroing, but you need to create a PutObjectRequest and use the setReadLimit on that: PutObjectRequest putObjectRequest = new PutObjectRequest(bucketName, key, fileInputStream, objectMetadata); putObjectRequest.getRequestClientOptions().setReadLimit(xxx); s3Client.putObject(putObjectRequest); If you look in the implementation of the putObjectRequest(String, String, InputStream, ObjectMetadata), you can see that it just creates a PutObjectRequest and passes that to putObject(PutObjectRequest)