My aim is to create an AWS-AppSync GraphQL API that implements the Relay Cursor Connections Specification. Specifically I want to add opaque cursors for some Elasticsearch queries. My cursor would be the sort values array for the search_after API. But to make them opaque I would like to stringify and base64 encode them before sending them to the client.
I know that AppSync exposes base64 encode and decode through its $util object. But decode returns a type of byte[] and not String. So my Question boils down to how to convert byte[] to String in Velocity?
Can we use the underlying Java for something like this:
new String(bytes)
Unfortunately I don't know which classes are in the AppSync Velocity Context.
Thanks in advance :)
Have you tried Base64.getEncoder().encodeToString(bytes)?
Related
Is it possible to access the raw url using AWS API Gateway (and Lambda)?
Alternatively, is it possible to access the original, undecoded query string paramters?
We are integrating against a third party service, that calls our API and encodes the query string params from Windows-1252. (E.g. the finnish letter Ä is encoded as %C4 instead of %C3%84). API Gateway seems to automatically decode the query string parameters and assume UTF-8, which means, that Ä (and Ö and Å) result in \ufffd.
For reference: https://www.w3schools.com/tags/ref_urlencode.asp
Damn, it really doesn't look possible...
I started off writing how you can use Lambda Proxy Integration with event.queryStringParameters, but that parses the data into a key-value object.
Then I went down the road of Mapping Templates in API Gateway, but again there doesn't seem to be any property that shows the whole querystring.
As much as I didn't want it to be true, I can only conclude that it is not possible...
I think your best option is to encode the parameter as base64 on the client, then decode in the Lambda function using Object.keys(event.queryStringParameters)[0].
I have an iOS app that calls a AWS Lambda function. I would like the Lambda function to get some files from a server and send it back to the iOS app, through the Lambda proxy feature.
I am directly invoking the Lambda function from my app using the generated SDK. I couldn't find a documentation explaining how to exchange data other than JSON encoded requests.
How should I go about this?
Lambda's only interface to the outside world is JSON.
To return text data from Lambda, you have to return it as a string, and deserialize the string from the JSON response.
To return binary data from Lambda, the data must be first transformed (encoded) using an encoding that cannot ever produce a series of bytes that is not also a valid sequence of UTF-8 characters, because JSON cannot serialize non-character data (and not every possible combination of bytes corresponds to a valid character or characters). The common strategy for doing this is to use base-64 encoding. Base-64 losslessly converts a series of bytes (octets) into a different series of bytes that are also always valid 7 bit ASCII characters, using an 8:6 conversion ratio (coding 6 bits per byte). You would then need to decode this data back from base-64 to binary.
You can do this decoding of JSON and base-64 on the client, but there are also a couple of other options, if you don't like that idea.
Both API Gateway and CloudFront's Lambda#Edge feature provide a built-in conversion capability to convert the base-64 payload (from the JSON Lambda response) back to binary, if you don't want to do it on the client.
API Gateway supports all Lambda runtimes, and expects this format...
"isBase64Encoded": true,
"body": "the-base-64-encoded-body",
Lambda#Edge supports only Node.js Lambda functions, but is less expensive than API Gateway. It expects a base-64 response to include...
"body": "the-base-64-encoded-body",
"bodyEncoding": "base64",
The viability of either approach depends on your security needs. API Gateway supports authentication via IAM as well as other mechanisms. CloudFront + Lambda#Edge doesn't support IAM auth but can be used with CloudFront signed URLs or Cognito or other custom authorization mechanisms.
If the "files" you mentioned are coming from a server, API Gateway can also proxy these files directly from the server, without a Lambda function handling the content (though depending on your security needs, a Lambda Custom Authorizer might be desirable, to authenticate the request but then simply tell API Gateway to allow the request to be forwarded to the backend).
Or, if the files are objects from S3, then you can access S3 directly, similiar to the way you are accessing Lambda, now.
My thing would publish string payload something like "[v1:ThingName]" to a topic (someTopic/topic1). And I have a rule applied in the rule engine to capture this topic and send to a Lambda function.
SELECT * FROM 'someTopic/+'
I want to send the topic1 part which will be captured from Rule engine along with the payload. So it can be captured from topic(2).
How do I concatenate topic(2) with * in select statement?
After so much of trial and error, figured that you cannot do it like that because payload is string.
But how I got away with this was using the encode function. Essentially I encode the whole payload into base64 as a json object and then have the topic(2) value as well.
So my Rule SQL query would like this.
SELECT encode(*, 'base64') as encode, topic(2) as topic FROM 'someTopic/+'
Subsequently you decode the payload in the Lambda function.
Frankly i'm not sure if this feature exist?sorry for that
My requirement is to send spark analysed data to file server on daily basis, file server supports file transfer through SFTP and REST Webservice post call.
Initial thought was to save Spark RDD to HDFS and transfer to fileserver through SFTP.
I would like to know is it possible to upload the RDD directly by calling REST service from spark driver class without saving to HDFS.
Size of the data is less than 2MB
Sorry for my bad english!
There is no specific way to do that with Spark. With that kind of data size it will not be worth it to go through HDFS or another type of storage. You can collect that data in your driver's memory and send it directly. For a POST call you can just use plain old java.net.URL, which would look something like this:
import java.net.{URL, HttpURLConnection}
// The RDD you want to send
val rdd = ???
// Gather data and turn into string with newlines
val body = rdd.collect.mkString("\n")
// Open a connection
val url = new URL("http://www.example.com/resource")
val conn = url.openConnection.asInstanceOf[HttpURLConnection]
// Configure for POST request
conn.setDoOutput(true);
conn.setRequestMethod("POST");
val os = conn.getOutputStream;
os.write(input.getBytes);
os.flush;
A much more complete discussion of using java.net.URL can be found at this question. You could also use a Scala library to handle the ugly Java stuff for you, like akka-http or Dispatch.
Spark itself does not provide this functionality (it is not a general-purpose http client).
You might consider using some existing rest client library such as akka-http, spray or some other java/scala client library.
That said, you are by no means obliged to save your data to disk before operating on it. You could for example use collect() or foreach methods on your RDD in combination with your REST client library.
I took a look through the documentation, but couldn't find it. Can anybody who has used the service answer this?
DynamoDB objects' attributes can be int, string, int set or string set.
JSONP is string representing a javascript code that can be parsed.
What is the use you intend for it? What object are you going to store?
Are you invoking an AJAX call to the dynamodb api to retrieve an element and you want that element to follow JSONP pattern? or you wish to call DynamoDB in your server (perhaps using a servlet) and prepare a JSONP object to be sent to the client