JItterBit HTTP Endpoint - endpoint

I am working to set up a HTTP Endpoint in JitterBit, for this end point we have a system that will call this Endpoint and pass parameters through the URL to it.
example...
http://[server]:[server port]/EndPoint?Id={SalesForecID}&Status={updated status in SF}
Would i need to use the Text File, JSON or XML Method for this? Follow up question would be if it is JSON or XML what would the file look like that is uploaded during creating the endpoint. I have tired with no success with the text file version.
any help would be great.

I'm just seeing your question now. You may have found a solution, but this took me a while to figure out, so I'll respond anyway.
To get the passed values, go ahead and create your HTTP Endpoint and add a new operation triggered by it. Then, in your new operation create a script with something like the following:
$SalesForceID = $jitterbit.networking.http.query.Id
$UpdatedStatus = $jitterbit.networking.http.query.Status
You can then use these variables elsewhere in your operation chain.
If you want to use these values to feed into another RESTful web service (i.e. an HTTP Source), you'll have to create a separate transformation operation with the HTTP Source. You'd set that source URL to be: http://mysfapp.com/call?Id=[SalesForceID]&Status=[UpdatedStatus]. I'm not sure why, but you can't have the script that extracts the parameters from the Endpoint and the HTTP Source that uses those in the same operation.
Cheers

Related

Google CDF: Can we set the value of a column as a runtime argument?

I am getting a value returned by hitting a HTTP endpoint which I am storing in a column. Now a want to trigger another Http Endpoint with the value in the column. But the HTTP endpoint takes hardcoded values or macros only. So I want to know if I can set a run-time argrument based on the column value
Please suggest How I can do this.
There are few ways of setting runtime arguments:
Using Argument Setter Plugin - type of Action plugin that allows one to create reusable pipelines by dynamically substituting the configurations that can be served by an HTTP Server.
Specifying runtime arguments as a JSON map in the request body, when starting an program.
CDAP Preferences HTTP RESTful API.
I recommend you check the official documentation for Datafusion, where you can find a way of setting variables/macros using the GCS bucket.
Additionally, have a look at the following thread on SO. It describes the method with specifying runtime arguments as a JSON map in the request body.

How to facilitate downloading both CSV and PDF from API Gateway connected to S3

In the app I'm working on, we have a process whereby a user can download a CSV or PDF version of their data. The generation works great, but I'm trying to get it to download the file and am running into all sorts of problems. We're using API Gateway for all the requests, and the generation happens inside a Lambda on a POST request. The GET endpoint takes in a file_name parameter and then constructs the path in S3 and then makes the request directly there. The problem I'm having is when I'm trying to transform the response. I get a 500 error and when I look at the logs, it says Execution failed due to configuration error: Unable to transform response. So, clearly that's where I've spent most of my time. I've tried at least 50 different iterations of templates and combinations with little success. The closest I've gotten is the following code, where the CSV downloads fine, but the PDF is not a valid PDF anymore:
CSV:
#set($contentDisposition = "attachment;filename=${method.request.querystring.file_name}")
$input.body
#set($context.responseOverride.header.Content-Disposition = $contentDisposition)
PDF:
#set($contentDisposition = "attachment;filename=${method.request.querystring.file_name}")
$util.base64Encode($input.body)
#set($context.responseOverride.header.Content-Disposition = $contentDisposition)
where contentHandling = CONVERT_TO_TEXT. My binaryMediaTypes just has application/pdf and that's it. My goal is to get this working without having to offload the problem into a Lambda so we don't have that overhead at the download step. Any ideas how to do this right?
Just as another comment, I've tried CONVERT_TO_BINARY and just leaving it as Passthrough. I've tried it with text/csv as another binary media type and I've tried different combinations of encoding and decoding base64 and stuff. I know the data is coming back right from S3, but the transformation is where it's breaking. I am happy to post more logs if need be. Also, I'm pretty sure this makes sense on StackOverflow, but if it would fit in another StackExchange site better, please let me know.
Resources I've looked at:
https://docs.aws.amazon.com/apigateway/latest/developerguide/request-response-data-mappings.html
https://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-mapping-template-reference.html#util-template-reference
https://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-payload-encodings-workflow.html
https://docs.amazonaws.cn/en_us/apigateway/latest/developerguide/api-gateway-payload-encodings-configure-with-control-service-api.html.
(But they're all so confusing...)
EDIT: One Idea I've had is to do CONVERT_TO_BINARY and somehow base64 encode the CSVs in the transformation, but I can't figure out how to do it right. I keep feeling like I'm misunderstanding the order of things, specifically when the "CONVERT" part happens. If that makes any sense.
EDIT 2: So, I got rid of the $util.base64Encode in the PDF one and now I have a PDF that's empty. The actual file in S3 definitely has things in it, but for some reason CONVERT_TO_TEXT is not handling it right or I'm still not understading how this all works.
Had similar issues. One major thing is the Accept header. I was testing in chrome which sends Accept header as text/html,application/xhtml.... api-gateway ignores everything except the first one(text/html). It will then convert any response from S3 to base64 to try and conform to text/html.
At last after trying everything else I tried via Postman which defaults the Accept header to */*. Also set your content handling on the Integration response to Passthrough. And everything was working!
One other thing is to pass the Content-Type and Content-Length headers through(Add them in method response first and then in Integration response):
Content-Length integration.response.header.Content-Length
Content-Type integration.response.header.Content-Type

How to modify a query parameter in AWS API Gateway in the integration request for an HTTP Proxy

I have a client facing API that takes a query parameter Time. The format is 14:00:00. Originally it would pass through this query parameter to the back end endpoint. However the back end endpoint (that I do not control) is now expecting time in the format 0001-01-01T14:00:00.
Is it possible to to modify the value of the query param before passing it on in AWS API Gateway?
I know you can modify the request body with a mapping template, and in the template you can access the queryParameters, but can you change them so that it modifies the actual request made to the back end?
I saw this:
https://forums.aws.amazon.com/thread.jspa?messageID=696524&#696524
but the user said he gave up trying to modify
Potential workaround I can think of right now are pass the parameters to a lambda and have the lambda build and make the request with modified values, with the response as the return value for the lambda
For now one can overrides query string in Mapping Templates using velocity templates e.g:
$context.requestOverride.querystring.time="_your_transformed_data_"
There is docs
I think you basically answered your own question :)
There is no way to transform query or header request parameters. All transformations need to happen in the body mapping template.
Best workaround would be to forward request on to a Lambda function to massage the parameters into the expected shape

Consuming RSS feed with AWS Lambda and API Gateway

I'm a newbie rails programmer, and I have even less experience with all the AWS products. I'm trying to use lambda to subscribe to and consume an rss feed from youtube. I am able to send the subscription request just fine with HTTParty from my locally hosted rails app:
query = {'hub.mode':'subscribe', 'hub.verify':'sync', 'hub.topic': 'https://www.youtube.com/feeds/videos.xml?channel_id=CHANNELID', 'hub.callback':'API Endpoint for Lambda'}
subscribe = 'HTTParty.post(https://pubsubhubbub.appspot.com/subscribe, :query=>query)
and it will ping the lambda function with a get request. I know that I need to echo back a hub.challenge string, but I don't know how. The lambda event is empty, I didn't see anything useful in the context. I tried formatting the response in the API gateway but that didn't work either. So right now when I try to subscribe I get back a 'Challenge Mismatch' error.
I know this: https://pubsubhubbub.googlecode.come/git/pubsubhubbub-core-0.3.html#subscribing explains what I'm trying to do better than what I just did, and section 6.2.1 is where the breakdown is. How do I set up either the AWS Lambda function and/or the API Gateway to reflect back the 'hub.challenge' verification token string?
You need to use the parameter mapping functionality of API Gateway to map the parameters from the incoming query string to a parameter passed to your Lambda function. From the documentation link you provided, it looks like you'll at least need to map the hub.challenge query string parameter, but you may also need the other parameters (hub.mode, hub.topic, and hub.verify_token) depending on what validation logic (if any) that you're implementing.
The first step is to declare your query string parameters in the method request page. Once you have declared the parameters open the integration request page (where you specify which Lambda function API Gateway should call) and use the "+" icon to add a new template. In the template you will have to specify a content type (application/json), and then the body you want to send to Lambda. You can read both query string and header parameters using the params() function. In that input mapping field you are creating the event body that is posted to AWS Lambda. For example: { "challenge": "$input.params('hub.challenge')" }
Documentation for mapping query string parameters

How To Pass Different Parameters To web Service Every Time It Is Called With Jmeter

I am testing web service using Jmeter, web service has several methods which takes some parameters. What I need to do is to pass different parameters every time user (thread) "calls" web service method.
I know, that I can do something like that, if I write Soap Messages in xml files and then to "give" Jmeter path of folder including this xml files, but Jmeter will take randomly those files and there is probability to use the same file twice or more. But I want Jmeter to use every request time different unique Soap Message.
Can Anyone help me?
Use CSV Data Set Config, prepare csv file containing your parameters.
Set CSV config recycle on EOF to false.
Set CSV config stop thread on EOF to true.
Set CSV config Sharing mode to all threads.