I am getting a value returned by hitting a HTTP endpoint which I am storing in a column. Now a want to trigger another Http Endpoint with the value in the column. But the HTTP endpoint takes hardcoded values or macros only. So I want to know if I can set a run-time argrument based on the column value
Please suggest How I can do this.
There are few ways of setting runtime arguments:
Using Argument Setter Plugin - type of Action plugin that allows one to create reusable pipelines by dynamically substituting the configurations that can be served by an HTTP Server.
Specifying runtime arguments as a JSON map in the request body, when starting an program.
CDAP Preferences HTTP RESTful API.
I recommend you check the official documentation for Datafusion, where you can find a way of setting variables/macros using the GCS bucket.
Additionally, have a look at the following thread on SO. It describes the method with specifying runtime arguments as a JSON map in the request body.
Related
Is there a simple way to retrieve all items from a DynamoDB table using a mapping template in an API Gateway endpoint? I usually use a lambda to process the data before returning it but this is such a simple task that a Lambda seems like an overkill.
I have a table that contains data with the following format:
roleAttributeName roleHierarchyLevel roleIsActive roleName
"admin" 99 true "Admin"
"director" 90 true "Director"
"areaManager" 80 false "Area Manager"
I'm happy with getting the data, doesn't matter the representation as I can later transform it further down in my code.
I've been looking around but all tutorials explain how to get specific bits of data through queries and params like roles/{roleAttributeName} but I just want to hit roles/ and get all items.
All you need to do is
create a resource (without curly braces since we dont need a particular item)
create a get method
use Scan instead of Query in Action while configuring the integration request.
Configurations as follows :
enter image description here
now try test...you should get the response.
to try it out on postman deploy the api first and then use the provided link into postman followed by your resource name.
API Gateway allows you to Proxy DynamoDB as a service. Here you have an interesting tutorial on how to do it (you can ignore the part related to index to make it work).
To retrieve all the items from a table, you can use Scan as the action in API Gateway. Keep in mind that DynamoDB limits the query sizes to 1MB either for Scan and Query actions.
You can also limit your own query before it is automatically done by using the Limit parameter.
AWS DynamoDB Scan Reference
How can I set stageVariables via request headers or query string
parameters in API Gateway?
I have added a stageVariable to the lambda function name in the integration request. When I set the stage variable fixed in the stage this does work.
However, when I try to set the variable via a query string I get a server error.
How can I set a lambda version in an API request dynamically?
I think Stage Variables are just that - they vary on stage, and nothing else. If there's a way of mapping a query parameter into a stage variable I'd like to know about it - but I think it's not possible.
I have a client facing API that takes a query parameter Time. The format is 14:00:00. Originally it would pass through this query parameter to the back end endpoint. However the back end endpoint (that I do not control) is now expecting time in the format 0001-01-01T14:00:00.
Is it possible to to modify the value of the query param before passing it on in AWS API Gateway?
I know you can modify the request body with a mapping template, and in the template you can access the queryParameters, but can you change them so that it modifies the actual request made to the back end?
I saw this:
https://forums.aws.amazon.com/thread.jspa?messageID=696524򪃌
but the user said he gave up trying to modify
Potential workaround I can think of right now are pass the parameters to a lambda and have the lambda build and make the request with modified values, with the response as the return value for the lambda
For now one can overrides query string in Mapping Templates using velocity templates e.g:
$context.requestOverride.querystring.time="_your_transformed_data_"
There is docs
I think you basically answered your own question :)
There is no way to transform query or header request parameters. All transformations need to happen in the body mapping template.
Best workaround would be to forward request on to a Lambda function to massage the parameters into the expected shape
I'm a newbie rails programmer, and I have even less experience with all the AWS products. I'm trying to use lambda to subscribe to and consume an rss feed from youtube. I am able to send the subscription request just fine with HTTParty from my locally hosted rails app:
query = {'hub.mode':'subscribe', 'hub.verify':'sync', 'hub.topic': 'https://www.youtube.com/feeds/videos.xml?channel_id=CHANNELID', 'hub.callback':'API Endpoint for Lambda'}
subscribe = 'HTTParty.post(https://pubsubhubbub.appspot.com/subscribe, :query=>query)
and it will ping the lambda function with a get request. I know that I need to echo back a hub.challenge string, but I don't know how. The lambda event is empty, I didn't see anything useful in the context. I tried formatting the response in the API gateway but that didn't work either. So right now when I try to subscribe I get back a 'Challenge Mismatch' error.
I know this: https://pubsubhubbub.googlecode.come/git/pubsubhubbub-core-0.3.html#subscribing explains what I'm trying to do better than what I just did, and section 6.2.1 is where the breakdown is. How do I set up either the AWS Lambda function and/or the API Gateway to reflect back the 'hub.challenge' verification token string?
You need to use the parameter mapping functionality of API Gateway to map the parameters from the incoming query string to a parameter passed to your Lambda function. From the documentation link you provided, it looks like you'll at least need to map the hub.challenge query string parameter, but you may also need the other parameters (hub.mode, hub.topic, and hub.verify_token) depending on what validation logic (if any) that you're implementing.
The first step is to declare your query string parameters in the method request page. Once you have declared the parameters open the integration request page (where you specify which Lambda function API Gateway should call) and use the "+" icon to add a new template. In the template you will have to specify a content type (application/json), and then the body you want to send to Lambda. You can read both query string and header parameters using the params() function. In that input mapping field you are creating the event body that is posted to AWS Lambda. For example: { "challenge": "$input.params('hub.challenge')" }
Documentation for mapping query string parameters
I am working to set up a HTTP Endpoint in JitterBit, for this end point we have a system that will call this Endpoint and pass parameters through the URL to it.
example...
http://[server]:[server port]/EndPoint?Id={SalesForecID}&Status={updated status in SF}
Would i need to use the Text File, JSON or XML Method for this? Follow up question would be if it is JSON or XML what would the file look like that is uploaded during creating the endpoint. I have tired with no success with the text file version.
any help would be great.
I'm just seeing your question now. You may have found a solution, but this took me a while to figure out, so I'll respond anyway.
To get the passed values, go ahead and create your HTTP Endpoint and add a new operation triggered by it. Then, in your new operation create a script with something like the following:
$SalesForceID = $jitterbit.networking.http.query.Id
$UpdatedStatus = $jitterbit.networking.http.query.Status
You can then use these variables elsewhere in your operation chain.
If you want to use these values to feed into another RESTful web service (i.e. an HTTP Source), you'll have to create a separate transformation operation with the HTTP Source. You'd set that source URL to be: http://mysfapp.com/call?Id=[SalesForceID]&Status=[UpdatedStatus]. I'm not sure why, but you can't have the script that extracts the parameters from the Endpoint and the HTTP Source that uses those in the same operation.
Cheers