My task is to write a REST WebService that can take a csv. This csv should then be modified via SpringBatch. I either worked with WebServices nor did i work with SpringBatch.
Can you give me a short intro of how to achieve my task?
Short example on rest webservice you can find here
You can get spring batch exampe here. You can use merge example 2 and example 4 of itemreader and itemwriter and itemprocessor.
Refer here to call spring batch from rest service layer
Related
My pipeline is IoTCore -> Pub/Sub -> Dataflow -> BigQuery. Initially the data I was getting was Json format and the pipeline was working properly. Now I need to shift to csv and the issue is the Google defined dataflow template I was using uses Json input instead of csv. Is there an easy way of transfering csv data from pub/sub to bigquery through dataflow. The template can probably be changed but it is implemented in Java which I have never used so would take a long time to implement. I also considered implementing an entire custom template in python but that would take too long.
Here is a link to the template provided by google:
https://github.com/GoogleCloudPlatform/DataflowTemplates/blob/master/src/main/java/com/google/cloud/teleport/templates/PubSubToBigQuery.java
Sample: Currently my pub/sub messages are JSON and these work correctly
"{"Id":"123","Temperature":"50","Charge":"90"}"
But I need to change this to comma seperated values
"123,50,90"
Very easy: Do nothing!! If you have a look to this line you can see that the type of the messages used is the PubSub message JSON, not your content in JSON.
So, to prevent any issues (to query and to insert), write in another table and it should work nicely!
Can you please share your existing python code where you are parsing JSON format data and new & old data sample, So that it can be customized accordingly.
Moreover you can refer Python code here, it has performed word count transformation logic over PCollection, hopefully it can give you some refence to customize your code accrdingly.
I have a situation where I have added my testdata in csv sheet. Moreover, I have created automated tests in js and I am passing this csv data with csv at the time of Collection Runner. It is working fine with postman on my local but when I run these testcases on postman monitor(https://monitor.getpostman.com/) these testcases is failing. Surely because of csv file unavailability. Is there any way I can pass my csv to monitor mode?
Unfortunately there is no such functionality in place right now. You can use csv files with collection runs and directly via the newman cli tool as described here.
You can file a feature request for this and look at the current product roadmap here.
Is it possible to use the mapping technique with the dynamic input/output?. Because WSO2 documentation shows only the input/output from the file system.
Basically I would like to map output of one webserive as input of the other webservice
As long as you know the schema of the output and the expected schema for the input, this can be done. [1] provides a sample where an output from a web service is written to a csv file using the data mapper. In a similar manner if the schema is known of the input and the output your requirement can be achieved.
[1] http://chathurikaerandi.blogspot.com/2016/07/mapping-to-csv-and-writing-output-to.html
Thanks
Erandi
This can be solved by writing the custom sequence and use as a mediator in the WSO2 ESB service creation. No straight forward way yet
I am working to set up a HTTP Endpoint in JitterBit, for this end point we have a system that will call this Endpoint and pass parameters through the URL to it.
example...
http://[server]:[server port]/EndPoint?Id={SalesForecID}&Status={updated status in SF}
Would i need to use the Text File, JSON or XML Method for this? Follow up question would be if it is JSON or XML what would the file look like that is uploaded during creating the endpoint. I have tired with no success with the text file version.
any help would be great.
I'm just seeing your question now. You may have found a solution, but this took me a while to figure out, so I'll respond anyway.
To get the passed values, go ahead and create your HTTP Endpoint and add a new operation triggered by it. Then, in your new operation create a script with something like the following:
$SalesForceID = $jitterbit.networking.http.query.Id
$UpdatedStatus = $jitterbit.networking.http.query.Status
You can then use these variables elsewhere in your operation chain.
If you want to use these values to feed into another RESTful web service (i.e. an HTTP Source), you'll have to create a separate transformation operation with the HTTP Source. You'd set that source URL to be: http://mysfapp.com/call?Id=[SalesForceID]&Status=[UpdatedStatus]. I'm not sure why, but you can't have the script that extracts the parameters from the Endpoint and the HTTP Source that uses those in the same operation.
Cheers
I am testing web service using Jmeter, web service has several methods which takes some parameters. What I need to do is to pass different parameters every time user (thread) "calls" web service method.
I know, that I can do something like that, if I write Soap Messages in xml files and then to "give" Jmeter path of folder including this xml files, but Jmeter will take randomly those files and there is probability to use the same file twice or more. But I want Jmeter to use every request time different unique Soap Message.
Can Anyone help me?
Use CSV Data Set Config, prepare csv file containing your parameters.
Set CSV config recycle on EOF to false.
Set CSV config stop thread on EOF to true.
Set CSV config Sharing mode to all threads.