I'm trying to write my first code with Apache Camel right now. I try to follow the examples from Camel in Action, but I want to use my own example data.
What I want to do
Right now I want to read from a CSV file and get each line as a java bean.
Here is my junit test:
#Test
public void testCsvWithBindy() throws Exception {
MockEndpoint mock = getMockEndpoint("mock:queue.csv");
mock.expectedMessageCount(2);
assertMockEndpointsSatisfied();
CsvBean line1 = mock.getReceivedExchanges().get(0).getIn()
.getBody(CsvBean.class);
assertEquals("row 01", line1.getFirst());
}
public RouteBuilder createRoute() {
return new RouteBuilder() {
public void configure() throws Exception {
context.setTracing(true);
from("file://src/test/resources?noop=true&fileName=test.csv")
.unmarshal().bindy(BindyType.Csv, "my.package.for.csvrecord")
.to("mock:queue.csv");
}
};
}
The CSV contains this:
row 01,row 02,,row 04
row 11, row 12, row 13, row 14
And this is my CsvRecord:
#CsvRecord(separator = ",")
public class CsvBean {
#DataField(pos = 1)
private String first;
#DataField(pos = 2)
private String second;
#DataField(pos = 3)
private String third;
#DataField(pos = 4)
private String fourth;
public String getFirst() {
return first;
}
public void setFirst(String first) {
this.first = first;
}
public String getSecond() {
return second;
}
public void setSecond(String second) {
this.second = second;
}
public String getThird() {
return third;
}
public void setThird(String third) {
this.third = third;
}
public String getFourth() {
return fourth;
}
public void setFourth(String fourth) {
this.fourth = fourth;
}
}
My Problem
When I run this test, the context is started and the route is loaded. But nothing is coming through. After about 10s the context is automatically stopped and my test fails. This is the stacktrace:
java.lang.AssertionError: mock://queue.csv Received message count. Expected: <2> but was: <0>
at org.apache.camel.component.mock.MockEndpoint.fail(MockEndpoint.java:1086)
at org.apache.camel.component.mock.MockEndpoint.assertEquals(MockEndpoint.java:1068)
at org.apache.camel.component.mock.MockEndpoint.doAssertIsSatisfied(MockEndpoint.java:367)
at org.apache.camel.component.mock.MockEndpoint.assertIsSatisfied(MockEndpoint.java:346)
at org.apache.camel.component.mock.MockEndpoint.assertIsSatisfied(MockEndpoint.java:334)
at org.apache.camel.component.mock.MockEndpoint.assertIsSatisfied(MockEndpoint.java:172)
at org.apache.camel.test.junit4.CamelTestSupport.assertMockEndpointsSatisfied(CamelTestSupport.java:391)
at my.package.for.unittests.CsvToBeanWithBindyTest.testCsvWithBindy(CsvToBeanWithBindyTest.java:20)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
Need help with
I guess I'm missing something obvious, maybe something that has to do with the test setup and not so much with my CsvRecord or my route. Can you give me a tip or maybe an URL to a better tutorial? The book is not very helpful at this point... :-(
Again, right after posting my question, I found the answer myself. ;-) Here is a working junit test:
public class CsvToBeanWithBindyTest extends CamelTestSupport {
#Test
public void testCsv() throws Exception {
MockEndpoint mock = getMockEndpoint("mock:queue.csv");
mock.expectedMessageCount(1);
assertMockEndpointsSatisfied();
List line1 = (List) mock.getReceivedExchanges().get(0).getIn()
.getBody();
Map map1 = (Map) line1.get(0);
CsvBean csv1 = (CsvBean) map1.get("my.package.CsvBean");
assertEquals("row 01", csv1.getFirst());
Map map2 = (Map) line1.get(1);
CsvBean csv2 = (CsvBean) map2.get("my.package.CsvBean");
assertEquals("row 11", csv2.getFirst());
}
#Override
protected RouteBuilder createRouteBuilder() throws Exception {
return new RouteBuilder() {
#Override
public void configure() throws Exception {
context.setTracing(true);
from("file://src/test/resources?noop=true&fileName=test.csv")
.unmarshal(new BindyCsvDataFormat("my.package"))
.to("mock:queue.csv");
}
};
}
}
The unexpected thing for me is that I get a List from my endpoint route which in turn holds many Maps. Each map has a key my.package.MyBeanClass with the value set to the actual unmarshalled row from my CSV file.
Related
I am trying to use aws sdk2 java for s3 select operations but not able to get extract the final data. Looking for an example if someone has implemented it. I got some idea from [this post][1] but not able to figure out how to get and read the full data .
Fetching specific fields from an S3 document
Basically, equivalent of v1 sdk:
``` InputStream resultInputStream = result.getPayload().getRecordsInputStream(
new SelectObjectContentEventVisitor() {
#Override
public void visit(SelectObjectContentEvent.StatsEvent event)
{
System.out.println(
"Received Stats, Bytes Scanned: " + event.getDetails().getBytesScanned()
+ " Bytes Processed: " + event.getDetails().getBytesProcessed());
}
/*
* An End Event informs that the request has finished successfully.
*/
#Override
public void visit(SelectObjectContentEvent.EndEvent event)
{
isResultComplete.set(true);
System.out.println("Received End Event. Result is complete.");
}
}
);```
///IN AWS SDK2, how do get ResultOutputStream ?
```public byte[] getQueryResults() {
logger.info("V2 query");
S3AsyncClient s3Client = null;
s3Client = S3AsyncClient.builder()
.region(Region.US_WEST_2)
.build();
String fileObjKeyName = "upload/" + filePath;
try{
logger.info("Filepath: " + fileObjKeyName);
ListObjectsV2Request listObjects = ListObjectsV2Request
.builder()
.bucket(Constants.bucketName)
.build();
......
InputSerialization inputSerialization = InputSerialization.builder().
json(JSONInput.builder().type(JSONType.LINES).build()).build()
OutputSerialization outputSerialization = null;
outputSerialization = OutputSerialization.builder().
json(JSONOutput.builder()
.build()
).build();
SelectObjectContentRequest selectObjectContentRequest = SelectObjectContentRequest.builder()
.bucket(Constants.bucketName)
.key(partFilename)
.expression(query)
.expressionType(ExpressionType.SQL)
.inputSerialization(inputSerialization)
.outputSerialization(outputSerialization)
.scanRange(ScanRange.builder().start(0L).end(Constants.limitBytes).build())
.build();
final DataHandler handler = new DataHandler();
CompletableFuture future = s3Client.selectObjectContent(selectObjectContentRequest, handler);
//hold it till we get a end event
EndEvent endEvent = (EndEvent) handler.receivedEvents.stream()
.filter(e -> e.sdkEventType() == SelectObjectContentEventStream.EventType.END)
.findFirst()
.orElse(null);```
//Now, from here how do I get the response bytes ?
///////---> ISSUE: How do I get ResultStream bytes ????
return <bytes>
}```
// handler
private static class DataHandler implements SelectObjectContentResponseHandler {
private SelectObjectContentResponse response;
private List receivedEvents = new ArrayList<>();
private Throwable exception;
#Override
public void responseReceived(SelectObjectContentResponse response) {
this.response = response;
}
#Override
public void onEventStream(SdkPublisher<SelectObjectContentEventStream> publisher) {
publisher.subscribe(receivedEvents::add);
}
#Override
public void exceptionOccurred(Throwable throwable) {
exception = throwable;
}
#Override
public void complete() {
}
} ```
[1]: https://stackoverflow.com/questions/67315601/fetching-specific-fields-from-an-s3-document
i came to your post since I was working on the same issue as to avoid V1.
After hours of searching i ended up with finding the answer at. https://github.com/aws/aws-sdk-java-v2/pull/2943/files
The answer is located at SelectObjectContentIntegrationTest.java File
services/s3/src/it/java/software/amazon/awssdk/services/SelectObjectContentIntegrationTest.java
The way to get the bytes is by using the RecordsEvent class, please note for my use case I used CSV, not sure if this would be different for a different file type.
in the complete method you have access to the receivedEvents. this is where you get the first index to get the filtered returned results and casting it to the RecordsEvent class. then this class provides the payload as bytes
#Override
public void complete() {
RecordsEvent records = (RecordsEvent) this.receivedEvents.get(0)
String result = records.payload().asUtf8String();
}
testclass.java
#Test
public void testgetDictionaryValueListById() {
DictionaryValue dictionaryValue = new DictionaryValue();
dictionaryValue.setId(1);
dictionaryValue.setValueName("Test Dictionary Value");
dictionaryValue.setValueKey("12345678");
dictionaryValue.setStatus("Active");
dictionaryValue.setCreatedOn(new Date());
dictionaryValue.setUpdatedOn(new Date());
Mockito.when(dictionaryValueRepo.findById(1).get()).thenReturn(dictionaryValue);
assertThat(dictionaryService.getDictionaryValueListById(1)).isEqualTo(dictionaryValue);
}
Service.java
public DictionaryValue getDictionaryValueListById(int id) {
return dictionaryValueRepo.findById(id).get();
}
Repo.java
#Repository
public interface DictionaryValueRepo extends JpaRepository<DictionaryValue, Integer> {
}
I am getting no such value present again and again on executing test case in testclass.java. I don't know why? but when I am running my service method from the controller it is working as expected - fetching records from the database but not working in a test case.
Your test should be like this and please check out the naming. You need to Mock the step findId() befor the `get().
#InjectMocks
Service cut;
#Mock
DictionaryValueRepo dictionaryValueRepoMock;
// Can skipped by adding a #RunWith... on Testclass
#Before
public init() {
Mockito.initMocks(this);
}
#Test
public void testgetDictionaryValueListById() {
// Prepare Data
final int testId = 1;
DictionaryValue dictionaryValue = new DictionaryValue();
dictionaryValue.setId(testId);
dictionaryValue.setValueName("Test Dictionary Value");
dictionaryValue.setValueKey("12345678");
dictionaryValue.setStatus("Active");
dictionaryValue.setCreatedOn(new Date());
dictionaryValue.setUpdatedOn(new Date());
// config mocking
Mockito.when(dictionaryValueRepo.findById(testId)).thenReturn(<VALUE>);
Mockito.when(dictionaryValueRepo.findById(testId).get()).thenReturn(dictionaryValue);
// Call yout method for Testing
cut.getDictionaryValueListById(testId);
// verifies (if wanted) + assertions....
}
I concur with LenglBoy, so the right answer should be given to him.
The thing you need to be careful is what "VALUE" means in this line:
Mockito.when(dictionaryValueRepo.findById(testId)).thenReturn(VALUE);
The findById returns an Optional, so that is what you should build and pass to Mockito. Something like this:
Mockito.when(dictionaryValueRepo.findById(testId))
.thenReturn(Optional.ofNullable(dictionaryValue));
And for a scenario where the id does not exists in BD, passing Optional.empty() should be good enough.
I hope this scenario is bit confused me lot. I want to run a few test cases using junit or testng with different set of data from csv file. The code snippet i have tried is given below but it dint work,
private static CSVReader csvReader = null;
#BeforeClass
public static void setUp() {
csvReader = new CSVReader(new FileReader(fileName));
}
#Test
public void test1() {
.......
.......
System.out.println(csvReader[0]);
}
#Test
public void test2() {
.......
.......
System.out.println(csvReader[1]);
}
#Test
public void test3() {
.......
.......
System.out.println(csvReader[2]);
}
#Test
public void test4() {
.......
.......
System.out.println(csvReader[3]);
}
My problem is that i need to use data from each column in different test cases and i need to iterate all the test cases again if i have multiple rows in csv file. I have tried using Theories and Datapoints, but it works in a way that first cases runs with all rows in csv file and its moves to next test case and runs again with all rows in csv.
I want the solution to run test1() with first column of first row, test2() with second column of first row, test3() with third column of first row and test4() with fourth column of first row and then same need to be iterated with second row and so on. Is it possible to iterate the test cases like this ? As far i searched we can iterate a particular test cases in many ways. My question is, is this possible to iterate all the test in a class with one set of data and again reiterate the class with another set of data from csv.
Can we accomplish this using junit or testng? if so, please proved some sample code. Thanks in advance!
Well, there are parameterized tests... You could use them.
#RunWith(Parameterized.class)
public class YourTest {
#Parameters
public static Collection<Object[]> data() {
try( FileReader read = new FileReader(fileName)) {
CSVReader csvReader = new CSVReader(reader);
List<CSVRecord> records = ... read data;
Object[][] parameters = new Object[records.length][1];
for(int i=0; i<records.length; i++) {
parameters[i][0] = records.get(i);
}
return parameters;
}
}
private CsvRecord record; // [0] from the array goes here
public YourTest (CsvRecord record) {
this.record = record;
}
#Test
public void test() {
...do something with the record
}
}
And the TestNG solution is:
public class YourTest {
#DataProvider
public static Object[][] data() {
try( FileReader read = new FileReader(fileName)) {
CSVReader csvReader = new CSVReader(reader);
List<CSVRecord> records = ... read data;
Object[][] parameters = new Object[records.length][1];
for(int i=0; i<records.length; i++) {
parameters[i][0] = records.get(i);
}
return parameters;
}
}
#Test(dataProvider="data")
public void test(CsvRecord record) {
...do something with the record
}
}
Here is an example of my webservice:
#Override
#RequestMapping(value="{width}/{height}/{limit}", method = RequestMethod.POST)
public #ResponseBody ThumbnailResponse createThumbnails(#PathVariable int width, #PathVariable int height, #PathVariable int limit) {
List<String> list = thumbnailService.processImages(width,height,limit);
return new ThumbnailResponse(list.size(),list);
}
What I want to do is to answer a 400 BAD REQUEST to the user if the value of width or height is less than 10.
I need a kind of "REST Validator" before sending the response to the user. How can I do that ?
Thanks.
The approach I would follow is based on Spring MVC exception handling
What it means in practice is that:
You make a custom exception
Change your controller method accordingly
Catch the exception in your exception handler, and set the proper status and message
In terms of code it would mean
public class HeightNotAllowedException extends Exception {
}
Change in the controller
#Override
#RequestMapping(value="{width}/{height}/{limit}", method = RequestMethod.POST)
public #ResponseBody ThumbnailResponse createThumbnails(#PathVariable int width, #PathVariable int height, #PathVariable int limit) throws HeightNotAllowedException {
if (height < 10) {
throw new HeightNotAllowedException();
}
List<String> list = thumbnailService.processImages(width,height,limit);
return new ThumbnailResponse(list.size(),list);
}
Global exception handler (though you can go for per controller one)
#ControllerAdvice
public class GlobalErrorHandler {
#ExceptionHandler(value = HeightNotAllowedException.class)
#ResponseBody
#ResponseStatus(value = HttpStatus.BAD_REQUEST, reason = "Invalid Height")
public String heightError(HeightNotAllowedException ex) {
LOGGER.error("Invalid height");
return "Invalid Height";
}
}
Although there are couple of ways, for this case you can do just like below.
#Override
#RequestMapping(value="{width}/{height}/{limit}", method = RequestMethod.POST)
public #ResponseBody ResponseEntity createThumbnails(#PathVariable int width, #PathVariable int height, #PathVariable int limit) {
Gson gson = new Gson();
if(width <10 || height <10){
return new ResponseEntity<String>(HttpStatus.BAD_REQUEST);
}
List<String> list = thumbnailService.processImages(width,height,limit);
ThumbnailResponse response = new ThumbnailResponse(list.size(),list);
return new ResponseEntity<String>(gson.toJson(response), HttpStatus.OK);
}
However the standard way is to write Validator for the object and there is a method validate,Inside which you can write your validation logic and
class YourValidator implements Validator{
public void validate(Object target, Errors errors) {
//write your validation logic
}
}
and from your controller you can use it and validate the response like below.
if(error.hasErrors()){
return new ResponseEntity<String>(HttpStatus.BAD_REQUEST);
}
We've created a custom webservice in Umbraco to add (async) files and upload them. After upload the service is called with node and file-information to add a new node to the content tree.
At first our main problem was that the service was running outside of the Umbraco context, giving strange errors with get_currentuser.
Now, we inherit the umbraco BaseWebService from the umbraco.webservices dll and we've set all acces information in the settings file; we authenticatie before doing anything else using (correct and ugly-hardcoded) administrator.
When we now execute the webservice (from the browser or anything else) we get:
at umbraco.DataLayer.SqlHelper`1.ExecuteReader(String commandText, IParameter[] parameters)
at umbraco.cms.businesslogic.CMSNode.setupNode()
at umbraco.cms.businesslogic.web.Document.setupNode()
at umbraco.cms.businesslogic.CMSNode..ctor(Int32 Id)
at umbraco.cms.businesslogic.Content..ctor(Int32 id)
at umbraco.cms.businesslogic.web.Document..ctor(Int32 id)
at FileUpload.AddDocument(String ProjectID, String NodeID, String FileName)*
Where AddDocument is our method. The node (filename w/o extension) does not exist in the tree (not anywhere, it's a new filename/node). We've cleared the recycle bin, so it's not in there either.
Are we missing something vital, does anyone has a solution?
Below is the source for the webservice;
using umbraco.cms.businesslogic.web;
using umbraco.BusinessLogic;
using umbraco.presentation.nodeFactory;
using umbraco.cms.businesslogic.member;
using umbraco.cms;
/// <summary>
/// Summary description for FileUpload
/// </summary>
[WebService(Namespace = "http://umbraco.org/webservices/")]
[WebServiceBinding(ConformsTo = WsiProfiles.BasicProfile1_1)]
public class FileUpload : umbraco.webservices.BaseWebService //System.Web.Services.WebService
{
private string GetMimeType(string fileName)
{
string mimeType = "application/unknown";
string ext = System.IO.Path.GetExtension(fileName).ToLower();
Microsoft.Win32.RegistryKey regKey = Microsoft.Win32.Registry.ClassesRoot.OpenSubKey(ext);
if (regKey != null && regKey.GetValue("Content Type") != null)
mimeType = regKey.GetValue("Content Type").ToString();
return mimeType;
}
[WebMethod]
public string HelloWorld() {
return "Hello World";
}
[WebMethod]
public void AddDocument(string ProjectID, string NodeID, string FileName)
{
Authenticate("***", "***");
string MimeType = GetMimeType(FileName); //"application/unknown";
// Create node
int nodeId = 1197;
string fileName = System.IO.Path.GetFileNameWithoutExtension(#"*****\Upload\" + FileName);
string secGroups = "";
//EDIT DUE TO COMMENT: Behavior remains the same though
Document node = umbraco.cms.businesslogic.web.Document.MakeNew(fileName.Replace(".", ""), new DocumentType(1049), umbraco.BusinessLogic.User.GetUser(0), nodeId);
secGroups = "Intern";
StreamWriter sw = null;
try
{
//EXCEPTION IS THROWN SOMEWHERE HERE
Document doc = NodeLevel.CreateNode(fileName, "Bestand", nodeId);
doc.getProperty("bestandsNaam").Value = fileName;
byte[] buffer = System.IO.File.ReadAllBytes(#"****\Upload\" + FileName);
int projectId = 0;
int tempid = nodeId;
//EXCEPTION IS THROWN TO THIS POINT (SEE BELOW)
try
{
Access.ProtectPage(false, doc.Id, 1103, 1103);
Access.AddMembershipRoleToDocument(doc.Id, secGroups);
}
catch (Exception ex)
{
// write to file
}
try
{
doc.Publish(umbraco.BusinessLogic.User.GetUser(0));
umbraco.library.UpdateDocumentCache(doc.Id);
umbraco.content.Instance.RefreshContentFromDatabaseAsync();
}
catch (Exception ex)
{
// write to file
}
System.IO.File.Delete(FileName);
}
catch (Exception ex)
{
// THIS EXCEPTION IS CAUGHT!!
}
}
public override umbraco.webservices.BaseWebService.Services Service
{
get { return umbraco.webservices.BaseWebService.Services.DocumentService; }
}
}
If anyone has a solution, pointer, hint or whatever; help is appreciated!!
TIA,
riffnl
We've rewritten the whole procedure (dumped all code and restart) and we've got it working now.
I think we've been messing around with the old code so much in trying to get it to work we were missing some key issues, because it functions.
Thanks for thinking along anyway!