I am testing a service which essentially is mostly serializing an object and sending it via a service to an external system.
If I create the typical unittest I would mock the response of the serializer and of the service, which contacts the external system. In fact there would be not much left to test except calling a bunch of setter Methods in my object.
The alternative would be using a KernelTestCase and creating a functional test, which would be fine except I don't want to contact the external system, but to use a mock only for this "external" service.
Is there any possibility to achieve this in Symfony 4?
Or is there another approach to this?
What I am doing now is the following:
<?php
namespace App\Tests\Service;
use App\Service\MyClassService;
use App\Service\ExternalClient\ExternalClient;
use JMS\Serializer\Serializer;
use JMS\Serializer\SerializerInterface;
use Psr\Http\Message\RequestInterface;
use Psr\Log\LoggerInterface;
use Symfony\Bundle\FrameworkBundle\Test\KernelTestCase;
use Symfony\Component\HttpFoundation\Request;
class MyClassServiceTest extends KernelTestCase
{
/** #var LoggerInterface */
private $logger;
/** #var Serializer */
private $serializer;
/** #var ExternalClient */
private $externalClient;
/** #var RequestInterface */
private $request;
/** #var MyClassService */
private $myClassService;
public function setUp()
{
$kernel = self::bootKernel();
$this->logger = $kernel->getContainer()->get(LoggerInterface::class);
$this->serializer = $kernel->getContainer()->get(SerializerInterface::class);
$this->externalClient = $this->createMock(ExternalClient::class);
}
public function testPassRegistrationData()
{
$getParams = [
'amount' => '21.56',
'product_id' => 867,
'order_id' => '47t34g',
'order_item_id' => 2,
'email' => 'kiki%40bubu.com',
];
$this->generateMyClassService($getParams);
$userInformation = $this->myClassService->passRegistrationData();
var_dump($userInformation);
}
/**
* generateMyClassService
*
* #param $getParams
*
* #return MyClass
*/
private function generateMyClassService($getParams)
{
$this->request = new Request($getParams, [], [], [], [], [], null);
$this->myClassService = new MyClassService(
$this->logger,
$this->serializer,
$this->externalClient,
$this->request
);
}
}
Give back this error:
Symfony\Component\DependencyInjection\Exception\RuntimeException: Cannot autowire service "App\Service\MyClassConfirmationService": argument "$request" of method "__construct()" references class "Symfony\Component\HttpFoundation\Request" but no such service exists.
You shouldn't inject Request into your services. You should use Symfony\Component\HttpFoundation\RequestStack instead of Request. Also, you should check if $requestStack->getCurrentRequest() doesn't return null. I suppose you get such error in process of container's initialization but you execute just a script (test) and of course, you don't have Request on it.
Since the whole thing needed some more tweak here I post my solution to which Nikita's answer lead me:
As he suggested I replaced "request in my service with RequestStack which worked out fine:
/**
* #param LoggerInterface $logger
* #param SerializerInterface $serializer
* #param ExternalClient $externalClient
* #param RequestStack $requestStack
*/
public function __construct(
LoggerInterface $logger,
SerializerInterface $serializer,
ExternalClient $externalClient,
RequestStack $requestStack
) {
$this->logger = $logger;
$this->serializer = $serializer;
$this->externalClient = $externalClient;
$this->request = $requestStack->getCurrentRequest();
$this->params = $this->request->query;
}
In my test I faked the request like this:
$this->request = new Request($getParams, [], [], [], [], [], null);
$this->requestStack = new RequestStack();
$this->requestStack->push($this->request);
However with that having fixed my next problems arised, since my class also asks for the logger and the serializer.
For the Logger I used a general Loggerclass I created especially for this test situations. But that leaves me to get the serializer and I wanted the real one or else I could have stuck to a mostly useless UnitTest.
That is what I did:
public function setUp()
{
$kernel = self::bootKernel();
$container = self::$container;
$this->serializer = $container->get('jms_serializer.serializer');
}
This then gave me the real serializer from the container.
Now I can let my mocked external client give me mocked answers and I can test my service for reaction without bothering an external service.
I'm using the spring schedule to do some full treatment at night, it should invoke a soap webServices and nothing else.
The program works fine on my local server, but not on PROD :
scheduler :
#Configuration
#EnableScheduling
public class ActivationScheduler {
private static final Logger LOGGER = LoggerFactory.getLogger(ActivationScheduler.class);
#Autowired
private ActivationService activationService;
/**
* un cron qui se lance automatiquement tous les soirs à 04h:00
*/
#Scheduled(cron = "0 20 11 * * *")
public void execute() {
try {
activationService.relanceAll();
} catch (Exception e) {
LOGGER.error(Constants.EXCEPTION_LABEL, e);
}
}
}
Exception :
2017-11-10 11:20:00 ERROR TaskUtils$LoggingErrorHandler:95 - Unexpected error occurred in scheduled task.
java.lang.NoClassDefFoundError: org/bouncycastle/jce/provider/JCEBlockCipher$AEADGenericBlockCipher
at org.bouncycastle.jce.provider.JCEBlockCipher.engineSetMode(Unknown Source)
at javax.crypto.Cipher$Transform.setModePadding(Cipher.java:374)
at javax.crypto.Cipher.getInstance(Cipher.java:533)
at sun.security.ssl.JsseJce.getCipher(JsseJce.java:229)
at sun.security.ssl.CipherBox.<init>(CipherBox.java:179)
at sun.security.ssl.CipherBox.newCipherBox(CipherBox.java:263)
at sun.security.ssl.CipherSuite$BulkCipher.newCipher(CipherSuite.java:505)
at sun.security.ssl.CipherSuite$BulkCipher.isAvailable(CipherSuite.java:572)
at sun.security.ssl.CipherSuite$BulkCipher.isAvailable(CipherSuite.java:527)
at sun.security.ssl.CipherSuite.isAvailable(CipherSuite.java:194)
at sun.security.ssl.SSLContextImpl.getApplicableCipherSuiteList(SSLContextImpl.java:346)
at sun.security.ssl.SSLContextImpl.getDefaultCipherSuiteList(SSLContextImpl.java:304)
at sun.security.ssl.SSLSocketImpl.init(SSLSocketImpl.java:626)
at sun.security.ssl.SSLSocketImpl.<init>(SSLSocketImpl.java:537)
at sun.security.ssl.SSLSocketFactoryImpl.createSocket(SSLSocketFactoryImpl.java:72)
at sun.net.www.protocol.https.HttpsClient.createSocket(HttpsClient.java:405)
at sun.net.NetworkClient.doConnect(NetworkClient.java:162)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:432)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:527)
at sun.net.www.protocol.https.HttpsClient.<init>(HttpsClient.java:264)
at sun.net.www.protocol.https.HttpsClient.New(HttpsClient.java:367)
at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.getNewHttpClient(AbstractDelegateHttpsURLConnection.java:191)
at sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1105)
at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:999)
at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:177)
at sun.net.www.protocol.http.HttpURLConnection.getOutputStream0(HttpURLConnection.java:1283)
at sun.net.www.protocol.http.HttpURLConnection.getOutputStream(HttpURLConnection.java:1258)
at sun.net.www.protocol.https.HttpsURLConnectionImpl.getOutputStream(HttpsURLConnectionImpl.java:250)
at org.springframework.ws.transport.http.HttpUrlConnection.getRequestOutputStream(HttpUrlConnection.java:89)
at org.springframework.ws.transport.AbstractSenderConnection$RequestTransportOutputStream.createOutputStream(AbstractSenderConnection.java:87)
at org.springframework.ws.transport.TransportOutputStream.getOutputStream(TransportOutputStream.java:41)
at org.springframework.ws.transport.TransportOutputStream.write(TransportOutputStream.java:64)
at com.sun.xml.internal.messaging.saaj.soap.MessageImpl.writeTo(MessageImpl.java:1314)
at org.springframework.ws.soap.saaj.SaajSoapMessage.writeTo(SaajSoapMessage.java:275)
at org.springframework.ws.transport.AbstractWebServiceConnection.send(AbstractWebServiceConnection.java:46)
at org.springframework.ws.client.core.WebServiceTemplate.sendRequest(WebServiceTemplate.java:658)
at org.springframework.ws.client.core.WebServiceTemplate.doSendAndReceive(WebServiceTemplate.java:606)
at org.springframework.ws.client.core.WebServiceTemplate.sendAndReceive(WebServiceTemplate.java:555)
at org.springframework.ws.client.core.WebServiceTemplate.marshalSendAndReceive(WebServiceTemplate.java:390)
at org.springframework.ws.client.core.WebServiceTemplate.marshalSendAndReceive(WebServiceTemplate.java:383)
at org.springframework.ws.client.core.WebServiceTemplate.marshalSendAndReceive(WebServiceTemplate.java:373)
I googled the exception, so I found some poeple talking about bouncycastle, I added the last version but it does'nt solve my problem and i had the same exception.
I'm using also dynamicreports 5.0.0 in the project and i excluded bouncycastle (bcprov-jdk14, bcmail-jdk14, bctsp-jdk14)
The last info that can be utile, the ws invoked have SSL security.
Thanks,
I have an implementation of GRPC-java server code, but I didn't find the example code to unit test the StreamObserver. Does anyone know the right way to unit test the function?
public class RpcTrackDataServiceImpl implements TrackDataServiceGrpc.TrackDataService {
#Override
public void getTracks(GetTracksRequest request, StreamObserver < GetTracksResponse > responseObserver) {
GetTracksResponse reply = GetTracksResponse
.newBuilder()
.addTracks(TrackInfo.newBuilder()
.setOwner("test")
.setTrackName("test")
.build())
.build();
responseObserver.onNext(reply);
responseObserver.onCompleted();
}
}
Unit testing is very straight forward using the InProcess transport mentioned by Eric above. Here is an example a bit more explicit on code:
We test a service based on this protobuff definition:
syntax = "proto3";
option java_multiple_files = true;
option java_package = "servers.dummy";
option java_outer_classname = "DummyProto";
option objc_class_prefix = "DMYS";
package dummy;
import "general.proto";
// The dummy service definition.
service DummyService {
// # Misc
// Returns the server version
rpc getVersion (Empty) returns (ServerVersion) {}
// Returns the java version
rpc getJava (Empty) returns (JavaVersion) {}
}
// Transmission data types
(The following file is included above:)
syntax = "proto3";
option java_multiple_files = true;
option java_package = "general";
option java_outer_classname = "General";
option objc_class_prefix = "G";
// Transmission data types
message Empty {} // Empty Request or Reply
message ServerVersion {
string version = 1;
}
message JavaVersion {
string version = 1;
}
The DummyService based on the generated Java from the Protoc compiler is the following:
package servers.dummy;
import java.util.logging.Logger;
import general.Empty;
import general.JavaVersion;
import general.ServerVersion;
import io.grpc.stub.StreamObserver;
public class DummyService extends DummyServiceGrpc.DummyServiceImplBase {
private static final Logger logger = Logger.getLogger(DummyService.class.getName());
#Override
public void getVersion(Empty req, StreamObserver<ServerVersion> responseObserver) {
logger.info("Server Version-Request received...");
ServerVersion version = ServerVersion.newBuilder().setVersion("1.0.0").build();
responseObserver.onNext(version);
responseObserver.onCompleted();
}
#Override
public void getJava(Empty req, StreamObserver<JavaVersion> responseObserver) {
logger.info("Java Version Request received...");
JavaVersion version = JavaVersion.newBuilder().setVersion(Runtime.class.getPackage().getImplementationVersion() + " (" + Runtime.class.getPackage().getImplementationVendor() + ")").build();
responseObserver.onNext(version);
responseObserver.onCompleted();
}
}
Now we build an InProcessServer that runs our Dummy service (or any other service you want to test):
package servers;
import io.grpc.Server;
import io.grpc.inprocess.InProcessServerBuilder;
import java.io.IOException;
import java.util.logging.Logger;
import servers.util.PortServer;
/**
* InProcessServer that manages startup/shutdown of a service within the same process as the client is running. Used for unit testing purposes.
* #author be
*/
public class InProcessServer<T extends io.grpc.BindableService> {
private static final Logger logger = Logger.getLogger(PortServer.class.getName());
private Server server;
private Class<T> clazz;
public InProcessServer(Class<T> clazz){
this.clazz = clazz;
}
public void start() throws IOException, InstantiationException, IllegalAccessException {
server = InProcessServerBuilder
.forName("test")
.directExecutor()
.addService(clazz.newInstance())
.build()
.start();
logger.info("InProcessServer started.");
Runtime.getRuntime().addShutdownHook(new Thread() {
#Override
public void run() {
// Use stderr here since the logger may have been reset by its JVM shutdown hook.
System.err.println("*** shutting down gRPC server since JVM is shutting down");
InProcessServer.this.stop();
System.err.println("*** server shut down");
}
});
}
void stop() {
if (server != null) {
server.shutdown();
}
}
/**
* Await termination on the main thread since the grpc library uses daemon threads.
*/
public void blockUntilShutdown() throws InterruptedException {
if (server != null) {
server.awaitTermination();
}
}
}
We can now test the service using the following unit test:
package servers;
import static org.junit.Assert.*;
import general.ServerVersion;
import io.grpc.ManagedChannel;
import io.grpc.StatusRuntimeException;
import io.grpc.inprocess.InProcessChannelBuilder;
import java.io.IOException;
import java.util.concurrent.TimeUnit;
import java.util.logging.Level;
import java.util.logging.Logger;
import org.junit.After;
import org.junit.Before;
import org.junit.Test;
import servers.dummy.DummyService;
import servers.dummy.DummyServiceGrpc;
import servers.dummy.DummyServiceGrpc.DummyServiceBlockingStub;
import servers.dummy.DummyServiceGrpc.DummyServiceStub;
public class InProcessServerTest {
private static final Logger logger = Logger.getLogger(InProcessServerTest.class.getName());
private InProcessServer<DummyService> inprocessServer;
private ManagedChannel channel;
private DummyServiceBlockingStub blockingStub;
private DummyServiceStub asyncStub;
public InProcessServerTest() {
super();
}
#Test
public void testInProcessServer() throws InterruptedException{
try {
String version = getServerVersion();
assertEquals("1.0.0", version);
} finally {
shutdown();
}
}
/** Ask for the server version */
public String getServerVersion() {
logger.info("Will try to get server version...");
ServerVersion response;
try {
response = blockingStub.getVersion(null);
} catch (StatusRuntimeException e) {
logger.log(Level.WARNING, "RPC failed: {0}", e.getStatus());
fail();
return "";
}
return response.getVersion();
}
#Before
public void beforeEachTest() throws InstantiationException, IllegalAccessException, IOException {
inprocessServer = new InProcessServer<DummyService>(DummyService.class);
inprocessServer.start();
channel = InProcessChannelBuilder
.forName("test")
.directExecutor()
// Channels are secure by default (via SSL/TLS). For the example we disable TLS to avoid
// needing certificates.
.usePlaintext(true)
.build();
blockingStub = DummyServiceGrpc.newBlockingStub(channel);
asyncStub = DummyServiceGrpc.newStub(channel);
}
#After
public void afterEachTest(){
channel.shutdownNow();
inprocessServer.stop();
}
public void shutdown() throws InterruptedException {
channel.shutdown().awaitTermination(5, TimeUnit.SECONDS);
}
}
The test does only test one of two methods, as it is just for illustration purposes. The other method can be tested accordingly.
See the RouteGuideExample for more information on how to test both server and client:
https://github.com/grpc/grpc-java/blob/master/examples/src/test/java/io/grpc/examples/routeguide/RouteGuideServerTest.java
I'd suggest using the InProcess transport. The InProcess transport is very lightweight but also is using much of the "real" code, so the behavior closely matches a real transport. If you also use directExecutor() for the Channel and Server then the test is essentially single-threaded and will be deterministic. (Although another thread would still be used for deadline handling.)
Although the question is for unit testing a service, InProcess is also great for unit testing a client.
I ended up with a solution to create a FakeStreamObserver that implements the StreamObserver interface.
The FakeStreamObserver is passed in to execute onNext, onCompleted etc.
I'm not sure if this is the best way or not.
I will insert snippets from the official gRPC examples. I have successfully created tests based on these HelloWorld example.
The HelloWorldService:
/*
* Copyright 2015, gRPC Authors All rights reserved.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package io.grpc.examples.helloworld;
import io.grpc.Server;
import io.grpc.ServerBuilder;
import io.grpc.stub.StreamObserver;
import java.io.IOException;
import java.util.logging.Logger;
/**
* Server that manages startup/shutdown of a {#code Greeter} server.
*/
public class HelloWorldServer {
private static final Logger logger = Logger.getLogger(HelloWorldServer.class.getName());
private Server server;
private void start() throws IOException {
/* The port on which the server should run */
int port = 50051;
server = ServerBuilder.forPort(port)
.addService(new GreeterImpl())
.build()
.start();
logger.info("Server started, listening on " + port);
Runtime.getRuntime().addShutdownHook(new Thread() {
#Override
public void run() {
// Use stderr here since the logger may have been reset by its JVM shutdown hook.
System.err.println("*** shutting down gRPC server since JVM is shutting down");
HelloWorldServer.this.stop();
System.err.println("*** server shut down");
}
});
}
private void stop() {
if (server != null) {
server.shutdown();
}
}
/**
* Await termination on the main thread since the grpc library uses daemon threads.
*/
private void blockUntilShutdown() throws InterruptedException {
if (server != null) {
server.awaitTermination();
}
}
/**
* Main launches the server from the command line.
*/
public static void main(String[] args) throws IOException, InterruptedException {
final HelloWorldServer server = new HelloWorldServer();
server.start();
server.blockUntilShutdown();
}
static class GreeterImpl extends GreeterGrpc.GreeterImplBase {
#Override
public void sayHello(HelloRequest req, StreamObserver<HelloReply> responseObserver) {
HelloReply reply = HelloReply.newBuilder().setMessage("Hello " + req.getName()).build();
responseObserver.onNext(reply);
responseObserver.onCompleted();
}
}
}
And the test:
/*
* Copyright 2016, gRPC Authors All rights reserved.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package io.grpc.examples.helloworld;
import static org.junit.Assert.assertEquals;
import io.grpc.examples.helloworld.HelloWorldServer.GreeterImpl;
import io.grpc.testing.GrpcServerRule;
import org.junit.Rule;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.junit.runners.JUnit4;
/**
* Unit tests for {#link HelloWorldServer}.
* For demonstrating how to write gRPC unit test only.
* Not intended to provide a high code coverage or to test every major usecase.
*
* <p>For more unit test examples see {#link io.grpc.examples.routeguide.RouteGuideClientTest} and
* {#link io.grpc.examples.routeguide.RouteGuideServerTest}.
*/
#RunWith(JUnit4.class)
public class HelloWorldServerTest {
/**
* This creates and starts an in-process server, and creates a client with an in-process channel.
* When the test is done, it also shuts down the in-process client and server.
*/
#Rule
public final GrpcServerRule grpcServerRule = new GrpcServerRule().directExecutor();
/**
* To test the server, make calls with a real stub using the in-process channel, and verify
* behaviors or state changes from the client side.
*/
#Test
public void greeterImpl_replyMessage() throws Exception {
// Add the service to the in-process server.
grpcServerRule.getServiceRegistry().addService(new GreeterImpl());
GreeterGrpc.GreeterBlockingStub blockingStub =
GreeterGrpc.newBlockingStub(grpcServerRule.getChannel());
String testName = "test name";
HelloReply reply = blockingStub.sayHello(HelloRequest.newBuilder().setName(testName).build());
assertEquals("Hello " + testName, reply.getMessage());
}
}
You can fin other examples, if you clone the examples repository as they describe it here:
https://grpc.io/docs/tutorials/basic/java.html
I hope it will help you, too.
Br,
Renato
#RunWith(JUnit4.class)
public class HelloWorldServerTest {
/**
* This rule manages automatic graceful shutdown for the registered servers and channels at the
* end of test.
*/
#Rule
public final GrpcCleanupRule grpcCleanup = new GrpcCleanupRule();
/**
* To test the server, make calls with a real stub using the in-process channel, and verify
* behaviors or state changes from the client side.
*/
#Test
public void greeterImpl_replyMessage() throws Exception {
// Generate a unique in-process server name.
String serverName = InProcessServerBuilder.generateName();
// Create a server, add service, start, and register for automatic graceful shutdown.
grpcCleanup.register(InProcessServerBuilder
.forName(serverName).directExecutor().addService(new GreeterImpl()).build().start());
GreeterGrpc.GreeterBlockingStub blockingStub = GreeterGrpc.newBlockingStub(
// Create a client channel and register for automatic graceful shutdown.
grpcCleanup.register(InProcessChannelBuilder.forName(serverName).directExecutor().build()));
HelloReply reply =
blockingStub.sayHello(HelloRequest.newBuilder().setName( "test name").build());
assertEquals("Hello test name", reply.getMessage());
}
}
https://github.com/grpc/grpc-java/blob/master/examples/src/test/java/io/grpc/examples/helloworld/HelloWorldServerTest.java
First, refactor the code so it's easier to unit test:
public class RpcTrackDataServiceImpl implements TrackDataServiceGrpc.TrackDataService {
#Override
public void getTracks(GetTracksRequest request, StreamObserver<GetTracksResponse> responseObserver) {
GetTracksResponse reply = getTracks(request);
responseObserver.onNext(reply);
responseObserver.onCompleted();
}
#VisibleForTesting
GetTracksResponse getTracks(GetTracksRequest request) {
return GetTracksResponse
.newBuilder()
.addTracks(TrackInfo.newBuilder()
.setOwner("test")
.setTrackName("test")
.build())
.build();
}
}
Small tests can then be written for each (more easily if using Spring Boot):
public class UnitTest {
private final ApplicationContextRunner applicationContextRunner = new ApplicationContextRunner();
#Configuration
public static class GetTracksConfiguration {
#Bean
public GetTracksService getTracksService() {
return new GetTracksService();
}
}
#Test
public void replyShouldBeSent() {
final GetTracksRequest request = GetTracksRequest.newBuilder().build();
final StreamObserver<GetTracksResponse> response = mock(StreamObserver.class);
applicationContextRunner
.withUserConfiguration(RequestTracksConfiguration.class)
.run(context -> {
assertThat(context)
.hasSingleBean(RequestTracksService.class);
context.getBean(RequestTracksService.class)
.getTracks(request, response);
verify(response, times(1)).onNext(any(GetTracksResponse.class));
verify(response, times(1)).onCompleted();
verify(response, never()).onError(any(Throwable.class));
});
}
#Test
public void shouldTestLogic {
assertLogicInFactoredOutMethodIsCorrect();
}
The larger test should then only test the startup and wiring:
#RunWith(SpringRunner.class)
#SpringBootTest(
classes = {GetTracksService.class}
)
#EnableAutoConfiguration
public class SmokeTest {
private GetTracksServiceGrpc.GetTracksServiceBlockingStub blockingStub;
#Test
public void springClientConnects() {
final GetTracksRequest request = GetTracksRequest.newBuilder()
.build();
assertNotNull(blockingStub.getTracks(request));
}
}
Note: The above code may not work OOTB since I've left out some annotations we use internally.
The major point is there's no need to pay for the cost of bringing up a server for unit tests that are meant to test logic.
I have been using org.elasticsearch.test.ElasticsearchIntegrationTesting class for testing elasticsearch functionality.
I Use below code to import data into Elasticsearch and to search that data :**
#Test
public void searchData() throws Exception{
logger.info("searchData : searching data from Elasticsearch");
try {
csvImporter.client = client();
csvImporter.importData(); // In the importData method after data is imported we close the client
//In the below line getting Exception and it is inconsistent. Some times works fine
SearchResponse searchResponse = csvImporter.client.prepareSearch().setQuery(matchAllQuery()).execute().actionGet();
for (SearchHit searchHit : searchResponse.getHits()){
logger.info("Source Data >>>>>>>>>>>>>>>>>>>>> " + searchHit.sourceAsString());
}
} finally {
csvImporter.client.close();
}
}
Exception:
org.elasticsearch.client.transport.NoNodeAvailableException: None of the configured nodes are available: []
at __randomizedtesting.SeedInfo.seed([86EB1B4AAE1739A5:99315B8D3D924DA1]:0)
at org.elasticsearch.client.transport.TransportClientNodesService.ensureNodesAreAvailable(TransportClientNodesService.java:305)
at org.elasticsearch.client.transport.TransportClientNodesService.execute(TransportClientNodesService.java:200)
at org.elasticsearch.client.transport.support.InternalTransportIndicesAdminClient.execute(InternalTransportIndicesAdminClient.java:86)
at org.elasticsearch.client.support.AbstractIndicesAdminClient.deleteTemplate(AbstractIndicesAdminClient.java:687)
at org.elasticsearch.action.admin.indices.template.delete.DeleteIndexTemplateRequestBuilder.doExecute(DeleteIndexTemplateRequestBuilder.java:40)
at org.elasticsearch.action.ActionRequestBuilder.execute(ActionRequestBuilder.java:91)
at org.elasticsearch.action.ActionRequestBuilder.execute(ActionRequestBuilder.java:65)
at org.elasticsearch.test.TestCluster.wipeTemplates(TestCluster.java:177)
at org.elasticsearch.test.TestCluster.wipe(TestCluster.java:76)
at org.elasticsearch.test.ElasticsearchIntegrationTest.beforeInternal(ElasticsearchIntegrationTest.java:292)
at org.elasticsearch.test.ElasticsearchIntegrationTest.before(ElasticsearchIntegrationTest.java:1946)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1665)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:898)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:914)
at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:59)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:873)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:775)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:809)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:820)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:59)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)
Since it is a Unit Testing I have not used configuration Settings to create client.
is there a way to recreate the client Object with same configuration settings??
I have a question and problem about writing a custom OutputAttributeProcessor.
I use WSO2 CEP 2.1.0 and siddhi 1.1.0.
I want to create a custom OutputAttributeProcessor, so I create two java classes, DiscomfortIndexAggregatorFactory implements OutputAttributeProcessorFactory and DiscomfortIndexAggregator implements OutputAttributeProcessor.
Package of two classes is org.wso2.siddhi.extention.aggregator.environment.
Two java programs are as follows.
DiscomfortIndexAggregatorFactory.java
package org.wso2.siddhi.extention.aggregator.environment;
import org.wso2.siddhi.core.query.projector.attribute.factory.OutputAttributeProcessorFactory;
import org.wso2.siddhi.core.query.projector.attribute.handler.OutputAttributeProcessor;
import org.wso2.siddhi.query.api.definition.Attribute.Type;
import org.wso2.siddhi.query.api.extension.annotation.SiddhiExtension;
#SiddhiExtension(namespace = "environment", function = "discomfortIndex")
public class DiscomfortIndexAggregatorFactory implements
OutputAttributeProcessorFactory {
#Override
public OutputAttributeProcessor createAggregator(Type type) {
return new DiscomfortIndexAggregator();
}
#Override
public ProcessorType getProcessorType() {
return OutputAttributeProcessorFactory.ProcessorType.AGGREGATOR;
}
}
DiscomfortIndexAggregator.java
package org.wso2.siddhi.extention.aggregator.environment;
import org.wso2.siddhi.core.query.projector.attribute.handler.OutputAttributeProcessor;
import org.wso2.siddhi.query.api.definition.Attribute.Type;
public class DiscomfortIndexAggregator implements OutputAttributeProcessor {
private static final long serialVersionUID = -5992266303998509661L;
#Override
public OutputAttributeProcessor createNewInstance() {
return new DiscomfortIndexAggregator();
}
#Override
public Type getType() {
return Type.DOUBLE;
}
#Override
public Object processInEventAttribute(Object obj) {
double discomfortIndex = -1D;
if (obj instanceof Object[]) {
Object[] objArray = (Object[]) obj;
double temperature = (double) objArray[0];
double humidity = (double) objArray[1];
discomfortIndex = 0.81 * temperature + 0.01 * humidity
* (0.99 * temperature - 14.3) + 46.3;
}
return discomfortIndex;
}
#Override
public Object processRemoveEventAttribute(Object obj) {
double discomfortIndex = -1D;
if (obj instanceof Object[]) {
Object[] objArray = (Object[]) obj;
double temperature = (double) objArray[0];
double humidity = (double) objArray[1];
discomfortIndex = 0.81 * temperature + 0.01 * humidity
* (0.99 * temperature - 14.3) + 46.3;
}
return discomfortIndex;
}
}
I created jar file consists of two java classes, added the jar file to the class path at /repository/components/lib, and added siddhi.extension file located at /repository/conf/siddhi.
Content of siddhi.extention is as follows.
org.wso2.siddhi.extention.aggregator.environment.DiscomfortIndexAggregatorFactory
I restarted after above configuration.
Error logs are not output after restart.
But I created following query of bucket,
from hiroshimaData
insert into disconfortIndexStream
environment:discomfortIndex(RTC_001_Temp, SHT_001_Humi) as discomfortIndex
and following error log is output.
ERROR {org.wso2.carbon.cep.core.BucketDeployer} - wrong configuration provided for adding HIROSHIMA.xml
org.wso2.carbon.cep.core.exception.CEPConfigurationException: Error in initializing Siddhi backend Runtime,null
at org.wso2.carbon.cep.core.internal.CEPBucket.init(CEPBucket.java:109)
at org.wso2.carbon.cep.core.internal.CEPService.deployBucket(CEPService.java:213)
at org.wso2.carbon.cep.core.internal.CEPService.deployBucket(CEPService.java:174)
at org.wso2.carbon.cep.core.BucketDeployer.deploy(BucketDeployer.java:95)
at org.apache.axis2.deployment.repository.util.DeploymentFileData.deploy(DeploymentFileData.java:136)
at org.apache.axis2.deployment.DeploymentEngine.doDeploy(DeploymentEngine.java:810)
at org.apache.axis2.deployment.repository.util.WSInfoList.update(WSInfoList.java:144)
at org.apache.axis2.deployment.RepositoryListener.update(RepositoryListener.java:377)
at org.apache.axis2.deployment.RepositoryListener.checkServices(RepositoryListener.java:254)
at org.apache.axis2.deployment.RepositoryListener.startListener(RepositoryListener.java:371)
at org.apache.axis2.deployment.scheduler.SchedulerTask.checkRepository(SchedulerTask.java:59)
at org.apache.axis2.deployment.scheduler.SchedulerTask.run(SchedulerTask.java:67)
at org.wso2.carbon.core.deployment.CarbonDeploymentSchedulerTask.runAxisDeployment(CarbonDeploymentSchedulerTask.java:67)
at org.wso2.carbon.core.deployment.CarbonDeploymentSchedulerTask.run(CarbonDeploymentSchedulerTask.java:112)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
Caused by: java.lang.NullPointerException
at org.wso2.siddhi.core.query.projector.attribute.processor.AbstractAggregationAttributeProcessor.load(AbstractAggregationAttributeProcessor.java:88)
at org.wso2.siddhi.core.persistence.PersistenceService.restoreRevision(PersistenceService.java:79)
at org.wso2.siddhi.core.persistence.PersistenceService.restoreLastRevision(PersistenceService.java:104)
at org.wso2.siddhi.core.SiddhiManager.restoreLastRevision(SiddhiManager.java:276)
at org.wso2.carbon.cep.siddhi.backend.SiddhiBackEndRuntime.init(SiddhiBackEndRuntime.java:259)
at org.wso2.carbon.cep.core.internal.CEPBucket.init(CEPBucket.java:107)
Can you tell me how to do it?
Thank you in advance.