Elasticsearch 6.2 start a localhost http node in unit test - unit-testing

My goal is to start a localhost elasticsearch node in my junit test, so I can index some docs and also test search queries through high level rest client(e.g. making http calls to localhost ES node). I'm using Elasticsearch 6.2.
Here is the code I tried
String clusterName = "test";
Settings settings = Settings.builder()
.put("path.home", ES_WORKING_DIR)
.build();
new Node(settings).start();
and this is the error message I got..
java.lang.IllegalStateException: Unsupported transport.type []
at __randomizedtesting.SeedInfo.seed([74752622FDACDD5:AB9FD863FD5A2A5F]:0)
at org.elasticsearch.common.network.NetworkModule.getTransportSupplier(NetworkModule.java:212)
at org.elasticsearch.node.Node.<init>(Node.java:427)
at org.elasticsearch.node.Node.<init>(Node.java:246)
at org.elasticsearch.node.Node.<init>(Node.java:242)
Basically, I'm not sure what parameters to set in Settings.
Thanks in advance.

These are no longer supported by elastic, but they work for Junit test purposes
I have the following
For 5.6.7
Settings.Builder nodeSettings = Settings.builder()
.put("cluster.name", "my-integration-test")
.put("http.enabled", "true")
.put("path.data", dataDirectory)
.put("path.home", "/")
.put("transport.type", "netty4")
.put("network.host", "_local_")
.put("transport.tcp.port", "19200-19400")
.put("http.port", "19200-19400")
.put("discovery.type", "single-node");
LOG.info("Starting up embedded Elasticsearch");
node = new LocalNode(nodeSettings.build(), Arrays.asList(Netty4Plugin.class,
ReindexPlugin.class));
node.start();
For elastic 6.8.3
Settings.Builder nodeSettings = Settings.builder()
.put("cluster.name", "integration-test")
.put("node.name", "node-test")
.put("path.data", dataDirectory)
.put("path.home", "/")
.put("transport.type", "netty4")
.put("network.host", "127.0.0.1")
.put("http.port", "19200");
LOG.info("Starting up embedded Elasticsearch");
node = new LocalNode(nodeSettings.build(), Arrays.asList(Netty4Plugin.class,
ReindexPlugin.class,
CommonAnalysisPlugin.class));
node.start();
where
private static class LocalNode extends Node {
LocalNode(Settings settings, Collection<Class<? extends Plugin>> classpathPlugins) {
super(new Environment(settings, null), classpathPlugins, true);
}
#Override
protected void registerDerivedNodeNameWithLogger(final String s) {}
}
For 6.8.x you will need
<dependency>
<groupId>org.codelibs.elasticsearch.module</groupId>
<artifactId>analysis-common</artifactId>
<version>6.8.2</version>
</dependency>

Related

Jetty - path requires trailing slash

So I have a Jetty container with a URL like this: http://localhost:7071/my-app-name/ui/. The problem I'm running into is that Jetty seems to require that last trailing slash.
So this works:
http://localhost:7071/my-app-name/ui/
But this doesn't:
http://localhost:7071/my-app-name/ui
It's really weird, I would want the slash-less path to go to the same place as the path with a slash. That path is the homepage of the entire application, which is why I want it to work.
Here is some of my Jetty configuration.
PS. I'm working in Kotlin. The principles of how it works should still be the same though.
server = Server(props.getProperty(AppProps.PORT).toInt())
val handler = ServletContextHandler(server, root)
val appServletHolder = ServletHolder("AppServlet", AppServlet::class.java)
handler.addServlet(appServletHolder, "/ui/*")
handler.addEventListener(AppCore(props))
server.start()
I know I'm specifying "/ui/*" in my servlet holder configuration. But I can't seem to figure out any way to change that, I've tried a few combinations and none of them work better.
You've told the servlet spec that you want to listen on /ui/* so it mandates that your URLs must have the /ui/ portion.
But all is not lost, just tell the servlet spec the other url-patterns you are also interested in. (Servlets can be mapped to as many url-patterns as you want)
aka:
server = Server(props.getProperty(AppProps.PORT).toInt());
val handler = ServletContextHandler(server, root);
val appServletHolder = ServletHolder("AppServlet", AppServlet::class.java);
handler.addServlet(appServletHolder, "/ui"); // <-- like that
handler.addServlet(appServletHolder, "/ui/*");
handler.addEventListener(AppCore(props));
server.start();
Instead of adding a bunch of duplicate servlet mappings, I added one at the root to redirect any requests ending with "/" or "/index.html":
...
servletHandler.addServletWithMapping(RedirectServlet.class, "/*");
servletHandler.addServletWithMapping(AnotherServlet.class, "/ui");
...
public class RedirectServlet extends HttpServlet {
#Override
protected void doGet(HttpServletRequest req, HttpServletResponse resp) {
String pathInfo = req.getPathInfo();
if(pathInfo.endsWith("/") || pathInfo.endsWith("/index.html")) {
String newPath = pathInfo.substring(0, pathInfo.lastIndexOf('/'));
resp.setStatus(HttpServletResponse.SC_MOVED_PERMANENTLY);
resp.setHeader("Location", newPath);
}
}
#Override
protected void doPut(HttpServletRequest req, HttpServletResponse resp) {
doGet(req, resp);
}
}

unit testing -Mock JNDI lookup in WebSphere LIberty

I am using WebSphere Liberty 8.5.5 server and writing some batch test cases on this server. I am trying to mock jndi lookup object for testing but while setting up dataSource object in test case I am getting below error, Does anyone know how to set INITIAL_CONTEXT_FACTORY
ERROR : javax.naming.NoInitialContextException: Need to specify class name in environment or system prop
erty, or as an applet parameter, or in an application resource file: java.naming.factory.initial
at javax.naming.spi.NamingManager.getInitialContext(NamingManager.java:662)
at javax.naming.InitialContext.getDefaultInitCtx(InitialContext.java:307)
at javax.naming.InitialContext.getURLOrDefaultInitCtx(InitialContext.java:344)
at javax.naming.InitialContext.lookup(InitialContext.java:411)
You need to set the class name of your initial context factory in the environment of the InitialContext constructor.
For example:
Hashtable<Object, Object> env = new Hashtable<>();
env.put(InitialContext.INITIAL_CONTEXT_FACTORY, "com.example.MockICF");
new InitialContext(env);
Where com.example.MockICF might be written as:
public class MockICF implements InitialContextFactory {
private static Context ctx = new Mockery().mock(Context.class);
#Override
public Context getInitialContext(Hashtable<?, ?> environment) throws NamingException {
return ctx;
}
}

Testing nhibernate Castle Windsor mappings in httpModules is not registered

I want to write test that verifies mappings in castle windsor.
I am using ASP MVC2 where i am using castle windsor to map my repositories.
I have read this article:
http://weblogs.asp.net/bsimser/archive/2008/06/04/the-first-spec-you-should-write-when-using-castle.aspx
and based on this i have created my MS Test
[TestMethod()]
public void GetContainerTest()
{
MooseMvc.Infrastructure.DependencyInjectionInitialiser target = new MooseMvc.Infrastructure.DependencyInjectionInitialiser(); // TODO: Initialize to an appropriate value
IWindsorContainer container = target.GetContainer();
foreach (IHandler assignableHandler in container.Kernel.GetAssignableHandlers(typeof(object)))
{
container.Resolve(assignableHandler.ComponentModel.Service);
}
}
The data for target.getcontainer() implements
this._windsorContainer.Register(Component.For<TInterfaceType>()
.ImplementedBy(typeof(TConcreteType)).LifeStyle.PerWebRequest);
I get message as follows:
Looks like you forgot to register the http module
Castle.MicroKernel.Lifestyle.PerWebRequestLifestyleModule Add '<add
name="PerRequestLifestyle"
type="Castle.MicroKernel.Lifestyle.PerWebRequestLifestyleModule,
Castle.Windsor" />' to the <httpModules> section on your web.config.
If you're running IIS7 in Integrated Mode you will need to add it to
<modules> section under <system.webServer>
I have had the same problem and I have found a solution: You can define an event in the contructor of the unit test to override the LifestyleType.
void Kernel_ComponentModelCreated(Castle.Core.ComponentModel model)
{
if (model.LifestyleType == LifestyleType.Undefined)
model.LifestyleType = LifestyleType.Transient;
if (model.LifestyleType == LifestyleType.PerWebRequest)
model.LifestyleType = LifestyleType.Transient;
}
public UnitTest1()
{
containerWithControllers = new WindsorContainer();
containerWithControllers.Kernel.ComponentModelCreated += new ComponentModelDelegate(Kernel_ComponentModelCreated);
}
i have found beautifull guide
http://docs.castleproject.org/Windsor.Windsor-tutorial-part-three-a-testing-your-first-installer.ashx
there is not much else to add..

Hadoop: How to unit test FileSystem

I want to run unit test but I need to have a org.apache.hadoop.fs.FileSystem instance.
Are there any mock or any other solution for creating FileSystem?
If you're using hadoop 2.0.0 and above - consider using a hadoop-minicluster
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-minicluster</artifactId>
<version>2.5.0</version>
<scope>test</scope>
</dependency>
With it, you can create a temporary hdfs on your local machine, and run your tests on it. A setUp method may look like this:
baseDir = Files.createTempDirectory("test_hdfs").toFile().getAbsoluteFile();
Configuration conf = new Configuration();
conf.set(MiniDFSCluster.HDFS_MINIDFS_BASEDIR, baseDir.getAbsolutePath());
MiniDFSCluster.Builder builder = new MiniDFSCluster.Builder(conf);
hdfsCluster = builder.build();
String hdfsURI = "hdfs://localhost:"+ hdfsCluster.getNameNodePort() + "/";
DistributedFileSystem fileSystem = hdfsCluster.getFileSystem();
And in a tearDown method you should shut down your mini hdfs cluster, and remove temporary directory.
hdfsCluster.shutdown();
FileUtil.fullyDelete(baseDir);
Take a look at the hadoop-test jar
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-test</artifactId>
<version>0.20.205.0</version>
</dependency>
it has classes for setting up a MiniDFSCluster and MiniMRCluster so you can test without Hadoop
Why not use a mocking framework like Mockito or PowerMock to mock your interations with the FileSystem? Your unit tests should not depend on an actual FileSystem, but should just be verifying behavior in your code in interacting with the FileSystem.
One possible way would be to use TemporaryFolder in Junit 4.7.
See.: http://www.infoq.com/news/2009/07/junit-4.7-rules or http://weblogs.java.net/blog/johnsmart/archive/2009/09/29/working-temporary-files-junit-47.
What I have done (until I will find better solution) I extended the FileSystem.
I tried Thirupathi Chavati and Alexander Tokarev solutions with sbt, and :
import org.apache.hadoop.hdfs.MiniDFSCluster
will only work by adding:
libraryDependencies += "org.apache.hadoop" % "hadoop-hdfs" % "2.8.1" classifier "tests"
libraryDependencies += "org.apache.hadoop" % "hadoop-common" % "2.8.1" classifier "tests"
My solution is to create a DummyFileSystem`` that extends abstract HadoopFileSystem`, so I can fake if a file exists or not, etc. Example of "all file exists":
#Override
public FileStatus getFileStatus(Path f) throws IOException {
return new FileStatus(10, false, 3, 128*1024*1024,1,1, null, null, null, f);
}
I found easier to keep control on faked data.
You might want to take a look at RawLocalFileSystem. Though I think you'd better just mock it.
You can use HBaseTestingUtility:
public class SomeTest {
private HBaseTestingUtility testingUtil = new HBaseTestingUtility();
#Before
public void setup() throws Exception {
testingUtil.startMiniDFSCluster(1);
}
#After
public void tearDown() throws IOException {
testingUtil.shutdownMiniDFSCluster();
}
#Test
public void test() throws Exception {
DistributedFileSystem fs = testingUtil.getDFSCluster().getFileSystem();
final Path dstPath = new Path("/your/path/file.txt);
final Path srcPath = new Path(SomeTest.class.getResource("file.txt").toURI());
fs.copyFromLocalFile(srcPath, dstPath);
...
}
}
add below dependency
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-minicluster</artifactId>
<version>2.7.3</version>
<!-- <scope>test</scope>-->
</dependency>
add the below code, it will create the FileSysetm.
import java.nio.file.{Files, Paths}
import org.apache.hadoop.conf.Configuration
import org.apache.hadoop.fs.FileSystem
import org.apache.hadoop.hdfs.MiniDFSCluster
object MiniClusterDemo extends App {
def sysDir: String = System.getProperty("user.dir")
if(miniCluster!=null) println("Cluster created and active") else println("something went wrong")
def miniCluster: FileSystem = {
val basePath = Paths.get(s"$sysDir")
val baseDir = Files.createTempDirectory(basePath,"hdfs_test").toFile.getAbsoluteFile
val conf = new Configuration()
conf.set(MiniDFSCluster.HDFS_MINIDFS_BASEDIR, baseDir.getAbsolutePath)
val hdfsCluster = new MiniDFSCluster.Builder(conf).build()
val hdfsURI = s"hdfs://localhost:${hdfsCluster.getNameNodePort}/"
val fileSystem = hdfsCluster.getFileSystem
//hdfsCluster.shutdown();
//FileUtil.fullyDelete(baseDir);
fileSystem
}
}
See the sample logs after creation of MiniCluster

Test #Webservice EJBs with WebServiceContext (using OpenEJB?)

I have some EJBs as JAX-WS Web Service:
#WebService
#Stateless
#Remote(MobileFacade.class)
public class MobileFacadeBean implements MobileFacade {
...
#Resource
WebServiceContext wsc;
...
}
Within this Web Service class, a WebServiceContext is injected via #Resource. I use this WebServiceContext to get the principal in the implementation. This works quite well, but now I wonder how to (Unit-)test this class!
So far, I was using OpenEJB to test my EJBs. Since the Web Service class also is an Stateless Session Bean, I would really like to use the same approach here. However, it does not work that easy - of course, it complains that there is no WebServiceContext when not called as a Web Service.
So the first question is: are there any ways to mock the WebServiceContext in OpenEJB?
And if no, what approach would you favour to test this kind of Web Service classes?
Cheers,
Frank
There are a handful of #WebService unit test examples in the OpenEJB examples zip file. Everything you want should work fine.
simple-webservice
webservice-attachments
webservice-security
webservice-ws-security
The webservice-security example sounds exactly like what you want. The version online uses #RolesAllowed to make the container do the security check rather than doing it in code, but it is possible to check the principle in code. Here's a slightly modified version of that example that worked for me with no issues.
The bean
#DeclareRoles(value = {"Administrator"})
#Stateless
#WebService(
portName = "CalculatorPort",
serviceName = "CalculatorWsService",
targetNamespace = "http://superbiz.org/wsdl",
endpointInterface = "org.superbiz.calculator.CalculatorWs")
public class CalculatorImpl implements CalculatorWs, CalculatorRemote {
#Resource
private WebServiceContext webServiceContext;
#RolesAllowed(value = {"Administrator"})
public int sum(int add1, int add2) {
// maybe log the principal or something -- prints "jane" in the test
System.out.print(webServiceContext.getUserPrincipal());
return add1 + add2;
}
#RolesAllowed(value = {"Administrator"})
public int multiply(int mul1, int mul2) {
return mul1 * mul2;
}
}
The Test
public class CalculatorTest extends TestCase {
private InitialContext initialContext;
protected void setUp() throws Exception {
Properties properties = new Properties();
properties.setProperty(Context.INITIAL_CONTEXT_FACTORY, "org.apache.openejb.client.LocalInitialContextFactory");
properties.setProperty("openejb.embedded.remotable", "true");
initialContext = new InitialContext(properties);
}
/**
* Create a webservice client using wsdl url
*
* #throws Exception
*/
public void testCalculatorViaWsInterface() throws Exception {
URL url = new URL("http://127.0.0.1:4204/CalculatorImpl?wsdl");
QName calcServiceQName = new QName("http://superbiz.org/wsdl", "CalculatorWsService");
Service calcService = Service.create(url, calcServiceQName);
assertNotNull(calcService);
CalculatorWs calc = calcService.getPort(CalculatorWs.class);
((BindingProvider) calc).getRequestContext().put(BindingProvider.USERNAME_PROPERTY, "jane");
((BindingProvider) calc).getRequestContext().put(BindingProvider.PASSWORD_PROPERTY, "waterfall");
assertEquals(10, calc.sum(4, 6));
assertEquals(12, calc.multiply(3, 4));
}
}
Libraries
If using maven, switch your normal openejb-core dependency to openejb-cxf like so. This will add Apache CXF and the OpenEJB/CXF integration code to your classpath.
<dependency>
<groupId>org.apache.openejb</groupId>
<artifactId>openejb-cxf</artifactId>
<version>3.1.4</version>
<scope>test</scope>
</dependency>
If not using maven, simplest approach is to just add all the jars from the lib/ directory of the OpenEJB zip file.
David,In your answer in CalculatorTest you have used CalculatorWs.class, Is it same interface as it is used in webservice side implementation. Do we have to create web service client?
Also in David's example Instead of
QName calcServiceQName = new QName("http://superbiz.org/wsdl", "CalculatorWsService");
use
QName calcServiceQName = new QName("http://superbiz.org/wsdl", "CalculatorPort");