Cannot find org.codehaus.commons.compiler.properties resource - apache-calcite

I created a Maven project that includes a dependency to the Calcite JDBC driver, as well as source code for a Calcite CSV adapter.
<dependency>
<groupId>org.apache.calcite</groupId>
<artifactId>calcite-core</artifactId>
<version>1.20.0</version>
</dependency>
When I run from a JUnit test, I can query some CSV files using SQL. Very cool!
But I cannot get the JAR to work in SQL Workbench/J. The log file has this:
Caused by: java.lang.IllegalStateException: Unable to instantiate java compiler
at org.apache.calcite.rel.metadata.JaninoRelMetadataProvider.compile(JaninoRelMetadataProvider.java:434)
Caused by: java.lang.ClassNotFoundException: No implementation of org.codehaus.commons.compiler is on the class path. Typically, you'd have 'janino.jar', or 'commons-compiler-jdk.jar', or both on the classpath.
at org.codehaus.commons.compiler.CompilerFactoryFactory.getDefaultCompilerFactory(CompilerFactoryFactory.java:65)
SQL Workbench/J is successfully connecting, and I can see the list of CSV "tables" in the UI. But when I try to query them, I get the above error.
I found a link to someone having a similar problem, but did not see a resolution.
https://community.jaspersoft.com/questions/1035211/apache-calcite-jdbc-driver-jaspersoft
Also, here's the code that seems to be throwing the error:
public final
class CompilerFactoryFactory {
...
public static ICompilerFactory
getDefaultCompilerFactory() throws Exception {
if (CompilerFactoryFactory.defaultCompilerFactory != null) {
return CompilerFactoryFactory.defaultCompilerFactory;
}
Properties properties = new Properties();
InputStream is = Thread.currentThread().getContextClassLoader().getResourceAsStream(
"org.codehaus.commons.compiler.properties"
);
if (is == null) {
throw new ClassNotFoundException(
"No implementation of org.codehaus.commons.compiler is on the class path. Typically, you'd have "
+ "'janino.jar', or 'commons-compiler-jdk.jar', or both on the classpath."
);
}
From what I can tell, the org.codehaus.commons.compiler.properties resource is just not being found when running under SQL Workbench/J, but for some reason it works in my code.
If I unzip the JAR file, I do see org.codehaus.commons.compiler.properties in the directory structure, so not sure why it's not being found.
Anyone else run into this problem?
Thanks for any help.

Related

how to define #TestPropertySource(location)?

i'm trying to add a test.properties file in my test package.
the structure is
test->java-> -> test.java
-> resource -> test.properties
and my command is
#RunWith(SpringRunner.class)
#ContextConfiguration(classes = Publish.class)
#TestPropertySource(locations = "classpath:/src/test/java/resources/test.properties")
public class Test {
...
}
but i always keep getting this
java.lang.IllegalStateException: Failed to load ApplicationContext
Caused by: java.lang.IllegalStateException: Failed to add PropertySource to Environment
Caused by: java.io.FileNotFoundException: class path resource [/src/test/java/resources/test.properties] cannot be opened because it does not exist
i have few other variants and i still get this error, how can i solve this??
Other variants:
#TestPropertySource(locations = "classpath:/test.properties")
#TestPropertySource(locations = "classpath:/resources/test.properties")
#TestPropertySource(locations = "classpath:test.properties")
another similar question was asked but i din't seem to solve my problem stackOverflow Question
It should be
#TestPropertySource(locations = "classpath:/yourPackage/thatContainResource/test.properties")
src is not on runtime and is not a part of classpath.
But most importantly, your directory structure is wrong. resources should be next to java like this:
By putting your resources into sources (like you did), they got filtered out.

Embedding Jetty 9.3 with modular XmlConfiguration

I am migrating from Jetty 8.1.17 to Jetty 9.3.9. Our application embeds Jetty. Previously we had a single XML configuration file jetty.xml which contained everything we needed.
I felt that with Jetty 9.3.9 it would be much nicer to use the modular approach that they suggest, so far I have jetty.xml, jetty-http.xml, jetty-https.xml and jetty-ssl.xml in my $JETTY_HOME/etc; these are pretty much copies of those from the 9.3.9 distribution. This seems to work well when I use start.jar but not through my own code which embeds Jetty.
Ideally I would like to be able to scan for any jetty xml files in the $JETTY_HOME/etc folder and load the configuration. However for embedded mode I have not found a way to do that without explicitly defining the order that those files should be loaded in, due to <ref id="x"/> dependencies between them etc.
My initial attempt is based on How can I programmatically start a jetty server with multiple configuration files? and looks like:
final List<Object> configuredObjects = new ArrayList();
XmlConfiguration last = null;
for(final Path confFile : configFiles) {
logger.info("[loading jetty configuration : {}]", confFile.toString());
try(final InputStream is = Files.newInputStream(confFile)) {
final XmlConfiguration configuration = new XmlConfiguration(is);
if (last != null) {
configuration.getIdMap().putAll(last.getIdMap());
}
configuredObjects.add(configuration.configure());
last = configuration;
}
}
Server server = null;
// For all objects created by XmlConfigurations, start them if they are lifecycles.
for (final Object configuredObject : configuredObjects) {
if(configuredObject instanceof Server) {
server = (Server)configuredObject;
}
if (configuredObject instanceof LifeCycle) {
final LifeCycle lc = (LifeCycle)configuredObject;
if (!lc.isRunning()) {
lc.start();
}
}
}
However, I get Exceptions at startup if jetty-https.xml is loaded before jetty-ssl.xml or if I place a reference in jetty.xml to an object from a sub-configuration jetty-blah.xml which has not been loaded first.
It seems to me like Jetty manages to do this okay itself when you call java -jar start.jar, so what am I missing to get Jetty to not care about what order the config files are parsed in?
Order is extremely important when loading the Jetty XML files.
That's the heart of what the entire start.jar and its module system is about, have an appropriate set of properties, the server classpath is sane, and ensuring proper load order of the XML.
Note: its not possible to have everything in ${jetty.home}/etc loaded at the same time, as you will get conflicts on alternate implementations of common technologies (something start.jar also manages for you)

servlet connect to solr using solrj

I want my servlet to establish a connection with solr so as to perform query. using eclipse I have created a simple JSP file for the user to input their query, then it will call the doGet() method from the servlet. From the servlet i try to use solrj to connect with solr
my solrj code is as below.
HttpSolrServer solr = new HttpSolrServer("http://localhost:8983/solr");
SolrQuery query = new SolrQuery();
query.setQuery("sony digital camera");
query.addFilterQuery("cat:electronics","store:amazon.com");
query.setFields("id","price","merchant","cat","store");
query.setStart(0);
query.set("defType", "edismax");
QueryResponse response = solr.query(query);
SolrDocumentList results = response.getResults();
for (int i = 0; i < results.size(); ++i) {
System.out.println(results.get(i));
}
}
Its basically just simple code available online, there is no syntax error which means that the program detects the library files. However on run time the program returns classNotFoundException error.
This prompt me thinking whether i can implement solrj on servlet. Anyone can advise me where i can exactly use the solrj code? I think that the code is right but just that i implement solrj wrongly.
Thanks in advance!
The exception is saying that your runtime classpath is wrong.
Assuming you're running this class in a servlet container, most probably what you're missing is not a Java EE jar but instead has something to do with SolrJ.
Solrj has few dependencies: you can find them in the Solr distribution or if you have confidence with Apache Maven, you can use it for individuating those jars.
In any case, once you identified those libraries, you have to put them under WEB-INF/lib of your webapp
For a list of dependencies you can read here: http://mvnrepository.com/artifact/org.apache.solr/solr-solrj/4.10.3

Load MapReduce output data into HBase

The last few days I've been experimenting with Hadoop. I'm running Hadoop in pseudo-distributed mode on Ubuntu 12.10 and successfully executed some standard MapReduce jobs.
Next I wanted to start experimenting with HBase. I've installed HBase, played a bit in the shell. That all went fine so I wanted to experiment with HBase through a simple Java program. I wanted to import the output of one of the previous MapReduce jobs and load it into an HBase table. I've wrote a Mapper that should produce HFileOutputFormat files that should easily read into a HBase table.
Now, whenever I run the program (using: hadoop jar [compiled jar]) I get a ClassNotFoundException. The program seems unable to resolve com.google.commons.primitives.Long. Of course, I thought it was just a dependency missing but the JAR (Google's Guava) is there.
I've tried a lot of different things but can't seem to find a solution.
I attached the Exception that occurs and the most important classes. I would be truly appreciated if someone could help me out or give me some advice on where to look.
Kind regards,
Pieterjan
ERROR
12/12/13 09:02:54 WARN snappy.LoadSnappy: Snappy native library not loaded
12/12/13 09:03:00 INFO mapred.JobClient: Running job: job_201212130304_0020
12/12/13 09:03:01 INFO mapred.JobClient: map 0% reduce 0%
12/12/13 09:04:07 INFO mapred.JobClient: map 100% reduce 0%
12/12/13 09:04:51 INFO mapred.JobClient: Task Id : attempt_201212130304_0020_r_000000_0,Status : FAILED
Error: java.lang.ClassNotFoundException: com.google.common.primitives.Longs
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:423)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:356)
at org.apache.hadoop.hbase.KeyValue$KVComparator.compare(KeyValue.java:1554)
at org.apache.hadoop.hbase.KeyValue$KVComparator.compare(KeyValue.java:1536)
at java.util.TreeMap.compare(TreeMap.java:1188)
at java.util.TreeMap.put(TreeMap.java:531)
at java.util.TreeSet.add(TreeSet.java:255)
at org.apache.hadoop.hbase.mapreduce.PutSortReducer.reduce(PutSortReducer.java:63)
at org.apache.hadoop.hbase.mapreduce.PutSortReducer.reduce(PutSortReducer.java:40)
at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:176)
at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:650)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:418)
at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1136)
at org.apache.hadoop.mapred.Child.main(Child.java:249)
JAVA
Mapper:
public class TestHBaseMapper extends Mapper<LongWritable, Text, ImmutableBytesWritable, Put> {
#Override
public void map(LongWritable key, Text value, Context context) throws IOException, InterruptedException {
//Tab delimiter \t, white space delimiter: \\s+
String[] s = value.toString().split("\t");
Put put = new Put(s[0].getBytes());
put.add("amount".getBytes(), "value".getBytes(), value.getBytes());
context.write(new ImmutableBytesWritable(Bytes.toBytes(s[0])), put);
}
Job:
public class TestHBaseRun extends Configured implements Tool {
#Override
public int run(String[] args) throws Exception {
try {
Configuration configuration = getConf();
Job hbasejob = new Job(configuration);
hbasejob.setJobName("TestHBaseJob");
hbasejob.setJarByClass(TestHBaseRun.class);
//Specifies the InputFormat and the path.
hbasejob.setInputFormatClass(TextInputFormat.class);
TextInputFormat.setInputPaths(hbasejob, new Path("/hadoopdir/user/data/output/test/"));
//Set Mapper, MapperOutputKey and MapperOutputValue classes.
hbasejob.setMapperClass(TestHBaseMapper.class);
hbasejob.setMapOutputKeyClass(ImmutableBytesWritable.class);
hbasejob.setMapOutputValueClass(Put.class);
//Specifies the OutputFormat and the path. If The path exists it's reinitialized.
//In this case HFiles, that can be imported into HBase, are produced.
hbasejob.setOutputFormatClass(HFileOutputFormat.class);
FileSystem fs = FileSystem.get(configuration);
Path outputpath = new Path("/hadoopdir/user/data/hbase/table/");
fs.delete(outputpath, true);
HFileOutputFormat.setOutputPath(hbasejob, outputpath);
//Check if table exists in HBase and creates it if necessary.
HBaseUtil util = new HBaseUtil(configuration);
if (!util.exists("test")) {
util.createTable("test", new String[]{"amount"});
}
//Reads the existing (or thus newly created) table.
Configuration hbaseconfiguration = HBaseConfiguration.create(configuration);
HTable table = new HTable(hbaseconfiguration, "test");
//Write HFiles to disk. Autoconfigures partitioner and reducer.
HFileOutputFormat.configureIncrementalLoad(hbasejob, table);
boolean success = hbasejob.waitForCompletion(true);
//Load generated files into table.
LoadIncrementalHFiles loader;
loader = new LoadIncrementalHFiles(hbaseconfiguration);
loader.doBulkLoad(outputpath, table);
return success ? 0 : 1;
} catch (Exception ex) {
System.out.println("Error: " + ex.getMessage());
}
return 1;
}
ClassNotFoundException, it means that the required .jar that contains com.google.common.primitives.Longs cannot be found.
There are several ways to solve this issue:
If you're just playing with Hadoop, the simplest way to solve this issue is to copy the required .jar into /usr/share/hadoop/lib.
Add the path to the required .jar to HADOOP_CLASSPATH. To do so open /etc/hbase/hbase-env.sh and add:
export HADOOP_CLASSPATH="<jar_files>:$HADOOP_CLASSPATH"
Create a folder /lib in your root project folder. Copy your .jar into that folder. Create a package (.jar) for your project. The result will be a fat jar contained all the jars included in /lib.

Embed jetty in java app and export as jar

I'm making a java application which embeds a Jetty web server, which in turn serves content developed with Google Web Toolkit. It's all working fine when run in Eclipse, but when I export it as a jar file all I get is a Jetty error message saying "File not found".
The Jetty server is launched like this:
WebAppContext handler = new WebAppContext();
handler.setResourceBase("./war");
handler.setDescriptor("./war/WEB-INF/web.xml");
handler.setContextPath("/");
handler.setParentLoaderPriority(true);
server.setHandler(handler);
try {
server.start();
server.join();
} catch (Exception e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
I suspect that the problem is the relative paths used in handler.setResourceBase() and handler.setDescriptor(). I've googled and tested lots of solutions to this but so far to no avail. Particularly I've tried using something like getClass().getResource("./war").toExternalForm() but this just throws Null exceptions.
I also tried:
ProtectionDomain protectionDomain = Start.class.getProtectionDomain();
URL location = protectionDomain.getCodeSource().getLocation();
but that only results in a Jetty serving a directory listing of the java classes.
Is there any way to do this?
Copy all files of the compiled GWT application into one of your Java packages. For instance:
my.webapp.resources
html/MyPage.html
gwtmodule/gwtmodule.nocache.js
...
(html, gwtmodule will become Java packages as well).
Configure the embedded Jetty instance to serve these files.
final String webDir = this.getClass().
getClassLoader().getResource("my/webapp/resources").toExternalForm();
final Context root = new Context(server, "/", Context.SESSIONS);
root.setContextPath("/");
root.setResourceBase(webDir);
root.addServlet(MyGwtService.class, "/servlets/v01/gwtservice");
This approach works for me both when the application is run from eclipse as well as when it is deployed.
I've posted a detailed description on how to do this here:
http://h30499.www3.hp.com/t5/HP-Software-Developers-Blog/Embedding-Jetty-in-a-Java-Main-Application/ba-p/6107503
It looks like ClassLoader.getResource does not understand an empty string or . or / as an argument. In my jar file I had to move all stuf to WEB-INF. So the code looks like
`contextHandler.setResourceBase(EmbeddedJetty.class.getClassLoader().getResource("WEB-INF").toExternalForm());`
so the context looks like this then:
ContextHandler:744 - Started o.e.j.w.WebAppContext#48b3806{/,jar:file:/Users/xxx/projects/dropbox/ui/target/ui-1.0-SNAPSHOT.jar!/WEB-INF,AVAILABLE}