Java binding for LMDB, need help to refresh LMDB JNI - java-native-interface

we are using LMDB within java application.
The Java bindings that are available are 1 year old.
I would like to refresh the LMDBJNI
https://github.com/chirino/lmdbjni
However, the project owner, did not provide any instructions on how to build his project.
So I cannot just clone his git repository, and drop the new version of LMDB ( https://git.gitorious.org/mdb/mdb.git ) c and h files, and get it rebuilt
It seems that underneath LMDB JNI Is using hawkjni, but that's as far as I had gotten.
these are the steps I tried
a) git clone https://github.com/chirino/lmdbjni.git
b) cd lmdbjni; mvn install
It finishes successfully, however the resulting JAR does not have the actual lmdb library compiled.
So my test program fails with
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:293)
at java.lang.Thread.run(Thread.java:744)
Caused by: java.lang.UnsatisfiedLinkError: Could not load library. Reasons: [no lmdbjni64-99-vspmaster-SNAPSHOT in java.library.path, no lmdbjni-99-vspmaster-SNAPSHOT in java.library.path, no lmdbjni in java.library.path]
at org.fusesource.hawtjni.runtime.Library.doLoad(Library.java:182)
at org.fusesource.hawtjni.runtime.Library.load(Library.java:140)
at org.fusesource.lmdbjni.JNI.<clinit>(JNI.java:41)
at org.fusesource.lmdbjni.Env.create(Env.java:42)
at org.fusesource.lmdbjni.Env.<init>(Env.java:36)
at com.db.locrefdcache.App.main(App.java:27)
... 6 more
c) then I figured, I may not just be able to run mvn install for lmdbjni, and instead I need to explicitly build it's 64 bit Linux subsystem
so I did
cd lmdbjni/lmdbjni-linux64
mvn install
There I can I see that its trying to run configure script (generated by autotools), but I get the
...
[INFO] checking lmdb.h usability... no
[INFO] checking lmdb.h presence... no
[INFO] checking for lmdb.h... no
[INFO] configure: error: cannot find headers for lmdb
[INFO] rc: 1
[INFO] ------------------------------------------------------------------------
[ERROR] BUILD ERROR
So what I do not quite understand is whether lmdb files (lmdb.h , mdb.c, midl.h ) need to be explicitly dropped somewhere, or if hawkjni actually needs to be ran prior on them and create some sort of 'intermediate' c and h files that later on get dropped into this build environment.
Update with Compile Error I am getting, when using the LMDBJNI deephacks fork
Reason for recompile: DeepHacks's LMDBJNI project had produced a maven archive for LMDBJNI with latest LMDB, however it was compiled with Java 8 (which we do not yet use). So I need to recompile it with Java 7.
I modified pom.xml and changed Sources 1.8 to 1.7
Then, mvn install -P linux64 produces an error
...
[INFO] [hawtjni:build {execution: default}]
[INFO] Extracting /home/dev01/.m2/repository/org/deephacks/lmdbjni/lmdbjni/0.1.3-SNAPSHOT/lmdbjni-0.1.3-SNAPSHOT-native-src.zip to /home/dev01/devel/3dp/lmdbjni/lmdbjni-linux64/target/native-build-extracted
[INFO] executing: /bin/sh -c make install
[INFO] ------------------------------------------------------------------------
[ERROR] BUILD ERROR
[INFO] ------------------------------------------------------------------------
[INFO] build failed: org.apache.maven.plugin.MojoExecutionException: Make based build did not generate: /home/dev01/devel/3dp/lmdbjni/lmdbjni-linux64/target/native-build/target/lib/liblmdbjni.so

You might have better luck with this fork, which is actively being maintained
https://github.com/deephacks/lmdbjni
They've also provided LMDB itself in Maven, you can see how that was setup here
https://github.com/deephacks/lmdb

It builds fine on my machine with Java 7. Did you provide the correct profile when building the packages? For linux you must use: mvn install -P linux64

Related

Building gRPC C++ from source

I am trying to build gRPC c++ from source from Build gRPC C++
I have already installed bazel.
Though when I tried to build I am seeing following error.
$ bazel build :all
Starting local Bazel server and connecting to it...
ERROR: /home/bigz/.cache/bazel/_bazel_bigz/a68d37101a3d172c639dd67a1941f719/external/io_bazel_rules_python/python/pip.bzl:39:25: Traceback (most recent
call last):
File "/home/bigz/.cache/bazel/_bazel_bigz/a68d37101a3d172c639dd67a1941f719/external/io_bazel_rules_python/python/pip.bzl", line 37
repository_rule(<2 more arguments>)
File "/home/bigz/.cache/bazel/_bazel_bigz/a68d37101a3d172c639dd67a1941f719/external/io_bazel_rules_python/python/pip.bzl", line 39, in repositor
y_rule
attr.label(allow_files = True, <2 more arguments>)
'single_file' is no longer supported. use allow_single_file instead. You can use --incompatible_disable_deprecated_attr_params=false to temporarily disable th
is check.
ERROR: error loading package '': Extension file 'python/pip.bzl' has errors
ERROR: error loading package '': Extension file 'python/pip.bzl' has errors
INFO: Elapsed time: 7.505s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (0 packages loaded)
Could someone point what I am missing here.
When I tried to run bazel info, I am getting following error.
$ bazel info --incompatible_disable_deprecated_attr_params=false
ERROR: /home/bigz/.cache/bazel/_bazel_bigz/a68d37101a3d172c639dd67a1941f719/external/org_pubref_rules_protobuf/protobuf/internal/proto_compile.bzl:771:21: name 'FileType' is not defined
ERROR: error loading package '': in /home/bigz/.cache/bazel/_bazel_bigz/a68d37101a3d172c639dd67a1941f719/external/org_pubref_rules_protobuf/python/rules.bzl: in /home/bigz/.cache/bazel/_bazel_bigz/a68d37101a3d172c639dd67a1941f719/external/org_pubref_rules_protobuf/cpp/rules.bzl: in /home/bigz/.cache/bazel/_bazel_bigz/a68d37101a3d172c639dd67a1941f719/external/org_pubref_rules_protobuf/protobuf/rules.bzl: Extension 'protobuf/internal/proto_compile.bzl' has errors
Though bazel --version shows as bazel 2.0.0.
$ bazel --version
bazel 2.0.0
The latest commit of gRPC is 9dfbd34f5c0b20bd77658c73c59b9a3e4e8f4e14
$ git log -1
commit 9dfbd34f5c0b20bd77658c73c59b9a3e4e8f4e14 (HEAD, tag: v1.20.0)
Merge: 1b488f8361 ae72bf76b3
Author: Lidi Zheng <scallopsky#gmail.com>
Date: Mon Apr 15 15:38:24 2019 -0700
Merge pull request #18760 from lidizheng/v1.20.x
Bump version to v1.20.0
The commit of grpc you are trying to build is not compatible with the version of bazel you have installed (2.0.0).
The commit 9dfbd34f5c0b20bd77658c73c59b9a3e4e8f4e14 is quite old (15 April 19), where the HEAD of that repo (at the time of writing this) is at eba60d8dbe4099c34b8097b2c89998d4484740ac, which now in BUILDING.md shows you need at least version 1.0.0 (there is a bazel wrapper at tools/bazel that will intercept and use 1.0.0 anyway)
For your specific commit however, trying different versions, I was able to build :all label with bazel version 0.20.0
I'd recommend trying bazelisk. You can add it to your PATH, add a .bazelversion file to the repository and bazelisk will take care of downloading the required bazel version.

bazel-run does not load all tensorflow packages that are loaded at bazel build

I have a problem with run my c++ tensorflow app directly after reboot ubuntu.
normally I build app by bazel build
bazel build -c opt --local_resources 3072,4.0,1.0 --verbose_failures --config=monolithic //tensorflow/app/...
after build, when I execute
bazel-bin/tensorflow/app/run
everything is work perfectly but, when I reboot ubuntu and execute firstly bazel-bin I have spam of this errors.
May 11 12:29:05 tegra-ubuntu start.sh[800]: 2018-05-11 12:29:04.622723: E tensorflow/stream_executor/cuda/cuda_driver.cc:967] failed to alloc 2304 bytes on host: CUDA_ERROR_UNKNOWN
May 11 12:29:05 tegra-ubuntu start.sh[800]: 2018-05-11 12:29:04.622849: E tensorflow/stream_executor/cuda/cuda_driver.cc:967] failed to alloc 2304 bytes on host: CUDA_ERROR_UNKNOWN
I guess that is a problem in loaded packages. When bazel build is executing before bazel-run, loading packages is correct.
There is a way to only load missing packages without earlier building? I need to run app immediately after start but my building process is to long to building before run app
Solution:
SessionOptions opts;
opts.config.mutable_gpu_options()->set_allow_growth(true);
session->reset(tensorflow::NewSession(opts));
Does enabling gpu_options.allow_growth fix the problem?
https://devtalk.nvidia.com/default/topic/1029742/tensorflow-1-6-not-working-with-jetpack-3-2/

Grpc compilation for c++ with protobuf 2.6.1

We are not able to compile the grpc source code with protobuf version 2.6.1 on linux, when we try to compile with it states dependency on protobuf 3.0.0+. But we have our current code base existing with lower protobuf version, and we don’t want to upgrade to protobuf 3.0.0+ unless necessary for grpc compilation.
we are getting below error:
[MAKE] Generating /home/harman/sample_code/grpc_client/grpc/libs/opt/pkgconfig/grpc++.pc
[MAKE] Generating /home/harman/sample_code/grpc_client/grpc/libs/opt/pkgconfig/grpc++_unsecure.pc
DEPENDENCY ERROR
The target you are trying to run requires protobuf 3.0.0+
Your system doesn't have it, and neither does the third_party directory.
Please consult INSTALL to get more information.
If you need information about why these tests failed, run:
make run_dep_checks
Additionally, since you are in a git clone, you can download the
missing dependencies in third_party by running the following command:
git submodule update --init
Makefile:945: recipe for target 'stop' failed
make: *** [stop] Error 1

Can't install Bioperl Module (properly?)

I am not able to install a BioPerl Module via cpanm. With
sudo cpanm Bio::SeqIO
The output says:
--> Working on Bio::SeqIO
Fetching http://www.cpan.org/authors/id/C/CJ/CJFIELDS/BioPerl-1.6.922.tar.gz ... OK
Configuring BioPerl-1.006922 ... OK
Building and testing BioPerl-1.006922 ... FAIL
! Installing Bio::SeqIO failed. See /home/administrator/.cpanm/work/1398677477.16274/build.log for details. Retry with --force to force install it.
When adding --force it says:
--> Working on Bio::SeqIO
Fetching http://www.cpan.org/authors/id/C/CJ/CJFIELDS/BioPerl-1.6.922.tar.gz ... OK
Configuring BioPerl-1.006922 ... OK
Building and testing BioPerl-1.006922 ... FAIL
! Testing BioPerl-1.006922 failed but installing it anyway.
Successfully installed BioPerl-1.006922
1 distribution installed
So I am not sure whether I can use BioPerl now properly? Can I simply ignore that the testing failed?
I have already tried to reinstall CPAN with
o conf init
in the CPAN prompt, but the outcome was the same. I am working with Perl V5.14.2 and cpanm -v brings me:
cpanm (App::cpanminus) 1.6933 on perl 5.014002 built for x86_64-linux-gnu-thread-multi
Work directory is /home/administrator/.cpanm/work/1398677790.16651
You have make /usr/bin/make
You have LWP 6.05
You have /bin/tar: tar (GNU tar) 1.26
I am happy with any help, since I am a totally Linux newbie.
BioPerl uses many modules that you might not have installed. From BioPerl INSTALL file:
Remember that there are over 900 modules in BioPerl and the test
suite is running more than 12000 individual tests, a few failed tests
may not affect your usage of BioPerl
Most of the time you will not need these additional modules depending on what modules within BioPerl you are using. I'm sure that BioPerl has installed correctly on you system. I would work through http://www.bioperl.org/wiki/HOWTO:Beginners to learn BioPerl. If you get errors running these basic codes in the HOWTO, repost another question.
Good luck,
Paul

What are the steps needed to build WSO2 Carbon Platform Patch Release 4.0.x?

I'm trying to build the Carbon Platform from source.
I've tried this:
svn co https://svn.wso2.org/repos/wso2/carbon/platform/branches/4.0.0
cd patch-releases\4.0.9
set MAVEN_OPTS="-Xms512m -Xmx1024m -XX:MaxPermSize=1024m"
mvn clean install -Dmaven.test.skip=true
However, I had hit this problem: link
After fixing that problem, I have now run into this:
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:2.0:compile (default-compile) on project org.wso2.carbon.message.store: Compilation failure: Compilation failure:
[ERROR] C:\wso2\src\wso2_platform_branch_400\components\mediation\message-store\org.wso2.carbon.message.store\4.0.9\src\main\java\org\wso2\c
arbon\message\store\persistence\jms\JMSMessageStore.java:[29,39] cannot find symbol
[ERROR] symbol : class MessageStores
[ERROR] location: package org.apache.synapse.message.store
I must be doing something wrong - I keep hitting errors at each turn?
Question: What are the steps needed to build WSO2 Carbon Platform Patch Release 4.0.x?
The necessary steps are, (if you want to build patch-release 4.0.x)
1) build orbit 4.0.0/
Then build orbit/patch-release/4.0.x
2) build kernel 4.0.0/
Then build kernel/patch-release/4.0.x
3) build platform 4.0.0/
Then build platform/patch-release/4.0.x
This should work.