Unable to add remote source with Vora 1.4 and Spark controller 2.0 - vora

Question

I have been try to connect Vora 1.4 to HANA 1.0 SPS12 through Spark controller 2.0. Remote source can be added to HANA studio but Vora tables are invisible. When trying to refresh remote source, a class not found error appears in Spark controller error log. Following are the error messages:
*
17/05/23 10:11:46 ERROR HanaSQLContext: Failed to set up Exended Store
java.lang.ClassNotFoundException: at java.lang.Class.forName0(Native
Method) at java.lang.Class.forName(Class.java:264) at
org.apache.spark.sql.hana.hdfs.store.HDFSStore$$anonfun$apply$2.apply(HDFSSt>ore.scala:212)
at
org.apache.spark.sql.hana.hdfs.store.HDFSStore$$anonfun$apply$2.apply(HDFSSt>ore.scala:187)
at scala.Option.getOrElse(Option.scala:120) at
org.apache.spark.sql.hana.hdfs.store.HDFSStore$.apply(HDFSStore.scala:186)
at
org.apache.spark.sql.hana.HanaSQLContext.getExtendedStore(HanaSQLContext.sca>la:104)
at
com.sap.hana.spark.core.session.Session.(SessionManager.scala:191)
at
com.sap.hana.spark.core.session.Session.(SessionManager.scala:166)
at
com.sap.hana.spark.core.session.Session$.apply(SessionManager.scala:136)
at
com.sap.hana.spark.core.session.SessionManager$.startNewSession(SessionManag>er.scala:72)
at
com.sap.hana.spark.network.CommandHandler$$anonfun$receive$3.applyOrElse(Com>mandRouter.scala:549)
at akka.actor.Actor$class.aroundReceive(Actor.scala:467) at
com.sap.hana.spark.network.CommandHandler.aroundReceive(CommandRouter.scala:>432)
at akka.actor.ActorCell.receiveMessage(ActorCell.scala:516) at
akka.actor.ActorCell.invoke(ActorCell.scala:487) at
akka.dispatch.Mailbox.processMailbox(Mailbox.scala:238) at
akka.dispatch.Mailbox.run(Mailbox.scala:220) at
akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDis>patcher.scala:397)
at
scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
at
scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1>339)
at
scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
at
scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java>:107)
*
Any one has any idea?
Thanks!

Answer 1

I would recommend using the new VoraODBC connectivity (a.k.a. SAP HANA Wire connectivity) to connect HANA to Vora 1.4. This gives easy access to tables in the new relational engine and disk engine without installing anything in addition.
See Vora 1.4 Developer Guide chapter "7.9 Accessing SAP Vora from SAP HANA" (see https://help.sap.com/viewer/p/SAP_VORA).

Answer 2

To troubleshooting the issue, I uninstalled Spark controller 2.0 and installed Spark controller 1.6. There are no more error message in Spark controller error log. But the issue remains, the Vora tables are invisible from remote source in HANA studio. There is no error in index server log on HANA side either. It seems that Vora 1.4 has serious problem with Spark controller as 1.4 is not on the any version of Spark controller compatibility matrix. I am hoping that someone already found a fix for this.
Spark controller 1.6 log:
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/hdp/2.5.3.0-37/spark/lib/spark-assembly-1.6.2.2.5.3.0-37-hadoop2.7.3.2.5.3.0-37.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/hdp/2.5.3.0-37/hadoop/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
INFO [main] (HanaESConfig.scala:67) - Loaded HANA Extended Store Configuration
Found Spark Libraries. Proceeding with Current Class Path
17/06/07 13:20:48 INFO Server: Starting Spark Controller
17/06/07 13:21:01 INFO CommandRouterDefault: Connecting to Vora Engine
17/06/07 13:21:01 INFO CommandRouterDefault: Initialized Router!
17/06/07 13:21:01 INFO CommandRouterDefault: Server started
17/06/07 13:21:28 INFO VoraClientFactory: returning a Vora catalog client of this Vora catalog server: bdesxi14.localdomain:20298
17/06/07 13:21:28 INFO CatalogFactory: returning a Vora catalog client of this Vora catalog server: bdesxi14.localdomain:20298
17/06/07 13:21:28 INFO CBinder: searching for compat-sap-c++.so at /opt/rh/SAP/lib64/compat-sap-c++.so
17/06/07 13:21:28 INFO CBinder: loading compat package from /opt/rh/SAP/lib64/compat-sap-c++.so
17/06/07 13:21:28 INFO CBinder: searching for libpam.so.0 at /lib64/libpam.so.0
17/06/07 13:21:28 INFO CBinder: loading libpam.so.0 from /lib64/libpam.so.0
17/06/07 13:21:28 INFO CBinder: loading library libprotobuf.so
17/06/07 13:21:28 INFO CBinder: loading library libprotoc.so
17/06/07 13:21:28 INFO CBinder: loading library libtbbmalloc.so
17/06/07 13:21:28 INFO CBinder: loading library libtbb.so
17/06/07 13:21:28 INFO CBinder: loading library libkrb5support.so
17/06/07 13:21:28 INFO CBinder: loading library libk5crypto.so
17/06/07 13:21:28 INFO CBinder: loading library libcom_err.so
17/06/07 13:21:28 INFO CBinder: loading library libkrb5.so
17/06/07 13:21:28 INFO CBinder: loading library libgsasl.so
17/06/07 13:21:28 INFO CBinder: loading library libgssapi_krb5.so
17/06/07 13:21:28 INFO CBinder: loading library libgssrpc.so
17/06/07 13:21:28 INFO CBinder: loading library libv2runtime.so
17/06/07 13:21:28 INFO CBinder: loading library libv2auth.so
17/06/07 13:21:28 INFO CBinder: loading library libv2net.so
17/06/07 13:21:28 INFO CBinder: loading library libv2catalog_connector.so