Build error from Installing Apache Zeppelin - build

I am at my wits end trying to get Apache Zeppelin running on my linux VM. I am following this tutorial:
http://madhukaudantha.blogspot.ca/2015/03/building-apache-zeppelin.html
I have done the following:
git clone to the machine
now trying to run 'mvn clean package'
I get the following error summary, I really need to get this running.
[INFO] Reactor Summary:
[INFO]
[INFO] Zeppelin .......................................... SUCCESS [16.124s]
[INFO] Zeppelin: Interpreter ............................. SUCCESS [36.026s]
[INFO] Zeppelin: Zengine ................................. SUCCESS [27.440s]
[INFO] Zeppelin: Spark ................................... SUCCESS [53.438s]
[INFO] Zeppelin: Markdown interpreter .................... SUCCESS [6.041s]
[INFO] Zeppelin: Angular interpreter ..................... SUCCESS [7.397s]
[INFO] Zeppelin: Shell interpreter ....................... SUCCESS [4.140s]
[INFO] Zeppelin: Hive interpreter ........................ SUCCESS [1:09.114s]
[INFO] Zeppelin: Tajo interpreter ........................ SUCCESS [15.153s]
[INFO] Zeppelin: web Application ......................... FAILURE [0.492s]
[INFO] Zeppelin: Server .................................. SKIPPED
[INFO] Zeppelin: Packaging distribution .................. SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 3:57.426s
[INFO] Finished at: Wed Apr 29 12:18:39 EDT 2015
[INFO] Final Memory: 78M/266M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal com.github.eirslett:frontend-maven-plugin:0.0.23:install-node -and-npm (install node and npm) on project zeppelin-web: The plugin com.github.eirslett:fron tend-maven-plugin:0.0.23 requires Maven version 3.1.0 -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the follow ing articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/PluginIncompatibleExceptio n
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn <goals> -rf :zeppelin-web
I have tried contacting the apache incubator team through their email (users#zeppelin.incubator.apache.org) however the emails bounced back. Ill continue to bounce around looking for a solution.

Download , extract and install Maven3 (if not already installed) and set maven in PATH
export PATH=$PATH:$MAVEN_HOME/bin
even i have updated the PATH variable with latest maven. mvn -version shows 2.x
now manually updated the maven using update-alternatives.
bdalab#solai:/opt$ sudo update-alternatives --install /usr/bin/mvn mvn $MAVEN_HOME/bin/mvn 1
bdalab#solai:/opt$ sudo update-alternatives --config mvn
Now, select a number referring to the recent maven3 installation, from the list of choices

I tried mvn install -DskipTests, with the following system configurations:
1) Java 7
2) Maven 3.3 - Download tar file. extract and update MAVEN_HOME in /etc/profile. Then run 'source' command
3) Spark 0.9.0
4) CDH4 cluster
Build is successful for me.

you can test your node and npm download url with wget first:
wget XXXnode-12.4.tar.gz
if this url can't download the file may be your network issuel

I think that you might be missing the installation of nodejs, npm. Please install nodejs, npm.
sudo yum install nodejs npm --enablerepo=epel

In my case below git config worked :
git config --global url."https://".insteadOf git://
Make sure Maven > v3.0 , npm , nodejs and bower should be installed.

For the issue was that I was using maven 3.2.
Installed maven 3.3.3 and everything went fine, build again and everything went ok.

I've the same problem with zeppelin and found the root cause to be a failure with 'bower install' command.
First you need to install nodejs
yum install -y node
then install bower
npm install -g bower
then install grunt
npm install -g grunt
run the following commands with a non-root user (i.e., create a zeppelin user)
cd zeppelin-web
bower install (takes some time to complete)
cd ..
mvn clean package -rf :zeppelin-web

https://github.com/NFLabs/zeppelin/issues/320
Switching github from git:// to https:// will do the trick.

Related

AWS beanstalk fails installing pip requisites when downloading tensorflow==2.4.0

I'm trying to deploy a flask app that predicts whether a lung x-ray picture is COVID-19 positive using a Tensorflow CNN. It is running on python 3.7
This is my requirements.txt
tensorflow==2.4.0
Flask==1.1.2
Keras==2.4.3
numpy==1.19.5
Pillow==8.3.1
After deployment it, the environment's health status is labelled severe with a '502 Bad gateway' error.
These are the errors in the eb-engine.log
2021/08/23 10:47:40.676374 [INFO] Installing dependencies with requirements.txt by using Pip
2021/08/23 10:47:40.676387 [INFO] Running command /bin/sh -c /var/app/venv/staging-LQM1lest/bin/pip install -r requirements.txt
2021/08/23 10:47:48.296114 [INFO] Collecting tensorflow==2.4.0
Downloading tensorflow-2.4.0-cp38-cp38-manylinux2010_x86_64.whl (394.8 MB)
2021/08/23 10:47:48.300932 [ERROR] An error occurred during execution of command [app-deploy] - [InstallDependency]. Stop running the command. Error: fail to install dependencies with requirements.txt file with error Command /bin/sh -c /var/app/venv/staging-LQM1lest/bin/pip install -r requirements.txt failed with error exit status 2. Stderr:ERROR: Exception:
Traceback (most recent call last):
File "/var/app/venv/staging-LQM1lest/lib/python3.8/site-packages/pip/_internal/cli/base_command.py", line 180, in _main
status = self.run(options, args)
File "... repeats itself until here:
File "/var/app/venv/staging-LQM1lest/lib/python3.8/site-packages/pip/_vendor/msgpack/fallback.py", line 896, in _pack
return self._buffer.write(obj)
MemoryError
2021/08/23 10:47:48.300957 [INFO] Executing cleanup logic
2021/08/23 10:47:48.309000 [INFO] CommandService Response: {"status":"FAILURE","api_version":"1.0","results":[{"status":"FAILURE","msg":"Engine execution has encountered an error.","returncode":1,"events":[{"msg":"Instance deployment failed to install application dependencies. The deployment failed.","timestamp":1629715668,"severity":"ERROR"},{"msg":"Instance deployment failed. For details, see 'eb-engine.log'.","timestamp":1629715668,"severity":"ERROR"}]}]}
2021/08/23 10:47:48.312983 [INFO] Platform Engine finished execution on command: app-deploy
2021/08/23 12:34:34.689865 [INFO] Starting...
2021/08/23 12:34:34.689926 [INFO] Starting EBPlatform-PlatformEngine
2021/08/23 12:34:34.689965 [INFO] reading event message file
2021/08/23 12:34:34.690627 [INFO] no eb envtier info file found, skip loading env tier info.
2021/08/23 12:34:34.690716 [INFO] Engine received EB command cfn-hup-exec
why is AWS-Beanstalk failing to install Tensorflow?
I changed the environment EC2 capacity from t2.small to c5.xlarge and the problem was solved.
Your application needs more memory space.

Warnings about javac when building QuestDB from source

I'm trying to build QuestDB from source on a newly setup machine and I get warnings about javac:
[INFO] --- maven-compiler-plugin:3.8.1:compile (default-compile) # questdb ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 1292 source files to /home/pi/questdb/core/target/classes
[WARNING] Unable to autodetect 'javac' path, using 'javac' from the environment.
What's a fix for this?
It looks like JAVA_HOME env var is not set. You can check by running the following in your shell:
echo $JAVA_HOME
On macOS, I have the following:
/Library/Java/JavaVirtualMachines/openjdk-11.jdk/Contents/Home
To check which runtime maven (brew install) is referencing:
mvn --version
Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
Maven home: /Users/me/maven/apache-maven-3.6.3
Java version: 11.0.9, vendor: Oracle Corporation, runtime: /usr/local/Cellar/openjdk#11/11.0.9/libexec/openjdk.jdk/Contents/Home
In ~/.zshrc (you may use ~/.bashrc) I have set JAVA_HOME with
export JAVA_HOME="/Library/Java/JavaVirtualMachines/openjdk-11.jdk/Contents/Home"
then try building again:
source ~/.zshrc
mvn clean package -DskipTests -P build-web-console,build-binaries

Run Packages Installation Commands on Amazon Linux 2 using EB Extensions

I'm running ASP.NET Core 3.1 Application on Elastic Beanstalk with Amazon Linux 2 EC2 instances. I need my app to be able to generate QR codes and for this I'm using QRCoder library. It worked on my Windows machine out of the box, but to make it work on Amazon Linux 2 machine I had to run the following commands via SSH:
sudo amazon-linux-extras install epel
sudo yum install libgdiplus
After running these commands EB app servers also need to be restarted for image generation to work.
I tried to script it by creating .ebextensions/install_gdi.config file in my application repo root directory with the following contents:
commands:
00_install_extras:
command: amazon-linux-extras install -y epel
01_install_gdi:
command: yum install -y libgdiplus
I rebuilt my Beanstalk environment to have fresh EC2 instances redeployed version, and it seems that EB extensions config file isn't doing anything. Images still cannot be generated until I connect via SSH and run those command manually on EC2 instance.
I read that there are alternative ways to automate instance provisioning for Amazon Linux 2 but I wasn't able to find good examples to explore things like pre deployment hooks and Buildfile and it also seems that EB extensions is the right tool for the job in this case.
My application is deployed to Elastic Beanstalk using AWS CodePipeline, which is hooked to a GitHub repo.
I tried to replicate the issue on EB with Amazon Linux 2, but your commands worked perfectly fine. I used Python platform, not .NET, but since both are based on AL2 I don't see why would they be different in this aspect.
To troubleshoot, you can ssh into your EB instance, or download EB logs from console and inspect /var/log/cfn-init-cmd.log. It should have details regarding install_gdi.config execution.
For comparision, I attach my output:
2020-08-30 05:34:52,631 P3617 [INFO] Command 01_install_gdi
2020-08-30 05:35:08,423 P3617 [INFO] -----------------------Command Output-----------------------
2020-08-30 05:35:08,423 P3617 [INFO] Loaded plugins: extras_suggestions, langpacks, priorities, update-motd
2020-08-30 05:35:08,423 P3617 [INFO] http://csc.mcs.sdsmt.edu/epel/7/x86_64/repodata/0e811d5e2eb547161695f97c2fb8c2eaa0ccfe0ba7125321404db90dab5af5b3-updateinfo.xml.bz2: [Errno 12] Timeout on http://csc.mcs.sdsmt.edu/epel/7/x86_64/repodata/0e811d5e2eb547161695f97c2fb8c2eaa0ccfe0ba7125321404db90dab5af5b3-updateinfo.xml.bz2: (28, 'Operation too slow. Less than 1000 bytes/sec transferred the last 5 seconds')
2020-08-30 05:35:08,423 P3617 [INFO] Trying other mirror.
2020-08-30 05:35:08,423 P3617 [INFO] 200 packages excluded due to repository priority protections
2020-08-30 05:35:08,423 P3617 [INFO] Resolving Dependencies
2020-08-30 05:35:08,424 P3617 [INFO] --> Running transaction check
2020-08-30 05:35:08,424 P3617 [INFO] ---> Package libgdiplus.x86_64 0:2.10-10.el7 will be installed
2020-08-30 05:35:08,424 P3617 [INFO] --> Processing Dependency: libXrender.so.1()(64bit) for package: libgdiplus-2.10-10.el7.x86_64
2020-08-30 05:35:08,424 P3617 [INFO] --> Processing Dependency: libcairo.so.2()(64bit) for package: libgdiplus-2.10-10.el7.x86_64
2020-08-30 05:35:08,424 P3617 [INFO] --> Processing Dependency: libexif.so.12()(64bit) for package: libgdiplus-2.10-10.el7.x86_64
2020-08-30 05:35:08,424 P3617 [INFO] --> Processing Dependency: libgif.so.4()(64bit) for package: libgdiplus-2.10-10.el7.x86_64
#
# more logs
#
2020-08-30 05:35:08,433 P3617 [INFO]
2020-08-30 05:35:08,433 P3617 [INFO] Installed:
2020-08-30 05:35:08,434 P3617 [INFO] libgdiplus.x86_64 0:2.10-10.el7
2020-08-30 05:35:08,434 P3617 [INFO]
#
# more logs
#
2020-08-30 05:35:08,435 P3617 [INFO] Complete!
2020-08-30 05:35:08,435 P3617 [INFO] ------------------------------------------------------------
2020-08-30 05:35:08,435 P3617 [INFO] Completed successfully.
The problem was with my buildspec.yml file.
It used to look like this and it didn't include .ebxtensions folder into the publish artifact folder:
version: 0.2
phases:
build:
commands:
- dotnet publish WebApi.csproj -c Release
artifacts:
files:
- bin/Release/netcoreapp3.1/publish/*
discard-paths: yes
I changed it to look like this and it works now:
version: 0.2
phases:
build:
commands:
- dotnet publish WebApi.csproj -c Release
artifacts:
files:
- '**/*'
base-directory: bin/Release/netcoreapp3.1/publish

Zeppelin build failure

I am trying to install zeppelin for spark1.0.0 and hadoop1.0.3.
Possible duplicate: Build error from Installing Apache Zeppelin
Well, the above thread didn't help much. I have the exact same output as the above thread: the web application fails to compile.
The command I ran was 'mvn install -DskipTests -Dspark.version=1.0.0 -Dhadoop.version=1.0.3'
I have maven 3.0.4 / java 1.8 / npm 1.1.4
[INFO] Zeppelin .......................................... SUCCESS [2.880s]
[INFO] Zeppelin: Interpreter ............................. SUCCESS [6.780s]
[INFO] Zeppelin: Zengine ................................. SUCCESS [3.394s]
[INFO] Zeppelin: Spark ................................... SUCCESS [31.284s]
[INFO] Zeppelin: Markdown interpreter .................... SUCCESS [1.936s]
[INFO] Zeppelin: Angular interpreter ..................... SUCCESS [1.916s]
[INFO] Zeppelin: Shell interpreter ....................... SUCCESS [1.905s]
[INFO] Zeppelin: Hive interpreter ........................ SUCCESS [2.226s]
[INFO] Zeppelin: Tajo interpreter ........................ SUCCESS [2.061s]
[INFO] Zeppelin: Flink ................................... SUCCESS [3.358s]
[INFO] Zeppelin: web Application ......................... FAILURE [0.215s]
[INFO] Zeppelin: Server .................................. SKIPPED
[INFO] Zeppelin: Packaging distribution .................. SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 58.228s
[INFO] Finished at: Tue Jun 16 11:50:03 KST 2015
[INFO] Final Memory: 47M/485M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal com.github.eirslett:frontend-maven-plugin:0.0.23:install-node-and-npm (install node and npm) on project zeppelin-web: The plugin com.github.eirslett:frontend-maven-plugin:0.0.23 requires Maven version 3.1.0 -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/PluginIncompatibleException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn <goals> -rf :zeppelin-web
Yes, I did already switched from git:// to https:// and tried again and it did not work.
As the error message point out :
[ERROR] Failed to execute goal
com.github.eirslett:frontend-maven-plugin:0.0.23:install-node-and-npm
(install node and npm) on project zeppelin-web:
The plugin com.github.eirslett:frontend-maven-plugin:0.0.23
requires Maven version 3.1.0 -> [Help 1]
You'll need to update your maven version from 3.0.4 to at least 3.1.0.
But you'll encounter another error : zeppelin-web: Failed to run task: 'npm install --color=false' failed
To correct that one, you'll need to run your mvn command as a sudoer or an administrator:
sudo mvn install -DskipTests -Dspark.version=1.0.0 -Dhadoop.version=1.0.3
I added the following elemets to .bowerrc file under ~/incubator-zeppelin-master/zeppelin-web.
"proxy" : "http://:",
"https-proxy" : "http://:"
After that, I tried to build again and it successfully completed without using sudo command.
Please refer to the follwing URL:
https://github.com/NFLabs/zeppelin/tree/master/zeppelin-web
I tried mvn install -DskipTests, with the following system configurations:
1) Java 7
2) Maven 3.3
3) Spark 0.9.0
4) CDH4 cluster
Build is successful for me.

How do I run Guava's unit tests?

I have cloned Guava's repository via the following command:
git clone https://code.google.com/p/guava-libraries/
Because I am not entirely familiar with Guava's structure, I tried the following commands:
cd guava-libraries
mvn test
Unfortunately, I received the following error (the entire output is available viewed here).
[INFO] ------------------------------------------------------------------------
[INFO] Building Guava GWT compatible libs
[INFO] task-segment: [test]
[INFO] ------------------------------------------------------------------------
Downloading: https://oss.sonatype.org/content/repositories/snapshots/com/google/guava/guava/13.0-SNAPSHOT/guava-13.0-SNAPSHOT-sources.jar
[INFO] Unable to find resource 'com.google.guava:guava:jar:sources:13.0-SNAPSHOT' in repository sonatype-nexus-snapshots (https://oss.sonatype.org/content/repositories/snapshots)
[INFO] ------------------------------------------------------------------------
[ERROR] BUILD ERROR
[INFO] ------------------------------------------------------------------------
[INFO] Failed to resolve artifact.
Missing:
----------
1) com.google.guava:guava:jar:sources:13.0-SNAPSHOT
Try downloading the file manually from the project website.
Then, install it using the command:
mvn install:install-file -DgroupId=com.google.guava -DartifactId=guava -Dversion=13.0-SNAPSHOT -Dclassifier=sources -Dpackaging=jar -Dfile=/path/to/file
Alternatively, if you host your own repository you can deploy the file there:
mvn deploy:deploy-file -DgroupId=com.google.guava -DartifactId=guava -Dversion=13.0-SNAPSHOT -Dclassifier=sources -Dpackaging=jar -Dfile=/path/to/file -Durl=[url] -DrepositoryId=[id]
Path to dependency:
1) com.google.guava:guava-gwt:jar:13.0-SNAPSHOT
2) com.google.guava:guava:jar:sources:13.0-SNAPSHOT
----------
1 required artifact is missing.
for artifact:
com.google.guava:guava-gwt:jar:13.0-SNAPSHOT
from the specified remote repositories:
central (http://repo1.maven.org/maven2),
sonatype-nexus-snapshots (https://oss.sonatype.org/content/repositories/snapshots)
How do I run Guava's unit tests?
You can not run just mvn test before complete installation to local repository. Please run first time:
mvn clean install
Then you can run only mvn test. SNAPSHOT's libraries are not available yet in repository.
Why use a snapshot revision when a release candidate is available from Maven Central?
<dependency>
<groupId>com.google.guava</groupId>
<artifactId>guava</artifactId>
<version>13.0-rc2</version>
</dependency>