Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about programming within the scope defined in the help center.
Closed 5 years ago.
Improve this question
What is the simple and step by step guide for installing Apache spark 2.0 in Ubuntu(16.04) local machine accessed using python(pyspark). I have tried to search web most of them end up in one or other error.
Courtesy : https://www.santoshsrinivas.com/installing-apache-spark-on-ubuntu-16-04/
I am writing my steps here as I saw few steps are different than expected (at least for ubuntu 16.04)
So here are the steps
step 1:
Download the latest pre-built version from http://spark.apache.org/downloads.html
step 2:
Unzip and move Spark
cd ~/Downloads/
tar xzvf spark-2.0.1-bin-hadoop2.7.tgz
mv spark-2.0.1-bin-hadoop2.7/ spark
sudo mv spark/ /usr/lib/
step 3:
Install SBT
echo "deb https://dl.bintray.com/sbt/debian /" | sudo tee -a /etc/apt/sources.list.d/sbt.list
sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 2EE0EA64E40A89B84B2DF73499E82A75642AC823
sudo apt-get update
sudo apt-get install sbt
Note:- you might have some issues in this step. so please refer the link provided at the end.
step 4:
make sure Java is installed. if not
sudo apt-add-repository ppa:webupd8team/java
sudo apt-get update
sudo apt-get install oracle-java8-installer
step 5:
Configure Spark
cd /usr/lib/spark/conf/
cp spark-env.sh.template spark-env.sh
vi spark-env.sh
and add following
JAVA_HOME=/usr/lib/jvm/java-8-oracle
SPARK_WORKER_MEMORY=4g
step 6:
configure bash file.
open the bashrc file using below command
vi ~/.bashrc(if you want to use vi editor)
subl ~/.basrc(if you want to use sublime)
copy the below in the bashrc file
export JAVA_HOME=/usr/lib/jvm/java-8-oracle
export SBT_HOME=/usr/share/sbt-launcher-packaging/bin/sbt-launch.jar
export SPARK_HOME=/usr/lib/spark
export PATH=$PATH:$JAVA_HOME/bin
export PATH=$PATH:$SBT_HOME/bin:$SPARK_HOME/bin:$SPARK_HOME/sbin
start the bash file using below command
source ~/.bashrc
step 7:
try now
pyspark
Note: in case you get stuck with sbt installation (usual case for ubuntu 16.04) try the below link
https://askubuntu.com/questions/771839/problem-installing-package-git-all/771889#771889
Related
Synopsis
I'm trying to build a Docker image, but it fails because one of the packages I'm trying to get with apt install prompts the user during the install process. I would like to reply to this prompt, but I can't figure out how to do it non-interactively.
Description
I'm building a Docker image, and my Dockerfile has the following line:
RUN apt install -y texlive-latex-extra
(This package has some LaTeX libraries that I need.)
During installation, this halts with:
Setting up tzdata (2018d-1) ...
debconf: unable to initialize frontend: Dialog
debconf: (TERM is not set, so the dialog frontend is not usable.)
debconf: falling back to frontend: Readline
Configuring tzdata
------------------
Please select the geographic area in which you live. Subsequent configuration
questions will narrow this down by presenting a list of cities, representing
the time zones in which they are located.
1. Africa 6. Asia 11. System V timezones
2. America 7. Atlantic Ocean 12. US
3. Antarctica 8. Europe 13. None of the above
4. Australia 9. Indian Ocean
5. Arctic Ocean 10. Pacific Ocean
Geographic area:
At this point, it is waiting for some input. (There's another prompt after this for selecting timezone—I assume this is important to know for the \today directive in LaTeX files. ¯\_(ツ)_/¯)
How can I answer this non-interactively?
What I've tried so far
I've tried doing this:
apt install -y texlive-latex-extra <(echo 12 && echo 2)
and this:
echo 12 && echo 2 | apt install -y texlive-latex-extra
The first one died with this error:
apt install -y texlive-latex-extra <(echo 12 && echo 9)
and the second one seemed to have no effect.
For reference, here is my Dockerfile up until this point:
FROM ubuntu:latest
RUN apt update && apt upgrade -y && apt install -y curl bzip2 tar make gcc wget gnupg unzip
RUN apt install -y texlive
RUN apt install -y nodejs npm git
RUN npm install -g bower
RUN apt install -y texlive-latex-extra
UPDATE
I found something close here which suggested running apt install with DEBIAN_FRONTEND=noninteractive. This solved my problem sufficiently. :) However, I still would like to know how to respond to prompts, as the solution offered there only offered how to suppress them.
If you want to script a terminal interaction, you could use expect on Linux (that might not be very easy; you need to predict the interactions).
Remember that terminal emulators are complex and arcane things (because terminals like VT100 have been complex). See termios(3), pty(7) and read The Tty demystified.
The specific install prompt mentioned in the question is caused by the package tzdata. I managed to get it configured non-interactively in my docker build by setting these environment variables:
DEBIAN_FRONTEND=noninteractive TZ=Etc/UTC apt-get install -y texlive-luatex texlive-latex-extra texlive-font
I found this solution over here: https://serverfault.com/questions/949991/how-to-install-tzdata-on-a-ubuntu-docker-image
I need to install php-imap on amazon ec2 linux 2 instance.
All the php stuff is inside amzn2extra-lamp-mariadb10.2-php7.2 but php-imap package is missing.
Any advice ?
Thanks
I installed it like this.
I downloaded the php source code of the currently installed version in my Amazon Linux 2
wget http://php.net/get/php-7.2.8.tar.bz2/from/a/mirror
Unpacked it and went into php-7.2.8/ext/imap/
Compiled extension:
phpize
./configure
I got some errors.
Some U8T_CANONICAL stuff so
sudo yum install libc-client-devel
Then libc-client.a not found so created a symlink for it:
cd /usr/lib
sudo ln -s /usr/lib64/libc-client.a
some other imap library error so:
sudo yum install uw-imap-static
I got some other errors so the working configure line was:
./configure --with-kerberos --with-imap-ssl
make
SUCCESS!
cd php-7.2.8/ext/imap/modules
sudo cp imap.so /usr/lib64/php/modules/
Created an ini file to load it:
sudo vi /etc/php.d/30-imap.ini
added to the file this content:
extension=imap
restarted php service (you might need to restart httpd depending on your php installation):
sudo systemctl restart php-fpm
PHPinfo now contains:
imap
IMAP c-Client Version 2007f
SSL Support enabled
Kerberos Support enabled
For those who are facing the U8T_CANONICAL error and can't being able to install the libc-client-devel on Amazon Linux 2 AMI need to do something extra to resolve the U8T_CANONICAL error.
The main problem is that libc-client-devel isn't available so we need to do it in another way.
I found this link while searching for my problem. The solution that is posted here is solid. The following link that i gave is for those who are not being able to install the libc-client-devel will be benifited from the link.
Or you could just copy paste the commands i'm giving below.
1 - sudo yum groupinstall "Development Tools"
2 - sudo amazon-linux-extras install epel
3 - yum install epel-release
4 - sudo yum install libc-client-devel uw-imap-static openssl-devel
5 - sudo ln -s /usr/lib64/libc-client.a /usr/lib
The rest can be followed from respondent.
Again thanks to the person who answered.
Same problem here.
I came from debian/ubuntu and i miss a lot of php packages on aws linux2.
I know it's still a candidate and not a final LTS release. Only worked for a month with this distro and saw a lot of changes and updates. Some php dependencies can be solved with pear or pecl, but I couldn't get nothing working on php-imap.
php-imap depends on libc-client (maybe that's a problem?)
Here is my solution.
Search for the libc-client rpm that correspond to your version and
install manually.
Search for the rpm that corresponds to your php installed version. In my case it was 7.2.5**-2** about 2 weeks ago. (today is
7.2.5**-3** ) and isntall manually
CODE
wget https://dl.iuscommunity.org/pub/ius/stable/CentOS/7/x86_64//php72u-imap-7.2.5-2.ius.centos7.x86_64.rpm -P /tmp
wget http://dl.fedoraproject.org/pub/epel/7/x86_64/Packages/l/libc-client-2007f-16.el7.x86_64.rpm -P /tmp
rpm -i /tmp/libc-client-2007f-16.el7.x86_64.rpm
rpm -i /tmp/php72u-imap-7.2.5-2.ius.centos7.x86_64.rpm
If it fails because Failed dependencies:
rpm -ivh /tmp/php72u-imap-7.2.5-2.ius.centos7.x86_64.rpm --nodeps
I dont like this solution, but can't find for now any other solution more stable and easy. It will be a pain with updates....
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about a specific programming problem, a software algorithm, or software tools primarily used by programmers. If you believe the question would be on-topic on another Stack Exchange site, you can leave a comment to explain where the question may be able to be answered.
Closed 1 year ago.
Improve this question
Following AWS instruction to install Docker, After ssh to the ubuntu instance, I am getting the terminal as
ubuntu#ip-<private ip>:~$ but step 3 says
[ec2-user ~]$ sudo yum update -y
Not sure if I should continue wit the command or I need to create a user and re login as that user? I am new to AWS and my linux practice is 10 years old.
And if a user needs to be created, is that via the current ssh or the AWS console?
Thanks
You use simple curl command to install docker on any Linux machine.
curl -SsL https://get.docker.com | bash
Above command will automatically solve all the dependencies and install docker.
The documentation shows how to install Docker on Amazon Linux instances not ubuntu. The user youre logged in with doesnt matter, just replace the yum commands with the apt-get equivalents or switch to using an Amazon Linux AMI.
For Ubuntu, you can use:
$ sudo apt-get update
you might have been checking the documentation of RHEL which needs
$ sudo yum update -y
If you are working behind a proxy, Make sure you configure the proxy for Docker.
Hope it helps.. :)
Follow below commands on ubuntu ec2 :
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -;
sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable";
sudo apt-get update -y;
sudo apt-cache policy docker-ce; ( Here select the required one from this step for the next step)
sudo apt-get install docker-ce=5:18.09.2~3-0~ubuntu-xenial;
sudo service docker start;
HI everybody i am using CentOs7 and havae some troubles with the installation of the contextBroker. I followed the installation on the added the Fiware Repo in my yum repo.
Here is a list of my yum repo:
repo id repo name status
base/7/x86_64 CentOS-7 - Base 9,007
epel/x86_64 Extra Packages for Enterprise Linux 7 - x86_64 10,368
extras/7/x86_64 CentOS-7 - Extras 356
fiware Fiware Repository 176
mongodb MongoDB repo 279
updates/7/x86_64 CentOS-7 - Updates 2,070
repolist: 22,256
I am always getting an error message that there are some failed dependencies
libboost_filesystem-mt.so.5()(64bit) is needed by contextBroker-0.15.0-1.x86_64
libboost_system-mt.so.5()(64bit) is needed by contextBroker-0.15.0-1.x86_64
libboost_thread-mt.so.5()(64bit) is needed by contextBroker-0.15.0-1.x86_64
Here is the output of my config file for the fiware.repo
[fiware]
name=Fiware Repository
#baseurl=http://repositories.lab.fiware.org/repo/rpm/$releasever
baseurl=http://repositories.lab.fiware.org/repo/rpm/x86_64/
gpgcheck=0
enabled=1
I found a possible solution under this link, but unfortunately this is not working. In the comment section there is an entry which says that there is a problem with CentOs7 but that doesn`t solve my problem
Btw: this is also not working Boost-Libboost is needed in Centos7 (ContexBroker)
So my question is, did someone installed the contextBroker on CentOs7 via yum and if he/she did how can i solve this problem ?
As you mention, there isn't official support for CentOS 7 at the present moment. However, you could do the following to get a running binary with contextBroker in your CentOS 7 system (actually, this a "quick and dirty" receipt based on the build from sources procedure documented in the Orion manual):
sudo yum install git wget
wget https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
sudo rpm -i epel-release-latest-7.noarch.rpm
sudo yum install make cmake gcc-c++ scons
sudo yum install boost-devel libcurl-devel gnutls-devel libgcrypt-devel libuuid-devel
# Install mongoDB driver as described in the documentation
# Install rapidjson as described in the documentation
# Install libmicrohttpd as described in the documentation
mkdir ~/src
cd ~/src
git clone https://github.com/telefonicaid/fiware-orion.git
cd fiware-orion
# optionally, you can set an specific version with a checkout command
# e.g. 'git checkout 1.2.1'. Otherwise, you will build the latest
# code from develop branch
make
BUILD_RELEASE/src/app/contextBroker/contextBroker --version
Moreover, you can use the following to generate a RPM file:
sudo yum install rpm-build
make rpm
# The RPM is generated in rpm/RPMS/x86_64/contextBroker-1.2.0_next-dev.x86_64.rpm
# (version number may vary)
That RPM file is supposed to work in other CentOS 7 systems and you can install just using sudo yum install /path/to/contextBroker-1.2.0_next-dev.x86_64.rpm.
I am trying to install docker from the source code downloaded from github.com/docker/docker
I am unable to install it from the source code .
The Makefile present creates a image , but i want to install it in my system.
Can anyone suggest solution ?
I am using UBUNTU 14.04
Well, idk if this works for your linux distro. (looks like it is ubuntu) but i run kali linux and even if we have different commands to use the process is just as same in every linux distro.
first, before we jump on, we need to update our linux repos.(repositories)
sudo apt update
and,
sudo apt-get update
then,
sudo apt install git
[This installs git]
Now we can start cloning git repos. into our system
go to your desired folder/working directory and type:
sudo git clone "link of the git repo. without the commas"
i would better suggest you to just:
sudo apt install docker.io
[To install docker by apt]
it's better to install it via the docker package and update it to the last version. This is the best way to install docker.