I am unable to run yum command in DSX environment. I need yum command access to install some packages.
Here's the error I am seeing when I type in "!yum install sox" command in DSX notebook:
Could not find platform independent libraries <prefix>
Could not find platform dependent libraries <exec_prefix>
Consider setting $PYTHONHOME to <prefix>[:<exec_prefix>]
ImportError: No module named site
This is possible duplicate of this
Can I use MeCab on IBM Data Science Experience
You cannot use yum in DSX Notebook attach to Apache Spark service on Bluemix.
Given Apache Spark service on bluemix does not allow user to install any root level packages which are usually** installed using yum as well.
The only alternative is for you to either try to see if you can download source using !wget or !curl and then try to see if you can compile it, if the package doesn't need any root permission technically , you should be able to compile and install it using make.
You can also raise feature enhancement for getting this package installed by default.
http://ibm.biz/dsxideas
Thanks,
Charles.
Related
I am trying to install the lustre clients on Unbuntu 20.04 nodes I have in GCP. Im using linux kernel version 5.15.0-1021-gcp.
I'm trying to install the client with the following code:
cd /home/apps/
mkdir lustre
git clone git://git.whamcloud.com/fs/lustre-release.git
cd lustre-release
git checkout 2.15.0
sh autogen.sh
./configure --prefix=/home/apps/lustre --disable-server --enable-client ## doesnt run! Fails at ./configures with error message "error: Run make config in /lib/modules/5.15.0-1021-gcp/build"
make debs
The configure step fails with an error about running make config in /lib/modules/5.15.0-1021-gcp/build. I tried running make config in /lib/modules/5.15.0-1021-gcp/build but was asked to input some values that I was unsure of.
I also tried downloading the deb package of the client software at
https://downloads.whamcloud.com/public/lustre/lustre-2.15.0/ubuntu2004/client/lustre-client-modules-5.4.0-96-generic_2.15.0-1_amd64.deb. However this is for the wrong linux kernel and I'm not sure what env variables need to be set for this package.
Anyone know how to install the client modules for lustre on Ubuntu?
You need to have the kernel sources or kernel-devel package that exactly match the kernel that you are installing on. This should also include the .config file that describes all of the options used when building your kernel.
Alternately, you could try a pre-built package, but it isn't clear if this will install on your kernel or not.
https://build.whamcloud.com/job/lustre-b2_15/40/arch=x86_64,build_type=client,distro=ubuntu2204,ib_stack=inkernel/
Using gcloud app deploy to deploy to App Engine will fail if cron.yaml or cron.xml contains timezone information, when run using Cloud SDK 297.0.0.
For example:
[INFO] GCLOUD: ERROR: (gcloud.app.deploy) An error occurred while parsing file: [/path/to/file/cron.yaml]
[INFO] GCLOUD: Unable to assign value 'America/New_York' to attribute 'timezone':
[INFO] GCLOUD: timezone 'America/New_York' is unknown
The workaround is to downgrade to version 296.0.1 of the Cloud SDK. (Substitute 296.0.1 for VERSION below.)
If you installed the SDK directly (outside of a package manager), you should use gcloud components to update: gcloud components update --version VERSION. This includes all installation mechanisms on this page (including the interactive installer, static versions, Windows installer, and Homebrew) but excludes the two following bullets.
If you installed via the rapture repo for Debian/Ubuntu: sudo apt-get update && sudo apt-get install google-cloud-sdk=VERSION-0
If you installed via the rapture repo for RedHat/CentOS: sudo yum downgrade google-cloud-sdk-VERSION
If for any reason any of the above do not work, use the download archive to manually download an older version, and install using http://cloud/sdk/docs/downloads-versioned-archives.
I got the same problem and I wrote a ticket to the GCP support.
It seems that they are not well aware of the bug.
after installing the gcp package to my airflow(1.10.9) set-up, i tried to call on the GSheetHook
https://airflow.readthedocs.io/en/latest/_api/airflow/providers/google/suite/hooks/sheets/index.html
but i get an error No module named 'airflow.providers'.
looking into the installed python packages for airflow, i do not find the providers package.
is the gcp airflow packagge working or am i missingg a step before i am able to use it?
EDIT: I have installed the gcp package using the pip installer: pip install apache-airflow[gcp]
and here's the list of the installed packages
The "providers" package is only available in Airflow Master. We plan to release each provider as a separate packages as "backport packages", most likely in a week or two from today.
PR to do that: https://github.com/apache/airflow/pull/8807
You should be checking https://airflow.apache.org/docs/1.10.9/ for Airflow 1.10.9 docs. You are looking at the docs for "latest" which is for Master.
It looks backport packages (providers) can now be installed in v1.10* using with following pip command $pip install apache-airflow-backport-providers-PACKAGE-NAME
i had a case where i need to configure an AWS structure similar to the architecture that is described in this article, is but this article is old, when i followed the steps i couldn't pass the step at which i run the script "vip_monitor.sh".
so be specific, at the step 5 by running the script i got the following error
Can't open /etc/profile.d/aws-apitools-common.sh
that shell script doesn't exist in the whole machine, how to solve this issue?
Thanks in advance
You will have to set api tools manually.
Ubuntu makes their own AMI's for Amazon, and they don't build the apitools into the images.
You can use official ubuntu documentation to fix these:
Install ec2 api tools
sudo apt-add-repository ppa:awstools-dev/awstools
sudo apt-get update
sudo apt-get install ec2-api-tools
actually i installed the ec2-api-tools J.Parashar instructed, and when i ran the script vip_monitor.sh it gave me the same error so i just took the missing script aws-apitools-common.sh file from an Amazon Linux instance and paste it at the path /etc/profile.d/ and then changed the mode to the script to executable chmod +x aws-apitools-common.sh and ran the script 'vip_monitor.sh'.
if you had the error :Unexpected operator run the script with bash ./vip_monitor.sh
I have updated Google Cloud SDK to the latest version 135.0.0 from
After the update , I got the following message.
WARNING: There are older versions of Google Cloud Platform tools on
your system PATH. Please remove the following to avoid accidentally
invoking these old tools:
/usr/bin/git-credential-gcloud.sh
/usr/bin/bq
/usr/bin/gcloud
/usr/bin/gsutil
So I have deleted all the above folders.
After that gsutil stopped working.
Please help me how can I resolve the issue.
The issue that it was installed via
sudo apt-get update && sudo apt-get install google-cloud-sdk
see
https://cloud.google.com/sdk/docs/#deb
and you should have used the same mechanism to upgrade.
gcloud is also kind of a package manager, and is able to upgrade itself and its depended packages. Unfortunately if you use gcloud itself to upgrade it installs it in different location. It likely does not work because new location needs to be added to your path.
You can try to reinstall googcle-cloud-sdk package via apt-get.