I'd like to use caffe2 with GPU support. I succesfully installed caffe2 (Ubuntu 16.04, python2.7) with conda environment (command : conda install pytorch-nightly -c pytorch)
It is successfully installed (I checked it with the command: python2 -c 'from caffe2.python import core' 2>/dev/null && echo "Success" || echo "Failure" and it says "Success")
However, when I check caffe2 GPU build (command : python2 -c 'from caffe2.python import workspace; print(workspace.NumCudaDevices())), it returns 0.
I already have cuda, cuDNN, nccl and I don't understand why caffe2 does not detect available GPU..
I guess you are going to implement Detectron (otherwise nobody wanna use this dumb Caffe2 these days)
I'm pretty sure that it is caused by the mismatch of the CUDA version and CuDNN. I got stacked by this problem for a while (you don't know which version is correct for the Caffe2), finally, I got two solutions at almost the same time. Both of them work for me.
At first, just update the Nvidia driver to the latest version. My version is updated to 410.78, you can simply update the driver by selecting the certain driver in the System settings-> Software and update-> Additional Drivers.
Don't forget to restart your PC.
Then, there are two ways to implement it.
Build the environment with a Docker.
It is simple and fast. You just install Docker (as well as nvidia-docker for GPU usage) and use this pre-implemented environment with this command:
sudo docker pull ylashin/detectron
sudo nvidia-docker run --rm -it ylashin/detectron
Then you can test your Caffe2 with that NumCudaDeivce command.
It works for me!
See the whole introduction here, thanks to the efforts:
Build a Detectron environment with Docker
If you have some problem with the Docker installation (especially for the nvidia-docker), you can just skip it to the next one.
Use Detectron2
The newest Detectron is published recently (actually three days ago!). We can now work on that one now which is supported by Pytorch.
Here is the Detectron2:
Detectron2 link
Just go to the lastest one, you can even distribute everything in the Google Colab, it's much easier.
Related
Right now, I am tring to build Chromium on my own computer under instructions at https://chromium.googlesource.com/chromium/src/+/master/docs/linux/build_instructions.md#Build-Chromium
At first, it went smoothly. However when I run
$ autoninja -C out/Default chrome
when I run this to start build. It started. And then an error occured and the build process is terminated.
The error is
NameError: name 'unicode' is not defined" and I know it is a common error associated with the difference between python2 and python3.
However, I don't know how to let the ninja use only python2 and avoid this bug.
If you can help me, I will really appreciate it.
I recently ran into this issue when attempting to home build the Brave browser, which is chromium based. It appears that chromium build is based on python2, which is fine, but uses plain calls to "python" in the scripts. Obviously, essentially all modern distros put a symlink to python3 at python in the PATH (generally /usr/bin/python -> /usr/bin/python3). I still retain hope that there is a way to configure the "python path" in the build system, but have not found it and I am not too familiar with ninja.
The first way I "solved" this, and managed a successful build is one I consider a hack and do not recommend as a solution without the stipulation that it leaves your system broken so don't run too many other things. This is to manually temporarily update the symlink. Assuming you have both /usr/bin/python2 and /usr/bin/python3 on your system, remove the symlink at /usr/bin/python and replace it with one to python2:
sudo rm /usr/bin python
sudo ln -s /usr/bin/python2 /usr/bin/python
You can then run the chromium build, and reverse the above change when completed. Note that anything on your system that requires python3 to be at the symlink python cannot be run during this time. I have successfully used this technique to build Brave (and then reverse the change after the build completed).
I can offer you some other options too. One is to set up pyenv. If you search for that, you will find it is a suite that allows you to manage multiple python versions and have different directories use different ones. I researched this, and assume it can work, but have not done it myself.
The next option, which I am currently undertaking right now, is basically a manual watered down version of what pyenv does. I will let you know how it goes. The essence is to add a symlink in the path that will redirect python calls for that instance of the shell (and all subshells it creates). Assuming you are in the source directly at the obvious path:
mkdir .shims
ln -s /usr/bin/python2 .shims/python
export PATH=/home/yourname/src/Chromium/.shims:$PATH
python --version
This should print out whatever version of python2 you have rather than python. Anytime you come back to work here, you have to export the new PATH again. The rest of your system is unaffected, and still finds python3 at python. You can open up a new terminal and type python --version to verify that.
I have Chromium building on Fedora 33. Somewhere in the Chromium build, /usr/bin/python is getting run explicitly. On Fedora, that means Python 3.
If you change /usr/bin/python to point to python2.7, I bet your build will work.
Creating aliases for "python" won't work.
If this works, consider using the alternatives command to easily change /usr/bin/python:
alternatives command for /usr/bin/python
I want to use the C++ kernel with Jupyter Notebook and lab on Windows 10. I have tried following the instructions at https://github.com/QuantStack/xeus-cling?fbclid=IwAR3EOC0yxcpe3hf0Lo82V1ioCZYQzagirnY-y4uVvWy6dOIpUL5TthHcx7M.
I am seeing the following output:
(base) C:\Users\Rogan>conda activate cling
(cling) C:\Users\Rogan>conda install xeus-cling -c QuantStack
Collecting package metadata (current_repodata.json): done
Solving environment: failed with current_repodata.json, will retry with next repodata source.
Collecting package metadata (repodata.json): done
Solving environment: failed
PackagesNotFoundError: The following packages are not available from current channels:
- xeus-cling
Current channels:
---
To search for alternate channels that may provide the conda package you're
looking for, navigate to
https://anaconda.org
and the search bar at the top of the page is:
(cling) C:\Users\Rogan>
Can anyone tell me what the problem might be?
I would like to add Java, R, and C later on but am unable to even get C++ working.
cling is not available on windows environment, so you should install Linux on your windows. Download WSL , and try above codes on Linux kernel.
To give as much background as possible -
I have a machine learning model trained using keras i'm trying to embed on an nvidia jetson tx2.
I have set up tensorflow on there (a bit of a pain in itself) however when i run my script i'm hitting an error with protobuf.
Using TensorFlow backend.
[libprotobuf FATAL google/protobuf/stubs/common.cc:61] This program requires version 3.1.0 of the Protocol Buffer runtime library, but the installed version is 2.6.1. Please update your library. If you compiled the program yourself, make sure that your headers are from the same version of Protocol Buffers as your link-time library. (Version verification failed in "external/protobuf/src/google/protobuf/any.pb.cc".)
terminate called after throwing an instance of 'google::protobuf::FatalException'
what(): This program requires version 3.1.0 of the Protocol Buffer runtime library, but the installed version is 2.6.1. Please update your library. If you compiled the program yourself, make sure that your headers are from the same version of Protocol Buffers as your link-time library. (Version verification failed in "external/protobuf/src/google/protobuf/any.pb.cc".)
Aborted (core dumped)
So - i upgraded protobuf using pip at first but i thought the clash is because c++ version is taking priority and the version in linux was still stating 2.6.1 however after building in c++ the version is now shown as 3.1.0 however i am still getting the same error.
From the nvidia dev forums i received some feedback
"/usr/lib/aarch64-linux-gnu/libprotobuf.so.9.0.1
This means that C/C++ code will find version 2.6.
pip install protobuf-3.1.0-py2.py3-none-any.whl
This means that Python code will find version 3.1.
You need to upgrade the C++ (system) library to match version 3.1.
I don't think there's a default package on Ubuntu that does this, so you will have to either hack it by building your own and installing it on top of the system package, or you will have to find a deb package that has a newer version that will still install on your current system."
Ive really been struggling with this as i cant find a way to upgrade the system files. Any help would be much appreciated
Thanks
edit: i'm also wondering could this be a clash with GTK (i am also using openCV here so thats worth a mention!)
Please check your version of libprotobuf-dev.
Please try to uninstall your existing one on your PC with following commands:
apt-get remove --purge libprotobuf-dev
Then, build new version of libprotobuf-dev:
apt-get install autoconf automake libtool curl make g++ unzip
wget https://github.com/google/protobuf/releases/download/v3.5.0/protobuf-cpp-3.5.0.tar.gz
tar -xvf protobuf-cpp-3.5.0.tar.gz
cd protobuf-3.5.0
./autogen.sh
./configure
make
make check
sudo make install
sudo ldconfig
Good luck.
References:
https://github.com/BVLC/caffe/issues/5711
https://github.com/google/protobuf/issues/2979
step 1:
first, uninstall with purge protobuf
sudo apt-get remove --purge libprotobuf
step 2:
start a fresh one
$ wget https://raw.githubusercontent.com/jkjungavt/jetson_nano/master/install_protobuf-3.6.1.sh
sudo chmod +x install_protobuf-3.6.1.sh
./install_protobuf-3.6.1.sh #this time will take 30 min on my board.
I am using a virtualenv for a django setup. I am trying to build a view that pulls data from logs and then graphs the data. Eventually I would like to have this real-time and live. If you have any recommendations on other solutions that would suit my project best, please do not hesitate to include them in the comment fields below.
I have attempted to install matplotlib from pip using pip install matplotlib.
I receive the following message:
* The following required packages can not be built:
* freetype
I then validated that it was installed
yum install freetype
Package freetype-2.3.11-14.el6_3.1.x86_64 already installed and latest version
I then found that there is a python-matplotlib which is an older version .99. However, I want to keep this inside of the virtual environment and not system wide.
find / -name *freetype*
/var/lib/yum/yumdb/f/d2807dcfe3762c0b9f8ef1d9bf0f05788e73282a-freetype-2.3.11-14.el6_3.1- x86_64
/usr/lib64/libfreetype.so.6.3.22
/usr/lib64/libfreetype.so.6
/usr/share/doc/freetype-2.3.11
I searched all over stackoverflow and only saw solutions for ubuntu which did not transfer over to centos.
Thank you for your time,
John
pip is going to compile matlibplot on your local machine, so you'll need freetype development headers installed as well.
CentOS 6+, Fedora, etc.:
$ sudo yum -y install freetype freetype-devel libpng-devel
On older operating systems (e.g. CentOS 5), you may run into a more specific freetype versioning issue with newer releases of matlibplot. If you're version agnostic, sticking with a legacy 1.3.x release will negate these dependency issues:
$ pip install matplotlib==1.3.1
Please note, you may need to downgrade your numpy to 1.8 in order to make matplotlib 1.3 work.
$ pip install numpy==1.8
Good luck!
I have just had a similar (albeit not exactly the same) situation. I'll write it up here as this page comes up among the first search results.
CentOS 5
pip install matplotlib complains about freetype
Both freetype and freetype-devel are installed.
~/.pip/pip.log provides the explanation of the problem. There is the line:
freetype: no [Requires freetype2 2.3 or later. Found 2.2.1.]
Obviously, the solution is either to upgrade freetype or downgrade matplotlib.
The second is easier (assuming I am OK with the older version).
pip install matplotlib==1.3.1 works fine.
On the matplotlib installation, this is what I did. Not sure if this is going to help you. Just followed the steps here:
http://pkgs.org/centos-6/centos-x86_64/python-matplotlib-0.99.1.2-1.el6.x86_64.rpm.html
I did not use pip, btw and have CentOS 6.4.
I had this happen to me in two different situations, see if yours is one of them:
freetype was installed, but not in the $PATH yet. Just exiting the shell and starting a new one fixed this.
I was building matplotlib from source, and trying to build from the master branch. After I switched to v1.3.x it correctly detected freetype.
I am tearing my hair out trying to install Spatialite for GeoDjango!
I am already using Homebrew, it's generally easy and convenient so I initially tried to follow the Homebrew instructions for GeoDjango.
But this stops short of installing any database, i.e. Spatialite. The next step is to try and install Spatialite itself, but there are no Homebrew-specific instructions provided by Django docs.
I found this tutorial which looks perfect - a Homebrew and virtualenv-friendly install of Spatialite for GeoDjango.
But it doesn't work... it appears that my pysqlite is linked against the non-spatial-enabled version of SQLite that comes with OS X, rather than the Spatial-ised one I installed from Homebrew, I get this error when Django tried to connect to the db:
"The pysqlite library does not support C extension loading. Both SQLite and pysqlite must be configured to allow the loading of extensions to use SpatiaLite."
The author of pysqlite hasn't responded to my pleas for help on Github and I haven't found anything via Google.
So I went back to the drawing board and decided to follow the "Mac OS X-specific instructions" in the GeoDjango docs... by installing the various geo libs from the KyngChaos binary packages.
The docs say "Install the packages in the order they are listed above" but I found I couldn't install UnixImageIO without installing PROJ first. The link in the docs to download Spatialite binaries (http://www.gaia-gis.it/spatialite-2.3.1/binaries.html) is broken so I used the "Spatialite Tools v4.1" from KyngChaos instead.
Proceeding to the next step I get this error:
$ spatialite geodjango.db "SELECT InitSpatialMetaData();"
SQLite header and source version mismatch
2013-10-17 12:57:35 c78be6d786c19073b3a6730dfe3fb1be54f5657a
2013-09-03 17:11:13 7dd4968f235d6e1ca9547cda9cf3bd570e1609ef
Not really sure what's wrong at this point.
There is someone else here on SO who has gone the KyngChaos route and just ends up with the same "Both SQLite and pysqlite must be configured to allow the loading of extensions" error I got from the Homebrew route anyway.
I found this ticket #17756 for adding pyspatialite support to Django - pyspatialite is supposed to be an easier way to pip install everything but unfortunately it doesn't work either (see comments towards bottom of ticket).
I'm a bit reluctant to start trying to build everything from source by hand as it seems likely I'll just run into the same problems again, but spending hours Googling for info about cryptic compiler errors, magic flags and paths etc along the way.
I'm about ready to give up and just use Postgres/PostGIS.
I was able to get this working now, using the tip here:
https://github.com/ghaering/pysqlite/issues/60#issuecomment-50345210
I'm not sure if it was using the real paths that fixed it, or just the Homebrew kegs or underlying packages have been updated and now install cleanly. Still, it works now.
I reproduce below the steps I took:
brew update
brew install sqlite # 3.8.5
brew install libspatialite # 4.2.0
brew install spatialite-tools # 4.1.1
git clone https://github.com/ghaering/pysqlite.git
cd pysqlite
(where brew reported I had existing versions I unlinked them and installed the latest as commented above)
then edited setup.cfg to comment out #define=SQLITE_OMIT_LOAD_EXTENSION and specify the paths:
include_dirs=/usr/local/opt/sqlite/include
library_dirs=/usr/local/opt/sqlite/lib
activated the virtualenv where I want it installed, then
python setup.py build
python setup.py install
(build_static still fails with clang: error: no such file or directory: 'sqlite3.c')
(maybe I should have done pip install . as suggested in the github issue)
now the spatialite geodjango.db "SELECT InitSpatialMetaData();" succeeds, albeit with an ignorable error:
InitSpatiaMetaData() error:"table spatial_ref_sys already exists"
i.e. it's probably not even necessary to run that command
When I was istalling this i follow this instructions https://docs.djangoproject.com/en/dev/ref/contrib/gis/install/spatialite/#pysqlite2
pysqlite2
If you’ve decided to use a newer version of pysqlite2 instead of the sqlite3 Python stdlib module, then you need to make sure it can load external extensions (i.e. the required enable_load_extension method is available so SpatiaLite can be loaded).
This might involve building it yourself. For this, download pysqlite2 2.6, and untar:
$ wget https://pypi.python.org/packages/source/p/pysqlite/pysqlite-2.6.3.tar.gz
$ tar xzf pysqlite-2.6.3.tar.gz
$ cd pysqlite-2.6.3
Next, use a text editor (e.g., emacs or vi) to edit the setup.cfg file to look like the following:
[build_ext]
#define=
include_dirs=/usr/local/include
library_dirs=/usr/local/lib
libraries=sqlite3
#define=SQLITE_OMIT_LOAD_EXTENSION
I had the same error: SQLite header and source version mismatch.
For me it was enough to update libsqlite3-dev.
After that invoking $ spatialite geo.db "SELECT InitSpatialMetaData();" creates proper database.