I am deploying a Django app using Heroku.
When I run
git push heroku master
in my terminal I get the following error:
Could not find a version that satisfies the requirement command-not-found==0.3"
When I run
sudo apt-get install command-not-found
I find that command-not-found is version 20.04.2. However, pip freeze tells me command-not-found is version 0.3.
command-not-found doesn't seem to exist on PyPI, but it is a package in Ubuntu and Debian repositories. It doesn't look like anything that your application should depend on, and it certainly doesn't belong on Heroku.
I suspect
you're trying to create your dependencies file after the fact, by simply doing pip freeze > requirements.txt, and
that you're either not working in a virtual environment or you created your virtual environment with system packages.
This is an antipattern that will cause several packages that your application doesn't actually need to be included in your requirements.txt. In this case it is even including Python packages that come from system packages and aren't meant to be installed from PyPI. Your requirements.txt should contain only your actual dependencies.
Instead of creating it with pip freeze after the fact, add things to that file before, and install them into your virtual environment with the same pip install -r requirements.txt command that you'll use in production. I also very strongly urge you to use a virtual environment.
In this case, I suggest you edit your requirements.txt and remove anything you don't actually need, commit, and redeploy.
I'm using Django and Python 3.7. I want to write a script to help me easily migrate my application from my local machien (a Mac High Sierra) to a CentOS Linux instance. I'm using a virtual environment in both places. There are many things that need to be done here, but to keep the question specific, how do I determine on my remote machine (where I'm deploying my project to), what dependencies are lacking? I'm using rsync to copy the files (minus the virtual environment)
On the source system execute pip freeze > requirements.txt, then copy the requiremnts.txt to the target system and then on the target system install all the dependencies with pip install -r requirements.txt. Of course you will need to activate the virtual environments on both systems before execute the pip commands.
If you are using a source code management system like git it is a good idea to keep the requirements.txt up to date in your source code repository.
I'm coming from a .Net and Node background where packages from the package manager e.g. NuGet or NPM get installed into a specific /packages folder.
I'm learning python and when I do a pip install my packages either end up in a global store or I can tell pip to put them into a specific place.
Is it good practice to put them into a separate folder or does that upset the Zen of python?
I want to do this so I can docker ignore that folder and then pip install them inside the docker container based on my production operating system from a requirements.txt file.
Is there a good reason I shouldn't do this? Why does Python not do this as default as it seems to be a clean and portable approach to package management?
Usually you install them in the default location or within a python virtual environment.
virtual environment
Do the virtual environment when you don't want to clutter up your default python environment.
Why is Anaconda package manager necessary to install Xeus-cling?
I have only used pip so far, and switching to a new package manager seems like a lot of work. I want to start using C++ with Jupyter, but all the installation instructions assume conda package manager.
Does anyone know of Xeus-cling installation which doesn't require Anaconda?
Pip is a package manager for python. It is mostly meant for packaging pure Python packages even though shipping some binary extensions is possible.
xeus-cling is a native application, which relies on a number of other libraries, including LLVM, Clang, Cling, Xeus, OpenSSL, libzmq and other. Vendoring the entire LLVM stack in a pip package would be unreasonable.
The developers of xeus-cling have made the choice of conda because it is a general-purpose cross-platform package manager, but it could be packaged for other general-purpose package managers, such as Debian's dpkg, OSX's Homebrew, conan.io or others.
I am starting to work with TensorFlow library for deep learning, https://www.tensorflow.org/.
I found a explicit guide to work on it on linux and Mac but I did not find how to work with it under Windows. I try over the net, but the information are lacking.
I use Visual Studio 2015 for my projects, and I am trying to compile the library with Visual studio Compiler VC14.
How to install it and to use it under Windows?
Can I use Bazel for Windows for production use?
How to install TensorFlow and to use it under Windows?
Updated on 8/4/16
Windows 10 now has a Ubuntu Bash environment, AKA Bash on Ubuntu on Windows, available as a standard option (as opposed to Insider Preview updates for developers). (StackOverflow tag wsl) This option came with the Windows 10 anniversary update (Version 1607) released on 8/2/2016. This allows the use of apt-get to install software packages such as Python and TensorFlow.
Note: Bash on Ubuntu on Windows does not have access to the GPU, so all of the GPU options for installing TensorFlow will not work.
The dated installation instructions for Bash on Ubuntu on Windows are basically correct, but only these steps are necessary:
Prerequisites
Enable the Windows Subsystem for Linux feature (GUI)
Reboot when prompted
Run Bash on Windows
Steps no longer needed:
Turn on Developer Mode
Enable the Windows Subsystem for Linux feature (command-line)
Then install TensorFlow using apt-get
sudo apt-get install python3-pip python3-dev
sudo pip3 install --upgrade https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-0.8.0-cp34-cp34m-linux_x86_64.whl
and now test TensorFlow
$ python3
...
>>> import tensorflow as tf
>>> hello = tf.constant('Hello, TensorFlow!')
>>> sess = tf.Session()
>>> print(sess.run(hello))
Hello, TensorFlow!
>>> a = tf.constant(10)
>>> b = tf.constant(32)
>>> print(sess.run(a + b))
42
>>> exit()
and run an actual neural network
python3 -m tensorflow.models.image.mnist.convolutional
Earlier Answer
After learning about the developer preview of Bash on Windows.
See Playing with TensorFlow on Windows by Scott Hanselman which uses Bash on Windows 10
Original Answer
Bazel is the problem
TensorFlow is not made with build automation tools such as make, but with Google's in-house build tool Bazel. Bazel only works on systems based on Unix such as Linux and OS X.
Since the current published/known means to build TensorFlow uses Bazel and Bazel does not work on Windows, one can not install or run TensorFlow natively on Windows.
From Bazel FAQ
What about Windows?
Due to its UNIX heritage, porting Bazel to Windows is significant
work. For example, Bazel uses symlinks extensively, which has varying
levels of support across Windows versions.
We are currently actively working on improving Windows support, but
it's still ways from being usable.
Status
See: TensorFlow issue #17
See: Bazel issue #276
Solutions
The solutions are listed in the order of complexity and work needed; from about an hour to may not even work.
Docker
~ 1 hour
Docker installation
Docker is a system to build self contained versions of a Linux operating system running on your machine. When you install and run TensorFlow via Docker it completely isolates the installation from pre-existing packages on your machine.
Also look at TensorFlow - which Docker image to use?
OS X
~ 1 hour
If you have a current Mac running OS X then see: Installation for Mac OS X
Linux
The recommend Linux system tends to be Ubuntu 14.04 LTS (Download page).
a. Virtual Machine - Hardware Virtualization - Full Virtualization
~ 3 hours
Download and install a virtual machine such as the commercial VMware or the free Virtual Box, after which you can install Linux and then install TensorFlow.
When you go to install TensorFlow you will be using Pip - Python's package management system. Visual Studio users should think NuGet. The packages are known as wheels.
See: Pip Installation
If you need to build from the source then see: Installing From Sources
~ 4 hours
Note: If you plan on using a Virtual Machine and have never done so before, consider using the Docker option instead, since Docker is the Virtual Machine, OS and TensorFlow all packaged together.
b. Dual boot
~ 3 hours
If you want to run TensorFlow on the same machine that you have Windows and make use of the GPU version then you will most likely have to use this option as running on a hosted Virtual Machine, Type 2 hypervisor, will not allow you access to the GPU.
Remote machine
~ 4 hours
If you have remote access to another machine that you can install the Linux OS and TensorFlow software on and allow remote connections to, then you can use your Windows machine to present the remote machine as an application running on Windows.
Cloud Service
I have no experience with this. Please edit answer if you know.
Cloud services such as AWS are being used.
From TensorFlow Features
Want to run the model as a service in the cloud?
Containerize with Docker and TensorFlow just works.
From Docker
Running Docker on AWS provides a highly reliable, low-cost way to
quickly build, ship, and run distributed applications at scale. Deploy
Docker using AMIs from the AWS Marketplace.
Wait for Bazel to work on Windows.
Currently it appears the only hold up is Bazel, however Bazel's roadmap list working on Windows should be available this year.
There are two features listed for Windows:
2016‑02 Bazel can bootstrap itself on Windows without requiring admin privileges.
2016‑12 Full Windows support for Android: Android feature set is identical for Windows and Linux/OS X.
Build TensorFlow by hand.
A few days or more depending on you skill level. I gave up on this one; too many subprojects to build and files to locate.
Remember that Bazel is only used to build TensorFlow. If you get the commands Bazel runs and the correct source code and libraries you should be able to build TensorFlow on Windows. See: How do I get the commands executed by Bazel.
While I have not researched this more, you can look at the continuous integration info for needed files and info on how to they build it for testing. (Readme) (site)
Build Bazel on Windows
A few days or more depending on you skill level. I gave up on this one also; could not find the necessary source files needed for Windows.
There is a public experimental source code version of Bazel that boots on Windows. You may be able to leverage this into getting Bazel to work on Windows, etc.
Also these solutions require the use of Cygwin or MinGW which adds another layer of complexity.
Use alternative build system such as Make
If you get this one to work I would like to see in on GitHub.
This currently does not exist for TensorFlow. It is a feature request.
See: TensorFlow issue 380
Cross Build
If you get this one to work I would like to see in on GitHub.
You build TensorFlow on Linux using Bazel but change the build process to output a wheel that can be installed on Windows. This will require detailed knowledge of Bazel to change the configuration, and locating the source code and libraries that work with Windows. An option I would only suggest as a last resort. It may not even be possible.
Run on the new Windows Subsystem for Linux.
See: Windows Subsystem for Linux Overview
You will know as much as I do by reading the referenced article.
Can I use Bazel for Windows for production use?
Since it is experimental software I would not use on a production machine.
Remember that you only need Bazel to build TensorFlow. So use the experimental code on a non production machine to build the wheel, then install the wheel on a production machine. See: Pip Installation
TLDR;
Currently I have several versions for learning. Most use a VMWare 7.1 Workstation to host Ubuntu 14.04 LTS or Ubuntu 15 or Debian. I also have one dual boot of Ubuntu 14.04 LTS on my Windows machine to access the GPU as the machine with VMware does not have the proper GPU. I would recommend that you give these machines at least 8G of memory either as RAM or RAM and swap space as I have run out of memory a few times.
I can confirm that it works in the Windows Subsystem for Linux!
And it is also very straightforward.
In the Ubuntu Bash on Windows 10, first update the package index:
apt-get update
Then install pip for Python 2:
sudo apt-get install python-pip python-dev
Install tensorflow:
sudo pip install --upgrade https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-0.8.0-cp27-none-linux_x86_64.whl
The package is now installed an you can run the CNN sample on the MNIST set:
cd /usr/local/lib/python2.7/dist-packages/tensorflow/models/image/mnist
python convolutional.py
I just tested the CPU package for now.
I blogged about it: http://blog.mosthege.net/2016/05/11/running-tensorflow-with-native-linux-binaries-in-the-windows-subsystem-for-linux/
cheers
~michael
Sorry for the excavation, but this question is quite popular, and now it has a different answer.
Google officially announced the addition of Windows (7, 10, and Server 2016) support for TensorFlow:
developers.googleblog.com
The Python module can be installed using pip with a single command:
C:\> pip install tensorflow
And if you need GPU support:
C:\> pip install tensorflow-gpu
TensorFlow manual - How to install pip on windows
Another useful information are included in release notes:
https://github.com/tensorflow/tensorflow/releases
UPD: As #m02ph3u5 right mentioned in the comments TF for windows supports only Python 3.5.x Installing TensorFlow on Windows with native pip
Installing TensorFlow
TensorFlow currently supports only Python 3.5 64-bit. Both CPU and GPU are supported. Here are some installation instructions assuming you do not have Python 3.5 64-bit:
Download and install Microsoft Visual C++ 2015 Redistributable Update 3:
https://www.microsoft.com/en-us/download/details.aspx?id=53587 (required by Python 3.5 and TensorFlow)
Download and install Python 3.5 64-bit: https://www.python.org/ftp/python/3.5.2/python-3.5.2-amd64.exe
Install pip as follows: download https://bootstrap.pypa.io/get-pip.py, then run python get-pip.py
Install TensorFlow with either pip install tensorflow (CPU version) or pip install tensorflow-gpu (GPU version --> requires CUDA to be installed).
Testing TensorFlow
You can now run something like following to test whether TensorFlow is working fine:
import tensorflow as tf
hello = tf.constant('Hello, TensorFlow!')
sess = tf.Session()
print(sess.run(hello))
a = tf.constant(10)
b = tf.constant(32)
print(sess.run(a + b))
TensorFlow comes with a few models, which are located in C:\Python35\Lib\site-packages\tensorflow\models\ (assuming you installed python in C:\Python35). For example, you can run in the console:
python -m tensorflow.models.image.mnist.convolutional
or
python C:\Python35\Lib\site-packages\tensorflow\models\image\mnist\convolutional.py
Limitations of TensorFlow on Windows
Initial support for building TensorFlow on Microsoft Windows was added on 2016-10-05 in commit 2098b9abcf20d2c9694055bbfd6997bc00b73578:
This PR contains an initial version of support for building TensorFlow
(CPU only) on Windows using CMake. It includes documentation for
building with CMake on Windows, platform-specific code for
implementing core functions on Windows, and CMake rules for building
the C++ example trainer program and a PIP package (Python 3.5 only).
The CMake rules support building TensorFlow with Visual Studio 2015.
Windows support is a work in progress, and we welcome your feedback
and contributions.
For full details of the features currently supported and instructions
for how to build TensorFlow on Windows, please see the file
tensorflow/contrib/cmake/README.md.
The Microsoft Windows support was introduced in TensorFlow in version 0.12 RC0 (release notes):
TensorFlow now builds and runs on Microsoft Windows (tested on Windows 10, Windows 7, and Windows Server 2016). Supported languages include Python (via a pip package) and C++. CUDA 8.0 and cuDNN 5.1 are supported for GPU acceleration. Known limitations include: It is not currently possible to load a custom op library. The GCS and HDFS file systems are not currently supported. The following ops are not currently implemented: DepthwiseConv2dNative, DepthwiseConv2dNativeBackpropFilter, DepthwiseConv2dNativeBackpropInput, Dequantize, Digamma, Erf, Erfc, Igamma, Igammac, Lgamma, Polygamma, QuantizeAndDequantize, QuantizedAvgPool, QuantizedBatchNomWithGlobalNormalization, QuantizedBiasAdd, QuantizedConcat, QuantizedConv2D, QuantizedMatmul, QuantizedMaxPool, QuantizeDownAndShrinkRange, QuantizedRelu, QuantizedRelu6, QuantizedReshape, QuantizeV2, RequantizationRange, and Requantize.
Now Tensorflow is officially supported in Windows, you can install it using pip command of Python 3.5 without compile it yourself
CPU Version
pip install --upgrade https://storage.googleapis.com/tensorflow/windows/cpu/tensorflow-0.12.0-cp35-cp35m-win_amd64.whl
cp35 indicates python 3.5 wheel, 0.12.0 the version, you can edit these according your preference, or to install latest CPU version available you can use
pip install --upgrade tensorflow
GPU Version
pip install --upgrade https://storage.googleapis.com/tensorflow/windows/gpu/tensorflow_gpu-0.12.0-cp35-cp35m-win_amd64.whl
cp35 indicates python 3.5 wheel, 0.12.0 the version, you can edit these according your preference, or to install latest GPU version available you can use
pip install --upgrade tensorflow-gpu
More Info
Following may work for you: install Virtual Box, create Linux VM and install Linux into it. I'd recommend Ubuntu, because Google often uses it internally. Then, install TensorFlow in Linux VM.
You can't at the moment. The problem is that tensorflow uses the bazel build another Google internal tool that has been exposed as an open source project and it has only support for mac and unix.
Until bazel is ported to windows or another build system is added to tensorflow there is a little chance to run tensorflow natively on windows.
That said you can install virtualbox and then install docker-machine and run a linux container with tensorflow inside it.
I managed to install TensorFlow on Win8.1 without Docker using advice from
https://discussions.udacity.com/t/windows-tensorflow-and-visual-studio-2015/45636
I tried a lot of stuff before that, and i won't try to install it twice but here is what i did:
- install VS2015 (make sure Visual C++ installed as well)
- install Python Tools for VS2015
- install Python2.7 with Anaconda2
- install pip and conda for Python
- install numpy with pip inside VS2015
- install tensorflow with pip inside VS2015
i didn't manage to do it with Python3.5
I managed also to install on Win8.1 via Cloud9
There is a video tutorial on Youtube.
https://www.youtube.com/watch?v=kMtrOIPLpR0
EDIT: actually for the above, (not Cloud9 which is fine) i have problems:
TensorFlow LOOKS LIKE it's installed (i can see it in the list of modules installed in VS2015 when clicking in Solution Explorer on Python 64-bit 2.7)
but if i type in a script or in Python Interactive import tensorflow as TF then i get an error message
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Users\Fagui\Anaconda2\lib\site-packages\tensorflow\__init__.py", line 23, in <module>
from tensorflow.python import *
File "C:\Users\Fagui\Anaconda2\lib\site-packages\tensorflow\python\__init__.py", line 50, in <module>
from tensorflow.python.framework.framework_lib import *
File "C:\Users\Fagui\Anaconda2\lib\site-packages\tensorflow\python\framework\framework_lib.py", line 62, in <module>
from tensorflow.python.framework.ops import Graph
File "C:\Users\Fagui\Anaconda2\lib\site-packages\tensorflow\python\framework\ops.py", line 40, in <module>
from tensorflow.python.framework import versions
File "C:\Users\Fagui\Anaconda2\lib\site-packages\tensorflow\python\framework\versions.py", line 24, in <module>
from tensorflow.python import pywrap_tensorflow
File "C:\Users\Fagui\Anaconda2\lib\site-packages\tensorflow\python\pywrap_tensorflow.py", line 28, in <module>
_pywrap_tensorflow = swig_import_helper()
File "C:\Users\Fagui\Anaconda2\lib\site-packages\tensorflow\python\pywrap_tensorflow.py", line 20, in swig_import_helper
import _pywrap_tensorflow
As of writing this answer, I wasn't able to get tensorflow to install properly with python version 3.5.2. Reverting to python 3.5.0 did the trick.
Then I was able to install with
C:> pip install tensorflow
If you have already installed anaconda on your windows, there is an easier way as I found out:
conda create --name snakes python=3
Then
activate snakes
Then
pip install tensorflow
This is similar to virtualenv and I found this helpful.
Follow this link to install Tensorflow on Windows and you can also use it in Visual Studio