Running into an error when trying to implement Keras - python-2.7

I'm using Python 2.7, Windows 7, and Keras 1.2.1 (Keras2 seems to be having a lot of compatibility issues with different PC/Python configs, so I was recommened to use 1.2.1)
I'm using a script from Practical Deep Learning For Coders, Part 1 Course
import utils; reload(utils)
from utils import plots
The error I"m getting is as below
Problem occurred during compilation with the command line below:
"g++" -shared -g -DNPY_NO_DEPRECATED_API=NPY_1_7_API_VERSION -m64 -DMS_WIN64 -I"c:\python27\lib\site-packages\numpy\core\include" -I"c:\python27\include" -I"c:\python27\lib\site-packages\theano\gof" -L"c:\python27\libs" -L"c:\python27" -o C:\Users\Moondra\AppData\Local\Theano\compiledir_Windows-7-6.1.7601-SP1-Intel64_Family_6_Model_94_Stepping_3_GenuineIntel-2.7.12-64\lazylinker_ext\lazylinker_ext.pyd C:\Users\Moondra\AppData\Local\Theano\compiledir_Windows-7-6.1.7601-SP1-Intel64_Family_6_Model_94_Stepping_3_GenuineIntel-2.7.12-64\lazylinker_ext\mod.cpp -lpython27
I can't decipher what this means. Google search brought up a Chinese message board.
It seems to be a g++ problem as the previous warning I got was as follows:
WARNING (theano.configdefaults): g++ not available, if using conda:conda install m2w64-toolchain``. Despite installing m2w64-toolchain afterwards, I was continuing to get the same warning.

Some interesting things I learned while trying to configure theano and keras:
C++/g++
Depending on your python version, you have to install a corresponding version of the C++ compiler, or Visual Studio. Ex: Python 2.7 needs C++ 9.0 (Visual Studio 2008). But Python 3.5 needs C++ 14.0 (Visual Studio 2015) -- See here for more: https://wiki.python.org/moin/WindowsCompilers
I also had to install MinGW for g++ compilers: https://nuwen.net/mingw.html
Even with it correctly installed, you may see an error about hypot. This question shows two possible solutions: g++ error on import of Theano on Windows 7
I tried installing everything via Conda, but all calculations were slower than continents moving away from each other. Then I gave up conda and installed everything via standard python. (First python, then numpy, then scipy, then theano, then keras). Not sure if MinGW goes before of after numpy/scipy, but I think it goes before.
BLAS/LAPACK
Numpy and theano may warn you that no lapack/blas is installed. For that, I used the numpy and scipy libraries available here, they're great and fast: http://www.lfd.uci.edu/~gohlke/pythonlibs/
Another option is installing MKL directly from Intel, before installing numpy and scipy.
Path problems / Invalid Token - ldflags
Avoid installing anything, including the MKL, in folders containing spaces in their names! That will definitely confuse theano/numpy.
You may get "invalid token" errors if you have spaces in folders, and a ldflags var described in that error. To solve that, either reinstall your mkl libraries in folders without spaces, or configure the .theanorc file in the theano home folder with your [blas] configuration and the ldflags var. (See here: Receive AssertionError while optimizing convolution in theano)
Since I wasn't willing to reinstall MKL in another folder, I learned to create symbolic links to windows. (in the answer above)
Replace the folders with space in the theano config and also in the numpy config files (__config__.py), you can see its content with np.__config__.show(). (See answer above)

I solved the problem.
I reinstalled Anaconda.
And then via command line I wrote conda install m2w64-toolchain
This time I didn't get a message stating that m2w64 was already installed. I'm assuming it was a path issue and this time reinstalling everything from the start also created a new path?

Related

Python distutils refuses to recognise installed C++ compilers, `Unable to find vcvarsall.bat` under Python 3.5

I'm tearing my hair out trying to compile c though Python's distutils. Specifically I'm trying to get started with Cython following this guide: http://docs.cython.org/src/quickstart/build.html, using a setup.py.
I'm running Python 3.5.0 under Anaconda 2.4.0 (64-bit), Windows 7.
I get the Unable to find vcvarsall.bat error referred to here: Cannot find vcvarsall.bat when running a Python script and in various other questions.
None of the proposed solutions work for me. So far I have tried:
installing Visual Studio - I have all versions from 2008 to 2013, ie 9.0, 10.0, 11.0, and 12.0.
pointing environment variables (VS100COMNTOOLS etc) related to the above directly to the correct locations
installing Microsoft Visual C++ Compiler for Python (as above link). This seemed to work but I get a compilation error that seems to be an incompatibility; I think because the compiler is intended for Python 2.7.
Ravi Kumar's suggestion at the above link to update pip's setuptools. Pip gave me an error saying it couldn't update.
Installing MS Windows SDK for Windows 7 and .NET Framework 4 as https://github.com/cython/cython/wiki/CythonExtensionsOnWindows. The final step failed with an error message that the x64 compilers aren't currently installed.
The only thing I can think of is installing MinGW which I've seen suggested in some places, but that CythonExtensionsOnWindows link specifically recommended against it.
Any other suggestions would be very welcome! In particular, since I clearly have all the versions of VS I could possibly need, is there a way to force Python to recognise them, other than setting environment variables VS100COMNTOOLS or similar?
Interestingly conda does seem to recognise the installed compilers, as a separate pip install attempt that failed with this same error, worked with conda.
As it turns out, Python 3.5 needs Visual Studio 2015. Advice elsewhere on fixing this for other Python versions is outdated.
Get Visual Studio 2015 Community here: https://www.visualstudio.com/en-us/downloads/visual-studio-2015-downloads-vs.aspx, this worked for me with no further steps needed.
Many thanks to Ionel, apparently the only place on the web to find this information! http://blog.ionelmc.ro/2014/12/21/compiling-python-extensions-on-windows/

Having trouble getting Python to recognize location of vcvarsall.bat

I was trying to install RandomWords which requires ujson, at some point in the installation I received the following notification: "error: Unable to find vcvarsall.bat" I searched here, on stackoverflow, and came across some answers as the to root of this problem. In this answer https://stackoverflow.com/a/8705722, near the end, the OP pointed out that Microsoft had released a C++ compiler package for Python 2.7 (http://www.microsoft.com/en-us/download/details.aspx?id=44266).
I installed this package but still haven't had any success in installing ujson. So far I've tried the following:
Adding the location of vcvarsall.bat to the path environment variable
Manually tried loading the VC++ compiler's environment into the session by executing vcvars64.bat (aI've tried vcvars32.bat as well) as recommended here: https://stackoverflow.com/a/18045219
I've also tried installing the compilers found in "Windows SDK for Windows 7 and .NET Framework 3.5 SP1" found here: http://www.microsoft.com/en-us/download/details.aspx?id=3138
Not sure where to go from here, any suggestions/help would be greatly appreciated.
For reference here's some relevant specs:
Windows 8.1 (64-bit)
Python 2.7 (64-bit)
I've been trying to install said package using pip from the command prompt (e.g. '> pip install ujson')
I've uninstalled all the C++ compiler packages I've tried except this one: http://www.microsoft.com/en-us/download/details.aspx?id=44266
Really don't feel like installing mingw, but I'll try it if it seems like it's the only solution.
-UPDATE-
I ended up modifying "msvc9compiler.py" and manually setting the location of vcvarsall.bat by setting the value of the vcvarsall variable to the location of that batch file. Everything compiled fine but this is not a recommended solution. I'm pretty sure the compiler version I have matches the one python 2.7 uses, but as mentioned in an answer, in another question on this topic, if these don't match it can cause problems since "the compilers will probably have incompatible C runtime libraries."

How to point pyinstaller to the right versions of MSVC?90.dll?

I have a python application that I am trying to build as a pyinstaller distributable. A similar script builds successfully on Linux.
I am building it on Windows 7 x64, but want to build 32-bit binary for better compatibility, so I am using 32-bit python-2.7. Among my dependencies are matplotlib and pyside which require MSVC. I install a package called VCForPython27 from Microsoft.
I run into an error when I run my pyinstaller script. I get the following message:
1250 INFO: Adding Microsoft.VC90.CRT to dependent assemblies of final executable
7428 INFO: Searching for assembly x86_Microsoft.VC90.CRT_1fc8b3b9a1e18e3b_9.0.21022.8_none ...
7428 WARNING: Assembly not found
7428 ERROR: Assembly x86_Microsoft.VC90.CRT_1fc8b3b9a1e18e3b_9.0.21022.8_none not found
7475 WARNING: lib not found: MSVCR90.dll dependency of C:\Python27\python.exe
7553 INFO: Searching for assembly x86_Microsoft.VC90.CRT_1fc8b3b9a1e18e3b_9.0.21022.8_none ...
7553 WARNING: Assembly not found
7553 ERROR: Assembly x86_Microsoft.VC90.CRT_1fc8b3b9a1e18e3b_9.0.21022.8_none not found
7662 WARNING: lib not found: MSVCR90.dll dependency of C:\Windows\system32\python27.dll
7662 INFO: Analyzing C:\Python27\lib\site-packages\PyInstaller\loader\_pyi_boots
There are multiple messages like that about both the files MSVCP90.dll and MSVCR90.dll
I can see that I have a folder C:\Windows\winsxs\x86_microsoft.vc90.crt_1fc8b3b9a1e18e3b_9.0.30729.4148_none_5090ab56bcba71c2 that contains versions of both files.
This mismatch occurs both when I install my python packages from Christoph Gohlke's page and with pip (except for matplotlib, which I can't install with pip because of missing dependencies).
Strangely enough pyinstaller makes a binary. Yet, when I try to run it I get a popup saying:
WARNING: file already exists but should not:
C:\Users\Martin\AppData\Local\Temp\_MEI34922\Include\pyconfig.h
Does anyone know how I can do any of the following:
Install the precious x86_Microsoft.VC90.CRT_1fc8b3b9a1e18e3b_9.0.21022.8_none assembly? Where can I take this specific version from?
Tell python to look for the other version (x86_microsoft.vc90.crt_1fc8b3b9a1e18e3b_9.0.30729.4148_none_5090ab56bcba71c2)?
Solve the pyconfig.h unwanted presence issue? Doesn't seem to lead anywhere, but I thought I should try it too.
Find another way to build my code to a binary? It's a complicated code, running external binaries, but if I have to I will try py2exe, not sure that it would be any better though.
The redistributable package that contains version 9.0.21022.8 of msvcr90.dll and msvcp90.dll can be downloaded from the Microsoft website here. This will help PyInstaller to find the versions it wants and include them with the resulting executable.
Interestingly enough, I'm able to run the executable compiled by PyInstaller with redistributable version 9.0.30729.6161 installed, it just won't package these dlls. I tried to copy the msvc*90.dll's into the dist directory, even tried creating and modifying manifest files, but I would still get an error from python27.dll in the end. Installing any version of VC++ redistributable would fix the issue, but then my package wouldn't be self-containing. I wish I understood what exactly is going on here better...

Testing Scipy; get "No module named msvccompiler in numpy.distutils; trying from distutils"

I'm trying to test my scipy install by installing nose and entering
import scipy
scipy.test('1','10')
I get some 200 errors but one of the more common is:
.... No module named msvccompiler in numpy.distutils; trying from distutils
Missing compiler_cxx fix for MSVCCompiler
I have MS Virtual Studio 2009 installed (required for python 2.7)
Can anyone help me eliminate that error or tell my why I can't?
Thanks
You have to do 2 things to make it find the Visual Studio Compiler:
In your environment, set VS90COMNTOOLS=%VS__vvv__COMNTOOLS% where vvv is your installed version. See other related questions on stackoverflow about setup.py and vcvarsall.bat
Run python from a Visual Studio Developer Command Prompt. VS includes a link for this in the start menu. It sets up all the environment variables you need to run the compilers and related tools from the command line.
Best solution to this could be using the Anaconda Python distribution
It has most of the engineering and data analysis packages and automatically sets path of the C++ compiler. You can easily install numpy, scipy, matplotlib etc. using Anaconda. It will not show vcvarshall.bat as well as numpy.core.multiarray error for numpy

Install Numpy compiled for VS2010 for an embedded interpreter

Another of my queries about an embedded Python 2.7.3 interpreter on Windows. I need to install numpy and matplotlib as support packages for an embedded interpreter in a C++ program compiled in Visual Studio 2010.
Some background:
I've been trying to make this work for a few months. The main issue is that numpy for Python 2.7 is compiled against an earlier version of the MS C library (VS2008) whereas my application is compiled against the 2010 version (a fact beyond my control - that is the office compiler).
I worked around this by fiddling with the manifest for my application to let it load the correct C library for numpy and this got it to import. As an end-user has tried out new features, bugs have started to appear, including untraceable crashes.
My current state:
I pinned some of these down to a poor compilation on my part of the Python 2.7.3 source, in particular the dependencies (lots of the crashes seemed associated with Tkinter from matplotlib). So I started afresh on the Python compilation and now have a clean compiled version of all the core and dependencies of Python 2.7.3. Now I'm looking at numpy/matplotlib.
My questions
Is it worth attempting to compile numpy against VS2010 or is my earlier manifest mangling solution the only way?
If the former, what problems might I encounter in doing so (i.e. what should I watch ou for)?
In the latter, how do I get numpy to install in the lib/site-packages folder of my embedded intepreter?
use cygwin to build numpy on windows.
I am on windows-xp
Its working fine for me.. .!
In cygwin python2.6 comes as a default python. I have installed python2.7 but not installed numpy in python2.7.
To use it on python2.7 i have run python2.7 setup.py build
You can see that in screen-shot of my terminal. and in folder 'namit' i have placed 'numpy' dir that i have compiled on cygwin.
when i have imported numpy when i am in that dir.. it worked...you can see that in globals() from where it has imported.