I have a problem in synchronising imports between ipyparallel engines. I want to run stochastic simulations in parallel. For this I have written a function named 'gillespie'. I have two versions of the function, one using Python and to gain some speed one in Cython. Both are working.
The problem now is how to make the compiled cython function available to all engines. The files 'gillespie.c', 'gillespie.so' and 'gillespie.pyx' are all in the current directory together with the notebook from where I want to use the functions. I wanted to import it like this
import gillespie as gc
%px import gillespie as gc
But this gives me:
Traceback (most recent call last)<ipython-input-50-801274ebf64a> in <module>()
----> 1 import gillespie as gc
ImportError: No module named gillespie
Just doing
import gillespie as gc
works just fine and I can use the compiled function.
Using the %px magic to import numpy or scipy to the engines works just fine.
How can I make my imports available to all engines?
Thanks for any help!
baumgast
%px import gillespie executes the import statement on the engines. For that to work, you need to make sure that gillespie.so is available to those engines. You can either 'install' it by placing it in site-packages or on PYTHONPATH, or you can rely on the current-directory-based loading that you are already relying on in the notebook, in which case you need to make sure that gillespie.so is in the working directory of the engines. If you are on a single machine or shared filesystem, you can accomplish this with:
%px cd /path/to/dir/containing/gillespie.so
or send gillespie.so to the machines, if the filesystem is not shared.
Related
I tried to download Graphlab from Turi wit the following tutorial. I coded with their tools and tried to compute a Python script but it answered me an ImportError.
(gl-env)ubuntu#ip-172-hey-hey-hey:~/Eclipse-Stats$ source deactivate
discarding /home/ubuntu/anaconda2/envs/gl-env/bin from PATH
ubuntu#ip-172-hey-hey-hey:~/Eclipse-Stats$ unset PYTHONPATH
ubuntu#ip-172-hey-hey-hey:~/Eclipse-Stats$ python Main.py
2017-07-27 14:56:00.520425
/home/ubuntu/.local/lib/python2.7/site-packages/sklearn/cross_validation.py:44: DeprecationWarning: This module was deprecated in version 0.18 in favor of the model_selection module into which all the refactored classes and functions are moved. Also note that the interface of the new CV iterators are different from that of this module. This module will be removed in 0.20.
"This module will be removed in 0.20.", DeprecationWarning)
Traceback (most recent call last):
File "Main.py", line 3, in <module>
import prediction
File "/home/ubuntu/Eclipse-Stats/prediction.py", line 1, in <module>
from graphlab.toolkits.recommender import ranking_factorization_recommender
ImportError: No module named graphlab.toolkits.recommender
Actually it cames often on the server when I tried to download with pip numpy, scipy, sklearn... Like we can see in the following conversation (in Spanish) between FJSevilla and the man of my team I'm working with.
Two things: (1) check the version of your Python console and see if it matches or is higher than the compatibility with the packages. If you look at the depreciation message and read it through, you would understand what is going on a little more. (2) be careful of what you are importing and how you import them because your formatting might also be a syntactic all error. One thing you could do is find the package, manually download, unzip, then run the setup.py.
I have written codes but it's showing cannot import pygame module
Download pygame from here: http://www.pygame.org/download.shtml
If you are not sure which version of python you are running, execute the following code in your shell:
import sysconfig
version = sysconfig.get_python_version()
print(version) #Python3
print version #Python2
Usually, Python3 will also accept the print command without brackets, so using sysconfig is the safest way. Sysconfig will also tell you which minor version you are using (such as 3.4.5)
Once you have download pygame, tryimport pygame in your shell. If this does not work, type help('modules'). This will now list all your modules. If pygame is not in there, check in which folder all other modules are saved and move the pygame module to that file path.
You can also use sysconfig to display all the file paths python is using:
import sysconfig
paths = sysconfig.get_paths()
print(paths) #For Python2, see print method above
I don't quite understand what you mean with "And make games", but the pygame website has a great documentary and some example codes to help you learn progamming with the pygame module.
Hope I could help,
Narusan
I would like to have a script invoke numpy from a c++ embedded python runtime by setting the runtime path to know about the numpy module located within site-packages.
However I get the error:
cannot import name 'multiarray'
from \Lib\site-packages\numpy\core__init_.py on the line
from . import multiarrray
I have tried to set the os.path to be xxx\numpy\core but it still cannot seem to find the multiarray.pyd file during the import statement
I have read through similar questions posed but none of the answers seem relevant to my case.
I am using Python 3.4.4 (32 bit) and have installed Numpy 1.11.1 using the wheel
numpy-1.11.1-cp34-none-win32.whl
python -m pip install numpy-1.11.1-cp34-none-win32.whl
Completed without any errors.
Seems like the failure message maybe more general than just an incomplete PYTHONPATH?
Also think it might be broader than Numpy in that ANY .pyd based package that is imported from the embedded environment will have this problem?
Any help appreciated.
Did you ensure all your NumPy includes: \numpy\core\include\numpy\ were present during the build? That's the only time I get those types of errors was if the build couldn't find all the NumPy includes... although during embedding I found that the numpy entire directory (already built on your build machine) has to be inside a directory under Py_SetPath(python35.lib;importlibs); assuming importlibs is a directory with NumPy inside and anything else you want to bundle.
Seems like the answer was to install python 3.4.1 to match the python34.dll version of 3.4.1.
I have made a python script for calculations purposes, importing libraries, Tkinter, Pmw, sympy, math, tkfiledialog, webbrowser.
Now, by using Pyinstaller I convert it into an EXE application.
When I run it, it gives the error:
WindowsError: [Error 3] The system cannot find the path specified: 'C:\\Python27\\Earthing\\dist\\Earthing\\Pmw/*.*'
So, I copy and paste the entire Pmw directory on this location. However, after doing this, I get the error:
AttributeError: 'module' object has no attribute 'OptionMenu'
Now, how do I resolve this error? Please do help me sort this out.
I ran into the same problem. It is due to what I would call 'dynamic imports', made mostly in PmwLoader.py (placed in lib subfolder): PmwLoader loads all the files, and the they become attributes of Pmw global library.
The solution I found was to manually delete the line 'import Pmw' in all the wanted Pmw files (I only used PmwComboBox and PmwScrolledFrame). PmwCombobox and PmwScrolledFrame notably need to import other Pmw files, so I had to replace
import Pmw
by
import PmwBase
import PmwScrolledListBox
import PmwEntryField
import PmwTimeFuncs
and then do the same in PmwScrolledListBox and PmwEntryFiled.
The the fun is to solve the bugs --notably replace a lot of the MegaWidget by PmwBase.MegaWidget, and so on.
In the end, it does not take more than one hour.
Good luck!
t.
I have a Python pyd module written in C++. I also have a Python version of the module (which is much slower).
My question is, as I want the program to run on multiple OSs, can I try to import the C++ version in the program and import the slower Python version if that one fails (other OS, architecture)?
Yes, you can import some thing like this:
try:
import CppModule as Module
except ImportError:
import PurePythonModule as Module
Yes you can:
try:
import CppModule
except ImportError:
import PythonModule
Edit: This answer, while not incorrect, is not really useful. As #Best Games' answer shows, this only really useful if you import the module using a common name.