Save matplotlib.pyplot subplots without generating display [Python] [duplicate] - python-2.7

I am trying to use networkx with Python. When I run this program it get this error. Is there anything missing?
#!/usr/bin/env python
import networkx as nx
import matplotlib
import matplotlib.pyplot
import matplotlib.pyplot as plt
G=nx.Graph()
G.add_node(1)
G.add_nodes_from([2,3,4,5,6,7,8,9,10])
#nx.draw_graphviz(G)
#nx_write_dot(G, 'node.png')
nx.draw(G)
plt.savefig("/var/www/node.png")
Traceback (most recent call last):
File "graph.py", line 13, in <module>
nx.draw(G)
File "/usr/lib/pymodules/python2.5/networkx/drawing/nx_pylab.py", line 124, in draw
cf=pylab.gcf()
File "/usr/lib/pymodules/python2.5/matplotlib/pyplot.py", line 276, in gcf
return figure()
File "/usr/lib/pymodules/python2.5/matplotlib/pyplot.py", line 254, in figure
**kwargs)
File "/usr/lib/pymodules/python2.5/matplotlib/backends/backend_tkagg.py", line 90, in new_figure_manager
window = Tk.Tk()
File "/usr/lib/python2.5/lib-tk/Tkinter.py", line 1650, in __init__
self.tk = _tkinter.create(screenName, baseName, className, interactive, wantobjects, useTk, sync, use)
_tkinter.TclError: no display name and no $DISPLAY environment variable
I get a different error now:
#!/usr/bin/env python
import networkx as nx
import matplotlib
import matplotlib.pyplot
import matplotlib.pyplot as plt
matplotlib.use('Agg')
G=nx.Graph()
G.add_node(1)
G.add_nodes_from([2,3,4,5,6,7,8,9,10])
#nx.draw_graphviz(G)
#nx_write_dot(G, 'node.png')
nx.draw(G)
plt.savefig("/var/www/node.png")
/usr/lib/pymodules/python2.5/matplotlib/__init__.py:835: UserWarning: This call to matplotlib.use() has no effect
because the the backend has already been chosen;
matplotlib.use() must be called *before* pylab, matplotlib.pyplot,
or matplotlib.backends is imported for the first time.
if warn: warnings.warn(_use_error_msg)
Traceback (most recent call last):
File "graph.py", line 15, in <module>
nx.draw(G)
File "/usr/lib/python2.5/site-packages/networkx-1.2.dev-py2.5.egg/networkx/drawing/nx_pylab.py", line 124, in draw
cf=pylab.gcf()
File "/usr/lib/pymodules/python2.5/matplotlib/pyplot.py", line 276, in gcf
return figure()
File "/usr/lib/pymodules/python2.5/matplotlib/pyplot.py", line 254, in figure
**kwargs)
File "/usr/lib/pymodules/python2.5/matplotlib/backends/backend_tkagg.py", line 90, in new_figure_manager
window = Tk.Tk()
File "/usr/lib/python2.5/lib-tk/Tkinter.py", line 1650, in __init__
self.tk = _tkinter.create(screenName, baseName, className, interactive, wantobjects, useTk, sync, use)
_tkinter.TclError: no display name and no $DISPLAY environment variable
I get a different error now:
#!/usr/bin/env python
import networkx as nx
import matplotlib
import matplotlib.pyplot
import matplotlib.pyplot as plt
matplotlib.use('Agg')
G=nx.Graph()
G.add_node(1)
G.add_nodes_from([2,3,4,5,6,7,8,9,10])
#nx.draw_graphviz(G)
#nx_write_dot(G, 'node.png')
nx.draw(G)
plt.savefig("/var/www/node.png")
/usr/lib/pymodules/python2.5/matplotlib/__init__.py:835: UserWarning: This call to matplotlib.use() has no effect
because the the backend has already been chosen;
matplotlib.use() must be called *before* pylab, matplotlib.pyplot,
or matplotlib.backends is imported for the first time.
if warn: warnings.warn(_use_error_msg)
Traceback (most recent call last):
File "graph.py", line 15, in <module>
nx.draw(G)
File "/usr/lib/python2.5/site-packages/networkx-1.2.dev-py2.5.egg/networkx/drawing/nx_pylab.py", line 124, in draw
cf=pylab.gcf()
File "/usr/lib/pymodules/python2.5/matplotlib/pyplot.py", line 276, in gcf
return figure()
File "/usr/lib/pymodules/python2.5/matplotlib/pyplot.py", line 254, in figure
**kwargs)
File "/usr/lib/pymodules/python2.5/matplotlib/backends/backend_tkagg.py", line 90, in new_figure_manager
window = Tk.Tk()
File "/usr/lib/python2.5/lib-tk/Tkinter.py", line 1650, in __init__
self.tk = _tkinter.create(screenName, baseName, className, interactive, wantobjects, useTk, sync, use)
_tkinter.TclError: no display name and no $DISPLAY environment variable

The main problem is that (on your system) matplotlib chooses an x-using backend by default. I just had the same problem on one of my servers. The solution for me was to add the following code in a place that gets read before any other pylab/matplotlib/pyplot import:
import matplotlib
# Force matplotlib to not use any Xwindows backend.
matplotlib.use('Agg')
The alternative is to set it in your .matplotlibrc

Just as a complement of Reinout's answer.
The permanent way to solve this kind of problem is to edit .matplotlibrc file. Find it via
>>> import matplotlib>>> matplotlib.matplotlib_fname()
# This is the file location in Ubuntu
'/etc/matplotlibrc'
Then modify the backend in that file to backend : Agg. That is it.

The clean answer is to take a little bit of time correctly prepare your execution environment.
The first technique you have to prepare your execution environment is to use a matplotlibrc file, as wisely recommended by Chris Q., setting
backend : Agg
in that file. You can even control — with no code changes — how and where matplotlib looks for and finds the matplotlibrc file.
The second technique you have to prepare your execution environment is to use the MPLBACKEND environment variable (and inform your users to make use of it):
export MPLBACKEND="agg"
python <program_using_matplotlib.py>
This is handy because you don't even have to provide another file on disk to make this work. I have employed this approach with, for example, testing in continuous integration, and running on remote machines that do not have displays.
Hard-coding your matplotlib backend to "Agg" in your Python code is like bashing a square peg into a round hole with a big hammer, when, instead, you could have just told matplotlib it needs to be a square hole.

I got the error while using matplotlib through Spark. matplotlib.use('Agg') doesn't work for me. In the end, the following code works for me. More here
import matplotlib.pyplot as plt.
plt.switch_backend('agg')

I will just repeat what #Ivo Bosticky said which can be overlooked. Put these lines at the VERY start of the py file.
import matplotlib
matplotlib.use('Agg')
Or one would get error
*/usr/lib/pymodules/python2.7/matplotlib/__init__.py:923: UserWarning: This call to matplotlib.use() has no effect
because the the backend has already been chosen;
matplotlib.use() must be called *before* pylab, matplotlib.pyplot,*
This will resolve all Display issue

I found this snippet to work well when switching between X and no-X environments.
import os
import matplotlib as mpl
if os.environ.get('DISPLAY','') == '':
print('no display found. Using non-interactive Agg backend')
mpl.use('Agg')
import matplotlib.pyplot as plt

When signing into the server to execute the code
use this instead:
ssh -X username#servername
the -X will get rid of the no display name and no $DISPLAY environment variable
error
:)

import matplotlib
matplotlib.use('Agg')
import matplotlib.pyplot as plt
It works for me.

What system are you on? It looks like you have a system with X11, but the DISPLAY environment variable was not properly set. Try executing the following command and then rerunning your program:
export DISPLAY=localhost:0

One other thing to check is whether your current user is authorised to connect to the X display. In my case, root was not allowed to do that and matplotlib was complaining with the same error.
user#debian:~$ xauth list
debian/unix:10 MIT-MAGIC-COOKIE-1 ae921efd0026c6fc9d62a8963acdcca0
root#debian:~# xauth add debian/unix:10 MIT-MAGIC-COOKIE-1 ae921efd0026c6fc9d62a8963acdcca0
root#debian:~# xterm
source: http://www.debian-administration.org/articles/494 https://debian-administration.org/article/494/Getting_X11_forwarding_through_ssh_working_after_running_su

To make sure your code is portable across Windows, Linux and OSX and for systems with and without displays, I would suggest following snippet:
import matplotlib
import os
# must be before importing matplotlib.pyplot or pylab!
if os.name == 'posix' and "DISPLAY" not in os.environ:
matplotlib.use('Agg')
# now import other things from matplotlib
import matplotlib.pyplot as plt
Credit: https://stackoverflow.com/a/45756291/207661

For Google Cloud Machine Learning Engine:
import matplotlib as mpl
mpl.use('Agg')
from matplotlib.backends.backend_pdf import PdfPages
And then to print to file:
#PDF build and save
def multi_page(filename, figs=None, dpi=200):
pp = PdfPages(filename)
if figs is None:
figs = [mpl.pyplot.figure(n) for n in mpl.pyplot.get_fignums()]
for fig in figs:
fig.savefig(pp, format='pdf', bbox_inches='tight', fig_size=(10, 8))
pp.close()
and to create the PDF:
multi_page(report_name)

import matplotlib
matplotlib.use('Agg')
worked for me as long as I was calling the plotting functions/code directly. But if you are using
from joblib import Parallel, delayed
Parallel and delayed modules for parallelizing your code then you need to add
matplotlib.rcParams.update({'backend': 'Agg'})
line inside your function in order for agg backend to work.

Related

Python: Where does import come from?

I have had this strange problem with import in python 2.7
I have my app in a directory that has some subdirectoriers and more python apps running simultaneously using Pyro Name Server for communicating with each other.
When I run one of my apps, it crashes on import when calling one of sub methods.
Here is the exception:
Traceback (most recent call last):
File "ps_logic.py", line 15840, in <module>
ps_logic = PSLogic(pyro_objects, cfg_handler, status_distributor, voip_processing)
File "ps_logic.py", line 590, in __init__
self.smarthopper_initial_check()
File "ps_logic.py", line 12824, in smarthopper_initial_check
counters_compared = self.smarthopper_maintenance_action()
File "ps_logic.py", line 12928, in smarthopper_maintenance_action
status = self.smart_hopper_logic.status_get()
File "/home/app_core/flexcore/003-480/ps_logic/smart_devices_logic.py", line 203, in status_get
return SmartStatusAugmented(self.smart_obj.queue_status_get(), self.smart_obj)
File "/usr/lib/python2.7/site-packages/Pyro/core.py", line 381, in __call__
return self.__send(self.__name, args, kwargs)
File "/usr/lib/python2.7/site-packages/Pyro/core.py", line 456, in _invokePYRO
return self.adapter.remoteInvocation(name, Pyro.constants.RIF_VarargsAndKeywords, vargs, kargs)
File "/usr/lib/python2.7/site-packages/Pyro/protocol.py", line 497, in remoteInvocation
return self._remoteInvocation(method, flags, *args)
File "/usr/lib/python2.7/site-packages/Pyro/protocol.py", line 536, in _remoteInvocation
answer = pickle.loads(answer)
ImportError: No module named drivers.smart.smart_common_const
it clearly says that it cannot import drivers.smart.smart_common_const BUT the problem is, I do not have that line in my code.
If I try to find in which file that line is (cuz I have already fixed it in some), it finds me nothing:
app_core#003-481 ~/flexcore/003-480 $ grep -R "from drivers.smart.smart_common_const import" .
./drivers/.svn/pristine/23/23e13acbf9e604f179d4625e18b2b992116a98a1.svn-base:from drivers.smart.smart_common_const import *
./drivers/.svn/pristine/65/65655973d3c70a16cc982db59db8f2989366524b.svn-base:from drivers.smart.smart_common_const import *
./drivers/.svn/pristine/3b/3ba2e2518e64db9188b63247b763926544bddd90.svn-base:from drivers.smart.smart_common_const import *
app_core#003-481 ~/flexcore/003-480 $
but svn files.
I have been running my python app with -v option to find out where it is trying to import from that file. BUT it is not returning nay debug line before that exception, so I guess its something imported previously or showing nothing if import fails.
I have also deleted all *.pyc files and rebooted machine to be sure there is noting left in memory, but problem persisted.
Is there any other option how to find out where is the problem? I am starting to be desperate..
That PS_Logic (whatever it may be) seems to be using Pyro to do remote calls to a server. In particular the line with the following, seems to be a remote call:
self.smart_obj.queue_status_get()
The server sends back a custom object and because it uses pickle as serialization format , your client program tries to reconstruct that object. Apparently you don't have the correct modules available in your client code, because it is pickle that fails when it tries to import the required module for you (to reassemble the response into objects)
There's got to be something in the manual of that ps_logic module that tells you about how to use it correctly, and that you should have it installed in the client as well perhaps.
(It is advised to not use pickle by the way, and stick to Pyro's default serializer, but that's another story)

Can't seem to import pyqt_fit on spyder

I have been using Atom.io to run my python codes but recently I was not able to run codes that require pyqt_fit. I could not find any packages on Atom that allow me to run pyqt. As such, I downloaded anaconda and spyder. I used homebrew to intall qt, sip, and pyqt.
In spyder, I tried to do the following:
import pyqt_fit
from pyqt_fit import plot_fit
import numpy as np
from matplotlib import pylab
x = np.arange(0,3,0.01)
y = 2*x + 4*x**2 + np.random.randn(*x.shape)
def fct(params, x):
(a0, a1, a2) = params
return a0 + a1*x + a2*x*x
fit = pyqt_fit.CurveFitting(x, y, (0,1,0), fct)
result = plot_fit.fit_evaluation(fit, x, y)
print(fit(x)) # Display the estimated values
plot_fit.plot1d(result)
pylab.show()
but I received the following errors:
runfile('/Users/Griffin/Atom/FYP Bifurcations/untitled1.py', wdir='/Users/Griffin/Atom/FYP Bifurcations')
Reloaded modules: pyqt_fit.utils, pyqt_fit.compat
Traceback (most recent call last):
File "<ipython-input-24-effdfa66efe9>", line 1, in <module>
runfile('/Users/Griffin/Atom/FYP Bifurcations/untitled1.py', wdir='/Users/Griffin/Atom/FYP Bifurcations')
File "/anaconda/lib/python2.7/site-packages/spyder/utils/site/sitecustomize.py", line 710, in runfile
execfile(filename, namespace)
File "/anaconda/lib/python2.7/site-packages/spyder/utils/site/sitecustomize.py", line 93, in execfile
builtins.execfile(filename, *where)
File "/Users/Griffin/Atom/FYP Bifurcations/untitled1.py", line 9, in <module>
import pyqt_fit
File "pyqt_fit/__init__.py", line 14, in <module>
from . import functions, residuals
File "pyqt_fit/functions/__init__.py", line 6, in <module>
from .. import loader
File "pyqt_fit/loader.py", line 3, in <module>
from path import path
ImportError: cannot import name path
Does it have anything to do with where I place the pyqt_fit folders?
Thanks in advance!
According to the PyQt-Fit requirements file, the path.py package is a dependency. However, the PyQt-Fit package has not been updated since 2015-02-11, and there have been incompatible changes made to path.py which it has not tracked. The specific error in the question is caused by this change, which removed path as an alias for the Path class.
To fix this, you could try monkey-patching path.py like this:
# before pyqt_fit is first imported
import path
path.path = path.Path
import pyqt_fit
from pyqt_fit import plot_fit
Alternatively, you could downgrade path.py to any version >= 2.4 and < 8.0.

Python Scipy.stats operations do not work when copied to another machine (where there is no python installed) with the source folder

I am using Scipy.stats to perform distribution fits on my data. It works perfectly on my machine. But when I package my script with Python source folder, python27.dll and try to run it on another machine (where python is not installed and I do not want to also) by invoking my script like below
Path/to/Python.exe Path/to/script.py
I get the follwing error
Traceback (most recent call last):
File "test2.py", line 2, in <module>
import scipy.stats
File "C:\Users\User\Desktop\Python27\lib\site-packages\scipy\stats\__init__.py
", line 344, in <module>
from .stats import *
File "C:\Users\User\Desktop\Python27\lib\site-packages\scipy\stats\stats.py",
line 173, in <module>
import scipy.special as special
File "C:\Users\User\Desktop\Python27\lib\site-packages\scipy\special\__init__.
py", line 636, in <module>
from ._ufuncs import *
ImportError: DLL load failed: The specified module could not be found.
My script is
import sys
import scipy.stats
import numpy
dataarray= [2.45, 3.67, 1.90, 2.56, 1.78, 2.67]
desc = scipy.stats.describe(dataarray)
print desc
This occurs only on Scipy import and scipy operations, all other imports and operations work fine. My purpose is to run this script in different machines without having to actually install python

python ImportError: No module named cy_unity graphlab

I am new to python and I am trying to work on a project with deep learning and want to use graphlab library. I use sublime text for coding on windows 10. My code is only this line:
import graphlab
I get this error msg:
Traceback (most recent call last):
File "test.py", line 1, in
import graphlab
File "graphlab__init__.py", line 59, in
from graphlab.data_structures.sgraph import Vertex, Edge
File "graphlab\data_structures__init__.py", line 25, in
from . import sframe
File "graphlab\data_structures\sframe.py", line 19, in
from ..connect import main as glconnect
File "graphlab\connect\main.py", line 26, in
from ..cython.cy_unity import UnityGlobalProxy
ImportError: No module named cy_unity
Try using graphlab.get_dependencies() in your interpreter.
I think I made a mistake when installing, i re-installed and it worked fine. Thanks anyway

Can't (.exe) my 300 line program using Cx_freeze or PyInstaller. (Matplotlib/Scipy)

I created my very first Python program for work and I have it running in PyCharm.
The program has a GUI, it basically pops up a window, has a directory tree so the user can select a jpg file, it then binarizes the image then lets the user select the binary threshold with a slider. It also displays two other images that I perform image transformations on.
SO, my problem is that I cannot build it to an .exe at all. I've tried Py2Exe, PyInstaller, and CX_freeze. I've tried building my own .spec files in Pyinstaller, but all have no success.
I'm running 2.7.6. I initially had 64 bit Python installed so I uninstalled everything Python related and installed 32 bit Python.
Initially when I was using pyinstaller I would just get ImportError: %1 is not a valid Win32 application, however since building a .spec file now I get actual errors. I've tried using hooks (hiddenimports=["wx", "scipy", "skimage", "matplotlib", "numpy"]) to make sure all the right files are included, but still no luck.
The main error that I'm receiving right now is: File "_ufuncs.pyx", line 1, in init scipy.special._ufuncs(scipy\special_ufuncs.c:19992) ImportError: No module named _ufuncs_cxx
I also created a smaller code below that just pops up a Wx window with a plot inside, but I receive the same error there.
Like I mentioned, I'm new to Python, and I've done a lot of reading and haven't been able to figure this out. I've spent probably 10-20 hours just trying to get it to compile correctly.
Below isn't my actual program, but a small snippet using wxPython and MatPlotLib that produces the same error.
This is the sample code:
import wx
from PIL import Image
import matplotlib
matplotlib.use("WXAgg")
from matplotlib.figure import Figure
from matplotlib.backends.backend_wxagg import FigureCanvasWxAgg as FigCanvas#,NavigationToolbar2WxAgg as NavigationToolbar
import matplotlib.pyplot as plt
import matplotlib.image as mpimg
import numpy
from scipy import interpolate
from matplotlib.backends.backend_wx import NavigationToolbar2Wx
from skimage.filter.rank import entropy
from skimage.morphology import disk
from skimage.util import img_as_ubyte
from skimage import color
from skimage import io
import skimage
from numpy import arange, sin, pi
class CanvasPanel(wx.Panel):
def __init__(self, parent):
wx.Panel.__init__(self, parent)
self.figure = Figure()
self.axes = self.figure.add_subplot(111)
self.canvas = FigCanvas(self, -1, self.figure)
self.sizer = wx.BoxSizer(wx.VERTICAL)
self.sizer.Add(self.canvas, 1, wx.LEFT | wx.TOP | wx.GROW)
self.SetSizer(self.sizer)
self.Fit()
def draw(self):
t = arange(0.0, 3.0, 0.01)
s = sin(2 * pi * t)
self.axes.plot(t, s)
if __name__ == "__main__":
app = wx.PySimpleApp()
fr = wx.Frame(None, title='test')
panel = CanvasPanel(fr)
panel.draw()
fr.Show()
This is the error I receive:
Traceback (most recent call last):
File "<string>", line 11, in <module>
File "c:\python27_32bit\lib\site-packages\PyInstaller-2.1-py2.7.egg\PyInstalle\loader\pyi_importers.py", line 270, in
load_module
exec(bytecode, module.__dict__)
File "C:\users\iolvera\PycharmProjects\EL and Grayscale Analyzer\build\compilertest\out00-PYZ.pyz\scipy.interpolate",line 156, in <module>
File "c:\python27_32bit\lib\site-packages\PyInstaller-2.1-py2.7.egg\PyInstaller\loader\pyi_importers.py", line 270, in load_module
exec(bytecode, module.__dict__)
File "C:\users\iolvera\PycharmProjects\EL and Grayscale Analyzer\build\compilertest\out00-PYZ.pyz\scipy.interpolate.interpolate", line 12, in <module>
File "c:\python27_32bit\lib\site-packages\PyInstaller-2.1-py2.7.egg\PyInstaller\loader\pyi_importers.py", line 270, in load_module
exec(bytecode, module.__dict__)
File "C:\users\iolvera\PycharmProjects\EL and Grayscale Analyzer\build\compilertest\out00-PYZ.pyz\scipy.special", line 531, in <module>
File "c:\python27_32bit\lib\site-packages\PyInstaller-2.1-py2.7.egg\PyInstaller\loader\pyi_importers.py", line 409, in load_module
module = imp.load_module(fullname, fp, filename, self._c_ext_tuple)
File "_ufuncs.pyx", line 1, in init scipy.special._ufuncs (scipy\special\_ufuncs.c:19992)
ImportError: No module named _ufuncs_cxx
My Python Path is correct C:\Python27_32bit\ and I also have \lib\site-packages\ and \DLL included correctly.
Like I mentioned, both programs run correctly in PyCharm.
Any help would be greatly appreciated.
Thank you!
I had this problem, and I fixed it in py2exe by specifically telling it to include the offending module. So, in setup.py:
includes = ['scipy.special._ufuncs_cxx']
setup(...,
options={"py2exe":{"includes":includes}}
)
I had this happen with a couple of other SciPy modules, too, so my includes list has about six things in it.