Related
How do I fit a curve on a barplot?
I have an equation, the diffusion equation, which has some unknown parameters, these parameters make the curve larger, taller, etc. On the other hand I have a barplot coming from a simulation. I would like to fit the curve on the barplot, and find the best parameters for the curve, how can I do that?
This is what I obtained by 'manual fitting', so basically I changed manually all the parameters for hours. However is there a way to do this with python?
To make it simple, imagine I have the following code:
import matplotlib.pyplot as plt
list1 = []
for i in range(-5,6):
list1.append(i)
width = 1/1.5
list2 = [0,0.2,0.6,3.5,8,10,8,3.5,0.6,0.2,0]
plt.bar(list1,list2,width)
plt.show()
T = 0.13
xx = np.arange(-6,6,0.01)
yy = 5*np.sqrt(np.pi)*np.exp(-((xx)**2)/(4*T))*scipy.special.erfc((xx)/(2*np.sqrt(T))) + np.exp(-((xx)**2)/(4*T))
plt.plot(xx,yy)
plt.show()
Clearly the fitting here would be pretty hard, but anyway, is there any function or such that allows me to find the best coefficients for the equation: (where T is known)
y = A*np.sqrt(np.pi*D)*np.exp(-((x-E)**2)/(4*D*T))*scipy.special.erfc((x-E)/(2*np.sqrt(D*T))) + 300*np.exp(-((x-E)**2)/(4*D*T))
EDIT: This is different from the already asked question and from the scipy documentation example. In the latter the 'xdata' is the same, while in my case it might and might not be. Furthermore I would also be able to plot this curve fitting, which isn't shown on the documentation. The height of the bars is not a function of the x's! So my xdata is not a function of my ydata, this is different from what is in the documentation.
To see what I mean try to change the code in the documentation a little bit, to fall into my example, try this:
def func(x,a,b,c):
return a * np.exp(-b * x) + c
xdata = np.linspace(0,4,50)
y = func(xdata, 2.5, 1.3, 0.5)
ydata = [1,6,3,4,6,7,8,5,7,0,9,8,2,3,4,5]
popt, pcov = curve_fit(func,xdata,ydata)
if you run this, it doesn't work. The reason is that I have 16 elements for the ydata and 50 for the function. This happens because y takes values from xdata, while ydata takes values from another set of x values, which is here unknown.
Thank you
I stand by my thinking that this question is a duplicate. Here is a brief example of the typical workflow using curve_fit. Let me know if you still think that your situation is different.
import numpy as np
from scipy.optimize import curve_fit
import matplotlib.pyplot as plt
# bar plot data
list1 = range(-5, 6)
list2 = [0, 0.2, 0.6, 3.5, 8, 10,
8, 3.5, 0.6, 0.2, 0]
width = 1/1.5
plt.bar(list1, list2, width, alpha=0.75)
# fit bar plot data using curve_fit
def func(x, a, b, c):
# a Gaussian distribution
return a * np.exp(-(x-b)**2/(2*c**2))
popt, pcov = curve_fit(func, list1, list2)
x = np.linspace(-5, 5, 100)
y = func(x, *popt)
plt.plot(x + width/2, y, c='g')
I have the following plot in which the X range is very wide and the shape of the graph near 1 MeV to 0.1 MeV is suppressed.
I want a plot where the X scale has equal separation (or equal grid) between 10,1,0.1 MeV.
You can use matplotlib's semilogx function instead of plot to make the x axis logarithmic.
Here's a short example:
import matplotlib.pyplot as plt
import numpy as np
x = np.arange(0.01,14,0.01)
y = np.log(100*x)
fig,(ax1,ax2) = plt.subplots(2)
ax1.plot(x,y)
ax1.set_xlim(x[-1],x[0])
ax1.set_title('plot')
ax2.semilogx(x,y)
ax2.set_xlim(x[-1],x[0])
ax2.set_title('semilogx')
plt.show()
also consider
ax.set_xscale("log")
http://matplotlib.org/examples/pylab_examples/aspect_loglog.html
I am trying to follow a MATLAB example of meshgrid + interpolation. The example code is found HERE. On that site, I am going through the following example: Example – Displaying Nonuniform Data on a Surface.
Now, I would like to produce a similar plot in Python (Numpy + Matplotlib) to what is shown there in MATLAB. This is the plot that MATLAB produces:
I am having trouble with doing this in Python. Here is my code and my output in Python 2.7:
from matplotlib.mlab import griddata
import matplotlib.pyplot as plt
from matplotlib import cm
import numpy as np
from mpl_toolkits.mplot3d import Axes3D
x = np.random.rand(200)*16 - 8
y = np.random.rand(200)*16 - 8
r = np.sqrt(x**2 + y**2)
z = np.sin(r)/r
xi = np.linspace(min(x),max(x), 100)
yi = np.linspace(min(y),max(y), 200)
X,Y = np.meshgrid(xi,yi)
Z = griddata(x, y, z, X, Y, interp='linear')
fig = plt.figure()
ax = fig.gca(projection='3d')
surf = ax.plot_surface(X, Y, Z, rstride=1, cstride=1,cmap=cm.jet)
Here is the result of my attempt at doing this with matplotlib and NumPy..
Could someone please help me recreate the MATLAB plot in matplotlib, as either a mesh or a surface plot?
So it seems that the major differences in the look have to do with the default number of lines plotted by matlab, which can be adjusted by increasing rstride and cstride. In terms of color, in order for the colormap to be scaled properly it is probably best in this case to set your limits, vmin and vmax because when automatically set, it will use the min and max of Z, but in this case, they are both nan, so you could use np.nanmin and np.nanmax.
from matplotlib.mlab import griddata
import matplotlib.pyplot as plt
from matplotlib import cm
import numpy as np
from mpl_toolkits.mplot3d import Axes3D
x = np.random.rand(200)*16 - 8
y = np.random.rand(200)*16 - 8
r = np.sqrt(x**2 + y**2)
z = np.sin(r)/r
xi = np.linspace(min(x),max(x), 100)
yi = np.linspace(min(y),max(y), 200)
X,Y = np.meshgrid(xi,yi)
Z = griddata(x, y, z, X, Y, interp='linear')
fig = plt.figure()
ax = fig.gca(projection='3d')
surf = ax.plot_surface(X, Y, Z, rstride=5, cstride=5, cmap=cm.jet, vmin=np.nanmin(Z), vmax=np.nanmax(Z), shade=False)
scat = ax.scatter(x, y, z)
In matplotlib unfortunately I get some annoying overlapping/'clipping' problems, where Axes3d doesn't always properly determine the order in which object should be displayed.
I can use np.polyfit to fit a line in my scatter plot as shown bellow
a = np.array([1.08,2.05,1.56,0.73,1.1,0.73,0.34,0.73,0.88,2.05])
b=np.array([4.72131259, 6.60937492, 6.41485738, 6.82386894, 6.20293278, 7.22670489, 6.15681295, 5.91595178, 6.43917035, 6.64453907])
m1, b1 = np.polyfit(a, b, 1)
corr1 =a1.plot(a, m1*a+b1, '-', color='black')
a1.scatter(a, b)
Is there any way to fit a line using polyfit this time taking the errors for my points as shown bellow?
ae = np.empty(10)
ae.fill(0.15)
be = ae
sca1=a1.errorbar(a, b, ae, be, capsize=0, ls='none', color='black', elinewidth=1)
If you want to compute the fit and plot the (fixed at the given value) error bars over the fit points, like this:
Then this code will do the job:
import numpy as np
import matplotlib.pyplot as mp
a = np.array([1.08,2.05,1.56,0.73,1.1,0.73,0.34,0.73,0.88,2.05])
b=np.array([4.72131259, 6.60937492, 6.41485738, 6.82386894, 6.20293278, 7.22670489, 6.15681295, 5.91595178, 6.43917035, 6.64453907])
ae = np.empty(10)
ae.fill(0.15)
be = ae
m1, b1 = np.polyfit(a, b, 1)
mp.figure()
corr1 =mp.errorbar(a,m1*a+b1,ae,be, '-', color='black')
mp.scatter(a, b)
mp.show()
If you want to get the covariance of the fit, and use the standard deviation to set the error bars, instead use the code
import numpy as np
import matplotlib.pyplot as mp
import math
a = np.array([1.08,2.05,1.56,0.73,1.1,0.73,0.34,0.73,0.88,2.05])
b=np.array([4.72131259, 6.60937492, 6.41485738, 6.82386894, 6.20293278, 7.22670489, 6.15681295, 5.91595178, 6.43917035, 6.64453907])
coeff,covar = np.polyfit(a, b, 1,cov=True)
m1= coeff[0]
b1= coeff[1]
xe = math.sqrt(covar[0][0])
ye = math.sqrt(covar[1][2])
mp.figure()
corr1 =mp.errorbar(a,m1*a+b1,xe,ye, '-', color='black')
mp.scatter(a, b)
mp.show()
which gives a plot like this:
If you want to do a weighted fit, you can supply a weight vector to polyfit with the syntax
m2, b2 = np.polyfit(a, b, 1,w=weightvector)
According to the documentation the weightvector should contain 1 over the standard deviation of the data points.
If you want to do a least squares fit weighted by errors in BOTH x and y, I don't think polyfit does this - it will accept a weight vector for one dimension.
To supply errors in both dimensions as weights you would have to use scipy.optimize.leastsq.
There is a documentation page at this link of the Scipy documentation about doing fits with scipy.optimize.leastsq. The example talks about a fit to power law, but clearly a straight line could be done as well.
For errors in one dimension (Y) here, an example using leastsq is:
import numpy as np
import matplotlib.pyplot as mp
import math
from scipy import optimize
a = np.array([1.08,2.05,1.56,0.73,1.1,0.73,0.34,0.73,0.88,2.05])
b=np.array([4.72131259, 6.60937492, 6.41485738, 6.82386894, 6.20293278, 7.22670489, 6.15681295, 5.91595178, 6.43917035, 6.64453907])
aerr = np.empty(10)
aerr.fill(0.15)
berr=aerr
# fit a straight line with scipy scipy.optimize.leastsq
# define our (line) fitting function
fitfunc = lambda p, x: p[0] + p[1] * x
errfunc = lambda p, x, y, err: (y - fitfunc(p, x)) / err
pinit = [1.0, -1.0]
out = optimize.leastsq(errfunc, pinit,args=(a, b, aerr), full_output=1)
coeff = out[0]
covar = out[1]
print 'coeff', coeff
print 'covar', covar
m1= coeff[1]
b1= coeff[0]
xe = math.sqrt(covar[0][0])
ye = math.sqrt(covar[1][1])
# plot results
mp.figure()
corr2 =mp.errorbar(a,m1*a+b1,xe,ye, '-', color='red')
mp.scatter(a, b)
mp.show()
To take into account errors in both X and Y, you would have to change the definition of errfunc to reflect the specific technique you are using to do that. If a lambda isn't convenient you can instead define a function that will do that. I can't comment further on this without knowing what technique is being used to weight by X and Y errors, there are several in the literature.
I am using matplotlib.pyplot to create histograms. I'm not actually interested in the plots of these histograms, but interested in the frequencies and bins (I know I can write my own code to do this, but would prefer to use this package).
I know I can do the following,
import numpy as np
import matplotlib.pyplot as plt
x1 = np.random.normal(1.5,1.0)
x2 = np.random.normal(0,1.0)
freq, bins, patches = plt.hist([x1,x1],50,histtype='step')
to create a histogram. All I need is freq[0], freq[1], and bins[0]. The problem occurs when I try and use,
freq, bins, patches = plt.hist([x1,x1],50,histtype='step')
in a function. For example,
def func(x, y, Nbins):
freq, bins, patches = plt.hist([x,y],Nbins,histtype='step') # create histogram
bincenters = 0.5*(bins[1:] + bins[:-1]) # center bins
xf= [float(i) for i in freq[0]] # convert integers to float
xf = [float(i) for i in freq[1]]
p = [ (bincenters[j], (1.0 / (xf[j] + yf[j] )) for j in range(Nbins) if (xf[j] + yf[j]) != 0]
Xt = [j for i,j in p] # separate pairs formed in p
Yt = [i for i,j in p]
Y = np.array(Yt) # convert to arrays for later fitting
X = np.array(Xt)
return X, Y # return arrays X and Y
When I call func(x1,x2,Nbins) and plot or print X and Y, I do not get my expected curve/values. I suspect it something to do with plt.hist, since there is a partial histogram in my plot.
I don't know if I'm understanding your question very well, but here, you have an example of a very simple home-made histogram (in 1D or 2D), each one inside a function, and properly called:
import numpy as np
import matplotlib.pyplot as plt
def func2d(x, y, nbins):
histo, xedges, yedges = np.histogram2d(x,y,nbins)
plt.plot(x,y,'wo',alpha=0.3)
plt.imshow(histo.T,
extent=[xedges.min(),xedges.max(),yedges.min(),yedges.max()],
origin='lower',
interpolation='nearest',
cmap=plt.cm.hot)
plt.show()
def func1d(x, nbins):
histo, bin_edges = np.histogram(x,nbins)
bin_center = 0.5*(bin_edges[1:] + bin_edges[:-1])
plt.step(bin_center,histo,where='mid')
plt.show()
x = np.random.normal(1.5,1.0, (1000,1000))
func1d(x[0],40)
func2d(x[0],x[1],40)
Of course, you may check if the centering of the data is right, but I think that the example shows some useful things about this topic.
My recommendation: Try to avoid any loop in your code! They kill the performance. If you look, In my example there aren't loops. The best practice in numerical problems with python is avoiding loops! Numpy has a lot of C-implemented functions that do all the hard looping work.
You can use np.histogram2d (for 2D histogram) or np.histogram (for 1D histogram):
hst = np.histogram(A, bins)
hst2d = np.histogram2d(X,Y,bins)
Output form will be the same as plt.hist and plt.hist2d, the only difference is there is no plot.
No.
But you can bypass the pyplot:
import matplotlib.pyplot
fig = matplotlib.figure.Figure()
ax = matplotlib.axes.Axes(fig, (0,0,0,0))
numeric_results = ax.hist(data)
del ax, fig
It won't impact active axes and figures, so it would be ok to use it even in the middle of plotting something else.
This is because any usage of plt.draw_something() will put the plot in current axis - which is a global variable.
If you would like to simply compute the histogram (that is, count the number of points in a given bin) and not display it, the np.histogram() function is available