I'm trying to make Param mutable with initializer zero but when I tried to read the Param in the constraint its doesn't understand what is reading, the only way that I found to read is using .value but when the problem finishes declaring the whole constraint. the solver found the first solution but when I defined the new param value, it doesn't change the value of the param. I know if I am doing bad the declared or I haven't found the correct white to do that.
As a solver im using CPLEX
This is the way i defined the Param:
model.s_value = pe.Param(mutable=True, initialize=0)
But if I use the Param as a normal Param is a constraint promo sent this message:
ValueError: Constraint 'def_constraint[0]' encountered a strict inequality expression ('>' or '<'). All constraints must be formulated using using '<=', '>=', or '=='.
I think is because when pyomo tired to read the param that sends an object like this:
pyomo.core.base.param.IndexedParam object at 0x1939C6A0
After declaring all the variable I put the solver inside a for and there I redefine the value of the Param:
model.s_value.value= new_value
Please. Someone can explain to me who can I use correctly the mutable Param and how can I iterate the model.
This is a simple example of changing the value of a mutable parameter. You didn't post a full set of executable code above, so it is difficult to figure out what is going on. If this below doesn't answer your question, request you update (edit) your post above with a minimal reproducible example.
import pyomo.environ as pyo
m = pyo.ConcreteModel()
m.x = pyo.Var(domain=pyo.NonNegativeReals)
m.p = pyo.Param(mutable=True, initialize = 10)
# the problem
m.OBJ = pyo.Objective(expr=m.x)
m.c1 = pyo.Constraint(expr=m.x >= m.p)
solver = pyo.SolverFactory('glpk')
# solve it...
results = solver.solve(m)
m.display() # x=10
# change the mutable parameter
m.p = 5
# re-solve it
results = solver.solve(m)
m.display() # x=5
This topic describes how to fit multiple data-sets using lmfit:
Python and lmfit: How to fit multiple datasets with shared parameters?
However it uses a fitting/objective function written by the user.
I was wondering if it's possible to fit multiple data-sets using lmfit without writing an objective function and using model.fit() method of the model class.
As an example: Lets say we have multiple data sets of (x,y) coordinates that we want to fit using the same model function in order to find the set of parameters that on average fit all the data best.
import numpy as np
from lmfit import Model, Parameters
from lmfit.models import GaussianModel
def gauss(x, amp, cen, sigma):
return amp*np.exp(-(x-cen)**2/(2.*sigma**2))
x1= np.arange(0.,100.,0.1)
x2= np.arange(0.,100.,0.09)
y1= gauss(x1, 1.,50.,5.)+ np.random.normal(size=len(x1), scale=0.1)
y2= gauss(x2, 0.8,48.4.,4.5)+ np.random.normal(size=len(x2), scale=0.1)
mod= GaussianModel()
params= mod.make_params()
mod.fit([y1,y2], params, x= [x1, x2])
I guess if this is possible the data has to be passed to mod.fit in the right type. The documentation only says that mod.fit takes an array-like data input.
I tried to give it lists and arrays. If I pass the different data sets as a list I get a ValueError: setting an array element with a sequence
If I pass an array I get an AttributeError: 'numpy.ndarray' has no atribute 'exp'
So am I just trying to do something that isn't possible or am I doing something wrong?
Well, I think the answer is "sort of". The lmfit.Model class is meant to represent a model for an array of data. So, if you can map your multiple datasets into a numpy ndarray (say, with np.concatenate), you can probably write a Model function to represent this by building sub-models for the different datasets and concatenating them in the same way.
I don't think you could do that with any of the built-in models. I also think that once you start down the road of writing complex model functions, it isn't a very big jump to writing objective functions. That is, what would be
def model_function(x, a, b, c):
### do some calculation with x, a, b, c values
result = a + x*b + x*x*c
return result
might become
def objective_function(params, x, data):
vals = params.valuesdict()
return data - model_function(x, vals['a'], vals['b'], vals['c'])
If that do_calc() is doing anything complex, the additional burden of unpacking the parameters and subtracting the data is pretty small. And, especially if some parameters would be used for multiple datasets and some only for particular datasets, you'll have to manage that in either the model function or the objective function. In the example you link to, my answer included a loop over datasets, picking out parameters by name for each dataset. You'll probably want to do something like that. You could probably do that in a model function by thinking of it as modeling the concatenated datasets, but I'm not sure you'd really gain a lot by doing that.
I found the problem. Actually model.fit() will handle arrays of multiple data sets just fine and perform a proper fit. The correct call of model.fit() with multiple data sets would be:
import numpy as np
from lmfit import Model, Parameters
from lmfit.models import GaussianModel
import matplotlib.pyplot as plt
def gauss(x, amp, cen, sigma):
"basic gaussian"
return amp*np.exp(-(x-cen)**2/(2.*sigma**2))
x1= np.arange(0.,100.,0.1)
x2= np.arange(0.,100.,0.1)
y1= gauss(x1, 1.,50.,5.)+ np.random.normal(size=len(x1), scale=0.01)
y2= gauss(x2, 0.8,48.4,4.5)+ np.random.normal(size=len(x2), scale=0.01)
mod= GaussianModel()
params= mod.make_params()
params['amplitude'].set(1.,min=0.01,max=100.)
params['center'].set(1.,min=0.01,max=100.)
params['sigma'].set(1.,min=0.01,max=100.)
result= mod.fit(np.array([y1,y2]), params,method='basinhopping',
x=np.array([x1,x2]))
print(result.fit_report(min_correl=0.5))
fig, ax = plt.subplots()
plt.plot(x1,y1, lw=2, color='red')
plt.plot(x2,y2, lw=2, color='orange')
plt.plot(x1,result.eval(x=x1), lw=2, color='black')
plt.show()
The problem in the original code actually lies in the fact that my data sets don't have the same length. However I'm not sure at all how to handle this in the most elegant way?
I'm starting with pyomo and I have some questions.
I create an abstract model and the correspondenting data file. I would like to solve it in the script in order to use the solutions of the variables later.
I tried to do it with the next code:
data = DataPortal()
data.load(filename="Datos\Datos_reactor2.dat")
instance = model.create_instance(data)
opt = SolverFactory("ipopt")
results = opt.solve(instance)
And it solve it perfectly, but I had some problems with the data. In my model there aren't set, so the data are simple constants defined as:
param qv := 2.832;
param ci := 14.46;
...
However if I use this data file python tell me an error:
ValueError: Parameter 'qv' defined with '1' dimensions, but data has '1' values: [2.832]. Are you missing a value for a 1-dimensional index?
When I change my data file defining the parameters like:
param qv := 1 2.832;
param ci := 1 14.46;
...
I got another error:
RuntimeError: Failed to set value for param=qv, index=1, value=2.832. source error message="Error setting parameter value: Cannot treat the scalar Param 'qv' as an array"
Finally, to can solve the problem I have to define the constants as tables (see beolw), but I don't understand the previous errors.
table qv := 2.832;
table ci := 14.46;
...
To highlight just say that if I try to solve the problem with:
!pyomo solve Reactor2.py Datos_reactor.dat --solver=ipopt --summary
With the data defined as in the first case (param qv := 2.832; ...) I don't have any problem and I got the same solution.
Can anyone tell me the difference and explain me the errors?
Thank you!!
Maria
I haven't seen this specific scenario in my research for this error in Numba. This is my first time using the package so it might be something obvious.
I have a function that calculates engineered features in a data set by adding, multiplying and/or dividing each column in a dataframe called data and I wanted to test whether numba would speed it up
#jit
def engineer_features(engineer_type,features,joined):
#choose which features to engineer (must be > 1)
engineered = features
if len(engineered) > 1:
if 'Square' in engineer_type:
sq = data[features].apply(np.square)
sq.columns = map(lambda s:s + '_^2',features)
for c1,c2 in combinations(engineered,2):
if 'Add' in engineer_type:
data['{0}+{1}'.format(c1,c2)] = data[c1] + data[c2]
if 'Multiply' in engineer_type:
data['{0}*{1}'.format(c1,c2)] = data[c1] * data[c2]
if 'Divide' in engineer_type:
data['{0}/{1}'.format(c1,c2)] = data[c1] / data[c2]
if 'Square' in engineer_type and len(sq) > 0:
data= pd.merge(data,sq,left_index=True,right_index=True)
return data
When I call it with lists of features, engineer_type and the dataset:
engineer_type = ['Square','Add','Multiply','Divide']
df = engineer_features(engineer_type,features,joined)
I get the error: Failed at object (analyzing bytecode)
'DataFlowAnalysis' object has no attribute 'op_MAKE_FUNCTION'
Same question here. I think the problem might be the lambda function since numba does not support function creation.
I had this same error. Numba doesnt support pandas. I converted important columns from my pandas df into bunch of arrays and it worked successfully under #JIT.
Also arrays are much faster then pandas df, incase you need it for processing large data.
I have a tensor which I convert into a vector by flattening, now I want to remove the duplicate values in this vector. How can I do this? What is equivalent for numpy.unique() in theano?
x1 = T.itensor3('x1')
y1 = T.flatten(x1)
#z1 = T.unique() How do I do this?
For e.g. my tensor may be : [1,1,2,3,3,4,4,5,1,3,4]
and I want : [1,2,3,4,5]
EDIT: this is now available in Theano: http://deeplearning.net/software/theano/library/tensor/extra_ops.html#theano.tensor.extra_ops.Unique
This question was also asked on theano-user mailing list. The conclusion is that this is one of the function NumPy function that isn't wrapped in Theano. As he don't need the grad, it can be rapidly wrapped. Here is an example who expect the outputs to be the same as the input.
from theano.compile.ops import as_op
#as_op(itypes=[theano.tensor.imatrix],
otypes=[theano.tensor.imatrix])
def numpy_unique(a):
return numpy.unique(a)
More doc about as_op is available here: http://deeplearning.net/software/theano/tutorial/extending_theano.html#as-op-example