I've been trying to get the HiGHS solver to work with the Pyomo kernel interface, but when I try to solve it it gives me this error: "AttributeError: unreadable attribute 'domain'".
I created a simple example to reproduce is - does HiGHS not not support the kernel?
import pyomo.kernel as pmo
from pyomo.contrib import appsi
m = pmo.block()
m.x1 = pmo.variable(domain=pmo.NonNegativeReals)
m.c = pmo.constraint(m.x1 == 5)
m.o = pmo.objective(m.x1*10, sense=1)
opt = pmo.SolverFactory("appsi_highs")
#opt = appsi.solvers.Highs()
result = opt.solve(m)
Pyomo's interface to HiGHS does not currently support kernel.
Related
I am trying to create a generative network based on the pre-trained Inception_v3.
1) I fix all the weights in the model
2) create a Variable whose size is (2, 3, 299, 299)
3) create targets of size (2, 1000) that I want my final layer activations to become as close as possible to by optimizing the Variable.
(I do not set the batchsize of 1, because unlike VGG16, Inception_v3 doesn't take batchsize=1, but that's not the point).
The following code should work, but gives me the error: «RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation».
# minimalist code with Inception_v3 that throws the error:
import torch
from torch.autograd import Variable
import torch.optim as optim
import torch.nn as nn
import torchvision
torch.set_default_tensor_type('torch.FloatTensor')
Iv3 = torchvision.models.inception_v3(pretrained=True)
for i in Iv3.parameters():
i.requires_grad = False
criterion = nn.CrossEntropyLoss()
x = Variable(torch.randn(2, 3, 299, 299), requires_grad=True)
target = torch.empty(2, dtype=torch.long).random_(1000)
output = Iv3(x)
loss = criterion(output[0], target)
loss.backward()
print(x.grad)
This is very strange, because if I do the same thing with VGG16, everything works fine:
# minimalist working code with VGG16:
import torch
from torch.autograd import Variable
import torch.optim as optim
import torch.nn as nn
import torchvision
# torch.cuda.empty_cache()
# vgg16 = torchvision.models.vgg16(pretrained=True).cuda()
# torch.set_default_tensor_type('torch.cuda.FloatTensor')
torch.set_default_tensor_type('torch.FloatTensor')
vgg16 = torchvision.models.vgg16(pretrained=True)
for i in vgg16.parameters():
i.requires_grad = False
criterion = nn.CrossEntropyLoss()
x = Variable(torch.randn(2, 3, 229, 229), requires_grad=True)
target = torch.empty(2, dtype=torch.long).random_(1000)
output = vgg16(x)
loss = criterion(output, target)
loss.backward()
print(x.grad)
Please help.
Thanks to #iacolippo the issue is solved. Turns out the problem was due to Pytorch 1.0.0. No problem with Pytorch 0.4.1. though.
Below is the code for a simple Bayesian Linear regression. After I obtain the trace and the plots for the parameters, is there any way in which I can save the data that created the plots in a file so that if I need to plot it again I can simply plot it from the data in the file rather than running the whole simulation again?
import pymc3 as pm
import matplotlib.pyplot as plt
import numpy as np
x = np.linspace(0,9,5)
y = 2*x + 5
yerr=np.random.rand(len(x))
def soln(x, p1, p2):
return p1+p2*x
with pm.Model() as model:
# Define priors
intercept = pm.Normal('Intercept', 15, sd=5)
slope = pm.Normal('Slope', 20, sd=5)
# Model solution
sol = soln(x, intercept, slope)
# Define likelihood
likelihood = pm.Normal('Y', mu=sol,
sd=yerr, observed=y)
# Sampling
trace = pm.sample(1000, nchains = 1)
pm.traceplot(trace)
print pm.summary(trace, ['Slope'])
print pm.summary(trace, ['Intercept'])
plt.show()
There are two easy ways of doing this:
Use a version after 3.4.1 (currently this means installing from master, with pip install git+https://github.com/pymc-devs/pymc3). There is a new feature that allows saving and loading traces efficiently. Note that you need access to the model that created the trace:
...
pm.save_trace(trace, 'linreg.trace')
# later
with model:
trace = pm.load_trace('linreg.trace')
Use cPickle (or pickle in python 3). Note that pickle is at least a little insecure, don't unpickle data from untrusted sources:
import cPickle as pickle # just `import pickle` on python 3
...
with open('trace.pkl', 'wb') as buff:
pickle.dump(trace, buff)
#later
with open('trace.pkl', 'rb') as buff:
trace = pickle.load(buff)
Update for someone like me who is still coming over to this question:
load_trace and save_trace functions were removed. Since version 4.0 even the deprecation waring for these functions were removed.
The way to do it is now to use arviz:
with model:
trace = pymc.sample(return_inferencedata=True)
trace.to_netcdf("filename.nc")
And it can be loaded with:
trace = arviz.from_netcdf("filename.nc")
This way works for me :
# saving trace
pm.save_trace(trace=trace_nb, directory=r"c:\Users\xxx\Documents\xxx\traces\trace_nb")
# loading saved traces
with model_nb:
t_nb = pm.load_trace(directory=r"c:\Users\xxx\Documents\xxx\traces\trace_nb")
I have the following constraint in a simple MINLP model:
model.cm2=Constraint(expr = model.xB2 == log(1.0+model.xA2))
This works when I call bonmin (windows64 binary distribution from AMPL)
When swithing to the couenne solver I need to convert to log10 base
model.cm2=Constraint(expr = model.xB2 == 2.3*log10(1.0+model.xA2))
otherwise I get the error:
ApplicationError: Solver (asl) did not exit normally.
model.pprint() gives in the first case:
cm2 : Size=1, Index=None, Active=True
Key : Lower : Body : Upper : Active
None : 0.0 : xB2 - log( 1.0 + xA2 ) : 0.0 : True
I use the anaconda python installation and work with spyder.
Do anyone have an idea of the reason for this behaviour?
I have read the comment from jsiirola, but I do not think that the problem is evaluating the log of a negative number. Here is a complete test problem, that behaves the same way. If I solve with bonmin i can use log() if I use cuonne I have to use ln(10)*log10().
from pyomo.environ import *
solverpathb="..\\solversAMPL\\bonmin\\bonmin"
solverpathc="..\\solversAMPL\\couenne\\couenne"
model=ConcreteModel()
model.x = Var(within=NonNegativeReals,bounds=(1,2),doc='Nonnegative')
model.y = Var(within=NonNegativeReals,doc='Nonnegative')
model.obj = Objective(expr= model.x+model.y, sense=maximize)
model.c1=Constraint(expr = model.y == log(1.0+model.x))
#model.c2=Constraint(expr = model.y == 2.3*log10(1.0+model.x))
#Works with version c1 and c2 of the constraint
#solver = pyomo.opt.SolverFactory("bonmin", executable=solverpathb)
#only constraint c2 works with this solver
solver = pyomo.opt.SolverFactory("couenne", executable=solverpathc)
results = solver.solve(model, tee = True)
model.display()
The log file that should include errors only includes the model. This is the last part of the errors.
...
File "C:/../testproblem.py", line 24, in
results = solver.solve(model, tee = True)
File "C:\Users..\Python\Python36\site-packages\pyomo\opt\base\solvers.py", line 623, in solve
"Solver (%s) did not exit normally" % self.name)
ApplicationError: Solver (asl) did not exit normally.
Note: I can use the following code for the object function, also with couenne
model.obj = Objective(expr= model.x+log(1.0+model.x), sense=maximize)
First, the entire model would be most useful when debugging problems like this. Second, what error is being thrown by Couenne when it exits abnormally?
As to answers:
Pyomo uses the same interface for BONMIN and Couenne (they are both through the ASL), so you should never have to change your expression just because you switched solvers. log and log10 are not the same function, of course, so you are not solving the same problem.
I suspect the problem is that model.xA2 is going less than or equal to -1 (i.e., the solver is asking the ASL to evaluate log(0). The way to verify this is by looking at the solver log. Additionally, you will want to make sure that Pyomo sends "symbolic" labels to the solver so that the error will reference actual variable / constraint names and not just "x1, x2, x3, ..." and "c1, c2, c3, ..." in the error message.
Couenne is a global solver, whereas BONMIN is a local solver. As this is a nonconvex problem, Couenne is probably exploring parts of the solution space that BONMIN never went to.
Using a binary distribution from COIN-OR solved the problem, it was not a pyomo problem. The binary distribution of couenne downloaded from ampl.com doesn't, for some odd reason, accept the log-function, only log10.
[short summary: how to use TF high-level Estimator on Python with an external file reader? or with feed_dict?]
Been struggling with this for few days, couldn't find any solution on-line...
I'm using TF high-level modules (tf.contrib.learn.Estimator on tf1.0, or tf.estimator.Estimator on tf1.1),
features and targets (x/y) inputted through an input_fn, and the graph built on the model_fn.
Already trained a nn on 'small' data sets, in which the whole input is the part of the graph, using slice_input_producer etc. (I can push an example to github if it serves ppl here).
I try to train a larger nn on 'heavier' data-sets (10s-100s GB).
I have an external Python reader that does some nasty binary file reading, which I really don't want to get into.
This reader has its own queue.Queue with m1 samples. When I use it to extract the m1 {features} & {targets}, the net simply saves all these samples as const. in the first layer of the graph... completely undesired.
I try to either -
feed the output of the external file reader as input to my graph.
define a proper tf queue object that will keep updating the queue (each time a sample is dequeued, i want a completely other sample to be enqueued).
Reminding that I use the "high level", e.g.
self.Estimator = tf.contrib.learn.Estimator(
model_fn=self.model_fn,
model_dir=self.config['model_dir'],
config=tf.contrib.learn.RunConfig( ... ) )
def input_fn(self, mode):
batch_data = self.data[mode].next() # pops out a batch of samples, as numpy 4D matrices
... # some processing of batch data
features_dict = dict(data=batch_data.pop('data'))
targets_dict = batch_data
return features_dict, targets_dict
self.Estimator.fit(input_fn=lambda: self.input_fn(modekeys.TRAIN))
Attached is a final solution for integrating an external reader into the high-level TF api (tf.contrib.learn.Estimator / tf.estimator.Estimator).
Please note:
the architecture and "logic" is not important. it's a stupid simple net.
the external reader outputs a dictionary of numpy matrices.
the input_fn is using this reader.
In order to verify that the reader "pulls new values", I both
save the recent value to self.status (should be > 1.0)
save a summary, to be viewed in tensorboard.
Code example is in gist, and below.
import tensorflow as tf
import numpy as np
modekeys = tf.contrib.learn.ModeKeys
tf.logging.set_verbosity(tf.logging.DEBUG)
# Tested on python 2.7.9, tf 1.1.0
class inputExample:
def __init__(self):
self.status = 0.0 # tracing which value was recently 'pushed' to the net
self.model_dir = 'temp_dir'
self.get_estimator()
def input_fn(self):
# returns features and labels dictionaries as expected by tf Estimator's model_fn
data, labels = tf.py_func(func=self.input_fn_np, inp=[], Tout=[tf.float32, tf.float32], stateful=True)
data.set_shape([1,3,3,1]) # shapes are unknown and need to be set for integrating into the network
labels.set_shape([1,1,1,1])
return dict(data=data), dict(labels=labels)
def input_fn_np(self):
# returns a dictionary of numpy matrices
batch_data = self.reader()
return batch_data['data'], batch_data['labels']
def model_fn(self, features, labels, mode):
# using tf 2017 convention of dictionaries of features/labels as inputs
features_in = features['data']
labels_in = labels['labels']
pred_layer = tf.layers.conv2d(name='pred', inputs=features_in, filters=1, kernel_size=3)
tf.summary.scalar(name='label', tensor=tf.squeeze(labels_in))
tf.summary.scalar(name='pred', tensor=tf.squeeze(pred_layer))
loss = None
if mode != modekeys.INFER:
loss = tf.losses.mean_squared_error(labels=labels_in, predictions=pred_layer)
train_op = None
if mode == modekeys.TRAIN:
train_op = tf.contrib.layers.optimize_loss(
loss=loss,
learning_rate = 0.01,
optimizer = 'SGD',
global_step = tf.contrib.framework.get_global_step()
)
predictions = {'estim_exp': pred_layer}
return tf.contrib.learn.ModelFnOps(mode=mode, predictions=predictions, loss=loss, train_op=train_op)
def reader(self):
self.status += 1
if self.status > 1000.0:
self.status = 1.0
return dict(
data = np.random.randn(1,3,3,1).astype(dtype=np.float32),
labels = np.sin(np.ones([1,1,1,1], dtype=np.float32)*self.status)
)
def get_estimator(self):
self.Estimator = tf.contrib.learn.Estimator(
model_fn = self.model_fn,
model_dir = self.model_dir,
config = tf.contrib.learn.RunConfig(
save_checkpoints_steps = 10,
save_summary_steps = 10,
save_checkpoints_secs = None
)
)
if __name__ == '__main__':
ex = inputExample()
ex.Estimator.fit(input_fn=ex.input_fn)
You can use tf.constant if you have the training data already in python memory as shown in the abalone TF example: https://github.com/tensorflow/tensorflow/blob/r1.1/tensorflow/examples/tutorials/estimators/abalone.py#L138-L141
Note: copying the data from disk to Python to TensorFlow is often less efficient than constructing an input pipeline in TensorFlow (i.e. loading data from disk directly into TensorFlow Tensors), such as using tf.contrib.learn.datasets.base.load_csv_without_header.
Code
import numpy as np
import matplotlib.pyplot as plt
fig = plt.figure()
ax = fig.add_subplot(111)
t = np.arange(0.0, 5.0, 0.01)
s = np.cos(2*np.pi*t)
line, = ax.plot(t, s, lw=2)
ax.annotate('local max', xy=(2, 1), xytext=(3, 1.5),
arrowprops=dict(facecolor='black', shrink=0.05),
)
ax.set_ylim(-2,2)
plt.show()
from http://matplotlib.org/1.2.0/users/annotations_intro.html
return
TypeError: 'dict' object is not callable
I manged to fixed it with
xxx={'facecolor':'black', 'shrink':0.05}
ax.annotate('local max', xy=(2, 1), xytext=(3, 1.5),
arrowprops=xxx,
)
Is this the best way ?
Also what caused this problem ? ( I know that this started with Python 2.7)
So if somebody know more, please share.
Since the code looks fine and runs ok on my machine, it seems that you may have a variable named "dict" (see this answer for reference). A couple of ideas on how to check:
use Pylint.
if you suspect one specific builtin, try checking it's type (type(dict)), or look at the properties/functions it has (dir(dict)).
open a fresh notebook and try again, if you only observe the problem in interactive session.
try alternate syntax to initialise the dictionary
ax.annotate('local max', xy=(2, 1), xytext=(3, 1.5),
arrowprops={'facecolor':'black', 'shrink':0.05})
try explicitly instancing a variable of this type, using the alternate syntax (as you did already).