Here is my code for a Concrete Model of the Rosenbrock.
from pyomo.environ import *
from pyomo.opt import SolverFactory
import numpy as np
import math
import statistics
import time
m = ConcreteModel()
m.x = Var()
m.y = Var()
m.z = Var()
def rosenbrock(model):
return (1.0-m.x)2 + 100.0*(m.y - m.x2)2 + (1.0-m.y)2 + 100.0*(m.z - m.y2)2
m.obj = Objective(rule=rosenbrock, sense=minimize)
dist = 0.0
xval = yval = zval = error = times = []
for i in range(50):
m.x = np.random.uniform(low=-5.0, high=5.0)
m.y = np.random.uniform(low=-5.0, high=5.0)
m.z = np.random.uniform(low=-5.0, high=5.0)
solver = SolverFactory('ipopt')
t1 = time.time()
results = solver.solve(m, tee=True)
The solver.solve line when passed the tee=True prints out this beautiful display of all sorts of nice information. I want to access that information from the prinout and have scoured Pyomo and IPOPT documentation and cannot seem to understand how to access the values that are printed to the screen. I've also included a short example of the printout, I want to save the values from each run so that I can iterate and gather statistics over the total range.
Number of nonzeros in equality constraint Jacobian...: 0
Number of nonzeros in inequality constraint Jacobian.: 0
Number of nonzeros in Lagrangian Hessian.............: 5
Total number of variables............................: 3
variables with only lower bounds: 0
variables with lower and upper bounds: 0
variables with only upper bounds: 0
Total number of equality constraints.................: 0
Total number of inequality constraints...............: 0
inequality constraints with only lower bounds: 0
inequality constraints with lower and upper bounds: 0
inequality constraints with only upper bounds: 0
****OMITTED****
Number of objective function evaluations = 45
Number of objective gradient evaluations = 23
Number of equality constraint evaluations = 0
Number of inequality constraint evaluations = 0
Number of equality constraint Jacobian evaluations = 0
Number of inequality constraint Jacobian evaluations = 0
Number of Lagrangian Hessian evaluations = 22
Total CPU secs in IPOPT (w/o function evaluations) = 0.020
Total CPU secs in NLP function evaluations = 0.000
I need some of these values but I see no feasible interface to access them from my search of the documentation, any wizards know how to do this? Thanks.
See this Ipopt solver wrapper that was contributed to Pyomo. It's essentially a parser for the Ipopt output log and you should be able to generalize/expand it to collect any values that aren't currently collected.
https://github.com/Pyomo/pyomo/blob/master/pyomo/contrib/parmest/ipopt_solver_wrapper.py
Related
I am trying to write a multivariate Singular Spectrum Analysis with Monte Carlo test. To this extent I am working on a code piece that can reconstruct the input series using the lagged trajectory matrix and projection base (ST-PCs) that result from the pca/ssa decomposition of the input series. The attached code piece works for a lagged univariate (that is, single) time series, but I am struggling to make this reconstruction for a lagged multivariate time series. I don't quite get the procedure mathematically and - not surprisingly - I also did not manage to program it. Useful links are attached to the function descriptions of the accompanying code. Input data should be of the form (time * number of series), so say 288x3 implying 3 time series of 288 time levels.
I hope you can help me out!
import numpy as np
def lagged_covariance_matrix(data, M):
""" Computes the lagged covariance matrix using the Broomhead & King method
Background: Plaut, G., & Vautard, R. (1994). Spells of low-frequency oscillations and
weather regimes in the Northern Hemisphere. Journal of the atmospheric sciences, 51(2), 210-236.
Arguments:
data : pxn time series, where p denotes the length of the time series and n the number of channels
M : window length """
# explicitely 'add' spatial dimension if input is a single time series
if np.ndim(data) == 1:
data = np.reshape(data,(len(data),1))
T = data.shape[0]
L = data.shape[1]
N = T - M + 1
X = np.zeros((T, L, M))
for i in range(M):
X[:,:,i] = np.roll(data, -i, axis = 0)
X = X[:N]
# X constitutes the trajectory matrix and is a stacked hankel matrix
X = np.reshape(X, (N, M*L), order = 'C') # https://www.jstatsoft.org/article/viewFile/v067i02/v67i02.pdf
# choose the smallest projection basis for computation of the covariance matrix
if M*L >= N:
return 1/(M*L) * X.dot(X.T), X
else:
return 1/N * X.T.dot(X), X
def sort_by_eigenvalues(eigenvalues, PCs):
""" Sorts the PCs and eigenvalues by descending size of the eigenvalues """
desc = np.argsort(-eigenvalues)
return eigenvalues[desc], PCs[:,desc]
def Reconstruction(M, E, X):
""" Reconstructs the series as the sum of M subseries.
See: https://en.wikipedia.org/wiki/Singular_spectrum_analysis, 'Basic SSA' &
the work of Vivien Sainte Fare Garnot on univariate time series (https://github.com/VSainteuf/mcssa)
Arguments:
M : window length
E : eigenvector basis
X : trajectory matrix """
time = len(X) + M - 1
RC = np.zeros((time, M))
# step 3: grouping
for i in range(M):
d = np.zeros(M)
d[i] = 1
I = np.diag(d)
Q = np.flipud(X # E # I # E.T)
# step 4: diagonal averaging
for k in range(time):
RC[k, i] = np.diagonal(Q, offset = -(time - M - k)).mean()
return RC
#=====================================================================================================
#=====================================================================================================
#=====================================================================================================
# input data
data = None
# number of lags a.k.a. window length
M = 45 # M = 1 means no lag
covmat, X = lagged_covariance_matrix(data, M)
# get the eigenvalues and vectors of the covariance matrix
vals, vecs = np.linalg.eig(covmat)
eig_data, eigvec_data = sort_by_eigenvalues(vals, vecs)
# component reconstruction
recons_data = Reconstruction(M, eigvec_data, X)
The following works but does not make direct use of the projection base (ST-PCs). Hence the original question still stands, but this already helps a great lot and solves the problem for me. This code piece makes use of the similarity between the ST-PCs projection base and the u & vt matrices obtained from the single value decomposition of the lagged trajectory matrix. I think it gives back the same answer as one would obtain using the ST-PCs projection base?
def lag_reconstruction(data, X, M, pairs = None):
""" Reconstructs the series as the sum of M subseries using the lagged trajectory matrix.
Based on equation 2.9 of Plaut, G., & Vautard, R. (1994). Spells of low-frequency oscillations and weather regimes in the Northern Hemisphere. Journal of Atmospheric Sciences, 51(2), 210-236.
Inspired by work of R. van Westen and C. Wieners """
time = data.shape[0] # number of time levels of the original series
L = data.shape[1] # number of input series
N = time - M + 1
u, s, vt = np.linalg.svd(X, full_matrices = False)
rc = np.zeros((time, L, M))
for t in range(time):
counter = 0
for i in range(M):
if t-i >= 0 and t-i < N:
counter += 1
if pairs:
for k in pairs:
rc[t,:,i] += u[t-i, k] * s[k] * vt[k, i*L : i*L + L]
else:
for k in range(len(s)):
rc[t,:,i] += u[t-i, k] * s[k] * vt[k, i*L : i*L + L]
rc[t] = rc[t]/counter
return rc
I have the following code, which uses gradient descent to find the global minimum of y = (x+5)^2:
cur_x = 3 # the algorithm starts at x=3
rate = 0.01 # learning rate
precision = 0.000001 # this tells us when to stop the algorithm
previous_step_size = 1
max_iters = 10000 # maximum number of iterations
iters = 0 # iteration counter
df = lambda x: 2*(x+5) # gradient of our function
while previous_step_size > precision and iters < max_iters:
prev_x = cur_x # store current x value in prev_x
cur_x = cur_x - rate * df(prev_x) # grad descent
previous_step_size = abs(cur_x - prev_x) # change in x
iters = iters+1 # iteration count
print("Iteration",iters,"\nX value is",cur_x) # print iterations
print("The local minimum occurs at", cur_x)
The procedure is fairly simple, and among the most intuitive and brief for solving such a problem (at least, that I'm aware of).
I'd now like to apply this to solving a system of nonlinear equations. Namely, I want to use this to solve the Time Difference of Arrival problem in three dimensions. That is, given the coordinates of 4 observers (or, in general, n+1 observers for an n dimensional solution), the velocity v of some signal, and the time of arrival at each observer, I want to reconstruct the source (determine it's coordinates [x,y,z].
I've already accomplished this using approximation search (see this excellent post on the matter: ), and I'd now like to try doing so with gradient descent (really, just as an interesting exercise). I know that the problem in two dimensions can be described by the following non-linear system:
sqrt{(x-x_1)^2+(y-y_1)^2}+s(t_2-t_1) = sqrt{(x-x_2)^2 + (y-y_2)^2}
sqrt{(x-x_2)^2+(y-y_2)^2}+s(t_3-t_2) = sqrt{(x-x_3)^2 + (y-y_3)^2}
sqrt{(x-x_3)^2+(y-y_3)^2}+s(t_1-t_3) = sqrt{(x-x_1)^2 + (y-y_1)^2}
I know that it can be done, however I cannot determine how.
How might I go about applying this to 3-dimensions, or some nonlinear system in general?
I am trying to write a function to calculate the binomial coefficients using this formula:
The problem I am having is that I can not mange to get the correct answer. This is an example of two ways I have tried to write the function.
def binomial(n, i):
total = 0
for j in range(1, (n-i+1)):
n = float(n)
i = float(i)
j = float(j)
product = (i+j) / j
if total == 0:
total = product
else:
total = total * product
print '%.f' %total
or like this using numpy
import numpy as np
def binomial_np(n, i):
array = np.zeros(n-i+1)
for j in range(1, (n-i+1)):
s = float(j)
n = float(n)
i = float(i)
array[j] = (i+s)/s
array = array[1 : ]
array = np.prod(array)
print '%.f' %array
Both of the functions produces almost the correct result. After looking around a bit on the forum I did find some other examples that do produce the correct result, like this one from Python Binomial Coefficient
import math
x = int(input("Enter a value for x: "))
y = int(input("Enter a value for y: "))
if y == x:
print(1)
elif y == 1: # see georg's comment
print(x)
elif y > x: # will be executed only if y != 1 and y != x
print(0)
else: # will be executed only if y != 1 and y != x and x <= y
a = math.factorial(x)
b = math.factorial(y)
c = math.factorial(x-y) # that appears to be useful to get the correct result
div = a // (b * c)
print(div)
The real question I have from this is if there is something wrong with the way I have written the formulas, or if it just isnt possible to get the correct answer this way because of how float's and number of decimals work in Python. Hope someone can point me in the right direction on what I am doing wrong here.
The slight discrepancies seem to come from using floating point arithmetic. However, if you are sure that n and i are integers, there is no need at all for floating point values in your routine. You can just do
def binomial(n, i):
result = 1
for j in range(1, n-i+1):
result = result * (i+j) // j
return result
This works because the product of 2 consecutive numbers is divisible by 1*2, the product of 3 consecutive numbers is divisible by 1*2*3, ... the product of n-i consecutive numbers is divisible by (n-i)!. The calculations in the code above are ordered to that only integers result, so you get an exact answer. This because my code does not calculate (i+j)/j as your code does; it calculates result * (i+j) and only then divides by j. This code also does a fairly good job of keeping the integer values as small as possible, which should increase speed.
If course, if n or i is float rather than integer, this may not work. Also note this code does not check that 0 <= i <= n, which should be done.
I would indeed see float precision as the main problem here. You do floating point division, which means your integers may get rounded. I suggest you maintain the numerator and denominator as separate numbers, and do the division in the end. Or, if the numbers get too big using this approach, write some gcd computation and cancel common factors along the way. But only do integer divisions (//) to avoid loss of precision.
I have an image of land cover and I segmented it using K-means clustering. Now I want to calculate the accuracy of my segmentation algorithm. I read somewhere that dice co-efficient is the substantive evaluation measure. But I am not sure how to calculate it.
I use Python 2.7
Are there any other effective evaluation methods? Please give a summary or a link to a source. Thank You!
Edits:
I used the following code for measuring the dice similarity for my original and the segmented image but it seems to take hours to calculate:
for i in xrange(0,7672320):
for j in xrange(0,3):
dice = np.sum([seg==gt])*2.0/(np.sum(seg)+np.sum(gt)) #seg is the segmented image and gt is the original image. Both are of same size
Please refer to Dice similarity coefficient at wiki
A sample code segment here for your reference. Please note that you need to replace k with your desired cluster since you are using k-means.
import numpy as np
k=1
# segmentation
seg = np.zeros((100,100), dtype='int')
seg[30:70, 30:70] = k
# ground truth
gt = np.zeros((100,100), dtype='int')
gt[30:70, 40:80] = k
dice = np.sum(seg[gt==k])*2.0 / (np.sum(seg) + np.sum(gt))
print 'Dice similarity score is {}'.format(dice)
If you are working with opencv you could use the following function:
import cv2
import numpy as np
#load images
y_pred = cv2.imread('predictions/image_001.png')
y_true = cv2.imread('ground_truth/image_001.png')
# Dice similarity function
def dice(pred, true, k = 1):
intersection = np.sum(pred[true==k]) * 2.0
dice = intersection / (np.sum(pred) + np.sum(true))
return dice
dice_score = dice(y_pred, y_true, k = 255) #255 in my case, can be 1
print ("Dice Similarity: {}".format(dice_score))
In case you want to evaluate with this metric within a deep learning model using tensorflow you can use the following:
def dice_coef(y_true, y_pred):
y_true_f = tf.reshape(tf.dtypes.cast(y_true, tf.float32), [-1])
y_pred_f = tf.reshape(tf.dtypes.cast(y_pred, tf.float32), [-1])
intersection = tf.reduce_sum(y_true_f * y_pred_f)
return (2. * intersection + 1.) / (tf.reduce_sum(y_true_f) + tf.reduce_sum(y_pred_f) + 1.)
This is an important clarification if what you're using has more than 2 classes (aka, a mask with 1 and 0).
If you are using multiple classes, make sure to specify that the prediction and ground truth also equal the value which you want. Otherwise you can end up getting DSC values greater than 1.
This is the extra ==k at the end of each [] statement:
import numpy as np
k=1
# segmentation
seg = np.zeros((100,100), dtype='int')
seg[30:70, 30:70] = k
# ground truth
gt = np.zeros((100,100), dtype='int')
gt[30:70, 40:80] = k
dice = np.sum(seg[gt==k]==k)*2.0 / (np.sum(seg[seg==k]==k) + np.sum(gt[gt==k]==k))
print 'Dice similarity score is {}'.format(dice)
I have a method that creates a 2 different instances (M, N) in a given x of times (math.random * x) the method will create object M and the rest of times object N.
I have written unit-tests with mocking the random number so I can assure that the method behaves as expected. However I am not sure on how to (and if) to test that the probability is accurate, for example if x = 0.1 I expect 1 out of 10 cases to return instance M.
How do I test this functionality?
Split the test. The first test should allow you to define what the random number generator returns (I assume you already have that). This part of the test just satisfies the "do I get the expected result if the random number generator would return some value".
The second test should just run the random number generator using some statistical analysis function (like counting how often it returns each value).
I suggest to wrap the real generator with a wrapper that returns "create M" and "create N" (or possibly just 0 and 1). That way, you can separate implementation from the place where it's used (the code which creates the two different instance shouldn't need to know how the generator is initialized or how you turn the real result into "create X".
I'll do this in the form of Python.
First describe your functionality:
def binomial_process(x):
'''
given a probability, x, return M with that probability,
else return N with probability 1-x
maybe: return random.random() > x
'''
Then test for this functionality:
import random
def binom(x):
return random.random() > x
Then write your test functions, first a setup function to put together your data from an expensive process:
def setUp(x, n):
counter = dict()
for _ in range(n):
result = binom(x)
counter[result] = counter.get(result, 0) + 1
return counter
Then the actual test:
import scipy.stats
trials = 1000000
def test_binomial_process():
ps = (.01, .1, .33, .5, .66, .9, .99)
x_01 = setUp(.01, trials)
x_1 = setUp(.1, trials)
x_33 = setUp(.1, trials)
x_5 = setUp(.5, trials)
x_66 = setUp(.9, trials)
x_9 = setUp(.9, trials)
x_99 = setUp(.99, trials)
x_01_result = scipy.stats.binom_test(x_01.get(True, 0), trials, .01)
x_1_result = scipy.stats.binom_test(x_1.get(True, 0), trials, .1)
x_33_result = scipy.stats.binom_test(x_33.get(True, 0), trials, .33)
x_5_result = scipy.stats.binom_test(x_5.get(True, 0), trials)
x_66_result = scipy.stats.binom_test(x_66.get(True, 0), trials, .66)
x_9_result = scipy.stats.binom_test(x_9.get(True, 0), trials, .9)
x_99_result = scipy.stats.binom_test(x_99.get(True, 0), trials, .99)
setups = (x_01, x_1, x_33, x_5, x_66, x_9, x_99)
results = (x_01_result, x_1_result, x_33_result, x_5_result,
x_66_result, x_9_result, x_99_result)
print 'can reject the hypothesis that the following tests are NOT the'
print 'results of a binomial process (with their given respective'
print 'probabilities) with probability < .01, {0} trials each'.format(trials)
for p, setup, result in zip(ps, setups, results):
print 'p = {0}'.format(p), setup, result, 'reject null' if result < .01 else 'fail to reject'
Then write your function (ok, we already did):
def binom(x):
return random.random() > x
And run your tests:
test_binomial_process()
Which on last output gives me:
can reject the hypothesis that the following tests are NOT the
results of a binomial process (with their given respective
probabilities) with probability < .01, 1000000 trials each
p = 0.01 {False: 10084, True: 989916} 4.94065645841e-324 reject null
p = 0.1 {False: 100524, True: 899476} 1.48219693752e-323 reject null
p = 0.33 {False: 100633, True: 899367} 2.96439387505e-323 reject null
p = 0.5 {False: 500369, True: 499631} 0.461122365668 fail to reject
p = 0.66 {False: 900144, True: 99856} 2.96439387505e-323 reject null
p = 0.9 {False: 899988, True: 100012} 1.48219693752e-323 reject null
p = 0.99 {False: 989950, True: 10050} 4.94065645841e-324 reject null
Why do we fail to reject on p=0.5? Let's look at the help on scipy.stats.binom_test:
Help on function binom_test in module scipy.stats.morestats:
binom_test(x, n=None, p=0.5, alternative='two-sided')
Perform a test that the probability of success is p.
This is an exact, two-sided test of the null hypothesis
that the probability of success in a Bernoulli experiment
is `p`.
Parameters
----------
x : integer or array_like
the number of successes, or if x has length 2, it is the
number of successes and the number of failures.
n : integer
the number of trials. This is ignored if x gives both the
number of successes and failures
p : float, optional
The hypothesized probability of success. 0 <= p <= 1. The
default value is p = 0.5
alternative : {'two-sided', 'greater', 'less'}, optional
Indicates the alternative hypothesis. The default value is
'two-sided'.
So .5 is the default null hypothesis for test, and it makes sense not to reject the null hypothesis in this case.