Here is code :
def _hard_sigmoid(x):
return tf.clip_by_value(tf.rint((x*0.2)/0.5),tf.rint(0.0),tf.rint(1.0))
def binary_tanh(x): # activation binarization to 1 and 0
a = (_hard_sigmoid(x)/0.5)*0.2
b = tf.clip_by_value(x, 0, 1)
return K.stop_gradient(a-b)+b
binary_tanh(x) # we are calling this function
This related to squeezedet CNN network activation function for each layer.
My question
Which activation function it represents.
What value I can set for my demo application. Like here they are dividing with some value and multiply by some value.
Related
I have a model that I need to solve many times, with different objective function coefficients.
Naturally, I want to spend as little time as possible on updating the model.
My current setup is as follows (simplified):
Abstract model:
def obj_rule(m):
return sum(Dist[m, n] * m.flow[m,n] for (m,n) in m.From * m.To)
m.obj = Objective(rule=obj_rule, sense=minimize)
After solve, I update the concrete model instance mi this way, we new values in Dist:
mi.obj = sum(Dist[m, n] * mi.flow[m,n] for (m,n) in mi.From * mi.To)
However, I see that the update line takes a lot of time - ca. 1/4 of the overall solution time, several seconds for bigger cases.
Is there some faster way of updating the objective function?
(After all, in the usual way of saving an LP model, the objective function coefficients are in a separate vector, so changing them should not affect anything else.)
Do you have a reason to define an Abstract model before creating your Concrete model? If you define your concrete model with the rule you show above, you should be able to just update your data and re-solve the model without a lot of overhead since you do don't redefine the objective object. Here's a simple example where I change the values of the cost parameter and re-solve.
import pyomo.environ as pyo
a = list(range(2)) # set the variables define over
#%% Begin basic model
model = pyo.ConcreteModel()
model.c = pyo.Param(a,initialize={0:5,1:3},mutable=True)
model.x = pyo.Var(a,domain = pyo.Binary)
model.con = pyo.Constraint(expr=model.x[0] + model.x[1] <= 1)
def obj_rule(model):
return(sum(model.x[ind] * model.c[ind] for ind in a))
model.obj = pyo.Objective(rule=obj_rule,sense=pyo.maximize)
#%% Solve the first time
solver = pyo.SolverFactory('glpk')
res=solver.solve(model)
print('x[0]: {} \nx[1]: {}'.format(pyo.value(model.x[0]),
pyo.value(model.x[1])))
# x[0]: 1
# x[1]: 0
#%% Update data and re-solve
model.c.reconstruct({0:0,1:5})
res=solver.solve(model)
print('x[0]: {} \nx[1]: {}'.format(pyo.value(model.x[0]),
pyo.value(model.x[1])))
# x[0]: 0
# x[1]: 1
I have a UNet++(view in private, code for model at the bottom of the article) which I'm trying to reconfigure. I'm getting some artifacts in some images so I'm following this article which suggests doing upsampling then a convolution operation.
I'm replacing the up-sample layers with sequential operation shown below but my model isn't learning. I suspect its to do with how I've configured the channels so I'd like another opinion.
Old up-sample operation:
self.up = nn.Upsample(scale_factor=2, mode='bilinear', align_corners=True)
New operations:
class upConv(nn.Module):
"""
Up sampling/ deconv block by factor of 2
"""
def __init__(self, in_ch, out_ch):
super().__init__()
self.upc = nn.Sequential(
nn.Upsample(scale_factor=2, mode='bilinear', align_corners=True),
nn.Conv2d(in_ch, out_ch*2, 3, stride=1, padding=1),
nn.BatchNorm2d(out_ch*2),
nn.ReLU(inplace=True)
)
def forward(self, x):
out = self.upc(x)
return out
My question is do these two operations have the same output/function within my model?
Do these two operations have the same output/function within my model?
If out_ch*2 == in_ch, then: yes, they have the same output shape.
If the input x is the output of a BatchNorm+ReLU op, then they could be even more similar.
I have a model trained to classify rgb values into 1000 categories.
#Model architecture
model = Sequential()
model.add(Dense(512,input_shape=(3,),activation="relu"))
model.add(BatchNormalization())
model.add(Dense(512,activation="relu"))
model.add(BatchNormalization())
model.add(Dense(1000,activation="relu"))
model.add(Dense(1000,activation="softmax"))
I want to be able to extract the output before the softmax layer so I can conduct analyses on different samples of categories within the model. I want execute softmax for each sample, and conduct analyses using a function named getinfo().
Model
Initially, I enter X_train data into model.predict, to get a vector of 1000 probabilities for each input. I execute getinfo() on this array to get the desired result.
Pop1
I then use model.pop() to remove the softmax layer. I get new predictions for the popped model, and execute scipy.special.softmax. However, getinfo() produces an entirely different result on this array.
Pop2
I write my own softmax function to validate the 2nd result, and I receive an almost identical answer to Pop1.
Pop3
However, when I simply calculate getinfo() on the output of model.pop() with no softmax function, I get the same result as the initial Model.
data = np.loadtxt("allData.csv",delimiter=",")
model = load_model("model.h5")
def getinfo(data):
objects = scipy.stats.entropy(np.mean(data, axis=0), base=2)
print(('objects_mean',objects))
colours_entropy = []
for i in data:
e = scipy.stats.entropy(i, base=2)
colours_entropy.append(e)
colours = np.mean(np.array(colours_entropy))
print(('colours_mean',colours))
info = objects - colours
print(('objects-colours',info))
return info
def softmax_max(data):
# calculate softmax whilst subtracting the max values (axis=1)
sm = []
count = 0
for row in data:
max = np.argmax(row)
e = np.exp(row-data[count,max])
s = np.sum(e)
sm.append(e/s)
sm = np.asarray(sm)
return sm
#model
preds = model.predict(X_train)
getinfo(preds)
#pop1
model.pop()
preds1 = model.predict(X_train)
sm1 = scipy.special.softmax(preds1,axis=1)
getinfo(sm1)
#pop2
sm2 = softmax_max(preds1)
getinfo(sm2)
#pop3
getinfo(preds1)
I expect to get the same output from Model, Pop1 and Pop2, but a different answer to Pop3, as I did not compute softmax here. I wonder if the issue is with computing softmax after model.predict? And whether I am getting the same result in Model and Pop3 because softmax is constraining the values between 0-1, so for the purpose of the getinfo() function, the result is mathematically equivalent?
If this is the case, then how do I execute softmax before model.predict?
I've gone around in circles with this, so any help or insight would be much appreciated. Please let me know if anything is unclear. Thank you!
model.pop() does not immediately have an effect. You need to run model.compile() again to recompile the new model that doesn't include the last layer.
Without the recompile, you're essentially running model.predict() twice in a row on the exact same model, which explains why Model and Pop3 give the same result. Pop1 and Pop2 give weird results because they are calculating the softmax of a softmax.
In addition, your model does not have the softmax as a separate layer, so pop takes off the entire last Dense layer. To fix this, add the softmax as a separate layer like so:
model.add(Dense(1000)) # softmax removed from this layer...
model.add(Activation('softmax')) # ...and added to its own layer
I have constructed a regression type of neural net (NN) with dropout by Tensorflow. I would like to know if it is possible to find which hidden units are dropped from the previous layer in the output file. Therefore, we could implement the NN results by C++ or Matlab.
The following is an example of Tensorflow model. There are three hidden layer with one output layer. After the 3rd sigmoid layer, there is a dropout with probability equal to 0.9. I would like to know if it is possible to know which hidden units in the 3rd sigmoid layer are dropped.
def multilayer_perceptron(_x, _weights, _biases):
layer_1 = tf.nn.sigmoid(tf.add(tf.matmul(_x, _weights['h1']), _biases['b1']))
layer_2 = tf.nn.sigmoid(tf.add(tf.matmul(layer_1, _weights['h2']), _biases['b2']))
layer_3 = tf.nn.sigmoid(tf.add(tf.matmul(layer_2, _weights['h3']), _biases['b3']))
layer_d = tf.nn.dropout(layer_3, 0.9)
return tf.matmul(layer_d, _weights['out']) + _biases['out']
Thank you very much!
There is a way to get the mask of 0 and 1, and of shape layer_3.get_shape() produced by tf.nn.dropout().
The trick is to give a name to your dropout operation:
layer_d = tf.nn.dropout(layer_3, 0.9, name='my_dropout')
Then you can get the wanted mask through the TensorFlow graph:
graph = tf.get_default_graph()
mask = graph.get_tensor_by_name('my_dropout/Floor:0')
The tensor mask will be of same shape and type as layer_d, and will only have values 0 or 1. 0 corresponds to the dropped neurons.
Simple and idiomatic solution (although possibly slightly slower than Oliver's):
# generate mask
mask = tf.nn.dropout(tf.ones_like(layer),rate)
# apply mask
dropped_layer = layer * mask
I am trying to create a page in which has ten labels and then update the values using the after() method .But the screen hangs .
The start definition is bind to a button creates the labels , and then using after method i am trying to call the update method in recursive fashion to update the label values .
Is their another way to call the update() method from the main function (main) .
def update(self,i):
self.lab_hold_X=25
self.data_hold_mL_screen[i].place(x=self.lab_hold_X,y=self.y_place)
self.data_hold_mL_screen[i]['text']=str(int(self.data_hold_mL_screen[i]['text']) + 1)
self.y_place +=30
self.valuess=i+1
if self.valuess <=10:
self.after(1000,self.update(self.valuess))
else :
self.valuess=0
self.after(1000,self.update(self.valuess))
def start(self):
self.lab_hold_X=25
self.lab_hold_Y=10
for i in range(0,10):
self.lab_hold_Y +=30
self.data_hold_mL_screen[i].place(x=self.lab_hold_X,y=self.lab_hold_Y)
self.lab_hold_X =25
self.after(1000,update(0))
I basically wish to create a page wherein i can get values from any external device and then display it on the screen , screen shows 10 values , on 11th iteration a value is recieved the screen shifts upward and the 1st value is discarded and the second value is displayed in 1st label position ..
This:
self.after(1000,update(0))
... needs to be this:
self.after(1000,lambda: update(0))
The same is true for the other places you call after. after requires a reference to a function, but you are calling the function and passing the result of the function to after. And since your function returns nothing, nothing is getting executed after 1000 ms.
A simple example using a list to hold the last 10 values, and a second list to hold the Label's StringVars that are updated. And please do not use "i", "l", or "O" as single character variable names as they can look like numbers.
from Tkinter import *
from functools import partial
class LabelTest():
def __init__(self, master):
self.master=master
self.string_vars=[] ## contains pointers to the label vars
self.values=[]
self.start()
Button(self.master, text="Quit", bg="orange", width=15,
command=self.master.quit).grid(row=20)
self.update(1)
def update(self, ctr):
""" Change the StringVars to the new values
"""
if len(self.values) > 9:
## full so delete the first one
del self.values[0]
self.values.append(ctr)
## update the StringVar with the corresponding
## value from self.values
for offset in range(len(self.values)):
self.string_vars[offset].set(self.values[offset])
ctr += 1
if ctr < 20: ## let's put a limit on this
self.master.after(1000, partial(self.update, ctr))
def start(self):
""" create 10 labels with StringVars that can be changed
each time a new value is created
"""
for ctr in range(10):
var=StringVar()
lab=Label(self.master, textvariable=var).grid(row=ctr)
self.string_vars.append(var) ## the variables to be updated
## initial variable value
self.values.append("***") ## the values
master=Tk()
LabelTest(master)
master.mainloop()