How to manually initialize the values for the weights? - python-2.7

I would like to experiment the weights initialization recommended by Karpathy in his lecture notes,
the recommended heuristic is to initialize each neuron's weight vector
as: w = np.random.randn(n) / sqrt(n), where n is the number of its
inputs
source: http://cs231n.github.io/neural-networks-2/#init
I'm beginner in python, and I don"t know how to implement this :/
weights = tf.Variable(??)
Please help? ...

For a single value, use:
weights = tf.Variable(10)
For a vector with random values:
shape = [784, 625]
weights = tf.Variable(tf.random_normal(shape, stddev=0.01)/tf.sqrt(n))
Please note that you need to sess.run to evaluate the variables.
Also, please check out other Random Tensors: https://www.tensorflow.org/versions/r0.8/api_docs/python/constant_op.html#random-tensors

n = 10
init_x = np.random.randn(n)
x = tf.Variable(init_x)
sess = tf.InteractiveSession()
sess.run(tf.initialize_all_variables())
print(sess.run(x))

I do it in the following way:
self.w_full, self.b_full = [], []
n_fc_layers = len(structure)
structure.insert(0, self.n_inputs)
with vs.variable_scope(self.scope):
for lr_idx in range(n_fc_layers):
n_in, n_out = structure[lr_idx], structure[lr_idx+1]
self.w_full.append(
vs.get_variable(
"FullWeights{}".format(lr_idx),
[n_in, n_out],
dtype=tf.float32,
initializer=tf.random_uniform_initializer(
minval=-tf.sqrt(tf.constant(6.0)/(n_in + n_out)),
maxval=tf.sqrt(tf.constant(6.0)/(n_in + n_out))
)
)
)
self.b_full.append(
vs.get_variable(
"FullBiases{}".format(lr_idx),
[n_out],
dtype=tf.float32,
initializer=tf.constant_initializer(0.0)
)
)
after
structure.insert(0, self.n_inputs)
you'll have [n_inputs, 1st FC layer size, 2nd FC layer size ... output layer size]

Related

Looping in Mata with OLS

I need help with looping in Mata. I have to write a code for Beta coefficients for OLS in Mata using a loop. I am not sure how to call for the variables and create the code. Here is what I have so far.
foreach j of local X {
if { //for X'X
matrix XX = [mata:XX = cross(X,1 , X,1)]
XX
}
else {
mata:Xy = cross(X,1 , y,0)
Xy
}
I am getting an error message "invalid syntax".
I'm not sure what you need the loop for. Perhaps you can provide more information about that. However the following example may help you implement OLS in mata.
Load example data from bcuse:
ssc install bcuse
clear
bcuse bwght
mata
x = st_data(., ("male", "parity","lfaminc","packs"))
cons = J(rows(x), 1, 1)
X = (x, cons)
y = st_data(., ("lbwght"))
beta_hat = (invsym(X'*X))*(X'*y)
e_hat = y - X * beta_hat
s2 = (1 / (rows(X) - cols(X))) * (e_hat' * e_hat)
B = J(cols(X), cols(X), 0)
n = rows(X)
for (i=1; i<=n; i++) {
B =B+(e_hat[i,1]*X[i,.])'*(e_hat[i,1]*X[i,.])
}
V_robust = (n/(n-cols(X)))*invsym(X'*X)*B*invsym(X'*X)
se_robust = sqrt(diagonal(V_robust))
V_ols = s2 * invsym(X'*X)
se_ols = sqrt(diagonal(V_ols))
beta_hat
se_robust
end
This is far from the only way to implement OLS using mata. See the Stata Blog for another example using quadcross, I like my example because it preserves a little more of the matrix algebra in the code.

nested list of lists of inegers - doing arithmetic operation

I have a list like below and need to firs add items in each list and then multiply all results 2+4 = 6 , 3+ (-2)=1, 2+3+2=7, -7+1=-6 then 6*1*7*(-6) = -252 I know how to do it by accessing indexes and it works (as below) but I also need to do it in a way that it will work no matter how many sublist there is
nested_lst = [[2,4], [3,-2],[2,3,2], [-7,1]]
a= nested_lst[0][0] + nested_lst[0][1]
b= nested_lst[1][0] + nested_lst[1][1]
c= nested_lst[2][0] + nested_lst[2][1] + nested_lst[2][2]
d= nested_lst[3][0] + nested_lst[3][1]
def sum_then_product(list):
multip= a*b*c*d
return multip
print sum_then_product(nested_lst)
I have tried with for loop which gives me addition but I don't know how to perform here multiplication. I am new to it. Please, help
nested_lst = [[2,4], [3,-2],[2,3,2], [-7,1]]
for i in nested_lst:
print sum(i)
Is this what you are looking for?
nested_lst = [[2,4], [3,-2],[2,3,2], [-7,1]] # your list
output = 1 # this will generate your eventual output
for sublist in nested_lst:
sublst_out = 0
for x in sublist:
sublst_out += x # your addition of the sublist elements
output *= sublst_out # multiply the sublist-addition with the other sublists
print(output)

GMPL generic way to produce params

Hello is there a way to generate params in GMPL as example io have a funcion
min:c[i]*x[i] and constrains that looks like A[i][j]*x[i]=b[i]. Where A[i][j]=1/(i+j-1) and i,j=1,2....,n. c[i]=b[i]=sum(j=1,...,n)1/(i+j-1) where i=1,...,n.
So there is a question is there a way to generate matrix A from equation? or do i need to make this matrix manual in data section ? and one more question is there a good way to find (n) maximum size of this problem when with precision of 2 numbers without modifying objective function ?
param n := 3;
set I := 1..n;
param A{i in I, j in I} := 1/(i+j-1);
param c{i in I} := sum{j in I} 1/(i+j-1);
# or better (may be):
# param c{i in I} := sum{j in I} A[i,j];
display A,c;
end;
The output should look like:
Reading model section from x.mod...
9 lines were read
Display statement at line 8
A[1,1] = 1
A[1,2] = 0.5
A[1,3] = 0.333333333333333
A[2,1] = 0.5
A[2,2] = 0.333333333333333
A[2,3] = 0.25
A[3,1] = 0.333333333333333
A[3,2] = 0.25
A[3,3] = 0.2
c[1] = 1.83333333333333
c[2] = 1.08333333333333
c[3] = 0.783333333333333
GMPL is a subset of AMPL. You may want to read the AMPL book to understand more about the syntax.
I am afraid I don't understand your second question.

Search for sequences in multiple vectors

What is the easiest way to find the sequence I need in multiple vectors in R without using loops?
For example, I need to find vectors their "yahoo" comes after "google"(only order matters).
seq = c("google","yahoo")
Matches:
vec1 = c("smth","google","smth","yahoo","smth")
Not matches:
vec2 = c("smth","yahoo","smth","google","smth")
Check this assuming you have unique values for yahoo and google:
library(dplyr)
dt = data.frame(vec1 = c("smth","google","smth","yahoo","smth"))
dt = dt %>% mutate(row = row_number()) # get the row number for each value of vec1
dt$row[dt$vec1=="google"] < dt$row[dt$vec1=="yahoo"] # returns T/F
Modify this if you don't have unique vec1 values. This one uses the max row number:
dt = data.frame(vec1 = c("smth","google","smth","yahoo","smth"))
dt = dt %>% mutate(row = row_number()) %>%
group_by(vec1) %>% summarise(row = max(row)) # get the max row number for each unique value of vec1
dt$row[dt$vec1=="google"] < dt$row[dt$vec1=="yahoo"]
You can use which function to find the positions of your search terms within a given vector
which(vec1=="google")[1] < which(vec1=="yahoo")[1]
use [1] if you're interested only in the first occurrence of each search term.

Matlab Codegen build error

I am trying to convert the below Matlab code into C++ using codegen. However it fails at build and I get the error:
"??? Unless 'rows' is specified, the first input must be a vector. If the vector is variable-size, the either the first dimension or the second must have a fixed length of 1. The input [] is not supported. Use a 1-by-0 or 0-by-1 input (e.g., zeros(1,0) or zeros(0,1)) to represent the empty set."
It then points to [id,m,n] = unique(id); being the culprit. Why doesn't it build and what's the best way to fix it?
function [L,num,sz] = label(I,n) %#codegen
% Check input arguments
error(nargchk(1,2,nargin));
if nargin==1, n=8; end
assert(ndims(I)==2,'The input I must be a 2-D array')
sizI = size(I);
id = reshape(1:prod(sizI),sizI);
sz = ones(sizI);
% Indexes of the adjacent pixels
vec = #(x) x(:);
if n==4 % 4-connected neighborhood
idx1 = [vec(id(:,1:end-1)); vec(id(1:end-1,:))];
idx2 = [vec(id(:,2:end)); vec(id(2:end,:))];
elseif n==8 % 8-connected neighborhood
idx1 = [vec(id(:,1:end-1)); vec(id(1:end-1,:))];
idx2 = [vec(id(:,2:end)); vec(id(2:end,:))];
idx1 = [idx1; vec(id(1:end-1,1:end-1)); vec(id(2:end,1:end-1))];
idx2 = [idx2; vec(id(2:end,2:end)); vec(id(1:end-1,2:end))];
else
error('The second input argument must be either 4 or 8.')
end
% Create the groups and merge them (Union/Find Algorithm)
for k = 1:length(idx1)
root1 = idx1(k);
root2 = idx2(k);
while root1~=id(root1)
id(root1) = id(id(root1));
root1 = id(root1);
end
while root2~=id(root2)
id(root2) = id(id(root2));
root2 = id(root2);
end
if root1==root2, continue, end
% (The two pixels belong to the same group)
N1 = sz(root1); % size of the group belonging to root1
N2 = sz(root2); % size of the group belonging to root2
if I(root1)==I(root2) % then merge the two groups
if N1 < N2
id(root1) = root2;
sz(root2) = N1+N2;
else
id(root2) = root1;
sz(root1) = N1+N2;
end
end
end
while 1
id0 = id;
id = id(id);
if isequal(id0,id), break, end
end
sz = sz(id);
% Label matrix
isNaNI = isnan(I);
id(isNaNI) = NaN;
[id,m,n] = unique(id);
I = 1:length(id);
L = reshape(I(n),sizI);
L(isNaNI) = 0;
if nargout>1, num = nnz(~isnan(id)); end
Just an FYI, if you are using MATLAB R2013b or newer, you can replace error(nargchk(1,2,nargin)) with narginchk(1,2).
As the error message says, for codegen unique requires that the input be a vector unless 'rows' is passed.
If you look at the report (click the "Open report" link that is shown) and hover over id you will likely see that its size is neither 1-by-N nor N-by-1. The requirement for unique can be seen if you search for unique here:
http://www.mathworks.com/help/coder/ug/functions-supported-for-code-generation--alphabetical-list.html
You could do one of a few things:
Make id a vector and treat it as a vector for the computation. Instead of the declaration:
id = reshape(1:prod(sizI),sizI);
you could use:
id = 1:numel(I)
Then id would be a row vector.
You could also keep the code as is and do something like:
[idtemp,m,n] = unique(id(:));
id = reshape(idtemp,size(id));
Obviously, this will cause a copy, idtemp, to be made but it may involve fewer changes to your code.
Remove the anonymous function stored in the variable vec and make vec a subfunction:
function y = vec(x)
coder.inline('always');
y = x(:);
Without the 'rows' option, the input to the unique function is always interpreted as a vector, and the output is always a vector, anyway. So, for example, something like id = unique(id) would have the effect of id = id(:) if all the elements of the matrix id were unique. There is no harm in making the input a vector going in. So change the line
[id,m,n] = unique(id);
to
[id,m,n] = unique(id(:));