Looping in Mata with OLS - stata

I need help with looping in Mata. I have to write a code for Beta coefficients for OLS in Mata using a loop. I am not sure how to call for the variables and create the code. Here is what I have so far.
foreach j of local X {
if { //for X'X
matrix XX = [mata:XX = cross(X,1 , X,1)]
XX
}
else {
mata:Xy = cross(X,1 , y,0)
Xy
}
I am getting an error message "invalid syntax".

I'm not sure what you need the loop for. Perhaps you can provide more information about that. However the following example may help you implement OLS in mata.
Load example data from bcuse:
ssc install bcuse
clear
bcuse bwght
mata
x = st_data(., ("male", "parity","lfaminc","packs"))
cons = J(rows(x), 1, 1)
X = (x, cons)
y = st_data(., ("lbwght"))
beta_hat = (invsym(X'*X))*(X'*y)
e_hat = y - X * beta_hat
s2 = (1 / (rows(X) - cols(X))) * (e_hat' * e_hat)
B = J(cols(X), cols(X), 0)
n = rows(X)
for (i=1; i<=n; i++) {
B =B+(e_hat[i,1]*X[i,.])'*(e_hat[i,1]*X[i,.])
}
V_robust = (n/(n-cols(X)))*invsym(X'*X)*B*invsym(X'*X)
se_robust = sqrt(diagonal(V_robust))
V_ols = s2 * invsym(X'*X)
se_ols = sqrt(diagonal(V_ols))
beta_hat
se_robust
end
This is far from the only way to implement OLS using mata. See the Stata Blog for another example using quadcross, I like my example because it preserves a little more of the matrix algebra in the code.

Related

How to manually initialize the values for the weights?

I would like to experiment the weights initialization recommended by Karpathy in his lecture notes,
the recommended heuristic is to initialize each neuron's weight vector
as: w = np.random.randn(n) / sqrt(n), where n is the number of its
inputs
source: http://cs231n.github.io/neural-networks-2/#init
I'm beginner in python, and I don"t know how to implement this :/
weights = tf.Variable(??)
Please help? ...
For a single value, use:
weights = tf.Variable(10)
For a vector with random values:
shape = [784, 625]
weights = tf.Variable(tf.random_normal(shape, stddev=0.01)/tf.sqrt(n))
Please note that you need to sess.run to evaluate the variables.
Also, please check out other Random Tensors: https://www.tensorflow.org/versions/r0.8/api_docs/python/constant_op.html#random-tensors
n = 10
init_x = np.random.randn(n)
x = tf.Variable(init_x)
sess = tf.InteractiveSession()
sess.run(tf.initialize_all_variables())
print(sess.run(x))
I do it in the following way:
self.w_full, self.b_full = [], []
n_fc_layers = len(structure)
structure.insert(0, self.n_inputs)
with vs.variable_scope(self.scope):
for lr_idx in range(n_fc_layers):
n_in, n_out = structure[lr_idx], structure[lr_idx+1]
self.w_full.append(
vs.get_variable(
"FullWeights{}".format(lr_idx),
[n_in, n_out],
dtype=tf.float32,
initializer=tf.random_uniform_initializer(
minval=-tf.sqrt(tf.constant(6.0)/(n_in + n_out)),
maxval=tf.sqrt(tf.constant(6.0)/(n_in + n_out))
)
)
)
self.b_full.append(
vs.get_variable(
"FullBiases{}".format(lr_idx),
[n_out],
dtype=tf.float32,
initializer=tf.constant_initializer(0.0)
)
)
after
structure.insert(0, self.n_inputs)
you'll have [n_inputs, 1st FC layer size, 2nd FC layer size ... output layer size]

Why RcppArmadillo's fastLmPure produces NA's in output but fastLm doesn't?

I use rolling regression in R quite a lot and my initial setup is something like:
dolm <- function(x) coef(lm(x[,1] ~ x[,2] + 0, data = as.data.frame(x)))
rollingCoef = rollapply(someData, 100, dolm)
Above example works perfectly, except it is slow if you have a lot of iterations.
To speed it up I've decided to experiment with Rcpp package.
First I substituted lm with fastLm, result is a bit faster but still slow. So that pushed me to attempt to write the entire rolling regression's coefficients function in c++ as for loop and than integrate it in R with Rcpp help.
So I've changed original RcppArmadillo's function fastLm to this:
// [[Rcpp::depends(RcppArmadillo)]]
#include <RcppArmadillo.h>
using namespace Rcpp;
// [[Rcpp::export]]
List rollCoef(const arma::mat& X, const arma::colvec& y, double window ) {
double cppWindow = window - 1;
double matRows = X.n_rows;
double matCols = X.n_cols - 1;
arma::mat coef( matRows - cppWindow, X.n_cols); // matrix for estimated coefficients
//for loop for rolling regression.
for( double i = 0 ; i < matRows - cppWindow ; i++ )
{
coef.row(i) = arma::trans(arma::solve(X( arma::span(i,i + cppWindow), arma::span(0,matCols)) , y.rows(i,i + cppWindow)));
}
return List::create(_["coefficients"] = coef);
}
and than download it to R with sourceCpp(file=".../rollCoef.cpp")
So it's much faster than rollapply and it worked fine on small examples, but than I applied it to ~200000 observations of data it produced ~half of NA's in output, in the same time rollapply/fastLm combination didn't produce any.
So here I need some help. What is wrong with my function? Why are there NA's in my function output, and no NA's in rollapply/fastLm, however, if I understand right, them both based on arma::solve? Any help is highly appreciated.
UPDATE
Here is reproducible code:
require(Rcpp)
require(RcppArmadillo)
require(zoo)
require(repmis)
myData <- source_DropboxData(file = "example.csv",
key = "cbrmkkbssu5bn96", sep = ",", header = TRUE)
## in order to use my custom function "rollCoef" you should download it to R.
## The c++ code is presented above in the main question.
## Download it where you want as "rollCoef.cpp" and then download it to R with:
sourceCpp(file=".../rollCoeff.cpp"). # there should be your actual path.
myCoef = rollCoef(as.matrix(myData[,2]),myData[,1],260)
summary(unlist(myCoef)) # 80923 NA's
dolm = function(x) coef(fastLmPure(as.matrix(x[,2]), x[,1]))
myCoef2 = rollapply(myData, 260, dolm, by.column = FALSE)
summary(myCoef2) # 80923 NA's
dolm2 = function(x) coef(fastLm(x[,1] ~ x[,2] + 0, data = as.data.frame(x)))
myCoef3 = rollapply(myData, 260, dolm2, by.column = FALSE)
summary(myCoef3) # !!! No NA's !!!
head(unlist(myCoef)) ; head(unlist(myCoef2)) ; head(myCoef3)
So the output of my function is identical to output of RcppArmadillo's fastLmPure combined with rollapply and them both produce NA's, but rollapply with fastLm does not. As I understand, for example from HERE and HERE fastLm is basically calling to fastLmPure, but why is there no NA's in the third method than? Is there some additional capabilities in fastLm that prevent NA's that I didn't spotted?
There is an entire package RcppRoll to do just that custom rolling -- and you should be able to extend it and its rollit() function to do rolling lm() as well.

Optimize in Stata / Mata

I want write a loop in Stata with a Mata command 'optimize'. The basic syntax is (in .do file):
mata: x=runiform(100,2)
mata: F=J(rows(x),1,3)
mata: X=J(1,2,48)
mata: I=J(rows(x),1,1)
mata:
void mysolver(todo, p, x, X, I, F, lnf, S, H)
{
factor = F :* (I + x*p')
factor_bis= factor , factor
Cuenta = x :* factor_bis
Final=I'*Cuenta
vvv = Final - X
lnf = (vvv*vvv')[1,1]
}
mata:
S = optimize_init()
optimize_init_evaluator(S, &mysolver())
optimize_init_evaluatortype(S, "v0")
optimize_init_params(S, J(1,2,0.01))
optimize_init_which(S, "min" )
optimize_init_argument(S, 1, x)
optimize_init_argument(S, 2, X)
optimize_init_argument(S, 3, I)
optimize_init_argument(S, 4, F)
optimize_init_tracelevel(S,"none")
optimize_init_conv_ptol(S, 1e-16)
optimize_init_conv_vtol(S, 1e-16)
xx=optimize(S)
st_matrix("param_estim",xx)
end
How write a procedure 'optimizo' to be include in a loop:
forvalues i=(1)500 {
.....
optimizo
}
to repeat 500 times the optimization? (in my application, the matrices change in each cycle)
Thank You.
In your main code, instead of the line
xx=optimize(S)
Simply write this:
for (i = 1; i <= 500; i++) {
[Do your matrix changes, call your optimize_init_argument commands if you need to change them]
xx = optimize(S)
xx
}

removing from the list Prolog

Hi everyone I'm newbie in prolog and I have such a list: (actually It is output of my predicate not a list )
P = [1/1, 1/3] ;
P = [1/1, 2/3] ;
P = [1/3, 1/1] ;
P = [1/3, 2/1] ;
P = [2/1, 1/3] ;
P = [2/1, 2/3] ;
P = [2/3, 1/1] ;
P = [2/3, 2/1] ;
and I need to remove dublicete terms.For example [1/1,2/3] and [2/3,1/1]is same and I should remove one of them , which one is not important ,How could I do that in prolog ?? Thanks in advance
NOTE I LEARNT THAT findALL should be good way for this but still dont know the answer please help me .
Unless you actually show us your code, it's never going to be possible to give you precise answers.
I assume you have a predicate f/1 such that:
?- f(P).
produces the interactive result you show above. A simple solution is to change your query:
?- f([X,Y]), X < Y.
This will produce the following result:
X = 1/3, Y = 1/1 ;
X = 1/3, Y = 2/1 ;
X = 2/3, Y = 1/1 ;
X = 2/3, Y = 2/1 ;
findall/3 isn't sufficient to solve this particular situation, because you've defined uniqueness in a way that ignores the position in the list. In Prolog (and everything else) [X,Y] and [Y,X] are not equal, so you'd have to find a trick to get this to give you "unique" results.

implementing Bags of Words object recognition using VLFEAT

I am trying to implement a BOW object recognition code in matlab. The process is slightly complicated and I've had a lot of trouble finding proper documentation on the procedure. So could someone double check if my plan below makes sense?
I'm using the VLSIFT library extensively here
Training:
1. Extract SIFT image descriptor with VLSIFT
2. Quantize the descriptors with k-means(vl_hikmeans)
3. Take quantized descriptors and create histogram(VL_HIKMEANSHIST)
4. Create SVM from histograms(VL_PEGASOS?)
I understand step 1-3, but I'm not quite sure if the function for SVM is correct.
VL_PEGASOS takes the following:
W = VL_PEGASOS(X, Y, LAMBDA)
How exactly do I use this function with the histogram that I create?
Finally during the recognition stage, how do I match the image with a class defined by the SVM?
Did you look at their Caltech 101 example code, that is full implementation of an BoW approach.
Here is the part where they classify with pegasos and evaluate the results:
% --------------------------------------------------------------------
% Train SVM
% --------------------------------------------------------------------
lambda = 1 / (conf.svm.C * length(selTrain)) ;
w = [] ;
for ci = 1:length(classes)
perm = randperm(length(selTrain)) ;
fprintf('Training model for class %s\n', classes{ci}) ;
y = 2 * (imageClass(selTrain) == ci) - 1 ;
data = vl_maketrainingset(psix(:,selTrain(perm)), int8(y(perm))) ;
[w(:,ci) b(ci)] = vl_svmpegasos(data, lambda, ...
'MaxIterations', 50/lambda, ...
'BiasMultiplier', conf.svm.biasMultiplier) ;
model.b = conf.svm.biasMultiplier * b ;
model.w = w ;
% --------------------------------------------------------------------
% Test SVM and evaluate
% --------------------------------------------------------------------
% Estimate the class of the test images
scores = model.w' * psix + model.b' * ones(1,size(psix,2)) ;
[drop, imageEstClass] = max(scores, [], 1) ;
% Compute the confusion matrix
idx = sub2ind([length(classes), length(classes)], ...
imageClass(selTest), imageEstClass(selTest)) ;
confus = zeros(length(classes)) ;
confus = vl_binsum(confus, ones(size(idx)), idx) ;