Logistic Regression using Scipy's fmin_cg - python-2.7

Question

I am trying to implement a logistic classifier using python. The goal is to train the algo to identify digits 0-9 using the mnist handwritten digits data set. However, fmin_cg seems to be changing the dimensions of my input arguments. I have tried reshaping the arguements inside cost() and gradient() with no luck; just more errors.
from scipy.io import loadmat
from numpy import shape, zeros, ones, dot, hstack, vstack, log, transpose, kron
from scipy.special import expit as sigmoid
import scipy.optimize
def cost(theta, X, y):
h = sigmoid( X.dot(theta) )
pos_class = y.T.dot( log(h) )
neg_class = (1.0-y).T.dot( log(1.0-h) )
cost = ((-1.0/m)*(pos_class+neg_class))
return cost
def gradient(theta, X, y):
h = sigmoid( X.dot(theta) )
grad = (1.0/m)*(X.T.dot((h-y)))
return grad
def one_vs_all(X, y, theta):
# add x1 feature,x1 = 1, to each example set
X = hstack( (ones((m,1)),X) )
# train the classifier for digit 9.0
temp_y = (y == 9.0)+0
result = scipy.optimize.fmin_cg( cost, fprime=gradient, x0=theta, \
args=(X, temp_y), maxiter=50, disp=False, full_output=True )
print result[1]
# Load data from Matlab file
data = loadmat('data.mat')
X,y = data['X'],data['y']
m,n = shape(X)
theta = zeros((n+1, 1))
one_vs_all(X, y, theta)
The error I receive:
Traceback (most recent call last):
File "/Users/jkarimi91/Documents/Digit Recognizer/Digit_Recognizer.py", line 36, in <module>
one_vs_all(X, y, theta)
File "/Users/jkarimi91/Documents/Digit Recognizer/Digit_Recognizer.py", line 26, in one_vs_all
args=(X, temp_y), maxiter=50, disp=False, full_output=True )
File "/anaconda/lib/python2.7/site-packages/scipy/optimize/optimize.py", line 1092, in fmin_cg
res = _minimize_cg(f, x0, args, fprime, callback=callback, **opts)
File "/anaconda/lib/python2.7/site-packages/scipy/optimize/optimize.py", line 1156, in _minimize_cg
deltak = numpy.dot(gfk, gfk)
ValueError: shapes (401,5000) and (401,5000) not aligned: 5000 (dim 1) != 401 (dim 0)
[Finished in 1.0s with exit code 1]

Answer 1

With the current code, the cost & gradient function are each returning a 2-D array. For fmin_cg to function properly, these functions must each return a 1-D array (as noted by the documentation).

Answer 2

I know this might be a little late but this is supposed to work
.In your gradient function I got several memory errors,So I changed the code a little and added regularization, check it out
def gradients (theta,X,y,Lambda):
m,n = shape(X)
theta = reshape(theta,(n,1))
h = sigmoid(X.dot(theta))
h = h-y
theta[0,0] = 0
grad = ((X.T.dot(h)) / m) + (Lambda / m * theta)
return grad.ravel()