How is the size of each k folds defined? - python-2.7

Question

I am currently training my regression network using crossvalidation, I don't have any labels, but specific input that should be mapped to an specific output, the network should then generate the mapping.I seem to have some problems with how the folds are being defined.
the way i do crossvalidation is like this:
############################### Training setup ##################################
#Define 10 folds:
seed = 7
np.random.seed(seed)
kfold = StratifiedKFold(n_splits=10, shuffle=True, random_state=seed)
print "Splits"
cvscores_loss = []
for train, test in kfold.split(train_set_data_vstacked_normalized,train_set_output_vstacked):
print "Model definition!"
model = Sequential()
#act = PReLU(init='normal', weights=None)
model.add(Dense(output_dim=400,input_dim=400, init="normal",activation=K.tanh))
#act1 = PReLU(init='normal', weights=None)
model.add(Dense(output_dim=400,input_dim=400, init="normal",activation=K.tanh))
#act2 = PReLU(init='normal', weights=None)
model.add(Dense(output_dim=400, input_dim=400, init="normal",activation=K.tanh))
act4=ELU(10000)
model.add(Dense(output_dim=13, input_dim=300, init="normal",activation=act4))
print "Compiling"
model.compile(loss='mean_squared_error', optimizer='RMSprop', metrics=["accuracy"])
print "Compile done! "
print '\n'
print "Train start"
model.fit(train_set_data_vstacked_normalized[train],train_set_output_vstacked[train], nb_epoch=10, verbose=1)
loss, accuracy = model.evaluate(x=train_set_data_vstacked_normalized[test],y=train_set_output_vstacked[test],verbose=1)
print
print('loss: ', loss)
print('accuracy: ', accuracy)
print()
print model.summary()
print "New Model:"
cvscores_loss.append(loss)
print("%.2f%% (+/- %.2f%%)" % (numpy.mean(cvscores_loss), numpy.std(cvscores_loss)))
Problem with this code is that I never enter the for loop.. receive a warning message after "Splits" is printed... It being .
Splits
/home/k/.local/lib/python2.7/site-packages/sklearn/model_selection/_split.py:579: Warning: The least populated class in y has only 1 members, which is too few. The minimum number of groups for any class cannot be less than n_splits=10.
Which make it question how kfold, knows what the input and output dimensions is of my neural network?...
Should i define it somewhere? or how?..

Answer 1

The message tells you the problem. One of your target classes has only 1 member. When it stratifies into 10 folds it needs at least 10 members of each class so that it can put 1 in each fold.
You need to check the counts of the target classes to find the offending class and remove it.

Answer 2

I think you over complicates this. If you need to do cross-validation on Keras model, you can use keras scikit-learn API. To do this you need to:
Import some stuff:
from keras.wrappers.scikit_learn import KerasClassifier
from sklearn.model_selection import StratifiedKFold
from sklearn.model_selection import cross_val_score
create a function that defines your model:
def model_creation():
model = Sequential()
model.add(...)
...
model.compile(...)
return model
and use the wrapper:
model = KerasClassifier(build_fn=model_creation, nb_epoch=100, batch_size=100, verbose=0)
kfold = StratifiedKFold(n_splits=10, shuffle=True, random_state=42)
results = cross_val_score(model, X, y, cv=kfold)