Fully Convolutional Net Training in C++ - c++

Question

I'm trying to transfer FCN training from BrainScript to a C++ program. To start with I'm just loading and re-training an existing model. I'm getting somewhere, but trainer->TrainMinibatch() is throwing an exception (and I can't work out how to get a description of the exception either). Rough code below:
CNTK::DeviceDescriptor& device= CNTK::DeviceDescriptor::GPUDevice(gpuid);
FunctionPtr rootFunc = nullptr;
try {
rootFunc = Function::Load(modelname, device);
}
catch (char *err) {
printf("Load fail: %s\n",err);
return;
}
catch (...) {
printf("Load fail\n");
return;
}
std::cerr << "Loaded model ok" << std::endl;
MinibatchSourcePtr minibatchSource;
try {
minibatchSource = HG_CreateMinibatchSource(64);
}
catch (char* err) {
std::cerr << "Failed to init src: " << err << std::endl;
return;
}
catch (...) {
std::cerr << "Failed to init src " << std::endl;
return;
}
auto imageStreamInfo = minibatchSource->StreamInfo(L"features");
auto labelStreamInfo = minibatchSource->StreamInfo(L"labels"); // We don't use labels as is FCN
auto inputImageShape = imageStreamInfo.m_sampleLayout;
std::cerr << "Input Shape: " << inputImageShape.AsString() << std::endl;
auto imageInputName = L"features";
auto imageInput = InputVariable(inputImageShape, imageStreamInfo.m_elementType, imageInputName);
auto classifierOutput = rootFunc;
//EITHER - construct error from output+target
std::wstring outputLayerName = L"op";
FunctionPtr outputLayer = rootFunc->FindByName(outputLayerName);
std::wstring targetLayerName = L"opool3";
FunctionPtr targetLayer = rootFunc->FindByName(targetLayerName);
// OR - just get from network
std::wstring errLayerName = L"e";
FunctionPtr errLayer = rootFunc->FindByName(errLayerName);
std::cerr << "Setup-got op layer" << outputLayer->Output().Shape().AsString() << std::endl;
std::cerr << "Setup-got tgt layer" << targetLayer->Output().Shape().AsString() << std::endl;
std::cerr << "Setup-got err layer" << errLayer->Output().Shape().AsString() << std::endl;
auto trainingLoss = CNTK::SquaredError(outputLayer, targetLayer);
auto prediction = CNTK::SquaredError(outputLayer, targetLayer);
LearningRateSchedule learningRatePerSample = TrainingParameterPerSampleSchedule(5e-8);
// Either
auto trainer = CreateTrainer(classifierOutput, trainingLoss->Output(), prediction->Output(), { SGDLearner(classifierOutput->Parameters(), learningRatePerSample) });
// Or
//auto trainer = CreateTrainer(classifierOutput, errLayer, errLayer, { SGDLearner(classifierOutput->Parameters(), learningRatePerSample) });
const size_t minibatchSize = 1;
size_t numMinibatchesToTrain = 100;
size_t outputFrequencyInMinibatches = 10;
try {
for (size_t i = 0; i < numMinibatchesToTrain; ++i)
{
std::cerr << "Iteration: " << i << std::endl;
auto minibatchData = minibatchSource->GetNextMinibatch(minibatchSize, device);
std::cerr << " got data for "<< imageInput.AsString() << std::endl;
trainer->TrainMinibatch({ { imageInput, minibatchData[imageStreamInfo] } }, device); // This line throws exception!
std::cerr << "Eval=" << trainer->PreviousMinibatchEvaluationAverage() << "," << trainer->PreviousMinibatchLossAverage() << std::endl;
}
}
// Question edited as result of comment on exceptions below
catch (const std::exception & err) {
std::cerr << "Training error:" << err.what() << std::endl;
}
catch (...) {
std::cerr << "Training error" << std::endl;
}
It's not clear exactly how to define the loss function (I'm guessing here - there's no documentation really). The network has a loss ('e') used by CNTK.exe/Brainscript, which is the Squared error between output ('op') and a target ('opool3'). I tried both using e directly, and defining the error in C++ using CNTK::SquaredError(). Both give the same output, indicating an exception thrown by trainer->TrainMinibatch:
Loaded model ok
Input Shape:B[1024 x 1024 x 3]
Setup-got op layeB[63 x 127 x 3]
Setup-got tgt layeB[63 x 127 x 3]
Setup-got err layeB[]
Iteration: 0
got data forB,Input('features', [1024 x 1024 x 3], [*, #])
Training error:Values for 1 required arguments 'Input('features', [1024 x 1024 x 3], [, #])', that the requested output(s) 'Output('aggregateLoss', [], []), Output('Block233_Output_0', [], [, #]), Output('aggregateEvalMetric', [], [])' depend on, have not been provided.
What am I doing wrong here?
Thanks!
D.
Edit: Exception is:
Training error:Values for 1 required arguments 'Input('features', [1024 x 1024 x 3], [, #])', that the requested output(s) 'Output('aggregateLoss', [], []), Output('Block233_Output_0', [], [, #]), Output('aggregateEvalMetric', [], [])' depend on, have not been provided.
Update: Having looked at the cntk code (CompositeFunction.cpp), the issue seems to be a mismatch between the input and the required input:
Supplied variable: Input('features', [1024 x 1024 x 3], [*, #])
Required argument: Input('features', [1024 x 1024 x 3], [, #])
The difference being the [*. #] vs [, #]
Not sure how to fix it though!

Answer 1

This issue is because imageInput is a new variable that has nothing to do with the network's arguments. Instead you need to get the input variable(s) associated with the network's arguments and bind those to minibatchData e.g. something like
std::unordered_map<Variable, ValuePtr> inputDataMap = { { classifierOutput.Arguments()[0], minibatchData[imageStreamInfo] } }
Then pass inputDataMap to TrainMinibatch. See also this evaluation example (training and evaluation have pretty similar APIs)