I am currently developing an application for facial recognition.
The algorithms are implemented and trained using the MatConvnet library (http://www.vlfeat.org/matconvnet/). At the end, I have a Network (.mat file) which looks like that:
I would like to know if it were possible to extract the weights of the Network using its .mat file, write them in a XML file and read them with Caffe C++. I would like to reuse them in Caffe C++ in order to do some testing and hardware implementation. Is there an efficient and practical way to proceed so ?
Thank you for very much for your help.
The layer whose parameters you'd like to store, must be set as 'precious'. In net.var you can access the parameters and write them.
There is a conversion script that converts matconvnet models to caffe models here which you may find useful.
You can't use weights of the trained Network by matconvnet for caffe. You can merely import your model from matconvnet to caffe.(https://github.com/vlfeat/matconvnet/blob/4ce2871ec55f0d7deed1683eb5bd77a8a19a50cd/utils/import-caffe.py). But this script does not support all layers and you may have difficulties in employing it.
The best way is to define your caffe prototxt in python as the matconvnet model.
Related
Given a Pyomo model and corresponding data command file input.dat, I would like to verify the model I built with a 3rd party tool. To be more specific, it is a network flow model with the topology defined in the data command file. The strategy would then be to load the topology from that input.dat and use it as an input to the 3rd party tool.
The actual question: Is there a way to load the topology defined in the Pyomo input.dat directly into Python data structures (e.g. sets, dicts, etc.) instead of round tripping through Pyomo data structures (pyomo.environ.Set et al.) and then building the Python data structures from the Pyomo data structures - for the sake of convenience? Is there maybe an undocumented or unofficial internal API function that does this?
You might be able to do this using the DataPortal infrastructure in Pyomo. See the documentation here: https://pyomo.readthedocs.io/en/latest/working_abstractmodels/data/dataportals.html
I open this thread to discuss how to bring my NN model to deployment.
I build and trained a NN in Matlab with mdCNN, (mdCNN is a simple Matlab library for building NN for multiple dimension input, which is currently is not supported with Matlab - cov3x3x3). I trained my model in Matlab, Now I want to bring it to production.
After few hours of research, I plan to do the following
Train a NN model in Keras with TF backend. I choose Keras because I want to have backward compatibility with Matlab in the future.
Grab a tensorflow session from Keras model, there is an example how to do that here. Than Save the session in *.pd file
Load the NN model from openCV dnn model. there is a specific function that does that
cv::readNet()
Run the NN in C++ using OpenCV with
net.setInput(blob);
Mat prob = net.forward();
I want to check with you if this flow would really work. Are there any suggestions how to do the deployment better? Any suggestions or improvements for the flow ?
Maybe have a look at this question: Convert Keras model to C++
The general idea is to save the model in json and the weights in hdf5 and use this keras2cpp solution to convert it to C++.
When applying TF model with its C++ interface, freeze_graph operations, for the graph file and the cpk file, are required. This operation will then generate a dumped binary protobuffer file.
However, protobuffer does not support to dump a file that is larger than 2GiB.
In this case, what's the right way to load a big model in TF, with its C++ interface? Any clues will be appriciated.
You can still load a model if you don't freeze the graph and use savedmodel.
I've read caffe2 tutorials and tried pre-trained models. I knew caffe2 will leverge GPU to run the model/net. But the input data seems always be given from CPU(ie. Host) memory. For example, in Loading Pre-Trained Models, after model is loaded, we can predict an image by
result = p.run([img])
However, image "img" should be read in CPU scope. What I look for is a framework that can pipline the images (which is decoded from a video and still resides in GPU memory) directly to the prediction model, instead of copying it from GPU to CPU scope, and then transfering to GPU again to predict result. Is Caffe or Caffe2 provides such functions or interfaces for python or C++? Or should I need to patch Caffe to do so? Thanks at all.
Here is my solution:
I'd found in tensor.h, function ShareExternalPointer() can exactly do what I want.
Feed gpu data this way,
pInputTensor->ShareExternalPointer(pGpuInput, InputSize);
then run the predict net through
pPredictNet->Run();
where pInputTensor is the entrance tensor for the predict net pPredictNet
I don't think you can do it in caffe with python interface.
But I think that it can be accomplished using the c++: In c++ you have access to the Blob's mutable_gpu_data(). You may write code that run on device and "fill" the input Blob's mutable_gpu_data() directly from gpu. Once you made this update, caffe should be able to continue its net->forward() from there.
UPDATE
On Sep 19th, 2017 PR #5904 was merged into master. This PR exposes GPU pointers of blobs via the python interface.
You may access blob._gpu_data_ptr and blob._gpu_diff_ptr directly from python at your own risk.
As you've noted, using a Python layer forces data in and out of the GPU, and this can cause a huge hit to performance. This is true not just for Caffe, but for other frameworks too. To elaborate on Shai's answer, you could look at this step-by-step tutorial on adding C++ layers to Caffe. The example given should touch on most issues dealing with layer implementation. Disclosure: I am the author.
I used the matlab neural network to train on some data but I want to run this neural network in c++ program,how to do that?
You can use ML to generate your feature set (input layer) and then use an open source C++ NN implementation to do training/classification. (E.g., http://takinginitiative.net/2008/04/23/basic-neural-network-tutorial-c-implementation-and-source-code/) If you want to use ML to train and C++ to classify it shouldn't be too difficult to write some additional code to write out the trained network in a way that can be read in by the C++ classifier.
You can use the Matlab Compiler that generates code you can embed in your C++ application
I'm using Matlab R2013a. If you are still facing with this issue, try to look at this location
Matlab\R2013a\toolbox\nnet\nnet\nnderivative\+nnMex2
inside Matlab directory. I found there file "yy.cpp" which contains mexFunction which does the thing. Very likely that Matlab calls this function to simulate network.
Seems that it's possible to integrate this function into your project after slight refactoring. At least I plan to do so. :-)