I have trained a basic back-propagation neural network in R using a training data set and validated using a test set. The neural net is giving me satisfactory results.
Now what I want to do is to use this trained neural network (and the determined weights ) in a C++ code, so that I can put the input variables in the C++ code and it gives me the prediction output.
Most of the tutorial I find regarding the implementation of Neural Network in C++ is about training a network itself, and not really using for prediction.
How do I do this? I feel since the hard part(training the NN) is already done in R, implementing the NN in C++ shouldn't be very difficult. Maybe I'm missing some key concepts?
When training an ANN:-
1) Present input pattern
2) Forward propagate to get the current ANNs answer
3) Compare the ANNs answer with the correct answer
4) Find the difference (error), and backprop the error to train the network.
When using a trained network, just do 1) & 2) and hope the ANNs answer is correct (!).
I am trying to use lasgne to train a simple neural network, and to use my own C++ code to do inference. I use the weights generated by lasgne, but I am not able to get good results. Is there a way I can print the output of a hidden layer and/or the calculations themselves? I want to see who it works under the hood, so I can implement it the same way in C++.
I can help with Lasagne + Theano in Python, I am not sure from your question whether you fully work in C++ or you only need the results from Python + Lasagne in your C++ code.
Let's consider you have a simple network like this:
l_in = lasagne.layers.InputLayer(...)
l_in_drop = lasagne.layers.DropoutLayer(l_in, ...)
l_hid1 = lasagne.layers.DenseLayer(l_in_drop, ...)
l_out = lasagne.layers.DenseLayer(l_hid1, ...)
You can get the output of each layer by calling the get_output method on a specific layer:
lasagne.layers.get_output(l_in, deterministic=False) # this will just give you the input tensor
lasagne.layers.get_output(l_in_drop, deterministic=True)
lasagne.layers.get_output(l_hid1, deterministic=True)
lasagne.layers.get_output(l_out, deterministic=True)
When you are dealing with dropout and you are not in the training phase, it's important to remember to call get_output method with the deterministic parameter set to True, to avoid non-deterministic behaviours. This applies to all layers that are preceded by one or more dropout layers.
I hope this answers your question.
I currently trying to create a neural network of my own. I have managed to get the network to work to a degree but to the best of my knowledge I seem to be stuck in a local minima when I run the program or atleast output spitted out by the net seems to be changing at a very very slow pace. The pace changes/ or I get better output as I add more nodes in the hidden layer can anyone suggest or have a look at my code to see why it's not working with 2 nodes in hidden layer. And why do I have to add more node in the hidden layer for it to give decent mse.
Thank you in advance.
this is the mse at the end of 100000 iteration with 2 nodes :
0.07402
0.07402
0.07402
so on and on. As you can see mse should be around 0.001 but its quite high.
link to the code!
Alright, I solved it all it need was bias used this link to help myself code. link!
I'm looking for advices, for a personal project.
I'm attempting to create a software for creating customized voice commands. The goal is to allow user/me to record some audio data (2/3 secs) for defining commands/macros. Then, when the user will speak (record the same audio data), the command/macro will be executed.
The software must be able to detect a command in less than 1 second of processing time in a low-cost computer (RaspberryPi, for example).
I already searched in two ways :
- Speech Recognition (CMU-Sphinx, Julius, simon) : There is good open-source solutions, but they often need large database files, and speech recognition is not really what I'm attempting to do. Speech Recognition could consume too much power for a small feature.
- Audio Fingerprinting (Chromaprint -> http://acoustid.org/chromaprint) : It seems to be almost what I'm looking for. The principle is to create fingerprint from raw audio data, then compare fingerprints to determine if they can be identical. However, this kind of software/library seems to be designed for song identification (like famous softwares on smartphones) : I'm trying to configure a good "comparator", but I think I'm going in a bad way.
Do you know some dedicated software or parcel of code doing something similar ?
Any suggestion would be appreciated.
I had a more or less similar project in which I intended to send voice commands to a robot. A speech recognition software is too complicated for such a task. I used FFT implementation in C++ to extract Fourier components of the sampled voice, and then I created a histogram of major frequencies (frequencies at which the target voice command has the highest amplitudes). I tried two approaches:
Comparing the similarities between histogram of the given voice command with those saved in the memory to identify the most probable command.
Using Support Vector Machine (SVM) to train a classifier to distinguish voice commands. I used LibSVM and the results are considerably better than the first approach. However, one problem with SVM method is that you need a rather large data set for training. Another problem is that, when an unknown voice is given, the classifier will output a command anyway (which is obviously a wrong command detection). This can be avoided by the first approach where I had a threshold for similarity measure.
I hope this helps you to implement your own voice activated software.
Song fingerprint is not a good idea for that task because command timings can vary and fingerprint expects exact time match. However its very easy to implement matching with DTW algorithm for time series and features extracted with CMUSphinx library Sphinxbase. See Wikipedia entry about DTW for details.
http://en.wikipedia.org/wiki/Dynamic_time_warping
http://cmusphinx.sourceforge.net/wiki/download
I’m teaching an entry-level C++ programming class. We only use iostream in the class (No GUI). It seems like students are not so excited to printout strings and numbers to their console. (Most students even never used the console before.) It is hard to motivate or convey the excitement of programming by showing strings in their console.
What would be a good and exciting console program that can be written by C++ beginners? I’m looking for something doable with basic C++ skill + a bit challenging + very exciting, which can motivate students to learn programming languages.
Any comments will be appreciated.
When I taught an undergrad intro course, we did the Game of Fifteen in straight C as the third homework project. It's pretty well scoped, and it's a game, so there's some inherent motivation there.
Back when I taught, I made an early project be an ATM machine.
Text-only interface, with basic operations like withdraw, deposit, query balance, transfer between accounts, etc.
It was something that everyone was already familiar with, it didn't take huge amounts of programming time, but it did help students feel like it was a practical and realistic program.
Other similar ideas would be a cash-register (handle refunds, coupons, items priced by the pound, sales-tax, store specials, etc, etc), or a cell-phone billing program (separate daytime, night, and weekend minutes, bill text messages, picture messages separately, etc).
How about a system that generates a set of poker hands from a deck? While clearly defined, the intricacies of ensuring no duplicate cards etc, make it a good entry level challenge.
As an extension, you could have the system take an input as to whether you want to bet or fold, and effectively play a poker game.
Finally, a good design would allow them to switch the console for a gui front-end later on (e.g. intermediate class).
I always enjoyed problems where there's a real world purpose for doing it. Something like calculating a mathematical equation, or a range of prime numbers. A lot of stuff on ProjectEuler would be good, I would think. Not everybody likes math (but then again, it's kind of a necessary thing for computer science!).
Instead of just printing to the screen you could make ascii animations.
Introduce your students to pipes and filters. Create a useful utility that takes data from stdin and directs its output to stdout. Create another utility that does something else using that same protocol. Create a third utility. Demonstrate how robustly the utilities can work together.
For example, create a clone of the GNU head and tee utilities, and perhaps add a new utility called cap which capitalizes letters. Then demonstrate how you can get the first 3 lines of a text file capitalized and tee'd to a file and stdout. Next, demonstrate how you use the same utilities, without changing a single line of code, to take the first 5 lines of a file and output to the screen the capitalized letters and to a file the original letters.
When I took C++ we had to replicate a Theseus and the Minotaur game. It lends well to outputting multiple lines to the console to form something "graphical" and it's easily based on a set of implemented rules.
I've had to program a Tower of Hanoi game in the console before, I found that quite fun. It requires use of basic data structures, user input, checking for game end conditions so it would be good for a beginner I believe.
New programming students usually find graphical programs to be the most exciting.
It doesn't have to be anything really advanced, just being able to manipulate pixels and stuff should be enough to keep them interested. Making a simple graphic class around SDL should be ok. Maybe something like this:
int main()
{
GraphicWindow graphic;
graphic.setPixel(10,20,GraphicWindow::Red);
graphic.idle();
}
Then you give out assignments like "implement a drawRect function" etc.
Perhaps a text-only version of the Lunar Lander game. You could do full ASCII art and animations (with ncurses, perhaps) as an advanced exercise, but even in a pure text form it can be interesting.
I recall playing the version that ran on the HP 67 calculator, and it was entertaining with only a seven segment display to work with.
I vaguely recall a version that probably ran on an ALTAIR 8800 written in MITS/Microsoft BASIC that used the leading part of the line to show height above ground as ASCII art, with your prompt for the next tick's burn at the right.
Another traditional choice would be to implement Hunt the Wumpus, or for the ambitious, Battleship.
One of my first programming class had a long homework about implementing a (reduced) Monopoly game.
You can use chained lists for the board.
You can use Inheritance for board tiles.
You need some logic to process players turns.
It was probably the first project I've done in CS that I could talk about to my non-tech friends and generate some interest.