I'm attempting to use RBM neural network in sklearn, but I can't find a predict function, I see how you can train it (I think) but I can't seem to figure out how to actually predict a value.
http://scikit-learn.org/stable/auto_examples/neural_networks/plot_rbm_logistic_classification.html#example-neural-networks-plot-rbm-logistic-classification-py
I'm working on a class assignment. this is the assignment:
You will then use randomized hill climbing algorithm to find good weights for a neural network.
Is it possible to do this with SKLearn? Is there a better recommended tool to be able to select different weights for NN? (The goal is to experiment with around 3 different search optimization techniques, and learn about them, not necessarily write them, nor write a NN in this case).
RBM's do not do prediction tasks. They are generative models. You can use the transform method to get a hidden state transformation of the input, or your gan use the gibbs method to sample from the network.
You will then use randomized hill climbing algorithm to find good weights for a neural network.
No, this is not available in scikit-learn.
It sounds like your assignment might be meant for you to implement a simpler problem from scratch rather than use another library, as hill climbing isn't normally used for training a neural network. And they probably don't want you to do hill climbing for an RBM neural network.
You should probably consult your professor for more direction on what you really should be doing.
Related
In Pose Estimation Using Associative Embedding technique I still don't have clarity regarding How we can group the detected points from HeatMaps to Individual Human Poses using Associative Embeddings Layer. Is there any code that clearly gives Idea regarding this ? I'm Using EfficientHRNet approach for Pose Estimation.
Extracted KeyPoints from Heatmaps and need to group those points into individual poses using Embedding Layer Output.
From OpenVINO perspective, we could offer:
This model: human-pose-estimation-0007
This IE demo: Human Pose Estimation Python* Demo
This model utilized the Associative Embedding technique.
However, if you want to build it from scratch, you'll need to design your own Deep Learning architecture, implement and train the neural network.
This research paper might give you some insight into things that you need to decide (eg batch, optimization algorithm, learning rate, etc).
Has anybody tried developing a SLAM system that uses deep learned features instead of the classical AKAZE/ORB/SURF features?
Scanning recent Computer Vision conferences, there seem to be quite a few reports of successful usage of neural nets to extract features and descriptors, and benchmarks indicate that they may be more robust than their classical computer vision equivalent. I suspect that extraction speed is an issue, but assuming one has a decent GPU (e.g. NVidia 1050), is it even feasible to build a real-time SLAM system running say at 30FPS on 640x480 grayscale images with deep-learned features?
This was a bit too long for a comment, so that's why I'm posting it as an answer.
I think it is feasible, but I don't see how this would be useful. Here is why (please correct me if I'm wrong):
In most SLAM pipelines, precision is more important than long-term robustness. You obviously need your feature detections/matchings to be precise to get reliable triangulation/bundle (or whatever equivalent scheme you might use). However, the high level of robustness that neural networks provide is only required with systems that do relocalization/loop closure on long time intervals (e.g. need to do relocalization in different seasons etc). Even in such scenarios, since you already have a GPU, I think it would be better to use a photometric (or even just geometric) model of the scene for localization.
We don't have any reliable noise models for the features that are detected by the neural networks. I know there have been a few interesting works (Gal, Kendall, etc...) for propagating uncertainties in deep networks, but these methods seem a bit immature for deployment ins SLAM systems.
Deep learning methods are usually good for initializing a system, and the solution they provide needs to be refined. Their results depend too much on the training dataset, and tend to be "hit and miss" in practice. So I think that you could trust them to get an initial guess, or some constraints (e.g. like in the case of pose estimation: if you have a geometric algorithm that drifts in time, then you can use the results of a neural network to constrain them. But I think that the absence of a noise model as mentioned previously will make the fusion a bit difficult here...).
So yes, I think that it is feasible and that you can probably, with careful engineering and tuning produce a few interesting demos, but I wouldn't trust it in real life.
I need to perform some inferences on a Bayesian network, such as the example I have created below.
I was looking at doing something like something like this to solve an inference such as P(F| A = True, B = True). My initial approach was to do something like
For every possible output of F
For every state of each observed variable (A,B)
For every unobserved variable (C, D, E, G)
// Calculate Probability
But I don't think this will work because we actually need to go over many variables at once, not each at a time.
I have heard about Pearls algorithm for message passing but am yet to find a reasonable description that isn't extremely dense. For added information, these Bayesian networks are constrained as to not have more than 15-20 nodes, and we have all the conditional probability tables, the code doesn't really have to be fast or efficient.
Basically I am looking for a way to do this, not necessarily the BEST way to do this.
Your Bayesian Network (BN) does not seem to be particularly complex. I think you should easily get away with using exact inference method, such as junction tree algorithm. Of course, you can still just do brute force enumeration, but that would be a waste of CPU resources given that there are so many nice libraries out there that implement smarter ways of doing both exact and approximate inference in graphical models.
Since your tag mentions C++, my recommendation would be libDAI. It is a well written library that implements multiple exact and approximate inference on generic factor graphs. It does not have any weird dependencies and is very easy to integrate into your project. It is particularly well suited for discrete cases, such as yours, for which you have the probability tables.
Now, you noticed that I mentioned factor graphs. If you are not familiar with the concept, I will refer you to Wikipedia article on factor graphs as well as What are "Factor Graphs" and what are they useful for?. The principle is very simple, you represent your BN as a factor graph and then libDAI will do the inference for you.
EDIT:
Since CPU resources do not seem to be a problem for you and simplicity is the key, you can always go with brute force enumeration. The idea is straightforward.
Your Bayesian Network represents a joint probability distribution, which you can write down in terms of an equation, e.g.
P(A,B,C) = P(A|B,C) * P(B|C) * P(C)
Assuming that you have tables for all your conditional probability distributions, i.e. P(A|B, C) P(B|C) and P(C) then you can simply go over all the possible values of variables A, B, and C and calculate the output.
We're working on a machine learning project in which we'd like to see the influence of certain online sample embedding methods on SVMs.
In the process we've tried interfacing with Pegasos and dlib as well as designing (and attempting to write) our own SVM implementation.
dlib seems promising as it allows interfacing with user written kernels.
Yet kernels don't give us the desired "online" behavior (unless that assumption is wrong).
Therefor, if you know about an SVM library which supports online embedding and custom written embedders, it would be of great help.
Just to be clear about "online".
It is crucial that the embedding process will happen online in order to avoid heavy memory usage.
We basically want to do the following within Stochastic subGradient Decent(in very general pseudo code):
w = 0 vector
for t=1:T
i = random integer from [1,n]
embed(sample_xi)
// sample_xi is sent to sub gradient loss i as a parameter
w = w - (alpha/t)*(sub_gradient(loss_i))
end
I think in your case you might want to consider the Budgeted Stochastic Gradient Descent for Large-Scale SVM Training (BSGD) [1] by Wang, Crammer, Vucetic
This is because, as specified in the paper about the "Curse of Kernelization" you might want to explore this option instead what you have indicated in the pseudocode in your question.
The Shark Machine Learning Library implements BSGD. Check a quick tutorial here
Maybe you want to use something like dlib's empirical kernel map. You can read it's documentation and particularly the example program for the gory details of what it does, but basically it lets you project a sample into the span of some basis set in a kernel feature space. There are even algorithms in dlib that iteratively build the basis set, which is maybe what you are asking about.
i am trying to implement discrete curve evolution algorithm in c++ do any one help me with psudo code or c code or
some simple steps of your understanding
Discrete Curve Evolution is an algorithm to compute an everywhere convex curve from one that is concave. It moves concave sections of the curve outward along their normal in discrete steps until all concavities are eliminated. It is not a genetic algorithm, the term evolution refers to 'evolving' the position of the curve over time.
Having searched on this for quite some time the best source on the internet is here:
https://cis.temple.edu/~latecki/Software/Evo.zip
This is matlab code so it's not quite what you are looking for but you have three good options:
Port it to C++ (usually not to hard with matlab as long as it doesn't use matrix prims.)
Wrap the matlab code so you can call it from C (matlab provides libraries to do this)
Compile it to an executable and call that from C (matlab also allows this)
Option 2 would require anyone that want's to run it to have a copy of the matlab dynamic library on their computer which may be undesirable. I'm guessing option 3 would require this too, but I only have experience with options 1 and 2. Porting matlab to c++ is usually not that bad; it depends on how much the code utilizes matrix primitives and matrix operations which are easy to use in matlab and hard to use in C++ (because they aren't built-in). Still, I'd recommend giving it the old college try!
If you're just looking for DCE, check out the file evolution.m. That's the function that implements DCE. The full skeleton pruning algorithm this comes from can only be described simply at a high level. The individual steps and parts are QUITE complicated and DCE is only a small piece of that.
Hope this helps! I will be working with this code myself so if I do end up using it in C++ in some way that might help you I will let you know.
I'm not exactly sure what you mean by Discrete Curve evolutionary algorithm, but if you mean a Symbolic regression algorithm, you can start by reading about symbolic regression (or genetic programming in general):
http://en.wikipedia.org/wiki/Symbolic_Regression
There's also some nice existing programs. The Eureqa one has an open API:
http://code.google.com/p/eureqa-api/