Considering how the TREPAN XAI works according to:
https://proceedings.neurips.cc/paper/1995/file/45f31d16b1058d586fc3be7207b58053-Paper.pdf
I do not understand why the TREPAN algorithm for Neural Network explainations randomly produces new samples of points for every split it makes in the building of the explaination tree.
What is the purpose of introducing new points?
Related
I'm new in Opencv, I want to comparison two image by theirs corner features rather than others, I tried SURF, SIFT, ORB ..., but their are not suit of me, could someone give me some suggestion about this? for example, the below image image1 and image2 are similar, because their have a lot of same corners ( although it isn't accurate ), but the image3 isn't similar as the 1 and 2, thanks.
There are a lot of things to try here, and others can suggest you others solutions, but here goes mine:
Use the Minutiae algorithm (the algorithm used for fingerprint identification). In the minutiae algorithm, a set of common features are extracted, such as:
These are called Minutiae. As you can see these features are pretty similar to the corners of your images. I suggest you the following procedure:
1) Find the corners (You already did this)
2) Assign each corner to a class of minutiae (e.g. one of the classes of the image). You can do this using a local binary pattern algorithm or just follow the usual recipe from the fingerprint algorithm (see the link at the end or search on Google).
3) To compute the similarity just do some voting. For instance, let's assume an image (I will call it A) has 4 minutiae of type a) and two of type E). To compute the similarity in a new image I will have to compute these minutiae in the new image. Then, see how many items per class both images share. You can add as many features or kind of minutiae as you want to make your algorithm more robust (and also complicated).
In any case, you can look in Google at the Minutiae fingerprint recognition algorithm (is one of the most famous digital image processing algorithms). Here goes one of the multiple slides you can find explaining the algorithm
https://is.muni.cz/el/1433/jaro2008/PV204/um/finger/MinutiaeBasedFpMatching.pdf
Just take care to modify the main algorithm to suit your necessities
Hope it helps
I need to implement a simple image search in my app using TensorFlow.
The requirements are these:
The dataset contains around a million images, all of the same size, each containing one unique object and only that object.
The search parameter is an image taken with a phone camera of some object that is potentially in the dataset.
I've managed to extract the image from the camera picture and straighten it to rectangular form and as a result, a reverse-search image indexer like TinEye was able to find a match.
Now I want to reproduce that indexer by using TensorFlow to create a model based on my data-set (make each image's file name a unique index).
Could anyone point me to tutorials/code that would explain how to achieve such thing without diving too much into computer vision terminology?
Much appreciated!
The Wikipedia article on TinEye says that Perceptual Hashing will yield results similar to TinEye's. They reference this detailed description of the algorithm. But TinEye refuses to comment.
The biggest issue with the Perceptual Hashing approach is that while it's efficient for identifying the same image (subject to skews, contrast changes, etc.), it's not great at identifying a completely different image of the same object (e.g. the front of a car vs. the side of a car).
TensorFlow has great support for deep neural nets which might give you better results. Here's a high level description of how you might use a deep neural net in TensorFlow to solve this problem:
Start with a pre-trained NN (such as GoogLeNet) or train one yourself on a dataset like ImageNet. Now we're given a new picture we're trying to identify. Feed that into the NN. Look at the activations of a fairly deep layer in the NN. This vector of activations is like a 'fingerprint' for the image. Find the picture in your database with the closest fingerprint. If it's sufficiently close, it's probably the same object.
The intuition behind this approach is that unlike Perceptual Hashing, the NN is building up a high-level representation of the image including identifying edges, shapes, and important colors. For example, the fingerprint of an apple might include information about its circular shape, red color, and even its small stem.
You could also try something like this 2012 paper on image retrieval which uses a slew of hand-picked features such as SIFT, regional color moments and object contour fragments. This is probably a lot more work and it's not what TensorFlow is best at.
UPDATE
OP has provided an example pair of images from his application:
Here are the results of using the demo on the pHash.org website on that pair of similar images as well as on a pair of completely dissimilar images.
Comparing the two images provided by the OP:
RADISH (radial hash): pHash determined your images are not similar with PCC = 0.518013
DCT hash: pHash determined your images are not similar with hamming distance = 32.000000.
Marr/Mexican hat wavelet: pHash determined your images are not similar with normalized hamming distance = 0.480903.
Comparing one of his images with a random image from my machine:
RADISH (radial hash): pHash determined your images are not similar with PCC = 0.690619.
DCT hash: pHash determined your images are not similar with hamming distance = 27.000000.
Marr/Mexican hat wavelet: pHash determined your images are not similar with normalized hamming distance = 0.519097.
Conclusion
We'll have to test more images to really know. But so far pHash does not seem to be doing very well. With the default thresholds it doesn't consider the similar images to be similar. And for one algorithm, it actually considers a completely random image to be more similar.
https://github.com/wuzhenyusjtu/VisualSearchServer
It is a simple implementation of similar image searching using TensorFlow and InceptionV3 model. The code implements two methods, a server that handles image search, and a simple indexer that do Nearest Neighbor matching based on the pool3 features extracted.
Specifically, my question is every consequent frame has different number of points and KNN/SVM fails to implement unless I have the same number of points for each frame. So how to apply ml on 3D frames which have are different in size? My ply output file consists of x,y,z coordinates for each point and more than 10000 points per frame.
You can use open3d to downsample the points to a fixed number for all point clouds and then use deep learning libraries for classification or segmentation. PointNet developed by the Stanford AI Lab is one of the best algorithms for this.
If you have 10000points per pointCloud. It's a pretty decent data precision for a 3D object. As a 3D artist, not a scientist I would try to find a hack. Like if your second pointCloud has 10065 points more or less. I will just ignore randomly the extra 65points on the second pointCloud so they match in length ( sum all your points number divide by frame number to get the reference value). But that can damage your data maybe (depends of how much they vary in length).
If I had to use scan raw data I would use a strongh geometry processing library like the C++ pointCloud library ? http://pointclouds.org/
and its python binding: http://ns50.pointclouds.org/news/2013/02/07/python-bindings-for-the-point-cloud-library/
Or a 3D software ? (Or tensor Flow ?)
You can extract global descriptors from each pointcloud and train a machine learning algorithm like a SVM or an ANN with them.
There are a lot of different global descriptors, here you can take a look at a few of them: PCL Descriptors
Once you have them train a machine learning algorithm like the ones shown in Python Machine Learning Classification
I would like to do a comparison from a query with pictures in a database (about 2000).
Before posting on this website i read a lot of papers concerning methods for matching a picture in a big database and read a lot of posts on stackOverflow.
Concerning papers, there are some stuff interesting but quite technical and difficult to understand well the algorithms. (I just began to specialize myself in this field)
Posts (the most interesting) :
Simple and fast method to compare images for similarity ;
Nearest neighbors in high-dimensional data? ;
How to understand Locality Sensitive Hashing? ;
Image fingerprint to compare similarity of many images ;
C++/SIFT/SQL - If there a way to compare efficiently a SIFT descriptor of an image with a SIFT descriptor in a SQL database?
Papers :
Object retrieval with large vocabularies and fast spatial matching,
Image Similarity Search with Compact Data Structures,
LSH,
Near Duplicate Image Detection min-Hash and tf-idf Weighting
Vocabulary tree
Aggregating locals descriptors
But i'm still confusing.
The first thing i did is to implement BoW. I trained the Bag of Words (with ORB as detector and descriptor ,and used VLAD features) with 5 class in order to test its efficiency. After a long training, i launched it. It functioned well with an accuracy of 94 %. That's pretty good.
But there is a problem for me:
I don't want to do a classification. In my database, i'll have about 2000 differents pictures. I just want to find the best matches between my query and the database.
So if i have 2000 differents pictures,if i'm logical i have to consider these 2000 pictures as 2000 differents class and obviously that's impossible...
For this first thing, are you agree with me ? It's not obviously the best method to do what i would like ?
Maybe there is another way to use BoW in order to find similarities in the database ?
The second thing i did is « more simpler ».
I compute the descriptors of my query. Then i did a loop over all my database and i computed the descriptors of each picture and then added each descriptors in a vector.
std::vector<cv::Mat> all_descriptors_database;
for (i → 2000) :
cv::Mat request=cv::imread(img);
computeKeypoints(request) ;
computeDescriptors(request) ;
all_descriptors_database.pushback(descriptors_of_request)
At the end i have a big vector which contains all the descriptors of the all database. (The same with all the keypoints)
Then, this is here where i get confused.
At the beginning, i wanted to compute the matching inside the loop that is to say, for each image in the database, compute its descriptors and do a match with the query. But it tooks a lot of time.
So after reading a lot of paper about how find similarities in big databases, i found the LSH algorithm which seems to be appropriate for that kind of search.
Therefore i wanted to use this method.
So inside my loop i did something like that :
//Create Flann LSH index
cv::flann::Index flannIndex(all_descriptors_database.at(i), cv::flann::LshIndexParams(12, 20, 2), cvflann::FLANN_DIST_HAMMING);
cv::Mat results, dists;
int k=2; // find the 2 nearest neighbors
// search (nearest neighbor)
flannIndex.knnSearch(query_descriptors, results, dists, k, cv::flann::SearchParams() );
However i have some questions :
It tooks more than 5 seconds to loop all my database (2000) whereas i thought it will take less 1s (on the papers, they have huge databases not like me and LSH is more efficient). Did i do something wrong ?
I found on the internet some libraries which implement LSH like http://lshkit.sourceforge.net/ or http://www.mit.edu/~andoni/LSH/ . So what is the difference between these libraries and the four line of code i wrote using OpenCV ? Because i checked the libraries and for a kind of beginner like me, it was so difficult to try to use it.I got a bit confused.
The third thing :
I wanted to do a kind of fingerprint of each descriptors for each picture (in order to compute the Hamming distance with the database) but it seems to be impossible to do that. OpenCV / SURF How to generate a image hash / fingerprint / signature out of the descriptors?
So since 3 days, i'm blocked on that task. I don't know if i'm on the wrong way or not.
Maybe i missed something.
I hope it will be enough clear for you. Thank for reading
Your question is kind of big. I'll give you some hints, though.
Bag of Words can work but classification is unnecessary. BoW pipeline typically consists of:
keypoint detection - ORB
keypoint description (feature extraction) - ORB
quantization - VLAD (fisher encoding might be better, but plain old kmeans might be enough in your case)
classification - you probably can skip this stage
You can treat quantization results (e.g. VLAD encoding) for each image as its fingerprint. Computing distance between fingerprints will yield a similarity measure. Still you have to do a 1 vs all matching, which is going to be tremendously expensive when your database gets big enough.
I didn't get your point.
I'd suggest reading G. Hinton's papers (e.g. this one) on dimensionality reduction with deep autoencoders and convolutional neural networks. He boasts of beating LSH. As for the tools, I'd recommend taking a look on BVLC's Caffe, a great neural network library.
Papers have been written describing how the Hough transform can be generalised to detect shapes like circles and parabolas. I'm new to computer vision though and find these papers pretty tough going. There is also code out there that does this detection but this is more than I want. I was wondering if anyone could briefly describe in bullet points or pseudo-code really simply how Hough Transforms are used to detect parabolas in images. That would be amazing. Or if anyone knows any basic explanations online that I haven't come across than that would be good enough too :).
Thanks very much :).
Interesting question. This looks like a great resource. I included a summary (loosely quoted). Also see the source from Mathworks at the bottom of this answer - Matlab has houghlines and houghpeaks functions which will be useful for you. Hope it helps.
Run edge detection algorithm, such as the Canny edge detector, on subject image
Input edge/boundary points into Hough Transform (line detecting)
Generate a curve in polar space (radius, angle) for each point
in Cartesian space (also called
accumulator array)
Extract local maxima from the accumulator array, for example using a
relative threshold
In other words, we take only those local maxima in the accumulator
array whose values are equal to or
greater than some fixed percentage of
the global maximum value.
De-Houghing into Cartesian space yields a set of line descriptions of the image subject
cs.jhu.edu:
http://www.cs.jhu.edu/~misha/Fall04/GHT1.pdf
Code from Mathworks: http://www.mathworks.com/help/toolbox/images/ref/hough.html