Histogram for big data set - mapreduce

I want to draw histogram for 1GB data using map reduce. Not able to get any hold after googling. Please give suggestions for any specific library in python or java.

ValueHistogram in hadoop can be used for drawing histogram.
https://hadoop.apache.org/docs/stable2/api/org/apache/hadoop/mapreduce/lib/aggregate/ValueHistogram.html

Related

Is there a way to construct and store a 3D Map from point cloud and depth data?

I am currently working on a SLAM algorithm, and I succeeded in gathering the depth and RGB data on the form of a point cloud. However, I only display the frames that my Kinect 2.0 received to the screen and that is all.
I would like to gather those frames and as I move the Kinect, I construct a more elaborate Map (either 2D or 3D) so that it will help me in the localization or mapping.
My idea of the map construction would be just like when we create a Panorama image from many single snapshots.
Anyone has a clue, idea or an algorithm to do it?
You can use rtabmap to create 3D map and localizing your device. Its very simple to use and supports different devices.

Sparse coding and dictionary learning using opencv and c++

I am trying to perform text image restoration and I can find no proper documentation on how to perform OMP or K-SVD in C++ using opencv.
I have over 1000 training images of different sizes so do I divide images into equal sized patches or resize all images? How do I construct the signal matrix X?
What other pre-processing steps are required for sparse coding? How to actually perform K-SVD on color images?
What data type is available in OpenCV for an image dictionary and how do I initialize the Dictionary D?
I have these very basic questions and have tried to use various libraries but they don't make the working very clear.
I found this code useful. This is the only implementation in opencv I have come across so far. I guess it uses a single image for dictionary learning whereas I have to use at least 1000 images. But it certainly provides a good guideline.

How to set classification colors in GDAL output files

I am using the GDAL C++ library to reclassify raster map images and then create an output image of the new data. However when I create the new the new image and open it, the classification values don't seem to have a color defined, so I just get a black image. I can fix this by going into the image properties and setting a color for each of the 10 classification values I'm using, but that is extremely time consuming for the amount of maps and trials I am doing.
My question is, is there a way to set metadata info through the GDAL API to define a color for each classification value? Just the name of the right function would be great, I can figure it out from there.
I have tried this using ArcGIS and QuantumGIS, and both have the same problem. Also the file type I am using is Erdas Imagine (called "HFA" in GDAL).
You can use SetColorTable() method on your raster band. Easiest to do is to fetch one pre-existing raster using GetColorTable(), and pass it to your new raster.

volume image data in the XTK toolkit

I have read here:
Is there a way to use a Custom cross-sectional slicer of 3d image data?
... that the nrrd parser stores the image data as a 3D array. I want to be able to access this array in my scripts. How can this be done? I would like to use this data to do image statistics, and subsets to do region of interest statistics. I believe the data is a private variable which is just used by the slice function to create the volume slices, is that correct? If so how can I save it for later use as a public variable, or as a property of the volume object?
Please explain as simply as possible how to proceed as I am quite a novice at javascript.
Many thanks,
We didn't store the array for all volume parsers yet to slim down the memory usage. This can certainly be added since the infrastructure is there under the hood.
I assigned the issue to me
https://github.com/xtk/X/issues/84

stitching aerial images to create a map

I am working on a project to stitch together around 400 high resolution aerial images around 36000x2600 to create a map. I am currently using OpenCV and so far I have obtained the match points between the images. Now I am at a lost in figuring out how to get the matrix transformation of the images so I can begin the stitching process. I have absolutely no background in working with images nor graphics so this is a first time for me. Can I get some advice on how I would approach this?
The images that I received also came with a data sheet showing longitude, latitude, airplane wing angle, altitude, etc. of each image. I am unsure how accurate these data are, but I am wondering if I can use these information to perform the proper matrix transformation that I need.
Thanks
Do you want to understand the math behind the process or just have an superficial idea of whats going on and just use it?
The regular term for "image snitching" is image alignment. Feed google with it and you'll find tons of sources.
For example, here.
Best regards,
zhengtonic
In recent opencv 2.3 release...they implemented a whole process of image stitching. Maybe it is worth looking at.