What kind of debugging is available for image processing/computer vision/computer graphics applications in C++? What do you use to track errors/partial results of your method?
What I have found so far is just one tool for online and one for offline debugging:
bmd: attaches to a running process and enables you to view a block of memory as an image
imdebug: enables printf-style of debugging
Both are quite outdated and not really what I would expect.
What would seem useful for offline debugging would be some style of image logging, lets say a set of commands which enable you to write images together with text (probably in the form of HTML, maybe hierarchical), easy to switch off at both compile and run time, and the least obtrusive it can get.
The output could look like this (output from our simple tool):
http://tsh.plankton.tk/htmldebug/d8egf100-RF-SVM-RBF_AC-LINEAR_DB.html
Are you aware of some code that goes in this direction?
I would be grateful for any hints.
Coming from a ray tracing perspective, maybe some of those visual methods are also useful to you (it is one of my plans to write a short paper about such techniques):
Surface Normal Visualization. Helps to find surface discontinuities. (no image handy, the look is very much reminiscent of normal maps)
color <- rgb (normal.x+0.5, normal.y+0.5, normal.z+0.5)
Distance Visualization. Helps to find surface discontinuities and errors in finding a nearest point. (image taken from an abandoned ray tracer of mine)
color <- (intersection.z-min)/range, ...
Bounding Volume Traversal Visualization. Helps visualizing a bounding volume hierarchy or other hierarchical structures, and helps to see the traversal hotspots, like a code profiler (e.g. Kd-trees). (tbp of http://ompf.org/forum coined the term Kd-vision).
color <- number_of_traversal_steps/f
Bounding Box Visualization (image from picogen or so, some years ago). Helps to verify the partitioning.
color <- const
Stereo. Maybe useful in your case as for the real stereographic appearance. I must admit I never used this for debugging, but when I think about it, it could prove really useful when implementing new types of 3d-primitives and -trees (image from gladius, which was an attempt to unify realtime and non-realtime ray tracing)
You just render two images with slightly shifted position, focusing on some point
Hit-or-not visualization. May help to find epsilon errors. (image taken from metatrace)
if (hit) color = const_a;
else color = const_b
Some hybrid of several techniques.
Linear interpolation: lerp(debug_a, debug_b)
Interlacing: if(y%2==0) debug_a else debug_b
Any combination of ideas, for example the color-tone from Bounding Box Visualization, but with actual scene-intersection and lighting applied
You may find some more glitches and debugging imagery on http://phresnel.org , http://phresnel.deviantart.com , http://picogen.deviantart.com , and maybe http://greenhybrid.deviantart.com (an old account).
Generally, I prefer to dump bytearray of currently processed image as raw data triplets and run Imagemagick to create png from it with number e.g img01.png. In this way i can trace the algorithms very easy. Imagemagick is run from the function in the program using system call. This make possible do debug without using any external libs for image formats.
Another option, if you are using Qt is to work with QImage and use img.save("img01.png") from time to time like a printf is used for debugging.
it's a bit primitive compared to what you are looking for, but i have done what you suggested in your OP using standard logging and by writing image files. typically, the logging and signal export processes and staging exist in unit tests.
signals are given identifiers (often input filename), which may be augmented (often process name or stage).
for development of processors, it's quite handy.
adding html for messages would be simple. in that context, you could produce viewable html output easily - you would not need to generate any html, just use html template files and then insert the messages.
i would just do it myself (as i've done multiple times already for multiple signal types) if you get no good referrals.
In Qt Creator you can watch image modification while stepping through the code in the normal C++ debugger, see e.g. http://labs.qt.nokia.com/2010/04/22/peek-and-poke-vol-3/
Related
I am currently developing an application based on OpenCV/C++ to track small animals:
Here is a example of the kind of video it should process. The program simply output the x,y position of the animal and the time for each area. This is graphical representation of the result onto the original.
My question is a bit awkward --- and maybe off topic --- in that I am not asking how to improve my program, but how to assess it. I am aware of the existence Bonn Benchmark on Tracking dataset, but it is not appropriate to my case.
The program is meant to process very long video, therefore, I cannot realistically ask independent humans to assess the position of the animals and compare human vs. program.
I have also considered using robots or putting transponders on bigger animals in order to have the precise positions, but I do not really have the resources.
I came out with the idea of using a program to generate videos of blobs moving in a 2d environment. My question is simple:
Are you aware of any programmable high-level framework that I could use to graphically simulate the motion of an object moving stochastically in a parametrisable background ?
My dream would be to have a command line tool that would work like this:
$ program [BACKGROUND_OPTIONS] [OBJECT_OPTIONS] -V VIDEO_OUTPUT -P POSITIONS_OUTPUT
The background texture could be manipulated as well as the shape, colour and motion pattern of the object moving.
I know that I could probably "easily" make it myself (and I will if I cannot find anything), but I would prefer if the program assessed and the reference were as much independent as they can (for instance not both made by the same person).
One thing I've seen several motion detection/tracking projects do is create test videos with some 3D rendering software such as Blender. It doesn't have the simple interface of your dream test creator, but it's a good testing tool for lots of reasons:
You can set up whatever scenario you want (varying perspective, number of objects, test length, motion paths, etc.)
You completely control lighting parameters, shapes, sizes, etc.
You can design simple tests to verify basic functionality (solid color background with solid colored moving spheres makes a good starting point), then branch into more complex scenarios (other static objects, objects occluding other objects, background images, and so on).
I recently saw the virtual mirror concept on you tube, I tried it out and researched about it. It seems that the creators have used augmented reality so that people can see the output on their screens. On researching I found out that we identify a pattern on which a 3D image is superimposed.
Question 1:How are they able to superimpose the jewellery and track the face of the person without identifying any pattern?
I also tried to check various libraries that I can use to make a program similar to the one they show. Seems to me that a lot of people are using Android phones and iPhones and making apps that use augmented reality.
Question 2:Is there any way that I can use c++ and try to make a program that uses augmented reality?
Oh, and the most important thing, the link to the application is provided below:
http://www.boutiqueaccessories.com.au/virtual-mirror/w1/i1001664/
Do try it out. Its a good experience. :D
I'm not able to actually try the live demo, but the linked video suggests that they either use some simplified pattern recognition (get the person's outline), or they simply track you based on the initial image (with your position/texture being determined by the outline being shown.
Following the video, it's easy to see that there's no real/advanced AR behind this. The images are simply overlayed or hidden (e.g. in case it's missing track of one ear due to you looking to the side) and they're not transformed (no perspective or resizing happening). They definitely seem to track the head (or features like ears, neck, etc.). depending on your background and surroundings that's actually a rather trivial task.
Question 2: Sure! There are lots of premade toolsets out there, but you could as well use some general image processing library such as OpenCV to do the math. Augmented reality usually uses some kind of pattern (e.g. a card or page with a known pattern) to determine the correct position and transformation for the contents to be added to the image. There are also approaches using the device's orientation and perspective changes in camera images to determine depth/position (I really like this demo).
I'm doing an implementation for a path planning algorithm. I'd really like to be able to load in a 2d "environment" in vector graphics (svg) format, so that complex obstacles can be used. This would also make it fairly easy to overlay the path onto the environment and export another file with the result of the algorithm.
What I'm hoping to be able to do is use some kind of library in my collision test method so that I can simply ask, "is there an obstacle at x, y?" and get back true or false. And then of course I'd like to be able to add the path itself to the file.
A brief search and a couple of downloads left me with libraries which either create svg's or render them but none really gave me what I need. Am I better off just parsing the xml and hacking through everything manually? That seems like a lot of wasted effort.
1.This may be a bit heavyhanded, but Qt has a really great set of tools called the Graphics View Framework. Using these tools, you can create a bunch of QGraphicsItems (polygons, paths, etc..) in a QGraphicsScene, and then query the scene by giving it a position. Using this you'll never actually have to render the scene out to a raster bitmap.
http://doc.trolltech.com/4.2/graphicsview.html, http://doc.trolltech.com/4.2/qgraphicsscene.html#itemAt
2.Cairo has tools to draw all sorts of shapes as well, but I believe you'll have to render the whole image and then check the pixel values. http://cairographics.org/
The SVG specification includes some DOM interfaces for collision detection:
http://www.w3.org/TR/SVG/struct.html#_svg_SVGSVGElement__getEnclosureList
http://www.w3.org/TR/SVG/struct.html#_svg_SVGSVGElement__getIntersectionList
Using these methods, all "obstacles" (which can be groups of primitive elements, using the <g> element) should be labelled as a target of pointer events.
These methods are bounding-box based, however, so may not be sophisticated enough for your requirements.
Thanks for the responses. I didn't manage to get anything working in QT (it's massive!) or Cairo, and I ended up going with PNGwriter, which does pretty much what I wanted except, of course, that it reads and writes PNG's instead of vector graphics. The downside here is that my coordinates must be rounded off to even pixels. Maybe I'll continue to look into vector graphics, but this solution is satisfactory for this project.
is there anyway to build rich animation with C++?
I have been using OpenCV for object detection, and I want to show the detected object with rich animation, Is there any easy way to realize this?
I know flash can be used to easily build rich animation. But can flash be reliably integrated with C++ and How?
Also, Can OpenGL help me with this? To my knowledge, OpenGL is good for 3D rendering. But I am more interested in showing 2D animations in an image. So I am not sure whether this is a right way to go.
Another question, how are those animations in augmented reality realized? What kind of library are they using?
Thank you in advance.
Its difficult to tell if this answer will be relevant, but depending on what sort of application you are creating you may be able to use Simple DirectMedia Layer.
This is a cross-platform 2D and 3D (via OpenGL) media library for C, C++ and many other compatible languages.
It appears to me that you wish to produce an animated demo of your processing results. If I am wrong, let me know.
The simplest way to produce a demo of a vision algorithm is to dump the results to a distinct image file after each processed frame. After the processing session, these individual image files are employed to prepare the video using e.g. mencoder. I employed such procedure to prepare this.
Of course, your program can also produce OpenGL. Many people dealing with 3D reconstruction do that. However, in my opinion that would be an overkill for simple 2D detection. Producing flash would be an even greater overkill.
How can I make a screensaver in C++ that fades an image in and out at random places on the screen with a specified time delay on the fade out?
Multimonitor support would be awesome.
If you have a working code or know where I can get it, it would be great. Otherwise point me in the right direction. I'm looking for a method that has a smooth and not laggy og flickery fade. The screensaver is for Windows XP.
I dont know C++, but I do know AS3, Javascript, and PHP. So I was hoping to relate some of that knowledge to C++.
What should I use to compile it?
First off if you're starting out in C++, don't start with a windows specific compiler like visual c++. Grab a nice cross-platform IDE that comes with a compiler like eclipse or code::blocks. Like any project, you are going to want to split it up into smaller tasks you can complete in stages. Sounds like you have a couple of hurdles.
Lack of C++ Knowledge (we were all here once)
Lack of knowledge about images (very common affliction)
Lack of experience (we'll work on this)
DO NOT let others discourage you. You CAN do this, and probably faster than you think possible. DO read a book or two about C++, you can get by for one project without knowing much but you WILL get frustrated often. Let's break up your project into a set of small goals. As you complete each goal your confidence in C++ will rise.
Image blending
Windows Screen Saver w/ multi-monitor support
Screen Canvas (directx, opengl, bitmaps?)
Timers
First, let's look at the problem of image blending. I assume that you'll want to "fade" the image in question into whatever windows background you have active. If you're going to have a solid-color background, you can just do it by changing the alpha transparency of the image in question between canvas refreshes. Essentially you'll want to average the color values of each pixel in the two images based on your refresh timer. In more direct terms to find the red, green, and blue pixel elements for any resultant pixel (P3)
N = timer ticks per interval (seconds/milliseconds/etc)
T = ticks that have occurred this interval
P1r = red pixel element from image 1
P2r = red pixel element from image 2
P3r = resultant red pixel element for blended image
P1g = green pixel element from image 1
P2g = green pixel element from image 2
P3g = resultant green pixel element for blended image
P1b = blue pixel element from image 1
P2b = blue pixel element from image 2
P3b = resultant blue pixel element for blended image
P3r = ((T/N * P1r) + ((N-T)/N * P2r))/2
P3g = ((T/N * P1g) + ((N-T)/N * P2g))/2
P3b = ((T/N * P1b) + ((N-T)/N * P2b))/2
Now let's look at the problem of windows screen savers and multi-monitor support. These are really two separate problems. Windows screensavers are really only regular .exe compiled files with a certain callback function. You can find many tutorials on setting up your screensaver functions on the net. Multi-monitor support will actually be a concern when you set up your Screen Canvas, which we'll discuss next.
When I refer to the Screen Canvas, I am referring to the area upon which your program will output visual data. All image rendering apps or tutorials will basically refer to this same concept. If you find this particularly interesting, please consider a graduate program or learning course in Computer Vision. Trust me you will not regret it. Anyway, considering the basic nature of your app I would reccommend against using openGL or DirectX, just because each have their own layer of app-specific knowledge you'll need to acquire before they are useful. On the other hand if you want built-in 3d niceties like double buffering and gpu offloading, I'd go with openGL (more platform agnostic). Lots of fun image libraries come as gimmes as well.
As for multi monitor support, this is a tricky but not complicated issue. You're basically just going to set your canvas bounds (screen boundary) to match the geometry of your multiple monitors. Shouldn't be an issue with multiple monitors of the same resolution, may get tricky with mismatched monitor resolutions (might have canvas areas not displayed on screen etc. There are well known algorithms and workarounds for these issues, but perhaps this is beyond the scope of this question.
Finally as to the timers required, this should be the easiest part. Windows and Linux handle time in their own strange ways (WHY doesn't MS implement STRPTIME), so If you are interested in portability I'd use a 3rd party library. Otherwise just use the windows settimer() basic functionality and have your image rendered in the callback function.
Advanced Topic: Hey if you're still reading there are a number of interesting algorithmic improvements you can make to this program. For instance, with your timer going off a set quanta each second, you could cache the first few blended images and save some processing time (our eyes are not terribly good at noticing differentiating between changing color gradients). If you are using openGL, you could cache sets of blended images as display lists (gotta use all that graphics card memory for something right?). Fun stuff, good luck!
"I dont know C++, but I do know AS3, Javascript, and PHP. So I was hoping to relate some of that knowledge to C++."
Oh boy, I suppose you're going to be surprised. To learn C++:
Buy at least one or two very good books (see here). Do not buy books that aren't very good. They'll teach you habits that you would have to unlearn later in order to make further progress.
Expect having to do plenty of hands-on code writing with a lot of reading in between. In the first few weeks, the compiler will spit legions of incomprehensible error messages at you, your code will keep crashing, and seasoned C++ programmers looking at your code will throw up their hands in disgust. And it's always your fault.
Plan a lot of time to learn. And I mean a real lot. No matter how much time you devote, it will take at least a couple of months until you upgrade from "dummy" to "novice".
Imagine having to hammer at it for a couple of years in order to become a real professional, constantly reading books and articles, devoting plenty of time to newsgroups and discussion forums, learning from others, banging your head against every wall surrounding your desk.
Be prepared to learn something you haven't known before at least once every week even after a decade of programming in C++ for a living.
Just for a moment, imagine I might not overstate this.
Edit for clarification: I have up-voted both Hunter Davis' and Elemental's answers, because they're very good, pretty much to the point, and in general the encouragement is supported by me despite my rant up there. In fact, I do not doubt that many are able to hack something together in C/C++ when given a skeleton example even if actually they don't know much of C++. Go ahead and try, there's a good chance you'll come up with a screen saver within reasonable time. But. Being able to "hack something together in C/C++" is far from "having learned C++". (And I mean very far.)
C++ is an incredible complex, mean, and stubborn beast. I also consider it almost breathtakingly beautiful, but in that it probably mirrors the view from a very high mountain: It's pretty hard to get to the point where you're able to appreciate the beauty.
C++ is a complex language and all that sbi indicates is probably true (certainly after 10 years of commercial C++ programming there is still some to learn) BUT I think that if you are confident in another language it should only take a couple of days to:
a) Install one of the compilers mentioned here (I would suggest VC as a quick in to windows programming)
b) Find some sample code of a screen saver that does something simple using the windows GDI (there is some code within the MS documentation on screen savers)
c) Modify this code to do what you want.
In terms of your limited requirement I think you will find that the window GDI++ libraries have sufficient power to do the alpha fades you require smoothly.
I'd look at microsoft c++ express. It's free and pretty capable. You may need to hack it, or get an older version, to produce unmanaged executables.
There are a lot of tutorials available for the rest