Hey, I am coding up a simple chess playing robot's vision system, I am trying to improve on some previous research to allow camera and a standard chess set be used and both be allowed to move during the game. So far I can locate the board in an image acquired via web-cam, and I want to detect moves by taking difference of successive images to determine what has changed then use previous information about the board occupancy to detect moves.
My problem is that I can't seem to reliably detect changes at the moment, my current pipeline goes like this:
Subtract two images -> Histogram equalize the difference image -> erode and dilate diff image to remove minor changes -> make a binary copy and do distance transform -> Get the largest blob(corresponding to the highest value after DT and flood fill that blob) -> repeat again until DT returns a value small enough to ignore change.
I am coding all this in OpenCV and C++. but my flood fill seem to always either not fill the blobs, hence most cases I just get one change detected. I have tried also using cv::inpaint but that didn't help either. So my question is; am I just using the wrong approach or somehow turing can make the change detection more reliable. In case of the former, could people suggest alternative routes, preferable codable in C++/Python and/or OpenCV in a reasonable time?
thanks
The problem of getting a fix on the board and detecting movement of pieces can be solved independently, assuming one does not move the board while also moving pieces around..
Some thoughts on how I would approach it:
Detecting the orientation of the board
You have to be able to handle the board being rotated in place, as well as moved around as long as some angle is maintained that lets you see the pieces. It would help if there were something on the board that you could easily identify (e.g. a marker on each corner) so that if you lose orientation (e.g. someone moves the board away from the camera completely) you could easily find it again.
In order to keep track of the board you need to model the position of the camera relative to the board in 3D space. This is the same problem as determining the location of the camera being moved around a fixed board. A problem of Egomotion. Once you solve that you can move on to the next stage, which is detecting movement and tracking objects.
Detecting movement of pieces
This probably the simpler part of the problem. There are lots of algorithms out there for object detection in video. I would only add that you can use "key" frames. What I mean by that is to identify those frames in which you see only the board before and after a single move. e.g. you don't see the hand moving it obscuring the pieces, etc. Once you have the before/after frame you can figure out what moved and where it is positioned relative to the board.
You can probably get away with not being able to recognize the shape of each piece if you assume continuity (i.e. that you've tracked all movements since the initial arrangement of the board, which is well known).
Related
Im building a DIY laser target.
So far on the software side I managed to find when is a laser shot at the target by calculating differences and threshold, and then find where did it hit in relation to the whole image.
Now the next problem I am facing is finding where it hit in relation to the target center.
I have a proper box assembled for the target, but while testing this is what I have (the inner circle should have been filled with black):
Its not pretty and has lots of unrelated noise but it should be enough to test. This is a best possible scenario, from within the box:
To find where it hit in relation to the target center I suppose I'll have to make a binary image out of those two black rings and calculate the average point. (how?)
Then (supposing the target is perfectly parallel, not even a little bit in perspective), with the center known, I would need to know how far the outer ring is from it.
With that known it should be a matter of trigonometry to tell the punctuation (11 to 0, being 0 the outside of the outer ring)
Resuming the question:
Need to detect a known figure(ring), tell where its center is, and determine its radius.
I am not worried about performance since this will only be done once at each program run to calibrate.
Some sample C++ code (besides pointing to the functions I can use) would be great since I'm new to both C++ and OpenCV.
I have positions of an object moving around an image. I think I'm detecting it the best I can where most of the time I'm detecting the center of the object. However, I'm still getting the odd detection of around the center caused by the frame rate not being fast enough, and the frame containing two positions of the object.
As I can't control the frame rate, how can I minimise the effects of the noise in the jittery positions.
As this is a common issue in computer vision, are there any filters in opencv to deal with noisy position data?
I asked for the comment by Berak to be made into an answer, but my request was "declined". So, yes, the answer that I found most useful was to use the Kalman filter which is implemented in opencv.
I am at the start of developing a software using OpenCV in Microsoft Visual 2010 Express. Now what I need to know before i get into coding is the procedures i have to follow.
Overview:
I want to develop software that detects simple boxing moves such as (Left punch, right punch) and outputs the results.
Now where am struggling is what approach should i take how should i tackle this development i.e.
Capture Video Footage and be able to extract lets say every 5th frame for processing.
Do i have to extract and store this frame perhaps have a REFERENCE image to subtract the capture frame from it.
Once i capture a frame what would be the best way to process it:
* Threshold it, then
* Detect the edges, then
* Smooth the edges using some filter, then
* Draw some BOUNDING boxes....?
What is your view on this guys or am i missing something or are there better simpler ways...? Any suggestions...?
Any answer will be much appreciated
Ps...its not my homework :)
I'm not sure if analyzing only every 5th frame will be enough, because usually punches are so fast that they could be overlooked.
I assume what you actually want to find is fast forward (towards camera) movements of fists.
In case of OpenCV I would first start off with such movements of faces, since some examples are already provided on how to do that in software package.
To detect and track faces you can use CvHaarClassifierCascade, but since this won't be fast enough for runtime detection, continue tracking such found face with Lukas-Kanade. Just pick some good-to-track points inside previously found face, remember their distance from arbitrary face middle, and at each frame update it. See this guy http://www.youtube.com/watch?v=zNqCNMefyV8 - example of just some random points tracked with Lukas-Kanade. Note that unlike faces, fists may not be so easy to track since their surface is rather uniform, better check Lukas-Kanade demo in OpenCV.
Of course with each frame actual face will drift away, once in a while re-run CvHaarClassifierCascade and interpolate to it your currently held face position.
You should be able to do above for fists also, but that will require training classifier with pictures of fists (classifier trained with faces is already provided in OpenCV).
Now having fists/face tracked you may try observing what happens to the points - when someone punches they move rapidly in some direction, while on the fist that remains still they don't move to much. And so, when you calculate average movement of single points in recent frames, the higher the value, the bigger chance that there was a punch. Alternatively, if somehow you've managed to track them accurately, if distance between each of them increases, that means object is closer to camera - and so a likely punch.
Note that without at least knowing change of a size of the fist in picture, it might be hard to distinguish if a movement of hand was forward or backward, or if the user was faking it by moving fists left or right. You may have to come up with some specialized algorithm (maybe with trial and error) to detect that, like say, increase a number of screen color pixels in location that previously fist was found.
What you are looking for is the research field of action recognition e.g. www.nada.kth.se/cvap/actions/ or an possible solution is e.g the STIP ( Space-time interest points) method www.di.ens.fr/~laptev/actions/ . But finally this is a tough job if you have to deal with occlusion or different point of views.
I'm working on a Minecraft like game for educational purposes. The rendering is great so far even with 1024x1204 blocks but now that I started integrating the player collision I'm having problems.
I have a aabb for the player and aabb's for all the blocks around him. These are created dynamically and it works out pretty fast.
My problem goes as following:
I have speed vector and the current position. For each axis I calculate the potential position and make out an aabb. I check for collisions and it's free I move there otherwise I set the speed for that component to 0. I separate the axis since I want my player to slide in a direction of partially facing a wall.
The order for the axis is y,x,z. The collision response is great but I'm having some problems with the corners as it sometimes get's stuck in the world without being able to move. Not sure what the reason is for this.
I do not want to implement actual physics since those are more demanding and basically just too much for what I need.
Do you guys have any suggestions on how to implement this in a nice way? I did some searching but I didn't find anything useful for this particular situation.
This is a bit abstract in a sense that the cause of your problem can be related to many things. From the top of my head, maybe a bug in your collision detection code: somehow it allows the objects to cross boundaries by 1 (or more) unit. So when the next collision is computed 1 or more dimension is stuck (imagine having an arm already inside the wall when collision is detected. You can't get your arm out because it collide with the interior of the wall boundary)
For Operating Systems class I'm going to write a scheduling simulator entitled "Jurrasic Park".
The ultimate goal is for me to have a series of cars following a set path and passengers waiting in line at a set location for those cars to return to so they can be picked up and be taken on the tour. This will be a simple 2d, top-down view of the track and the cars moving along it.
While I can code this easily without having to visually display anything I'm not quite sure what the best way would be to implement a car moving along a fixed track.
To start out, I'm going to simply use OpenGL to draw my cars as rectangles but I'm still a little confused about how to approach updating the car's position and ensuring it is moving along the set path for the simulated theme park.
Should I store vertices of the track in a list and have each call to update() move the cars a step closer to the next vertex?
If you want curved track, you can use splines, which are mathematically defined curves specified by two vector endpoints. You plop down the endpoints, and then solve for a nice curve between them. A search should reveal source code or math that you can derive into source code. The nice thing about this is that you can solve for the heading of your vehicle exactly, as well as get the next location on your path by doing a percentage calculation. The difficult thing is that you have to do a curve length calculation if you don't want the same number of steps between each set of endpoints.
An alternate approach is to use a hidden bitmap with the path drawn on it as a single pixel wide curve. You can find the next location in the path by matching the pixels surrounding your current location to a direction-of-travel vector, and then updating the vector with a delta function at each step. We used this approach for a path traveling prototype where a "vehicle" was being "driven" along various paths using a joystick, and it works okay until you have some intersections that confuse your vector calculations. But if it's a unidirectional closed loop, this would work just fine, and it's dead simple to implement. You can smooth out the heading angle of your vehicle by averaging the last few deltas. Also, each pixel becomes one "step", so your velocity control is easy.
In the former case, you can have specially tagged endpoints for start/stop locations or points of interest. In the latter, just use a different color pixel on the path for special nodes. In either case, what you display will probably not be the underlying path data, but some prettied up representation of your "park".
Just pick whatever is easiest, and write a tick() function that steps to the next path location and updates your vehicle heading whenever the car is in motion. If you're really clever, you can do some radius based collision handling so that cars will automatically stop when a car in front of them on the track has halted.
I would keep it simple:
Run a timer (every 100msec), and on each timer draw each ones of the cars in the new location. The location is read from a file, which contains the 2D coordinates of the car (each car?).
If you design the road to be very long (lets say, 30 seconds) writing 30*10 points would be... hard. So how about storing at the file the location at every full second? Then between those 2 intervals you will have 9 blind spots, just move the car in constant speed (x += dx/9, y+= dy/9).
I would like to hear a better approach :)
Well you could use some path as you describe, ether a fixed point path or spline. Then move as a fixed 'velocity' on this path. This may look stiff, if the car moves at the same spend on the straight as cornering.
So you could then have speeds for each path section, but you would need many speed set points, or blend the speeds, otherwise you'll get jerky speed changes.
Or you could go for full car simulation, and use an A* to build the optimal path. That's over kill but very cool.
If there is only going forward and backward, and you know that you want to go forward, you could just look at the cells around you, find the ones that are the color of the road and move so you stay in the center of the road.
If you assume that you won't have abrupt curves then you can assume that the road is directly in front of you and just scan to the left and right to see if the road curves a bit, to stay in the center, to cut down on processing.
There are other approaches that could work, but this one is simple, IMO, and allows you to have gentle curves in your road.
Another approach is just to have it be tile-based, so you just look at the tile before you, and have different tiles for changes in road direction an so you know how to turn the car to stay on the tile.
This wouldn't be as smooth but is also easy to do.