I am trying to improve my webcam based OpenCV mouse controller for disabled people (MFC C++ application): https://preability.com/face-controlled-mouse/
The cursor moves, when a person moves her/his head, clicks when smile, etc.
Controller finds face area then use goodFeaturesToTrack, cornerSubPix and calcOpticalFlowPyrLK.
In general, I managed to stabilize cursor if lighting is good.
What I use now:
Evaluating and filtering the direction of the each corner point movement.
Spreading the corner points all over the face area for cv::goodFeaturesToTrack() helped a little too.
EWMA (or Kalman) filter for the cursor position.
I’ve included equalizeHist() for the face ROI. The detector performed much better in low light conditions.
In addition, I tried morphology operations of OpenCV without improvement.
However, the corner points still dance in uneven lighting.
I can see that similar old program eViacam has preprocessing module for webcam Creavision (old too) and the corner points are more stable.
Please advise what can be done with the input Mat? Or how can the video be processed with reasonable CPU loading?
Now I can answer my own question. Christoph Rackwitz gave me some good advice:
don’t track the whole head. track each feature. and don’t use those trackers, they’re too complex. use MOSSE. it’s dumb but very precise, as long as the object (which should be a tiny feature on the face) doesn’t change much.
MOSSE approaches optical flow. methods to calculate optical flow work like MOSSE, except they use simpler math and smaller regions, hence the result is noisier. MOSSE uses a larger area (for a single track/point of course) and more sophisticated math, for a more accurate result.
When MOSSE algorithm tracks “corner points”, cursor moves much more smoothly. There was a slight issue with discrete movement as the object rectangles moved the same number of pixels at the same time. Cursor moved in leaps. So, I had to use filter on each tracked point. Anyway, as you can see in the video, the CPU load did not increase comparing to Lukas-Kanade optical flow algorithm + filtration only cursor position. In good light, the difference is also very noticeable.
https://www.youtube.com/watch?v=WKwuas0GVkA
Lucas-Kanade optical flow:
goodFeaturesToTrack,
cornerSubPix,
calcOpticalFlowPyrLK,
cursor EWMA filter
MOSSE object tracking:
goodFeaturesToTrack,
cornerSubPix,
TrackerMOSSE,
all points EWMA filtration
And of course I had to remember to include the tracking453.lib to Linker when add legacy Tracker. I spent half a day googling the “unresolved external symbol LNK2001 error”. For some reason including a tracker from the core library (cv::Tracker) does not result such compilation error, so it is confusing.
I have a movie in the after effects that dosent have a KEY color of background, but a still background.
I want to detect the motion of 2 persons walking in front of this still background and bring them to the front, so I can create effects on the background.
Is that possible to do? Witch Effect do I use?
This works with a technique called difference matting. It never works well even if you have a solid clean plate. your best bet is to rotobrush or just mask them out manually. if you want to try difference matting you can set the footage layer with the people in it to difference blending mode and put a clean plate below it.
you can use the rotobtush that allows to separate elements from a static background. Works better if:
you have a clean background
good quality videos
front object needs to be cutted as big as possible
This method works well with a locked down camera. Depending upon how active people are and how still the background - you can remove the people by cutting and assemble pieces of the background to create a background layer without the people... Once you have a clean plate - then use a difference matte to subtract the background from your clip which will leave anything that moves (read people). This will work well if the hue/values between the background and the people are different.
Doing this in conjuncture with animated masks gives good results and can give you a jump on your rotobrushing to improve your edges.
If all else fails the Mocha plug works well as long as you break your masks up into lots of parts to follow the planes of your actors.
I have never found doing this is a single tool solution.
It is not impossible, but I don't think it could give you the best result.
Anyway, you should mask them out manually. but not keyframe by keyframe. there is a technic for it. for setting the mask keyframes, first, you should set 3 keyframes in first, middle and end of your footage.
after masking three one, you do it the same for first keyframe and the middle one.
it means between the first keyframe and the middle keyframe, you should set a key.
Do the same for middle and end key.
this technic could save your time and make masking with less fault.
by the way, you can use Mocha pro as well for tracking the persons.
I am learning opengl es and am planning to make a program which will have a shape which can be cut into a smaller shape by removing a part of the shape dynamicly. The constraint is I must be able to tell if an object is inside or outside the cut shape.
The option I thought of are:
1) use a stencil buffer made up of just a black and white mask. This way I can also use the same map for collision detection.
2) the other option is to dynamicly change my mind renderd primitive an then tesselating it. This sounds more complex and is currently my least favorite option. It would also make the collision detection more difficult.
PS
I would like the part of the shape removed to be fall of in animation, I am not sure how choosing any of these methods will affect the ease of doing so. Please express your opinion.
What are your thoughts on this?
Keep in mind that I am new to opengl an might be making mistakes without realizing it.
Thanks, Jason
It is generally considered a good idea to issue only write-commands to the graphics card. Basically that is "dont use glGet* commands at all", because the latency of those commands might be somewhat high.
That said option 1) is great if you just want to mask out stuff. As you are trying to make the cut part fall off this is really not an option, as you have to retrieve/reconstruct the vertices of that part.
I don't quite get the "tesselation" part of your second option, but if your primitive is a polygon and your cuts are straight lines, it is easy to calculate the 2 polygons after the cut. In fact the viewport clipping routine in OpenGL does that all the time and there is a lot of literatur, for example http://en.wikipedia.org/wiki/Sutherland-Hodgman
In the long term it is often way better to first build a (non-visual) model of what is going on in the application before visualizing.
I am working on a simple 2D openGL project. It contains a main actor you can control with the keyboard arrows. I got that to work okay. What I am wanting is something that can help explain how to make another actor object follow the main actor. Maybe a tutorial on openGL. The three main things I need to learn are the actor following, collision detection, and some kind of way to create gravity. Any good books or tutorials to help get me in the right direction would be great.
You could use a physics library like Chipmunk Physics, which lets you attach springs and things between the two objects and detect when they hit each other and other things.
A pre-rolled library would be good, but the concepts you describe are ones you need to know if you are going to do any sort of game programming anyways:
A simple way to make one actor follow behind another is to have the lead actor store its position every time it moves. Feed these positions to a trailing actor with a delay of a few values - the longer the delay, the further behind they travel. Simple, but doesn't handle dynamic collision (other actors moving the block collision.)
Collision detection in 2D can simply be axis aligned (AA) bounding boxes. Search for this and you'll see the 4 ifs or so that are needed.
Gravity is just adding a fixed velocity (usually down) to every object every game loop. This is constant acceleration which is exactly how gravity works.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 6 years ago.
Improve this question
I'm looking for some good ideas for a simple 3d graphics program as my final project for an intro to computer graphics class. As for some background information, we'll be using opengl and will have a little over a month to work on it, so nothing too far-fetched. The simpler and "prettier" looking, the better. It does, however, require some sort of interface that the user can interact with (so a very simple game or similar is a good idea) and must be 3D.
My only idea so far is maybe a 3D version of Tetris (google for some examples).
Edit: I ended up going with 3D Tetris. For a less than a month's worth of time, you can see what I came up with here.
A 3D text/code editor. Text is 3D, errors stand out, code indentations not only indent, but protrude on z axis, pages/files are 3D and can be flipped like a ringpad.
Probably not useful, but fun and more interesting than a game IMO.
In university, for my parallel programming course, I did an openGL/MPI implementation of Conway's Game of Life. It was quite interesting. Wish I still had the code around somewhere. The advantage of using open GL is that you can lay out the grid in different orientations rather than a flat grid. Remember, code doesn't exist until it's checked into source control.
Putting some physics in makes it more interesting. How about implementing Labyrinth (the maze toy where you are supposed to guide a ball from the starting point to the goal by tilting it).
EDIT: Erik told me it's called Labyrinth.
A 3D minesweeper game similar to this one.
Rubik's cube.
Look at http://www.contextfreeart.org/ ... write something similar, but for 3d.
If you've ever played Missile Command I belive that this could be a good project to '3d-ify'.
Try a chicken crossing the road game.
You will probably need to demonstrate the bare minimum of:
textures
lighting
animation
interaction
collision detection
Do not include even simple physics if there are no marks for it. Prioritise tasks based on the marking scheme. Get something simple working first and back it up :)
Honestly it's actually pretty easy to load up a bunch of animated models and set up a simple first person shooter. I mean, to get a generic thing working you don't need all that much:
Either load and display a heightmap or a BSP tree as the level.
Load and render some simple MD2 models (keyframe animation, low amount of polys and simple format).
Draw a simple hud.
Ray/AABB intersection, every time the user clicks you'll need to cast a ray from the center of the screen and see if it intersects an the bounding boxes of the enemies.
Simple FPS camera system.
The above is pretty doable in a month for as far as I'm concerned. (It's probably doable in a week if you already know some of the stuff).
I tried to do a 3D Asteroids for a class once. I never completed the gameplay part, since it was a graphics class. The ship could move around, as could the asteroids, but there was no collision detection. The ship and the asteroids had 3D textures applied to them, and the asteroids were built out of ellipsoids, so they were actually 3D. The gameplay was all 2D, though.
How about one of those games that are a wooden maze with a ball rolling around the top. You tilt the board and try to get the ball round the maze without falling down the hole? It has the advantage that it's relatively simple to get started, but you could probably think of some extensions if you have time.
If you're looking for a true university size task, mine was to produce a small helicopter "game" where you could take off from an aircraft carrier in an ocean and fly around with some environmental effects, moving water etc. i.e. nothing too complicated. As another example, the task set for the year previous to mine was a little sans-opponent racing game.
I would worry that you may loose marks with tetris as it sounds like little would be done on the z-axis and may come across a little too 2d though it obviously depends on your brief.
Anyway, these will give you the chance to experiment with the basic OpenGL features such as fog, lighting, geometry, textures and some basic movement physics & collision detection/response.
Further on this, though often beyond the scope of such a university sized task you could then take this further add nicities such as animated geometry (e.g. people), environment mapping, reflections, shadows, particle systems, shaders, perhaps a heightmapped island.
Rewrite Blocks 3D. The graphics on this project look horrible now. I remember playing this game (or one like it) on a 386 with wireframe graphics... awesome. The game is basically 3D tetris.
I would check Panda3D or Pygame.
Panda3D is probably close to what you are looking for, and one idea that always works is to put the user's face in the main character or object. 3d-pong with the player's face? Use something unexpected... like a tetris made of burgers instead of bricks.
I love little self-organising alife applications like boids. They can be fun to code and always benefit from a nice UI, especially 3D ones. User input can modify aspects of the environment as well as moving around/through the environment.
I like exoplanets. Go read up on them. On Wikipedia and http://exoplanet.eu there's a lot of information. Astronomers and public outreach people could always use fresh 3D animations showing how the Doppler effect works, or how the planet transiting in front of the star makes it for example 0.5% dimmer.
Or, what I work on, is how when the planet passes behind its star. At Earth we receive just a teeny bit less infrared from that star. The user could adjust the orbit, size of planet, etc. and see how that affects what astronomers see. It could be fun, simple enough to do, and unlimited potential in extending the work for nicer textures, slick lighting effects, etc., and you could end up with something to contribute to science education.
I'd be making such 3D animations myself, if I weren't busy helping crunch numbers for the actual science. I'll be jealous!