I am new in the field of computer vision and try to get the extrinsic camera parameters from video frames.
I have the intrinsic paremeters of my camera (focal length etc.). However, to analyse my video stream I would like to get the exttrinsic camera paremeters.
Is this possible via e.g SURF/ORB? anyone has experience with it? and can recommend some tutorials, preferably in Pyhton.
Related
I wanted to capture a pair of stereo images with my iPhone camera. I wanted to know how could I do it. What ai have tried so far is, clicked one photo moved my phone a little to the right (almost 1.5 inches), but I did not get the desired results.
I want to rectify those images before I could apply opencv's StereoBM_create function to them. But I can't capture proper stereo images.
I have a custom USB camera with a custom driver on a custom board Nvidia Jetson TX2 that is not detected through openpose examples. I access the data using GStreamer custom source. I currently pull frames into a CV mat, color convert them and feed into OpenPose on a per picture basis, it works fine but 30 - 40% slower than a comparable video stream from a plug and play camera. I would like to explore things like tracking that is available for streams since Im trying to maximize the fps. I believe the stream feed is superior due to better (continuous) use of the GPU.
In particular the speedup would come at confidence expense and would be addressed later. 1 frame goes through pose estimation and 3 - 4 subsequent frames are just tracking the object with decreasing confidence levels. I tried that on a plug and play camera and openpose example and the results were somewhat satisfactory.
The point where I stumbled is that I can put the video stream into CV VideoCapture but I do not know, however, how to provide the CV video capture to OpenPose for processing.
If there is a better way to do it, I am happy to try different things but the bottom line is that the custom camera stays (I know ;/). Solutions to the issue described or different ideas are welcome.
Things I already tried:
Lower resolution of the camera (the camera crops below certain res instead of binning so cant really go below 1920x1080, its a 40+ MegaPixel video camera by the way)
use CUDA to shrink the image before feeding it to OpenPose (the shrink + pose estimation time was virtually equivalent to the pose estimation on the original image)
since the camera view is static, check for changes between frames, crop the image down to the area that changed and run pose estimation on that section (10% speedup, high risk of missing something)
I am studying about visual odometry and watched Prof. Dr. Cyrill Stachniss' video recordings which are available as YouTube 2015/16 Playlist about Photogrammetry I & II .
First, If I want to create my own dataset (like KITTI dataset for VO or like Oxford campus dataset) what should be the properties of the image that I take with a camera.
Are they just images? Or, does they have some special properties ? That is, how can I create my own dataset with a monocular or stereo camera.
Thank you.
To get extrinsic and intrinsic parameters from the image you must have a set of images of known shape from varying views. It's not trivial task to do on your own, by common CV libraries / solution have a built-in utilities for camera calibration (I have to deal with OpenCV library and Matlab CV package and they are generally the same).Usually it's done with a black and white checkboard or another simple geometric pattern.
Then with known camera parameters you can manipulate your own dataset.
Matlab camera calibration reference
OpenCV camera calibration tutorials
If you want to benchmark some visual odometry algorithms with your dataset, you will definitely need the intrinsic parameters of your camera as well as its pose.
As said in #f4f answer, the intrinsic calibration is typically done with some images of a checkerboard that you tilt and rotate (see opencv).
This will give you parameters such as focal length, optical center but also the distortion coefficients which can be important depending on your camera.
Getting the pose of the camera (i.e extrinsic parameters) at each frame is probably trickier. Usually the ground-truth is obtained using information from additional sensors (tracking system, IMU, GPS, ...). You can have a look at : TUM RGB-D SLAM Dataset and the corresponding paper. They explain how they used a motion-capture system to get the ground-truth pose.
Recording the time of acquisition of the camera frames can also be interesting (one timestamp per frame).
Creating your own visual odometry dataset is not trivial. If you just want to create a dataset "for fun" or to do some experiments and if you have only a camera available, I would say you can just try some methods that are known to work well (like ORB-SLAM). This will give you good approximate of the camera poses (you may have to manually fix the unknown scale).
I use libvlc with vlc-qt to load, modify and show various streams and videos. It works well with all videos and streams which have top-left orientation. I have a video created with a smartglass, and it has top-right orientation.
When i opened this video with the vlc media player, it showed correctly, but when i loaded it to my program, it was upside down (because of the orientation).
How can i set it in vlc-qt/libvlc to automatically adjust the frames to the orientation? Based on the vlc media player, it must be possible somehow.
If it is not possible, i would be content with knowing how to get the video orientation from libvlc.
i would be content with knowing how to get the video orientation from libvlc.
libvlc_video_get_track returns a struct containing a field with orientation info.
Don't think you can rotate the video from libvlc API, you will need to provide CLI arguments to VLC through your wrapper/libvlc.
See https://wiki.videolan.org/VLC_command-line_help/
Video transformation filter (transform)
Rotate or flip the video
--transform-type={90,180,270,hflip,vflip,transpose,antitranspose}
I am building a stereo vision system, but when I start my videoCapture with opencv, the frame is zoomed. Instead of showing the whole face with background for example, it only shows a big zoom to the chin.
It is USB camera bought on Amazon, just a PCB with lens and USB cable. I can only set the focus with a screw on the lens.
Any ideas how I can solve this?