I am using a Basler camera and want to write images that I grabbed at 1000x1000px to disk as fast as possible.
Keeping a movie container open and saving as .avi doesn't really help (slows down 2x) and saving single images is even slower.
I am currently using openCV cvVideoCreator and imwrite functions.
My computer should be fast enough (Intel Xeon 3.6 GHz, 64GB RAM, SSD).
Any ideas?
There are multiple factors (steps) you may want to consider in order to reach your goal (saving color images (1K x 1K x 3) with 50 FPS or 150 MB/s).
Image encoding: most of well-known image formats such as png, tif, jpg takes time to encode image (e.g., 5.7 ms for png and 11.5 ms for tif in OpenCV with 3.6 GHz 4-core CPU) to the specific format even before saving the encoded format data to a disk.
File opening: Independent of file size, this may take time (e.g., 1.5 ms on 3.5" WD Black)
File writing: Dependent of file size, this may take time (e.g., 2.0 ms on 3.5" WD Black)
File closing: Dependent of file size, this may take a lot of time (e.g., 14.0 ms on 3.5" WD Black)
This means you have to finish all of the steps in 20 ms per image for your goal, and as I gave examples of timing, you may not be able to achieve your goal with OpenCV imwrite in JPG because the library sequentially does all the steps above in a single thread.
I think you have a couple of options
imwrite into BMP format on SSD as its encoding time is virtually zero (e.g., less than 0.3 ms)
do some of steps above (e.g., encoding or file closing) in a separate thread.
Especially, file closing is a good candidate to be run in a separate thread because it can be asynchronously done with the other steps. I was able to reach 400 MB/s bandwidth of saving with the second option, BMP file format, and a better hard disk.
Hope this helps you and others with similar goals.
The specs you state in your question are related to your ability to process and buffer the data, but not about the speed you can dump to disk.
You're trying to write (some numbers assumed, just replace with your own)
1000*1000 (size) * 4 (data/pixel) * 25 (frame rate) bytes per second.
(or 100M/s)
This is around abouts the limit of a traditional HDD, but if the disk is fragmented or full at all it's unlikely to keep up.
As a result you must find a way to either speed up your write time (switch to SSD for example); reduce the data being written (compress, reduction in colour depth / quality / frame rate) or buffer what you want to write while a background thread saves to disk.
The question you must ask is how long do you plan to record for. If it's not long enough to fill up your RAM, then you have all the options available to you. If however you plan to record for extended periods of time then you will have to pick one of the other 2.
Related
I have an algorithm that generates a video frame by frame, which I am refactoring to stream the images to ffmpeg rather than save them all to disk first.
The algorithm currently generates and saves a bunch of QImages then executes ffmpeg (with QProcess). The new version will generate QImages then stream them out to ffmpeg's stdin via QProcess. I can do all that just fine.
The issue is, I'm not actually generating the images serially, I'm generating them on a thread pool, maybe 12-16 at a time (just using QtConcurrent). I'd like to keep this parallelism for performance reasons, since the operation (even excluding image encoding and file I/O) does take a long time.
So my question is, does ffmpeg have something like image2pipe but that can take the images slightly out of order (let's assume there's some sane maximum offset, +/- 20 or so)?
If not, my solution will be to divide the image set into small batches, then for each batch, run it on the pool, reorder it, and stream them in the correct order; sacrificing some fraction of performance depending on the batch size. I'd rather not do that just because the code right now is dirt simple, and simple is good. So I'm wondering if there's a way to get ffmpeg to accept the images out of order.
They're at a fixed frame rate, if that is useful (maybe there's some way to set PTS's or something, then I just find a codec + container that can store out-of-order frames).
I want to create the vector of matrices to stores as many images as possible.
I know that,it is possible as written below:
vector<Mat> images1;
and during the image acquisition from the camera and i would save the images at 100fps with resolution of 1600*800 as below:
images1.push_back(InputImage.clone());
Where InputImage is the Mat and given by the camera. Since creating video during the acquisition process either leads to frame missing in the video or reduction in aquisition speed.
Later after stopping the image acquisition and before stopping the program, I would write the images into video as written below:
VideoWriter writer;
writer = Videowriter("video.avi",-1,100,frameSize(1600,800),false);
for (vector<Mat>::iterator iter = images1.begin(); ier != images1.end(); iter++)
writer.write(*iter);
Is it correct, since I am not sure the images1 can store the images around 1500 images without overflow.
You don't really have to worry about "overflow", whatever that means in your context.
The bigger problem is memory. A single frame takes (at 8 bits per color, with 3 colors) 3 * 1600 * 800 == 3.84Mb. At 100fps, One second of footage requires 0.384Gb of memory. 8GB of memory can only hold about 20 seconds of footage. You'll need almost 24GB of memory before you can hold a whole minute. There's a reason that the vast, vast, vast majority of Video Encoding Software only keeps a few frames of video data in memory at any given time, and dumps the rest to the hard drive (or discards it, depending on what purpose the software is serving).
What you should probably be doing (which is what programs like FRAPS do) is dumping frames to the hard drive as soon as soon as you receive them. Then, when recording finishes, you can either call it a day (if raw video footage is what you need) or you can begin a process of reading the file and encoding it into a more compressed format.
Pre-allocate your image vector in memory so that you just need to copy the frames without real-time allocation.
If you have memory problems, try dumping the frames to a file, the OS will hopefully be able to handle the I/O. If not try memory mapped files.
I am making a game with a large number of sprite sheets in cocos2d-x. There are too many characters and effects, and each of them use a sequence of frames. The apk file is larger than 400mb. So I have to compress those images.
In fact, each frame in a sequence only has a little difference compares with others. So I wonder if there is a tool to compress a sequence of frames instead of just putting them into a sprite sheet? (Armature animation can help but the effects cannot be regarded as an armature.)
For example, there is an effect including 10 png files and the size of each file is 1mb. If I use TexturePacker to make them into a sprite sheet, I will have a big png file of 8mb and a plist file of 100kb. The total size is 8.1mb. But if I can compress them using the differences between frames, maybe I will get a png file of 1mb and 9 files of 100kb for reproducing the other 9 png files during loading. This method only requires 1.9mb size in disk. And if I can convert them to pvrtc format, the memory required in runtime can also be reduced.
By the way, I am now trying to convert .bmp to .pvr during game loading. Is there any lib for converting to pvr?
Thanks! :)
If you have lots of textures to convert to pvr, i suggest you get PowerVR tools from www.imgtec.com. It comes with GUI and CLI variants. PVRTexToolCLI did the job for me , i scripted a massive conversion job. Free to download, free to use, you must register on their site.
I just tested it, it converts many formats to pvr (bmp and png included).
Before you go there (the massive batch job), i suggest you experiment with some variants. PVR is (generally) fat on disk, fast to load, and equivalent to other formats in RAM ... RAM requirements is essentially dictated by the number of pixels, and the amount of bits you encode for each pixel. You can get some interesting disk size with pvr, depending on the output format and number of bits you use ... but it may be lossy, and you could get artefacts that are visible. So experiment with limited sample before deciding to go full bore.
The first place I would look at, even before any conversion, is your animations. Since you are using TP, it can detect duplicate frames and alias N frames to a single frame on the texture. For example, my design team provide me all 'walk/stance' animations with 5 pictures, but 8 frames! The plist contains frame aliases for the missing textures. In all my stances, frame 8 is the same as frame 2, so the texture only contains frame 2, but the plist artificially produces a frame8 that crops the image of frame 2.
The other place i would look at is to use 16 bits. This will favour bundle size, memory requirement at runtime, and load speed. Use RGBA565 for textures with no transparency, or RGBA5551 for animations , for examples. Once again, try a few to make certain you get acceptable rendering.
have fun :)
I am building an application which takes a great many number of screenshots during the process of "recording" operations performed by the user on the windows desktop.
For obvious reasons I'd like to store this data in as efficient a manner as possible.
At first I thought about using the PNG format to get this done. But I stumbled upon this: http://www.olegkikin.com/png_optimizers/
The best algorithms only managed a 3 to 5 percent improvement on an image of GUI icons. This is highly discouraging and reveals that I'm going to need to do better because just using PNG will not allow me to use previous frames to help the compression ratio. The filesize will continue to grow linearly with time.
I thought about solving this with a bit of a hack: Just save the frames in groups of some number, side by side. For example I could just store the content of 10 1280x1024 captures in a single 1280x10240 image, then the compression should be able to take advantage of repetitions across adjacent images.
But the problem with this is that the algorithms used to compress PNG are not designed for this. I am arbitrarily placing images at 1024 pixel intervals from each other, and only 10 of them can be grouped together at a time. From what I have gathered after a few minutes scanning the PNG spec, the compression operates on individual scanlines (which are filtered) and then chunked together, so there is actually no way that info from 1024 pixels above could be referenced from down below.
So I've found the MNG format which extends PNG to allow animations. This is much more appropriate for what I am doing.
One thing that I am worried about is how much support there is for "extending" an image/animation with new frames. The nature of the data generation in my application is that new frames get added to a list periodically. But I do have a simple semi-solution to this problem, which is to cache a chunk of recently generated data and incrementally produce an "animation", say, every 10 frames. This will allow me to tie up only 10 frames' worth of uncompressed image data in RAM, not as good as offloading it to the filesystem immediately, but it's not terrible. After the entire process is complete (or even using free cycles in a free thread, during execution) I can easily go back and stitch the groups of 10 together, if it's even worth the effort to do it.
Here is my actual question that everything has been leading up to. Is MNG the best format for my requirements? Those reqs are: 1. C/C++ implementation available with a permissive license, 2. 24/32 bit color, 4+ megapixel (some folks run 30 inch monitors) resolution, 3. lossless or near-lossless (retains text clarity) compression with provisions to reference previous frames to aid that compression.
For example, here is another option that I have thought about: video codecs. I'd like to have lossless quality, but I have seen examples of h.264/x264 reproducing remarkably sharp stills, and its performance is such that I can capture at a much faster interval. I suspect that I will just need to implement both of these and do my own benchmarking to adequately satisfy my curiosity.
If you have access to a PNG compression implementation, you could easily optimize the compression without having to use the MNG format by just preprocessing the "next" image as a difference with the previous one. This is naive but effective if the screenshots don't change much, and compression of "almost empty" PNGs will decrease a lot the storage space required.
Please advise the best way to compress satellite Image. Details
Uncompressed size - 60 gb
Uncompressed format - IMG
4 Bands (To be retained after compression)
Preferred compression format - JPEG2000
Lossy enough to aid in Visual analysis.
Thanks
Monika
If I'm interpreting your question correctly, you'll already need to look at the file format defined in JPEG2000 Part 2 ("JPX") because of the multiple bands. Beyond that, because you are asking for "visually lossless" (e.g. lossy compression but tuned to the point where you can't see it), you'll need to experiment with various settings using your own files until you achieve what you want. For a small discussion on how the Internet Archive did this with print materials, see A Status Report on JPEG 2000 Implementation for Still Images: The UConn Survey.
You should check out the Kakadu JPEG 2000 software. It's really fast.
The summary of one of their time tests, which sounds in line with our observed results:
'"There is an example on the spreadsheet involving a 13.3K x 13.3K RGB image (531 MBytes), being compressed to 2 bits/pixel, at 0.145 pictures/second, using the speedpack on a standard 2.4 GB core-2 duo machine. From this, it can be inferred that compressing 60 MB down to 5 MB (12:1 is 2 bits/pixel for RGB, although colour properties were not specified in the original request) should occur at a rate of 1.2 pictures/second. Compressing to the slightly lower target size of 4 MB will be a little bit faster."
http://www.kakadusoftware.com/