I'm trying to write a TCP client/server application that transmits objects containing OpenCv Mat. I'd like to serialize these objects using JSON. I found some libraries that help me in doing that (rapidjson), but they of course do not take into account images as object members.
What would you suggest to serialize in a JSON object a cv::Mat variable? How can I use RapidJson, for example, to achieve that?
imencode can be used to encode an viewable image (with CV_8UC1 or CV_8UC3 pixel formats) into a std::vector<uchar>. Link to documentation.
The vector<uchar> will contain the same bytes as if OpenCV had saved the image into one of the supported image file formats (such as JPEG or PNG) and then have the file bytes loaded back into a byte array.
imencode can be found in highgui module when using OpenCV 2.x, or imgcodecs module when using OpenCV 3.x.
With the compressed data in a vector<uchar>, you can use Base64 encoding to format it into a string, which can then be added as a JSON value inside a JSON object.
When using JSON to transmit large amounts of data, consider very very carefully the character encoding format that the JSON library is instructed to emit. Normally, If a large portion of the data is going to be Base64, you will want to make sure the JSON is emitted in UTF8.
If you have the option of sending in binary (which requires an "out-of-band" design in the web service, something not always doable), it should be seriously considered.
When considering different serialization choices for images, these things should be taken into account:
Typical image sizes (total number of pixels)
Size efficiency is less of a concern if images are small.
Pixel format (number of channels and precision)
Most common image file formats will only allow 8-bit grayscale and 24-bit RGB pixel data. Trying to save higher-precision pixel data into these image formats will result in partial loss of precision.
Available transmission bandwidth (if it is scarce enough to be a concern). With less available bandwidth, compression becomes more important.
Compression options.
Typical (photographic or synthetic) images are highly compressible due to the common sense that images that are too "dense" will be too hard to comprehend when viewed by human eyes.
Compression can be lossless or lossy.
Choice of compression may depend on the statistical characteristics of the pixel values (image content).
As mentioned above, if compression is performed by encoding into some image formats, you have to make sure the image format can satisfy the pixel value precision requirements of your application.
If no existing image format meets your requirements and you still want to perform lossless compression, consider using the zlib API that is integrated into the OpenCV Core module.
If you are good at image processing and data compression theory, you may be able to devise an application-specific compression method based on your own needs.
Remember that reducing the image resolution can be a powerful (and super-lossy) way of reducing the transmission file size. Consider carefully what minimum image resolution is actually needed for your application.
Other considerations
Binary or text
Endianness
Availability of highgui, imgcodecs or an image decoder for the chosen image format on the receiving end.
Information source: just did this a few months ago.
Related
I am writing an application that generates a huge amount of images. Each frame is 1280x800 pixels large and has 1 byte per pixel for color information (greyscale). Each of the frames must be written to disk.
Currently I simply dump the raw pixel data to a binary file on the disk. The file can then be viewed with a special viewer I also created.
This is a very unsatisfactory solution, since the images can't be viewed/processed directly. They always have to run through my custom viewer/converter.
Is there an image format I could use to write my images to disk that:
Is fast to be written (no compression etc.)
Does not increase the final file's size much
Supports dumping my raw pixel buffer in there (no alignemnt changes etc.)
Can be read by common applications (Windows Explorer, Paint, Photoshop etc.)
I already tried to use .png, but the file generation takes much too long due to the compression.
Have a look at the binary Portable GrayMap (P5) format. It consists of an extremely simple header followed by raw image data (without any alignment requirements), and is widely supported by image viewers.
Both bmp and tiff can be used to save raw data. Bmp has the oddity of having image upside down, unless height is negative. And tiff has plenty of encoding options. It should be anyway feasible to reverse engineer the format to be used as a template, where the image data is copy pasted. So no need to use a library: just a header, image data and an optional footer concatenated.
I want to write a script to extract the PixelDATA of a DICOM file using c or c ++, I don't want to use external libraries like dicomsdl... if anyone can help me to write algorithm for extract and show image .
Just extracting the image data under the pixel data is not enough to interpret the DICOM image properly. You will need other attributes from DICOM file such as Rows, Columns, Bit Allocated, Bit Stored, High Bit, Photometric Interpretation, Sample Per Pixel to Number of Frames information just to interpret the raw uncompressed image data. Also, stored image data can be in Little Endian or Big Endian byte order. In addition, image data can be encapsulated or compressed (e.g. compressed using different compression algorithms such as JPEG, JPEG 2000, JPEG LS, RLE etc)) and compressed streams are stored differently than the uncompressed image data. Even the PixelData element can exist in multiple locations in a single DICOM file (e.g. one under the Icon Image Sequence (thumbnail) and one at the top level (actual image).
It can get more complicated when you need to account for Palette Color (segmented vs un-segmented), modality LUT, VOI LUT etc. My recommendation is to use an existing DICOM SDK and there are many open source and commercial SDK available for different platforms and programming environments.
In order to accomplish some specific editing on some .avi files, I'd like to create an application (in C++) that is able to load, edit, and save those .avi files. But, what is the most efficient way? When first thinking about it, a simple 3D-Array containing a 2D-array of pixels for every frame seems the simplest solution; But then its size would be ENORMOUS. I mean, let's assume that a pixel only needs a color. One color would mean 3bytes (1char r, 1char b, 1char g). If I now have a 1920x1080 video format, this would mean 2MEGABYTES for only one frame! This data may or may not be smaller if using pointers for the colors, so that alreay used colors wont take more size - I don't really know, since I'm pretty new to C++ and the whole low-level stuff. (As a comparison: One of my AVI files recorded with Xvid codec is 40seconds long, 30fps, and only has 2MB.)
So how would you actually store the video data (Not even the audio, just the video) efficiently (while still being easily able to perform per-frame-changes on it)?
As you have realised, uncompressed video is enormous and it is not practical to store an entire video in this way.
Video compression is an extremely complex topic, but more-or-less, it works as follows: certain "key-frames" are compressed using fairly standard compression techniques similar or identical to still-photo compression such as JPEG. Frames following key-frames are compressed by comparing the frame with the previous one and looking for changes (such as moving blocks). Every now and again, a new key-frame is used.
You don't really have to worry much about that as you are not going to write your own video coder/decoder (codec). There are standard ones.
What will happen is that your program will decode the compressed video frame-by-frame and keep a certain number of frames in memory while you are working on them and then re-encode them when it is finished. In the uncompressed form, you will have access to the individual pixels and can work on them how you want.
You are probably not going to do that either by yourself - it is very hard. You probably need to use a framework, such as OpenCV. There are a huge number of standard filters and tools built in to these frameworks, and it may be that what you want to do is already implemented somewhere.
The OpenCV framework can return individual frames in a Mat object and you can then access the pixels. See this post Get Pixels from Mat
OpenCV
Tutorial page: Open CV Tutorial
I am trying to implement a steganographic algorithm where hidden message could survive jpeg compression.
The typical scenario is the following:
Hide data in image
Compress image using jpeg
The hidden data is not destroyed by jpeg compressiona nd could be restored
I was trying to use different described algorithms but with no success.
For example I was trying to use simple repetition code but the jpeg compression destroyed hidden data. Also I was trying to implementt algorithms described by the following articles:
http://nas.takming.edu.tw/chkao/lncs2001.pdf
http://www.securiteinfo.com/ebooks/palm/irvine-stega-jpg.pdf
Do you know about any algorithm that actually can survive jpeg compression?
You can hide the data in the frequency domain, JPEG saves information using DCT (Discrete Cosine Transform) for every 8x8 pixel block, the information that is invariant under compression is the highest frequency values, and they are arranged in a matrix, the lossy compression is done when the lowest coefficients of the matrix are rounded to 0 after the quantization of the block, these zeroes are arranged in the low-right part of the matrix and that is why the compression works and the information is lost.
Quite a few applications seem to implement Steganography on JPEG, so it's feasible:
http://www.jjtc.com/Steganography/toolmatrix.htm
Here's an article regarding a relevant algorithm (PM1) to get you started:
http://link.springer.com/article/10.1007%2Fs00500-008-0327-7#page-1
Perhaps the answer is late,but ...
You can do it in compressed domain steganography.Read image as binary file and analysis this file with libs like JPEG Parser. Based on your selected algorithm, find location of venues and compute new value of this venue and replace result bits in file data. Finally write file in same input extension.
I hope I helped.
What you're looking for is called watermarking.
A little warning: Watermarking algorithms use insane amounts of redundancy to ensure high robustness of the information being embedded. That means the amount of data you'll be able to hide in an image will be orders of magnitude lower compared to standard steganographic algorithms.
I'm trying to use OpenCV to read/write images for me. Currently, I have them in a different, non-standard format, and I know how to get them into OpenCV's containers. Here are the requirements:
The pixels are 1, or 3 bands, U8, U16, U32, or F32
The images have metadata, random stuff, like the camera ID that took the images. I would like the metadata to be vi/notepad editable
I want to write as little code as possible when it comes to low level stuff. My experience is that this stuff requires the most maintenance.
I can define the format. It's only to read and write for these programs.
I don't want the pixels to be anything but binary, '0.5873499082' is way too much data for one float.
Is there a way to describe to OpenCV how to read and write image types it doesn't know? Are there image types already available for the types of images I have?
My interim solution is to use boost to serialize the image, and save the metadata in a separate file.
Try using gdal library for reading images and then convert it to IplImage.
OpenCV can't do that for you, you can store the metadata in a separate file, or you can use for example the jpeg exif (that won't be notepad editable though).