We're working on a project using OpenCV 2.4.6 and Qt 5.1.1 in C++. We have to load images for image processing at several points in our code, which we did using cv::imread, as normal. However, we wanted to make software compatible with other language filesystems, and found that having file paths with foreign characters would fail to load.
The problem, we believe, has to do with the fact that imread can only take in a std::string (or char*), and casting a path with non Latin-1 symbols to a std::string results in characters that use multiple bytes in UTF-8 (the encoding used for QString, which is how we store the paths) being converted to multiple chars.
To confirm that the file paths are valid, we've opened them by passing a wstring to a regular ifstream, which successfully opens the file and reads bits. Our current hack is to load the image as a QImage and then copy the data to a cv::Mat, but this isn't a satisfying solution, for multiple reasons, chiefly that as we understand it, Qt::QImage loads images in 8bit format, and our images are of a higher bit depth.
Is there any "clean" way to get around this? I saw this questions, but toAscii is deprecated, and its replacements didn't work for us. We've tried the following ways of converting the QString to a std::string and passing them to imread.
QString::toStdString(), QString::toUtf8().data(), QString::toLocal8Bit().data(), QString::toLatin1().data(). They all appear to yield roughly the same results.
Thanks in advance.
You can try QString::toStdWString() and then convert the resulting std::wstring to std::string.
Related
I wrote a C++ program to retrieve some text data from MS-Active Directory and save them into a Sqlite3 database, however I have a problem, the utf-8 encoding.
According to some readings, data from active directory is UTF-8 encoded, but when reading from C++, it treats as a “wide char” (wchar_t) that Sqlite3 (which is utf-8 default) does not accept as UTF-8 because it uses only “char” into the parameter of its “sqlite3_bind_text” unless I use sqlite3_bind_text16, but I do not wish to do it because it increases the size of database.
I tried to convert from "wchar_t" to "char" using the function "wcstombs_s", but resulting data are not correct.
I read that the only way would be to use MultiByteToWideChar or WideCharToMultiByte, but I didn’t give a try because I read that the cost of convertion is quite expensive.
I would like to know if anybody of you had a similar situation and found a clean and effective solution for this matter.
Many Thanks!
Using sqlite3_bind_text16 on a database created with the UTF-8 encoding doesn't increase its size, the strings are converted on the fly to UTF-8.
See the paragraph "Support for UTF-8 and UTF-16" on this page.
So I see samples all around on saving into file. But I wonder if it is possible to save into char* or string instead of file - so to say keep it in memory?
There doesn't seem to be anything to facilitate this in boost itself. All I/O seems to be based on supplying filenames.
However, there seems to be an extension here called io_new that has streams based I/O.
See documentation here for an example (search for "Reading And Writing In-Memory Buffers").
edit:
Sorry, I guess my question was vague. I'd like to have a way to check if a file is not an image without wasting time loading the whole image, because then I can do the rest of the loading later. I don't want to just check the file extension.
The application just views the images. By 'checking the validity', I meant 'detecting and skipping the non-image files' also in the directory. If the pixel data is corrupt, I'd like to still treat it as an image.
I assign page numbers and pair up these images. Some images are the single left or right page. Some images are wide and are the "spread" of the left and right pages. For example, pagesAt(3) and pagesAt(4) could return the same std::pair of images or a std::pair of the same wide image.
Sometimes, there is an odd number of 'thin' images, and the first image is to be displayed on its own, similar to a wide image. An example would be a single cover page.
Not knowing which files in the directory are non-images means I can't confidently assign those page numbers and pair up the files for displaying. Also, the user may decide to jump to page X, and when I later discover and remove a non-image file and reassign page numbers accordingly, page X could appear to be a different image.
original:
In case it matters, I'm using c++ and QImage from the Qt library.
I'm iterating through a directory and using the QImage constructor on the paths to the images. This is, of course, pretty slow and makes the application feel unresponsive. However, it does allow me to detect invalid image files and ignore them early on.
I could just save only the paths to the images while going through the directory and actually load them only when they're needed, but then I wouldn't know if the image is invalid or not.
I'm considering doing a combination of these two. i.e. While iterating through the directory, reading only the headers of the images to check validity and then load image data when needed.
So,
Will just loading the image headers be much faster than loading the whole image? Or is doing a bit of i/o to read the header mean I might as well finish off loading image in full? Later on, I'll be uncompressing images from archives as well, so this also applies to uncompressing just the header vs uncompressing the whole file.
Also, I don't know how to load/read just the image headers. Is there a library that can read just the headers of images? Otherwise, I'd have to open each file as a stream and code image header readers for all the filetypes on my own.
The Unix file tool (which has been around since almost forever) does exactly this. It is a simple tool that uses a database of known file headers and binary signatures to identify the type of the file (and potentially extract some simple information).
The database is a simple text file (which gets compiled for efficiency) that describes a plethora of binary file formats, using a simple structured format (documented in man magic). The source is in /usr/share/file/magic (in Ubuntu). For example, the entry for the PNG file format looks like this:
0 string \x89PNG\x0d\x0a\x1a\x0a PNG image
!:mime image/png
>16 belong x \b, %ld x
>20 belong x %ld,
>24 byte x %d-bit
>25 byte 0 grayscale,
>25 byte 2 \b/color RGB,
>25 byte 3 colormap,
>25 byte 4 gray+alpha,
>25 byte 6 \b/color RGBA,
>28 byte 0 non-interlaced
>28 byte 1 interlaced
You could extract the signatures for just the image file types, and build your own "sniffer", or even use the parser from the file tool (which seems to be BSD-licensed).
Just to add my 2 cents: you can use QImageReader to get information about image files without actually loading the files.
For example with the .format method you can check a file's image format.
From the official Qt doc ( http://qt-project.org/doc/qt-4.8/qimagereader.html#format ):
Returns the format QImageReader uses for reading images. You can call
this function after assigning a device to the reader to determine the
format of the device. For example: QImageReader reader("image.png");
// reader.format() == "png" If the reader cannot read any image from
the device (e.g., there is no image there, or the image has already
been read), or if the format is unsupported, this function returns an
empty QByteArray().
I don't know the answer about just loading the header, and it likely depends on the image type that you are trying to load. You might consider using Qt::Concurrent to go through the images while allowing the rest of the program to continue, if it's possible. In this case, you would probably initially represent all of the entries as an unknown state, and then change to image or not-an-image when the verification is done.
If you're talking about image files in general, and not just a specific format, I'd be willing to bet there are cases where the image header is valid, but the image data isn't. You haven't said anything about your application, is there no way you could add in a thread in the background that could maybe keep a few images in ram, and swap them in and out depending on what the user may load next? IE: a slide show app would load 1 or 2 images ahead and behind the current one. Or maybe have a question mark displayed next to the image name until the background thread can verify that validity of the data.
While opening and reading the header of a file on a local filesystem should not be too expensive, it can be expensive if the file is on a remote (networked) file system. Even worse, if you are accessing files saved with hierarchical storage management, reading the file can be very expensive.
If this app is just for you, then you can decide not to worry about those issues. But if you are distributing your app to the public, reading the file before you absolutely have to will cause problems for some users.
Raymond Chen wrote an article about this for his blog The Old New Thing.
I need to get the image dimensions of a JPEG in C++. I'm looking for either a fairly simple way to do it or a smallish library that provides that functionality. I'm working in C++ on OpenVMS, so any external libraries may have to be adapted to compile on our systems - so please don't post me links to big, closed source libraries!
Has anyone come across anything that might do the trick, or understand the JPEG file format (I think I probably mean the JFIF file format here) to tell me how I might go about rolling my own solution?
You have this C function which may extract the relevant data for you.
This is a C routine but should compile fine with C++.
Pass it a normal FILE pointer (from fopen) to the beginning of a jpeg file and two int pointers to be set with the image height and width.
Or you may find in the Boost library a jpeg class which has the right function (From Adobe Generic Image Library).
jpeg_read_dimensions
boost::gil::jpeg_read_dimensions (const char *filename)
Returns the width and height of the JPEG file at the specified location. Throws std::ios_base::failure if the location does not correspond to a valid JPEG file.
libjpeg is reasonably small, open source and available on OpenVMS. It's probably quicker to install it than to handle JPEG yourself.
Maybe libjpeg?
You should be able to use this jpeg lib with this patch for OpenVMS
No need for full libjpeg library just to get this information (unless you need to do something else with the images). ImageInfo might help you. It is a Java class, but there are ports for other languages, including C++.
As pointed out, Exif might change these information (eg. with orientation setting).
You may want to try GDAL library which serves as an abstraction layer for large number of raster data formats, mostly used in geospatial applications for GIS/RS.
GDAL provides number of APIs, for C, C++ and scripting languages. Of course, it supports JPEG images and its variants like JPEG2000 and more.
Here is a very simple example how to open JPEG image and query its dimensions:
#include <gdal_priv.h>
GDALAllRegister(); // call ones in your application
GDALDataset* ds = (GDALDataset*)GDALOpen("my.jpeg", GA_ReadOnly);
int width = ds->GetRasterXSize();
int height = ds->GetRasterYSize(),
int nbands = ds->GetRasterCount();
Check GDAL API tutorial for more complete example.
I want to decode JPEG files and obtain uncompressed decoded output in BMP/RGB format.I am using GNU/Linux, and C/C++.
I had a look at libjpeg, but there seemed not to be any good documentation available.
So my questions are:
Where is documentation on libjpeg?
Can you suggest other C-based jpeg-decompression libraries?
The documentation for libjpeg comes with the source-code. Since you haven't found it yet:
Download the source-code archive and open the file libjpeg.doc. It's a plain ASCII file, not a word document, so better open it in notepad or another ASCII editor.
There are some other .doc files as well. Most of them aren't that interesting though.
Unfortunately I cannot recommend any other library besides libjpeg. I tried a couple of alternatives, but Libjpeg always won. Is pretty easy to work with once you have the basics done. Also it's the most complete and most stable jpeg library out there.
MagickWand is the C API for ImageMagick:
http://imagemagick.org/script/magick-wand.php
I have not used it, but the documentation looks quite extensive.
You should check out Qt's QImage. It has a pretty easy interface that makes this task really easy. Setup is pretty simple for every platform.
If Qt is overkill, you can try Magick++ http://www.imagemagick.org/Magick++/. It supports similar operations and is also well suited for that sort of task. The last time I used it, I struggled a bit with dependencies for it on Windows, but don't recall much trouble on Linux.
For Magick++'s Image class, the function you probably want is getConstPixels.
I have code that you can copy ( or just use as a reference ) for loading a jpeg image using the libjpeg library.
You can browse the code here: http://code.google.com/p/kgui/source/browse/trunk/kguiimage.cpp
Just look for the function LoadJPGImage.
The code is setup to handle c++ binding of my DataHandle class to it for loading the image, that way the image can be a file or data already in memory or whatever.
A slightly out of the box solution is to acquire a copy of the netpbm tools, which transform images from pretty much any format to any other format via one of several very simple intermediate formats. They work well from the shell, and are most often used in pipes to read some arbitrary image, perform an operation on it, and write it out to some other format.
The pbm formats can be as simple as a plain ASCII header followed by the RGB data in ASCII or binary. They are intended to be simple enough to use without required a library to implement.
JPEG is supported in netpbm by read and write filters that are implemented on top of libjpeg.