I've got functions (c++) that convert a game image (SDL2 SDL_Surface) from RGB through CIEXYZ to CIEL*a*b* so that adjustments to hue, brightness, saturation, will be more visually natural than in HSV space. That works, with the exception of those pixels that are adjusted out of the RGB gamut in the process.
While it is easy enough to force a value back into gamut by:
individually cropping subpixel values below 0 to 0 and above 255 to 255, or
compressing and moving the whole pixel or whole image into the 0-255 range by dividing by (max-min) and subtracting min/(max-min);
these options lead to gross artifacts when doing multiple operating on the same image. I am looking for the least destructive method of handling out-of-gamut subpixels in code. Digging through many pages of Google results least to hundreds of Photoshop links, a few design oriented links and references to CMSs like LittleCMS.
I need an algorithmic solution to put into c++ code.
Note: Just doing some basic experimentation, using linear compression on the entire image leads to massive loss of brightness over hundreds of iterations with calculations happening as floats. Further insight into the sigmoid compression comment below is most welcome.
The fundamental issue you face is multiple conversions between color spaces. If the conversion isn't lossless, then you will get cumulative artifacts.
The better solution is to maintain all of your imagery in one color space and do all of your manipulation within that color space. Treat conversion as a one-way street, converting a copy to RGB for display. Do not convert back and forth.
Related
My question is very short and very naïve but I found very different answers on Internet. What is the most effective (and actually used in the Computer Vision community) way of normalizing an RGB image.
This question comes from the fact that algorithm like PCA or even contrast normalization are described often in their 2D-versions.
Therefore for whitening/global contrast normalization or whatever methods you like to preprocess images to feed to a statistical method of your liking: do you consider each channel separately or do you reshape the depth-3 thing into a rectangular 2D-array (of depth-1) (and how to do that while preserving structure) do your thing and then split it back to its former shape ?
I think each method has its advantages considering the image as a whole seems more meaningful but applying to each channel separately is more simple.
There is no simple answer to your question.
For most tasks it is sufficient to operate on separate RGB or HSI channels. Most images in image processing are gray scale anyway so most algorithms expect gray scale input.
In most scenarios pre-processing serves the sole purpose of reducing information to the necessary minimum. So an RGB output "back in its former shape" with preserved structures does not exist. At least in my world.
If you want RGB output from RGB input you would implement a function that operates on RGB values. Unless you can reduce the calculations to grayscale and already have the necessary grayscale functions implemented.
How can you reshape an image into a 2D array? Usually an image is a 2D array...
I will make my answer specific to ZCA whitening but I guess it is the same for others:
As the input of the PCA has the shape of a 2D-matrix with (nsamplesxfeatures) dimension. I thought of using the RGB channels as nsamples and the image in those channel flattened as features.
The answer seems to be to use nsamples as nsamples (the numbers of images you have if you have several RGB images) and to use the RGB-image completely flattened as features.
This answer leads me to believe that if you want to normalise an image you should use the general mean of the image and general standard deviation and not to consider each channel separately.
If somebody disagrees he is free to comment, I agree that my question was a bit too broad.
Let's say I have a series of infrared pictures and the task is to isolate human body from other objects in the picture. The problem is a noise from other relatively hot objects like lamps and their 'hot' shades.
Simple thresholding methods like binary and/or Otsu didn't give good results on difficult (noisy) pictures, so I've decided to do it manually.
Here are some samples
The results are not terrible, but I think they can be improved. Here I simple select pixels by hue value of HSV. More or less, hot pixels are located in this area: hue < 50, hue > 300. My main concern here is these pink pixels which sometimes are noise from lamps but sometimes are parts of human body, so I can't simply discard them without causing significant damage to the results: e.g. on the left picture this will 'destroy' half of the left hand and so on.
As the last resort I could use some strong filtering and erosion but I still believe there's a way somehow to told to OpenCV: hey, I don't need these pink areas unless they are part of a large hot cluster.
Any ideas, keywords, techniques, good articles? Thank in advance
FIR data is presumably monotonically proportional (if not linear) to temperature, and this should yield a grayscale image.
Your examples are colorized with a color map - the color only conveys a single channel of actual information. It would be best if you could work directly on the grayscale image (maybe remap the images to grayscale).
Then, see if you can linearize the images to an actual temperature scale such that the pixel value represents the temperature. Once you do this you can should be able to clamp your image to the temperature range that you expect a person to appear in. Check the datasheets of your camera/imager for the conversion formula.
This question already has answers here:
Closed 10 years ago.
Possible Duplicate:
Is there a way to detect if an image is blurry?
How to calculate blurness and sharpness of a given image usig opencv? Is there any functions there in opencv to do it? If there is no functions in opencv how can I implement it? nay ideas would be great..
The input will be an image and the output should be the blurness and sharpness of the image.
I recommend you to make a frequential analysis of the image. Energy in high band will tell you that the image is quite sharpened, while energy in low band usually means that image is blurry. For computing spectrum, you can use FFTW library.
Regards,
I don't know about opencv.
If I were trying to get an approximate measurement of where an imagine is on the sharp-to-blurry spectrum, I'd start from the observation that the sharpness of parts of an image is evident from the contrast between adjacent pixels - something like max(c1 * abs(r1 - r2), c2 * abs(g1 - g2), c3 * abs(b1 - b2)) where c1-3 weigh perceptual importance of each of the red, green and blue channels, and the two pixels are (r1,g1,b1) and (r2,g2,b2)).
Many tweaks possible, such as raising each colour's contribution to a power to emphasise changes at the dark (power <1)or bright (power >1) end of the brightness scale. Note that the max() approach considers sharpness for each colour channel separately: a change from say (255,255,255) to (0,255,255) is very dramatic despite only one channel changing.
You may find it better to convert from RBG to another colour representation, such as Hue/Saturation/Value (there'll be lots of sites online explaining the HSV space, and formulas for conversions).
Photographically, we're usually interested in knowing that the in-focus part of the image is sharp (foreground/background blur/bokeh due to shallow depth of field is a normal and frequently desirable quality) - the clearest indication of that is high contrast in some part of the image, suggesting you want the maximum value of adjacent-pixel contrasts. That said, some focused pixtures can still have very low local contrasts (e.g. a picture of a solid coloured surface). Further, damaged pixel elements on the sensor, dirt on the lens/sensor, and high-ISO / long-exposure noise may all manifest as spots of extremely high contrast. So the validity of your result's always going to be questionable, but it might be ball-park right a useful percentage of the time.
I want to upsample an array of captured (from webcam) OpenCV images or corresponding float arrays (Pixel values don't need to be discrete integer). Unfortunately the upsampling ratio is not always integer, so I cannot figure myself how to do it with simple linear interpolation.
Is there an easier way or a library to do this?
Well, I dont know a library to to do framerate scaling.
But I can tell you that the most appropriate way to do it yourself is by just dropping or doubling frames.
Blending pictures by simple linear pixel interpolation will not improve quality, playback will still look jerky and even also blurry now.
To proper interpolate frame rates much more complicated algorithms are needed.
Modern TV's have build in hardware for that and video editing software like e.g. After-Effects has functions that do it.
These algorithms are able to create in beetween pictures by motion analysis. But that is beyond the range of a small problem solution.
So either go on searching for an existing library you can use or do it by just dropping/doubling frames.
The ImageMagick MagickWand library will resize images using proper filtering algorithms - see the MagickResizeImage() function (and use the Sinc filter).
I am not 100% familiar with video capture, so I'm not sure what you mean by "pixel values don't need to be discrete integer". Does this mean the color information per pixel may not be integers?
I am assuming that by "the upsampling ratio is not always integer", you mean that you will upsample from one resolution to another, but you might not be doubling or tripling. For example, instead of 640x480 -> 1280x960, you may be doing, 640x480 -> 800x600.
A simple algorithm might be:
For each pixel in the larger grid
Scale the x/y values to lie between 0,1 (divide x by width, y by height)
Scale the x/y values by the width/height of the smaller grid -> xSmaller, ySmaller
Determine the four pixels that contain your point, via floating point floor/ceiling functions
Get the x/y values of where the point lies within that rectangle,between 0,1 (subtract the floor/ceiling values xSmaller, ySmaller) -> xInterp, yInterp
Start with black, and add your four colors, scaled by the xInterp/yInterp factors for each
You can make this faster for multiple frames by creating a lookup table to map pixels -> xInterp/yInterp values
I am sure there are much better algorithms out there than linear interpolation (bilinear, and many more). This seems like the sort of thing you'd want optimized at the processor level.
Use libswscale from the ffmpeg project. It is the most optimized and supports a number of different resampling algorithms.
I have an array of raw rgb data on a 16 bit display with dimension of 320 * 480. The size of the array is 320*480*4 = 6144000.
I would like to know how can I scale this down (80 * 120) without losing image quality?
I found this link about scaling image in 2D array, but how can I apply that to my array of 16 bit display? It is not a 2D array (because of it has 16 bit color).
Image scaling and rotating in C/C++
Thank you.
If you are scaling down a big image to a smaller one, you WILL lose image quality.
The question, then, is how to minimize that loss.
There are many algorithms that do this, each with strengths and weaknesses.
Typically you will apply some sort of filter to your image, such as Bilinear or Nearest Neighbor. Here is a discussion of such filters in the context of ImageMagick.
Also, if the output is going to be less than 16 bits per pixel, you need to do some form of Color Quantization.
I assume that you mean a 16 bit rgb display, not a display that has each color (red, green, and blue) as 16 bits. I also assume you know how your r, g, and b values are encoded in that 16 bit space, because there are two possibilities.
So, assuming you know how to split your color space up, you can now use a series of byte arrays to represent your data. What becomes a tricky decision is whether to go with byte arrays, because you have a body of algorithms that can already do the work on those arrays but will cost you a few extra bits per byte that you may not be able to spend, or to keep everything crammed into that 16 bit format and then do the work on the appropriate bits of each 16 bit pixel. Only you can really answer that question; if you have the memory, I'd opt for the byte array approach, because it's probably faster and you'll get a little extra precision to make the images look smooth(er) in the end.
Given those assumptions, the question is really answerable by how much time you have on your device. If you have a very fast device, you can implement a Lanczos resampling. If you have a less fast device, bicubic interpolation works very well as well. If you have an even slower device, bilinear interpolation is your friend.
If you really have no speed, I'd do the rescaling down in some external application, like photoshop, and save a series of bitmaps that you load as you need them.
There are plenty of methods of scaling down images, but none can guarantee not losing "quality". Ultimately information is lost during the rescaling process.
You have 16bit colors = 2bytes, but in your calculations you use 4 multiplier.
Maybe you don't needed reducing image size?
in general it is impossible to scale raster image without loosing quality. Some algorithms make scaling almost without visible quality loosing.
Since you are scaling down by a factor of 4, each 4x4 block of pixels in your original image will correspond to a single pixel in your output image. You can then loop through each 4x4 block in the original image and then reduce this to a single pixel. A simple way (perhaps not the best way) to do this reduction could be to take the average or median of the RGB components.
You should note that you cannot do image scaling without losing image quality unless for all the blocks in the original image, each pixel is the exact same colour (which is unlikely).