How to get rid of small image artifacts (threshold by size) using ImageMagick? - computer-vision

I am trying to automate image conversion using ImageMagick CLI. One of the biggest problems with my image set is with tiny artifacts that should be cut out.
My images are generally consistent, with big objects (c.a. 50% of image space) on a white background. Unfortunately, sometimes tiny artifacts may just look bad and make trimming less efficient.
E.g. something like that:
In reality, the big object is not a solid color, it's just a simplified example. It is not necessarily a circle either, it can be a square, rectangle, or something irregular.
I cannot also use any morphology like opening, closing, or erosion. Filters like gaussian or median also are out of the question. I need to keep the big object untouched since the highest possible quality is required.
An ideal solution would be something similar to Contours known for example from OpenCV, where I could find all the uniform objects and if they don't meet certain rules (e.g. threshold of size greater than 5% of the whole image) - fill them with white color.
Is there any similar mechanism in ImageMagick CLI? I've gone through the docs and haven't found a suitable solution to my problem.
Thanks in advance!
EDIT (ImageMagick version):
Version: ImageMagick 7.1.0-47 Q16-HDRI x86_64 20393 https://imagemagick.org
Copyright: (C) 1999 ImageMagick Studio LLC
License: https://imagemagick.org/script/license.php
Features: Cipher DPC HDRI Modules OpenMP(5.0)
Delegates (built-in): bzlib fontconfig freetype gslib heic jng jp2 jpeg lcms lqr ltdl lzma openexr png ps raw tiff webp xml zlib
Compiler: gcc (4.2)
EDIT (Real-life example):
As requested, here is a real-life example. A picture of a coin on a white background, but with some artifacts:
noise under the coin (slightly on the left)
dot under the coin (slightly on the right)
gray irregular shape in the top right corner
The objects might not be necessarily circles like coins but we may assume that there always will be one object with a strong border (no white spaces on the border, like here) and the rest is noise.

Here is one way to do that im Imagemagick 7. First threshold the image so the background is white and the object(s) is black. That will likely be image dependent. NOTE: that JPG is a lousy format, since solid colors are not really truly solid due to the compression. If you can save your images in some non-lossy compressed or uncompress format, that would be better. Then decide on the largest area you need to remove. Use that with connected components processing so that you have only two regions, one white background and one black object. This will be a mask. If you have several objects that is fine also, but they need to be black. I show the textual output showing the two regions. The mask is just the object with the noise removed. So now use the original input, a white image and the mask to composite the first two images so that where the mask is black, the object is used and where the mask is white, the white image will be used. Note, I create the white image by making a copy (clone) of the input and colorizing it 100% with white. The following is in Unix syntax.
Input:
magick coin.jpg -negate -threshold 2% -negate -type bilevel \
-define connected-components:verbose=true \
-define connected-components:area-threshold=1000 \
-define connected-components:mean-color=true \
-connected-components 4 mask.png
Objects (id: bounding-box centroid area mean-color):
0: 1000x1000+0+0 525.8,555.7 594824 gray(255)
44: 722x720+101+58 460.9,417.0 405176 gray(0)
magick coin.jpg \
\( +clone -fill white -colorize 100 \) \
mask.png \
-compose over -composite \
coin_result.png
Mask
Result:
See https://imagemagick.org/script/connected-components.php
and https://imagemagick.org/Usage/compose/#compose and Composite Operator of Convert (-composite, -geometry) at https://imagemagick.org/Usage/layers/#convert

Related

imagemagick - compare two JPEGs with masking image

I want to compare two JPG images. I know the parameter "fuzz" because of JPG compensation.
Now I want to compare these two images with a mask - I dont want to compare the whole rectangle, only a part of it.
As a result I want the pixel difference to calculate the percentage difference and in step two I want an image, which shows the difference in red/black.
I found this article: New compare feature
My old way (compare the whole rectangle):
/usr/bin/compare -metric ae /mnt/usbstorage/tmp/15.jpg /mnt/usbstorage/tmp/14.jpg -compose src /dev/null 2>&1
=> 8432 Pixels different
My old way (compare the whole rectangle and paint it to an image):
/usr/bin/compare -metric ae /mnt/usbstorage/tmp/15.jpg /mnt/usbstorage/tmp/14.jpg -compose src /dev/null -highlight-color Red -lowlight-color Black /mnt/usbstorage/tmp/diff.gif
My new try (compare the images with a masking image):
/usr/bin/compare -read-mask /mnt/usbstorage/tmp/mask.png -metric ae /mnt/usbstorage/tmp/15.jpg /mnt/usbstorage/tmp/14.jpg -compose src /dev/null 2>&1
=> 8432 Pixels different
The different pixels won't change! 8432 <=> 8432...so my mask doesn't change anything!
When I do that command to save the difference in a new image there are gray places for the mask, black places for non-changed-pixels and red areas for different pixels -> perfect. The image is perfect, the pixel comparison stays the same. Why?
I don't get it - do you have an explanation / example? Am I doing anything wrong?
Here are the files:
Masked compare only works in IM 7.0.3.9 or higher. In IM 7, you must add magick before compare:
magick compare -metric rmse -read-mask mask.png 14.jpg 15.jpg 14_15_diff.png
The red areas are where they differ under the mask. The white areas are where they are the same under the mask. The gray areas are outside the mask.

Position detection of a defined mark in a picture

I am still a beginner in coding. I am currently working on a program in C/C++ that is determining pixel position of a defined mark (which is a black circle with white surroundings) in a photo.
I made a mask from the mark and a vector, which contains mask's every pixel value as it's elements (using Magick++ I summed values for Red, Green and Blue). Vector contains aprox. 10 000 values since the mask is 100x100px. I also used threshold functions for simplifying the image.
Than I made a grid, that is doing the same for the picture, where I want to find the coordinates of the mark. It is basically a loop, that is going throught the image and when the program knows pixel values in the grid it immediately compares them with the mask. Main idea is to find lowest difference between the mask and one of the grid positions.
The problem is however that this procedure of evaluating all grids position takes huge amount of time (e.g. the image has 1920x1080px so more than 2 million vectors containing 10 000 values). I decided to cycle the grid not every pixel but for example every 10th column and row, and than for the best corellation from this procedure I selected area where I used every pixel loop. But, this still takes lot of time.
I would like to ask you, if there is some way of improving this method for better (faster) results or this whole idea is not time efficient and I should use different approach.
Thanks for every advice!
Edit: The program will be used for processing multiple images and on all of them the size will be same. This is the picture after threshold, the mark is the big black dot.
Image
The idea that I find interesting is a pyramidal scheme - or progressive refinement: you find the spot at a lower size image then search only a small rectangle in the larger image.
If you reduce your image by 2 in each dimension then you would reduce the time by 4 plus some search effort in the larger image.
This has some problems: the reduction will affect accuracy I expect. You might miss the spot.
You have to cut the sample (template) by the same so you create a half-size template in this case. As you half half half... the template will get blurred into the surrounding objects so it will not be possible to have a valid template; for half size once I guess the dot has a couple of pixels around it.
As you haven't specified a tool or OS, I will choose ImageMagick which is installed on most Linux distros and is available for OSX and Windows. I am just using it at the command-line here but there are C, C++, Python, Perl, PHP, Ruby, Java and .Net bindings available.
I would use a "Connect Components Analysis" or "Blob Analysis" like this:
convert image.png -negate \
-define connected-components:area-threshold=1200 \
-define connected-components:verbose=true \
-connected-components 8 -auto-level result.png
I have inverted your image with -negate because in morphological operations, the foreground is usually white rather than black. I have excluded blobs smaller than 1200 pixels because your circles seem to have a radius of 22 pixels which makes for an area of 1520 pixels (Pi * 22^2).
That gives this output, which means 7 blobs - one per line - with the bounding box and area of each:
Objects (id: bounding-box centroid area mean-color):
0: 1358x1032+0+0 640.8,517.0 1296947 gray(0)
3: 341x350+1017+287 1206.5,468.9 90143 gray(255)
106: 64x424+848+608 892.2,829.3 6854 gray(255)
95: 38x101+44+565 61.5,619.1 2619 gray(255)
49: 17x145+1341+379 1350.3,446.7 2063 gray(0)
64: 43x43+843+443 864.2,464.1 1451 gray(255)
86: 225x11+358+546 484.7,551.9 1379 gray(255)
Note that, as your circle is 42x42 pixels you will be looking for a blob that is square-ish and close to that size - so I am looking at the second to last line. I can draw that in in red on your original image like this:
convert image.png -fill none -stroke red -draw "rectangle 843,443 886,486" result.png
Also, note that as you are looking for a circle, you would expect the area to be pi * r^2 or around 1500 pixels and you can check that in the penultimate column of the output.
That runs in 0.4 seconds on a reasonable spec iMac. Note that you could divide the image into 4 and run each quarter in parallel to speed things up. So, if you do something like this:
#!/bin/bash
# Split image into 4 (maybe should allow 23 pixels overlap)
convert image.png -crop 1x4# tile-%02d.mpc
# Do Blob Analysis on 4 strips in parallel
for f in tile-*mpc; do
convert $f -negate \
-define connected-components:area-threshold=1200 \
-define connected-components:verbose=true \
-connected-components 8 info: &
done
# Wait for all 4 to finish
wait
That runs in around 0.14 seconds.

C++ loading and looping through pixels of an image (JPEG format)

Hello there and thank you for at the very least, trying to help me.
I need to, firstly, load an image and then loop through all pixels of that image and check the color of each pixel.
I have never tried messing around with images or whatnot.
Any help is greatly appreciated.
Thank you.
Looking at the bigger picture, of counting the dots on a dice, I would look at using ImageMagick - with the C++ binding called Magick++ from here
I would be looking at using "Blob Analysis", or "Connected Component Analysis" to count the dots on a dice.
Using this dice...
If I use ImageMagick at the command line like this:
convert dice.png -colorspace gray -threshold 50% \
-define connected-components:verbose=true \
-define connected-components:area-threshold=10 \
-connected-components 8 -auto-level output.png
Output
Objects (id: bounding-box centroid area mean-color):
0: 380x362+0+0 189.6,180.0 103867 srgba(255,255,255,1)
2: 93x92+248+32 293.8,77.5 6743 srgba(0,0,0,1)
4: 92x93+39+241 84.8,286.7 6741 srgba(0,0,0,1)
5: 93x93+248+241 293.8,286.8 6738 srgba(0,0,0,1)
1: 92x92+39+32 84.8,77.5 6736 srgba(0,0,0,1)
3: 93x93+143+136 189.3,182.1 6735 srgba(0,0,0,1)
You can see it has found 5 dots (the first one is actually the whole, white image), and I can put a red box around each dot like this so you can see what it has found:
convert dice.png -stroke red -fill none -strokewidth 1 -draw "rectangle 248,32 341,124" -draw "rectangle 39,241 131,334" -draw "rectangle 248,241 341,334" -draw "rectangle 39,32 131,124" -draw "rectangle 143,136 236,229" result.png
I think you may use OpenCV image processing library. You have a detailed manual for installing for VS2013 here:
OpenCV installation for Visual Studio 2013
After you've installed it. You will get a lot of functions for image processing, including what you are looking for.
For example:
Mat inputImage = imread(filename, CV_LOAD_IMAGE_COLOR);
then:
Cycle through pixels with opencv
One of the methods you could use, is to interpret the file as binary. If you know how to interpret the header part and know what color depth the image has (the header has such info), then it wont be a longshot to just compare binary or hexadecimal color codes - probably hexadecimal since C++ doesnt have a built in binary variable.
if you dont think you can handle binary and need a library to work with, refer to
How do I read JPEG and PNG pixels in C++ on Linux?
EDIT - or just use any image processing libraries such as http://cimg.eu since the objective appears to be interpretation of dice from an image.

Image Fusion OpenCV

I am new to OpenCV and I am looking to fuse two images(Panchromatic and Multispectral) using OpenCV with C++. Note that I have already registered the reference image and now I just need to fuse the reference and the sensed image. I could not find any functions that could help me with this. Did I miss something or is there no direct way to fuse two images?
Please suggest any simple way to proceed with the fusion process.
Since you are trying to fuse together the panchromatic and multispectral images, you would need to :
Convert the input images into a suitable format (YUV works for me,
HSI might too).
Fuse the luminance or intensity values of the two images, leaving the color space untouched.
Combine the fused channel with the color information to produce the final image.
.
cvtColor(ref, tmp1, CV_BGR2GRAY, 0);
cvtColor(trans, tmp2, CV_BGR2GRAY, 0);
cv::Mat yuv;
cvtColor(ref, yuv, CV_BGR2YUV, 3);
vector <Mat> channels_ref;
split(yuv, channels_ref);
double alpha = 0.3;
double beta = 1 - alpha;
addWeighted(tmp1, alpha, tmp2, beta, 0.0, channels_ref[0]);
Mat merge[] = {channels_ref[0], channels_ref[1], channels_ref[2]};
cv::merge(merge, 3, output);
cvtColor(output, output, CV_YUV2BGR);
imshow("Linear Blend", output);
waitKey(0);
I revisited this question after a long time and decided to have a go at it as there was no sample imagery available before. In the meantime, I have generated some - see later.
So, let's say you have a hi-res, panchromatic image with 10m resolution something like this:
and a lo-res, multi-spectral image with 40m resolution of the same area, something like this:
Then, just using ImageMagick at the command-line for now (since it is installed on most Linux distros and is available for OSX and Windows), do what I was alluding to in the comments under your original question...
convert hi-res-panchromatic.tif \
\( lo-res-multispectral.tif -resize 400% -colorspace Lab -separate -delete 0 \) \
-set colorspace Lab -combine result.tif
So, that says... "Load up the hi-res image. Then, to one side, load the lo-res image and upsize it to 400% to account for the 40m resolution versus 10m resolution and convert it to Lab colorspace and separate the channels. Delete the Lightness (L) channel of the lo-res image. Now, returning to the main processing from the aside processing, we will have the hi-res image that we loaded first acting as the L channel along with the ab channels (i.e. colour information) of the lo-res image. Combine them from Lab back into RGB and save".
I see you haven't logged on in a year, so I will delay any OpenCV code-writing until anyone else expresses an interest in the question - but I hope the technique is understandable.
Note
As I don't happen to have any geo-registered panchromatic and multi-spectral imagery of the same place, I cheated somewhat... I took a single image and synthesised a panchromatic version using ImageMagick:
convert orig.tif -colorspace gray hi-res-panchromatic.tif
and I synthesised the lo-res multi-spectral image using:
convert orig.tif -resize 25% lo-res-multispectral.tif
Also, note that I just used Lab mode here to do the blending, because it is simpler, but in the comments I suggested using Principal Components Analysis. I may re-visit this again and implement that too...

jpegtran.exe not correctly rotating image

I have a freshly compiled libjpeg version 9 and tried running jpegtran.exe in command line with the arguments:
.\jpegtran.exe -rotate 180 -outfile test_output1.jpg testimg.jpg
testimg.jpg: test_output1.jpg:
As you can see it does rotate the image but it clips it and it's not put together correctly. The usage.txt file that comes with the package isn't totally up to date because I had to use the -outfile switch instead of what it says:
jpegtran uses a command line syntax similar to cjpeg or djpeg. On
Unix-like systems, you say:
jpegtran [switches] [inputfile] >outputfile
On most non-Unix systems, you say:
jpegtran [switches] inputfile outputfile
where both the input and output files are JPEG
files.
To specify the coded JPEG representation used in the output file,
jpegtran accepts a subset of the switches recognized by cjpeg:
-optimize Perform optimization of entropy encoding parameters.
-progressive Create progressive JPEG file.
-arithmetic Use arithmetic coding.
-restart N Emit a JPEG restart marker every N MCU rows, or every N MCU blocks if "B" is attached to the number.
-scans file Use the scan script given in the specified text file.
See the previous discussion of cjpeg for more details about these
switches. If you specify none of these switches, you get a plain
baseline-JPEG output file. The quality setting and so forth are
determined by the input file.
The image can be losslessly transformed by giving one of these
switches:
-flip horizontal Mirror image horizontally (left-right).
-flip vertical Mirror image vertically (top-bottom).
-rotate 90 Rotate image 90 degrees clockwise.
-rotate 180 Rotate image 180 degrees.
-rotate 270 Rotate image 270 degrees clockwise (or 90 ccw).
-transpose Transpose image (across UL-to-LR axis).
-transverse Transverse transpose (across UR-to-LL axis).
Oddly enough (or maybe not), if I execute .\jpegtran.exe -rotate 180 -outfile test_output2.jpg test_output1.jpg I get the original image back without any clipping issues. It's flipping the clipped parts but just not lining it up right with the rest of the image.
test_output2.jpg:
I get the same result by executing jpegtran.exe -rotate 90 twice.
Also, I tried it on a larger .jpg file which resulted in the same issue but the file size was 18KB smaller for the output. I imagine the issue is related to this.
Edit - I also found this blurb which seems to describe the problem:
jpegtran's default behavior when transforming an odd-size image is
designed to preserve exact reversibility and mathematical consistency
of the transformation set. As stated, transpose is able to flip the
entire image area. Horizontal mirroring leaves any partial iMCU
column at the right edge untouched, but is able to flip all rows of
the image. Similarly, vertical mirroring leaves any partial iMCU row
at the bottom edge untouched, but is able to flip all columns. The
other transforms can be built up as sequences of transpose and flip
operations; for consistency, their actions on edge pixels are defined
to be the same as the end result of the corresponding
transpose-and-flip sequence.
The -trim switch works, if you can call it that, and trims out the disorganized data but the image is smaller and lost data.
test_output5.jpg:
Adding the -perfect switch which supposedly stops the above from happening results in this: transformation is not perfect for output and no image.
So is it not possible to losslessly rotate a .jpg? I could, myself, go into paint and reconstruct the original image by simply moving the edge lines into their correct place. Is there a method to do this within libjpeg?
A lossless rotation works with whole DCT blocks contained within the JPEG file. These blocks are always 8x8 or 16x16 pixels (depending on the compression downsampling settings). The file contains a width and height so the extra pixels can be thrown away when the image is decoded, but there's no way to move the clipping from the right/bottom edge to the left/top edge. The software is doing the best it can with an impossible problem.
As you've discovered the way around this problem is to make the width and height evenly divisible by 16. You'll find that images from cameras for example will have this property.