Simple code to create an OpenGL cubemap from "cross layout" image file - opengl

I'm looking for a simple C subroutine (in Linux) that takes a filename parameter and makes the 6 calls to glTexImage2D to define a "cubemap" texture (GL_TEXTURE_CUBE_MAP).
I'd like to be able to read a "cross layout" file (horizontal or vertical) or a "Horizontal Strip" file, so this requires various cropping operations (and possibly some rotates to match the orientation that OpenGL expects).
I'm sure this code exists, but I Googled around and only found fragments.
I stumbled on the GEGL / BABL libraries. This looks good. A bit overkill, but it can read from a file ("gegl:load"), crop, rotate and BABL can do various pixel format operations (to match the OpenGL GL_RGB or GL_RGBA format).
Also, it might be useful to read a "rectilinear" texture file and convert to the gnomic projections (I'm using MMPS for this now).
I found this Python "gimpfu" code ( http://registry.gimp.org/files/cubemap-to-dds-v0.3.py.txt ) that converts from a "cross layout" to a multi-image DDS file, so that's similar to what I want, but I'd rather not have Python code in my app. GIMP uses GEGL internally, so that's why I'm using that.

Well... I wrote a very simple "vertical strip" cubemap reader using the Freeimage library. Here's the crux:
FIBITMAP *bitmap = FreeImage_Load(FreeImage_GetFileType(name0, 0), name0, BMP_DEFAULT);
FIBITMAP *pImage = FreeImage_ConvertTo32Bits(bitmap);
FreeImage_Unload(bitmap);
int width0=FreeImage_GetWidth(pImage);
int height0=FreeImage_GetHeight(pImage);
void *pixels0 = FreeImage_GetBits(pImage);
// "v-strip" (for simplicity, assume OpenGL order and orientation)
height0 /= 6;
for(int idx0=GL_TEXTURE_CUBE_MAP_NEGATIVE_Z; idx0 >= GL_TEXTURE_CUBE_MAP_POSITIVE_X ; idx0--) { // NB : index goes backwards because image files start at the bottom
glTexImage2D(idx0, 0, GL_RGB, width0, height0,
0, GL_BGRA_EXT, GL_UNSIGNED_BYTE, pixels0);
pixels0 += width0 * height0 * 4; // "4" : sizeof(RGBA pixel) ?
}
glGenerateMipmap(GL_TEXTURE_CUBE_MAP);
FreeImage_Unload(pImage);
That gets me started. I have a MMPS script that converts from rectilinear to v-strip, but I'm still fiddling with the orientations. As one book says, "The orientation of the texture coordinates of each face are counter-intuitive". I might convert the script to "nona" / PTstitcher (part of PanoTools).
FYI, there are many cubemap image file formats. V-strip is easiest to read. "Horizontal cross" seems to be popular. In general, there are {horizontal, vertical} x {cross, strip, tee} (and probably more).
I have some code that reads crosses, h-strips (it uses glPixelStorei(GL_UNPACK_ROW_LENGTH_EXT, width0) ), but it still doesn't deal with the orientations. I'll have to use something like GEGL or libpano13. Or perhaps hack an intermediate step that uses execl to run MMPS / nona / ImageMagick. Maybe libpano13 is way to go : I could generate the cubemaps on the fly from rectilinear files.
Cheers!
Update: here's the MMPS bash script to convert from a rectilinear image to v-strip (in OpenGL order and orientation) :
## NB : yum install ImageMagick # for convert & montage
infile0=$1
MMPS0=~/mmps-0-1-36
tmp0=/tmp/cube$$
convert $infile0 $tmp0.ppm
ARGS0="gnomonic -grid -gridcolor 255:255:255 -scale 2 -w 512 -h 512"
ARGS0="gnomonic -scale 2 -w 512 -h 512"
#
$MMPS0/project $ARGS0 -lat 0 -long 0 -f $tmp0.ppm -rotate -90 > $tmp0-xp.ppm & ## africa
$MMPS0/project $ARGS0 -lat 0 -long 180 -f $tmp0.ppm -rotate 90 > $tmp0-xn.ppm & ## pacific ocean
$MMPS0/project $ARGS0 -lat 0 -long 90 -f $tmp0.ppm -rotate 180 > $tmp0-yp.ppm & ## indian ocean
$MMPS0/project $ARGS0 -lat 0 -long 270 -f $tmp0.ppm -rotate 0 > $tmp0-yn.ppm & ## americas
$MMPS0/project $ARGS0 -lat 90 -long 0 -f $tmp0.ppm -rotate -90 > $tmp0-zp.ppm & ## north pole
$MMPS0/project $ARGS0 -lat -90 -long 0 -f $tmp0.ppm -rotate -90 > $tmp0-zn.ppm & ## south pole
wait
montage $tmp0-xp.ppm $tmp0-xn.ppm \
$tmp0-yn.ppm $tmp0-yp.ppm \
$tmp0-zp.ppm $tmp0-zn.ppm \
-mode Concatenate -tile 1x6 cube-vstrip-ogl.jpg
##$tmp0-yp.ppm $tmp0-yn.ppm \ ##### swap 'em ??? had to reverse
# also, for textures viewed from the "outside" (eg : earth texture on sphere, swap y in frag shader:
# gl_FragColor = textureCube(cubemap2, vec3(texCoord0.x, -texCoord0.y, texCoord0.z));
rm $tmp0-*
FYI, I found some test images ( https://www.terathon.com/wiki/index.php/Importing_a_Texture ) and the v-strip is in OpenGL order and orientation, so maybe that's typical.

Related

How can I split a raw buffer of uint8 I420 video data into YUV pointers?

I have a raw uint8 pointer to a buffer containing I420 formatted video data, and a buffer size. I also know the frame width / height. I want to feed the data into a library which can create video frames via this function signature:
Copy(int width, int height,
const uint8_t* data_y, int stride_y,
const uint8_t* data_u, int stride_u,
const uint8_t* data_v, int stride_v)
Is there some simple pointer arithmetic to resolve this?
The best site I know for describing the various YUV and RGB video formats is FOURCC - the YUV descriptions are here.
The I420 format you refer to is described here. That means that your raw data will be organised like this:
Y plane, of H x W bytes, with a line stride of W bytes
U plane, of H/2 x W/2 bytes, with a line stride of W/2 bytes
V plane, of H/2 x W/2 bytes, with a line stride of W/2 bytes
I find ffmpeg is the very best tool for generating YUV data to encode into and decode out of your own software. Here are some tips for working with ffmpeg and raw YUV.
You can get a list of the pixel formats it supports with:
ffmpeg -pix_fmts
So, to find yours, I looked for something with 420 in it:
ffmpeg -pix_fmts | grep 420p
IO... yuv420p 3 12 8-8-8
so I know I need -pix_fmt yuv420p to encode or decode your data. I can also get a decent description of how that is laid out by checking the ffmpeg source here. The 12 above means 12 bits per pixel.
Then I wanted to:
generate a sample I420 frame with ffmpeg
split it apart with dd and extract the Y, U and V channels with IMageMagick
recombine the Y, U and V channels with ImageMagick
recombine the Y, U and V channels with ffmpeg
so I made the following bash script:
#!/bin/bash
################################################################################
# User-editable values
################################################################################
# Define WIDTH and HEIGHT of the frame we want to generate...
# ... so we are consistent all the way through
W=640
H=480
PIX_FMT="yuv420p"
################################################################################
# Derived values - do not edit
################################################################################
BASENAME="${PIX_FMT}-${W}x${H}"
FILENAME="${BASENAME}.raw"
PNGNAME="${BASENAME}.png"
UVW=$((W/2)) # width of U plane, same as V plane
UVH=$((H/2)) # height of U plane, same as V plane
YBYTES=$((H*W)) # bytes in Y plane
UBYTES=$((UVW*UVH)) # bytes in U plane, same as in V plane
# Generate a sample frame
echo "Generating sample: ${FILENAME}, and viewable PNG equivalent: ${PNGNAME}"
ffmpeg -y -f lavfi -i testsrc=size=${W}x${H}:rate=1:duration=1 -vcodec rawvideo -pix_fmt "$PIX_FMT" -f image2pipe - > "$FILENAME"
ffmpeg -y -f lavfi -i testsrc=size=${W}x${H}:rate=1:duration=1 "$PNGNAME"
# Check its size in bytes
ls -l "$FILENAME"
# Extract Y plane from sample into "Y.png" using ImageMagick
echo "Extracting Y plane into Y.png"
dd if="$FILENAME" bs=$YBYTES count=1 | magick -depth 8 -size ${W}x${H} gray:- Y.png
# Extract U plane from sample into "U.png" using ImageMagick
echo "Extracting U plane into U.png"
dd if="$FILENAME" bs=1 skip=$YBYTES count=$UBYTES | magick -depth 8 -size ${UVW}x${UVH} gray:- U.png
# Extract V plane from sample into "V.png" using ImageMagick
echo "Extracting V plane into V.png"
dd if="$FILENAME" bs=1 skip=$((YBYTES+UBYTES)) count=$UBYTES | magick -depth 8 -size ${UVW}x${UVH} gray:- V.png
# Recombine with ImageMagick
echo "Combining Y.png, U.png, V.png into result.png"
magick Y.png \( U.png v.png -resize 200% \) -set colorspace YUV -combine result.png
# Create a PNG from the YUV420p raw data just the same with 'ffmpeg'
echo "Create PNG from the YUV420p raw data as 'extracted.png'"
ffmpeg -y -f rawvideo -video_size 640x480 -pixel_format yuv420p -i - extracted.png < "$FILENAME"
That creates this image as PNG and as I420 data for you to test with:
and these Y, U and V planes:

How to perform ImageMagick gradient call with python wand

Trying to convert this ImageMagick command to Python Wand code but I don't see a means to create a gradient image with Wand.
convert -size 800x800 gradient:"rgba(0,0,0,0.12)-rgba(0,0,0,1)" gradient_overlay.png
convert gradient_overlay.png background.png -compose Overlay -composite -depth 8 background_gradient.png
Does anyone know how I could achieve this with wand?
You will need to allocate an instance of wand, set canvas size, then read the pseudo-image format.
from wand.image import Image
from wand.api import library
with Image() as canvas:
library.MagickSetSize(canvas.wand, 800, 800)
canvas.read(filename="gradient:rgba(0,0,0,0.12)-rgba(0,0,0,1)")
canvas.save(filename="gradient_overlay.png")
ImageMagick Equivalent For Use With Wand
I have not used Wand, but I can show you how to do it by reference to ImageMagick. You can create a new transparent image. Then use the fx operator to modify the alpha channel into a gradient.
The equivalent Wand references are:
Create a new transparent image (the default) - see http://docs.wand-py.org/en/0.4.1/guide/read.html?highlight=new%20image#open-empty-image
Use fx to convert the alpha channel to a gradient - see http://docs.wand-py.org/en/0.4.1/wand/image.html
convert -size 800x800 xc:transparent -channel a -fx "(j*(1-0.12)/(h-1)+0.12)" +channel alpha.png
Here is how I got the fx formula:
You want a 12% gray at the top (0.12) and 100% gray/white at the bottom (1.0). So we take the formula:
c = a*j + b
At j=0 the top, you need 0.12, so
0.12 = a*0 + b --> b = 0.12
At j=(h-1) the bottom, you want 1, so
1 = a*(h-1) + 0.12 --> a = (1-0.12)/(h-1)
So the equation is:
c = j*(1-0.12)/(h-1) + 0.12
h is the height of the image (800).
Here is how I solved it based fmw42 feedback
with Color('black') as blackColor:
with Image(width=800, height=800, background=blackColor) as black:
blackPath = existing[:existing.rfind('/') + 1] + 'black.png'
with Color('white') as whiteColor:
with Image(width=800, height=800, background=whiteColor) as alpha:
fxFilter = "(j*(1-0.12)/(h-1)+0.12)"
with alpha.fx(fxFilter) as filteredImage:
black.composite_channel('default_channels', filteredImage, 'copy_opacity', 0, 0)
black.save(filename=blackPath)

Convert image magick command to magick++ c++ code

I'm working on an image preprocessing project in my university and used an image magick script to clean image background.Now I want to get the same output through Magick++ (c++ api for imageMagick).
ImageMagick Command: "convert -respect-parenthesis ( INPUT_IMAGE.jpg -colorspace gray -contrast-stretch 0 ) ( -clone 0 -colorspace gray -negate -lat 25x25+30% -contrast-stretch 0 ) -compose copy_opacity -composite -fill white -opaque none -alpha off -background white OUTPUT_IMAGE.jpg"
I tried to convert this code to Magick++ code and failed in "-lat", "-contrast-stretch" and "-compose" positions.
This is my c++ code so far:
Image backgroungImage;
backgroungImage.read("INPUT_IMAGE.jpg");
backgroungImage.colorSpace(GRAYColorspace);
backgroungImage.type(GrayscaleType);
backgroungImage.contrastStretch(0, QuantumRange);
backgroungImage.write("Partial_output.jpg");
If anyone has an idea or a better solution please let me know.
thanx in advance.
You're on the right track with -contrast-stretch. For -lat, remember that's an abbreviation of "Local Adaptive Threshold". So the C++ code would look like...
Image backgroundImage;
// INPUT_IMAGE.jpg
backgroundImage.read("INPUT_IMAGE.jpg");
// -colorspace gray
backgroundImage.colorSpace(GRAYColorspace);
// -contrast-stretch 0
backgroundImage.contrastStretch(0, QuantumRange);
// -clone 0
Image foregroundImage(backgroundImage);
// -negate
foregroundImage.negate();
// -lat 25x25+30%
foregroundImage.adaptiveThreshold(25, 25, QuantumRange * 0.30);
// -contrast-stretch 0
backgroundImage.contrastStretch(0, QuantumRange);
// -compose copy_opacity -composite
backgroundImage.composite(foregroundImage, 0, 0, CopyAlphaCompositeOp);
// -fill white -opaque none
backgroundImage.opaque(Color("NONE"), Color("WHITE"));
// -alpha off
backgroundImage.alpha(false);
// -background white
backgroundImage.backgroundColor(Color("WHITE"));
// OUTPUT_IMAGE.jpg
backgroundImage.write("OUTPUT_IMAGE.jpg");
Hope that helps!

finding spots and lines in image

I was working on following images to find lines and spots in these images. I am working with OpenCV, C++. I have tried HoughLineP, HoughLine, Contour and Canny methods but couldn't get the results. If someone can help or write a pseudo-code, I shall be grateful.
Thanks.
Image to detect line:
Image to detect spot:
Mmmmm - where did you get those awful images? Well, they were worth 2 minutes of effort... the spot is lighter than the rest, so if you divide the image into 100 rectangles and find the brightest ones, you will probably get it... I use ImageMagick here just at the command line - it is installed on most Linux distros and available for OSX and Windows:
convert noise.jpg -crop 10x10# -format "%[mean] %g\n" info: | sort -n
32123.3 640x416+384+291
32394.6 640x416+256+42
32442.2 640x416+320+125
32449.1 640x416+384+250
32459.6 640x416+192+374
32464.4 640x416+0+374
32486.5 640x416+448+125
32491.4 640x416+576+374
32493.7 640x416+576+333
32504.3 640x416+576+83
32520.9 640x416+576+0
32527 640x416+448+0
32621.8 640x416+384+333
32624.1 640x416+320+42
32631.3 640x416+192+333
32637.8 640x416+384+42
32643.4 640x416+512+0
32644.2 640x416+0+0
32652.6 640x416+384+83
32659.1 640x416+128+374
32660.4 640x416+320+208
32662.2 640x416+384+0
32668.5 640x416+256+208
32669.4 640x416+0+333
32676.7 640x416+256+250
32683.5 640x416+256+83
32699.7 640x416+0+208
32701.3 640x416+64+166
32704 640x416+576+208
32704 640x416+64+333
32707.5 640x416+512+208
32710.8 640x416+192+83
32729.8 640x416+320+83
32733.4 640x416+256+166
32735 640x416+576+250
32741 640x416+256+125
32745.4 640x416+0+166
32748.4 640x416+320+166
32751.4 640x416+512+166
32752.4 640x416+512+42
32755.1 640x416+384+208
32770.9 640x416+448+291
32776.8 640x416+128+166
32777.1 640x416+256+0
32795.8 640x416+512+125
32801.5 640x416+128+333
32803.3 640x416+192+125
32805.5 640x416+256+374
32809.6 640x416+448+166
32810 640x416+576+166
32822.2 640x416+0+291
32822.8 640x416+576+42
32826.8 640x416+320+333
32831.7 640x416+320+0
32834.8 640x416+192+42
32837.6 640x416+192+166
32843 640x416+384+125
32862 640x416+64+374
32865.8 640x416+0+42
32871.5 640x416+576+291
32872.5 640x416+0+83
32872.8 640x416+448+333
32873.6 640x416+320+291
32877.5 640x416+448+42
32880.5 640x416+64+208
32883.5 640x416+128+42
32883.9 640x416+192+208
32885.5 640x416+128+208
32889.2 640x416+256+333
32921 640x416+192+291
32923.3 640x416+64+291
32929.2 640x416+512+374
32935.4 640x416+192+250
32938.4 640x416+64+250
32943.5 640x416+448+374
32953.3 640x416+384+374
32954.7 640x416+320+374
32962 640x416+320+250
32966.9 640x416+448+83
32967.3 640x416+128+291
32968.3 640x416+0+250
32970.8 640x416+512+333
32974.5 640x416+64+0
32979.6 640x416+512+291
32983.6 640x416+256+291
32988.9 640x416+448+250
32993.3 640x416+576+125
33012.7 640x416+0+125
33057.3 640x416+512+250
33068.6 640x416+128+250
33102.9 640x416+64+42
33126.1 640x416+512+83
33127.9 640x416+384+166
33139.2 640x416+192+0
33141.4 640x416+64+83
33142.3 640x416+64+125
33181.5 640x416+448+208
33190.8 640x416+128+0
34693 640x416+128+125
36178.3 640x416+128+83
The last 2 rectangles are the brightest, so if I box them in in red and blue you can see what it has found:
convert noise.jpg -fill none -stroke red -draw "rectangle 128,83 192,123" -stroke blue -draw "rectangle 128,125 192,168" result.png
Alternatively, you could create a new image in which each pixel is the mean of the 50x50 square of surrounding pixels in the original image, like this:
convert noise.jpg -virtual-pixel edge -statistic mean 50x50 -auto-level result.png
Of course, you can also threshold that:
convert noise.jpg -virtual-pixel edge -statistic mean 50x50 -auto-level -threshold 80% result.png
As regards the lines, I want to use some type of mode to detect the frequently occurring values within small areas but as the colours vary, I need to reduce the palette of colours to find things that are just similarly coloured so I would go with an approach something like this which reduces the colours then calculates the mode:
convert noise2.jpg -colors 8 -statistic mode 8x8 result.jpg
It needs refinement, but you get the idea hopefully.
Alternatively, you could calculate a new image wherein each pixel is the standard deviation of the surrounding 3x3 pixels in the original image and then look for the ones where this value is lowest - i.e. where the image is darkest which corresponds to areas in the input image where there is least variation in the pixel colours:
convert noise2.png -statistic standarddeviation 3x3 -auto-level result.png

How to detect an inclination of 90 degrees or 180?

In my project I deal with images which I don't know if they are inclined or not.
I work with C++ and OpenCV. I try with Hough transformation to determine the angle of inclination: if it is 90 or 180. But it doesn't give a result.
A link to example image (full resolution TIFF) here.
The following illustration is the full-res image scaled down and converted to PNG:
If I want to attack your image with the Hough lines method, I would do a Canny edge detection first, then find the Hough lines and then look at the generated lines. So it would look like this in ImageMagick - you can transform to OpenCV:
convert input.jpg \
\( +clone -canny x10+10%+30% \
-background none -fill red \
-stroke red -strokewidth 2 \
-hough-lines 9x9+150 \
-write lines.mvg \
\) \
-composite hough.png
And in the lines.mvg file, I can see the individual detected lines:
# Hough line transform: 9x9+150
viewbox 0 0 349 500
line 0,-3.74454 349,8.44281 # 160
line 0,55.2914 349,67.4788 # 206
line 1,0 1,500 # 193
line 0,71.3012 349,83.4885 # 169
line 0,125.334 349,137.521 # 202
line 0,142.344 349,154.532 # 156
line 0,152.351 349,164.538 # 155
line 0,205.383 349,217.57 # 162
line 0,239.453 349,245.545 # 172
line 0,252.455 349,258.547 # 152
line 0,293.461 349,299.553 # 163
line 0,314.464 349,320.556 # 169
line 0,335.468 349,341.559 # 189
line 0,351.47 349,357.562 # 196
line 0,404.478 349,410.57 # 209
line 349.39,0 340.662,500 # 187
line 0,441.484 349,447.576 # 198
line 0,446.484 349,452.576 # 165
line 0,455.486 349,461.578 # 174
line 0,475.489 349,481.581 # 193
line 0,498.5 349,498.5 # 161
I resized your image to 349 pixels wide (to make it fit on Stack Overflow and process faster), so you can see there are lots of lines that start at 0 on the left side of the image and end at 349 on the right side which tells you they go across the image, not up and down it. Also, you can see that the right end of the lines is generally 16 pixels lower than the left, so the image is rotated tan inverse (16/349) degrees.
Here is a fairly simple approach that may help you get started, or give you ideas that you can adapt. I use ImageMagick, but the concepts and techniques should be readily applicable in OpenCV.
First, I note that the image is rotated a few degrees and that gives the black triangle at top right, so the first thing I would consider is cropping the middle out of the image - i.e. removing around 10-15% off each side.
The next thing I note is that, the image is poorly scanned with lots of noisy, muddy grey areas. I would tend to want to blur these together so that they become a bit more uniform and can be thresholded.
So, if I want to do those two things in ImageMagick, I would do this:
convert input.tif \
-gravity center -crop 75x75%+0+0 \
-blur x10 -threshold 50% \
-negate \
stage1.jpg
Now, I can count the number of horizontal black lines that run the full width of the image (without crossing anything white). I do this by squidging the image till it is just a single pixel wide (but still the full original height) and counting the number of black rows:
convert stage1.jpg -resize 1x! -threshold 1 txt: | grep -c black
1368
And I do the same for vertical black lines that run the full height of the image from top to bottom, uninterrupted by white. I do that by squidging the image till it is a single pixel tall and the full original width:
convert stage1.jpg -resize x1! -threshold 1 txt: | grep -c black
0
Therefore there are 1,368 lines across the image and none up and down it, so I can say the dark lines in the original image tend to run left-right across the image rather than top-bottom up and down the image.