Flatten array of structs efficiently - c++

I'm looking for the most efficient way to flatten an array of structs in C++ for passing the flattend 1D array data as input to a cv::Mat. The struct looks as follows:
struct Color3
{
uint8_t red, green, blue;
}
My code then looks like this:
// Update color frame
cv::Mat colorMat = cv::Mat::zeros(cv::Size(1920, 1080), CV_8UC3)
const Color3* colorPtr = colorFrame->getData(); // Get Frame from Library
std::vector<uchar> vecColorData;
data.reserve(1920 * 1080 * 3);
for (int i = 0; i < 1920 * 1080; ++i)
{
auto color = *colorPtr;
vecColorData.push_back(color.red);
vecColorData.push_back(color.green);
vecColorData.push_back(color.blue);
vecColorData++;
}
colorMat.data = vecColorData.data();
Is there a more efficient way than creating an intermediate std::vector and looping over the entire array? I guess I'm looking for something like:
colorMat.data = colorFrame->getData()
However, I'm getting the following error: a value of type Color3* cannot be assigned to an entity of type uchar*.

you don't need an intermediate vector.
If I understood, you want to assign the same RGB triple to all data.
It is also unclear to me if you have to allocate colorMat.data on your own or not.
If this is the case, once colorMat.data is allocated and sized 1920 * 1080 * 3, you can do something like the following:
uchar * data = colorMat.data;
for (int i = 0; i < 1920 * 1080; ++i)
{
*data++ = (uchar)colorPtr->red;
*data++ = (uchar)colorPtr->green;
*data++ = (uchar)colorPtr->.blue;
}

The following answer is not technically portable but will work on the vast majority of platforms you will encounter in real life.
It is extremely likely that your Color3 struct has no padding. You can veryify this by using a static_assert:
static_assert(sizeof(Color3) == sizeof(uint8_t) * 3);
With this confirmed you can cast an array of Color3 to an array of uint8_t and pass it directly to colorMat.data (assuming that member actually accepts uint8_t).
Your code therefore becomes:
cv::Mat colorMat = cv::Mat::zeros(cv::Size(1920, 1080), CV_8UC3)
const Color3* colorPtr = colorFrame->getData(); // Get Frame from Library
colorMat.data = reinterpret_cast<const uint8_t*>(colorPtr);
Bear in mind I have never used the cv library and know nothing about the ownership requirements of the data pointer. The above just replicates what you're doing without the unnecessary std::vector.

Related

sws_scale on raw yuv420p data

I am getting camera data from my raspberry pi in yuv420p format. I am trying to use sws_scale to convert them into RGBA format. So this is how I initialize my context:
_sws_context = sws_getContext(CAMERA_WIDTH, CAMERA_HEIGHT, AV_PIX_FMT_YUV420P,
CAMERA_WIDTH, CAMERA_HEIGHT, AV_PIX_FMT_RGBA, 0, nullptr, nullptr, nullptr);
I am now a bit confused on how to set the data and line size for sws_scale. From the camera I just get a plain array of bytes without further structure. I assume I have to subdivide that into the planes somehow. My first approach was not to separate it at all and essentially have something like this (based on the fact that :
const uint8_t *src_data[] = {data.data()};
const int src_strides[] = {(int) std::ceil((CAMERA_WIDTH * 6) / 8)};
This was based on:
there are 12 bits for a 2x2 grid of pixels
So I assumed one line would use half of this. But that causes a segmentation fault. So I think I somehow have to split src_data and src_strides into the respective YUV planes, but I am not sure how to do this, especially since one pixel for YUV420 data uses less than one byte per plane...
Turns out it is simpler than I thought! The planes are one after another which also makes the strides pretty obvious:
const auto y = data.data();
const auto u = y + CAMERA_WIDTH * CAMERA_HEIGHT;
const auto v = u + (CAMERA_WIDTH * CAMERA_HEIGHT) / 4;
const auto stride_y = CAMERA_WIDTH;
const auto stride_u = CAMERA_WIDTH / 2;
const auto stride_v = CAMERA_WIDTH / 2;

Using halide with HDR images represented as float array

that's my first post here so sorry if I do something wrong:). I will try to do my best.
I currently working on my HDR image processing program, and I wonna implement some basing TMO using Halide. Problem is all my images are represented as float array (with order like: b1,g1,r1,a1, b2,g2,r2,a2, ... ). Using Halide to process image require Halide::Image class. Problem is I don't know how to pass those data there.
Anyone can help, or have same problem and know the answer?
Edit:
Finally got it! I need to set stride on input and output buffer in generator. Thx all for help:-)
Edit:
I tried two different ways:
int halideOperations( float data[] , int size, int width,int heighy )
{
buffer_t input_buf = { 0 };
input_buf.host = &data[0];
}
or:
int halideOperations( float data[] , int size, int width,int heighy )
{
Halide::Image(Halide::Type::Float, x, y, 0, 0, data);
}
I was thinking about editing Halide.h file and changing uint8_t * host to float_t * host but i don't think it's good idea.
Edit:
I tried using code below with my float image (RGBA):
AOT function generation:
int main(int arg, char ** argv)
{
Halide::ImageParam img(Halide::type_of<float>(), 3);
Halide::Func f;
Halide::Var x, y, c;
f(x, y, c) = Halide::pow(img(x,y,c), 2.f);
std::vector<Halide::Argument> arguments = { img };
f.compile_to_file("function", arguments);
return 0;
}
Proper code calling:
int halideOperations(float data[], int size, int width, int height)
{
buffer_t output_buf = { 0 };
buffer_t buf = { 0 };
buf.host = (uint8_t *)data;
float * output = new float[width * height * 4];
output_buf.host = (uint8_t*)(output);
output_buf.extent[0] = buf.extent[0] = width;
output_buf.extent[1] = buf.extent[1] = height;
output_buf.extent[2] = buf.extent[2] = 4;
output_buf.stride[0] = buf.stride[0] = 4;
output_buf.stride[1] = buf.stride[1] = width * 4;
output_buf.elem_size = buf.elem_size = sizeof(float);
function(&buf, &output_buf);
delete output;
return 1;
}
unfortunately I got crash with msg:
Error: Constraint violated: f0.stride.0 (4) == 1 (1)
I think something is wrong with this line: output_buf.stride[0] = buf.stride[0] = 4, but I'm not sure what should I change. Any tips?
If you are using buffer_t directly, you must cast the pointer assigned to host to a uint8_t * :
buf.host = (uint8_t *)&data[0]; // Often, can be just "(uint8_t *)data"
This is what you want to do if you are using Ahead-Of-Time (AOT) compilation and the data is not being allocated as part of the code which directly calls Halide. (Other methods discussed below control the storage allocation so they cannot take a pointer that is passed to them.)
If you are using either Halide::Image or Halide::Tools::Image, then the type casting is handled internally. The constructor used above for Halide::Image does't exist as Halide::Image is a template class where the underlying data type is a template parameter:
Halide::Image<float> image_storage(width, height, channels);
Note this will store the data in planar layout. Halide::Tools::Image is similar but has an option to do interleaved layout. (Personally, I try not to use either of these outside of small test programs. There is a long term plan to rationalize all of this which will proceed after the arbitrary dimension buffer_t branch is merged. Note also Halide::Image requires libHalide.a to be linked where Halide::Tools::Image does not and is header file only via including common/halide_image.h .)
There is also the Halide::Buffer class which is a wrapper on buffer_t that is useful in Just-In-Time (JIT) compilation. It can reference passed in storage and is not templated. However my guess is you want to use buffer_t directly and simply need the type cast to assign host. Also be sure to set the elem_size field of buffer_t to "sizeof(float)".
For an interleaved float buffer, you'll end up with something like:
buffer_t buf = {0};
buf.host = (uint8_t *)float_data; // Might also need const_cast
// If the buffer doesn't start at (0, 0), then assign mins
buf.extent[0] = width; // In elements, not bytes
buf.extent[1] = height; // In elements, not bytes
buf.extent[2] = 3; // Assuming RGB
// No need to assign additional extents as they were init'ed to zero above
buf.stride[0] = 3; // RGB interleaved
buf.stride[1] = width * 3; // Assuming no line padding
buf.stride[2] = 1; // Channel interleaved
buf.elem_size = sizeof(float);
You will also need to pay attention to the bounds in the Halide code itself. Probably best to look at the set_stride and bound calls in tutorial/lesson_16_rgb_generate.cpp for information on that.
In addition to Zalman's answer above you also have to specify the strides for the inputs and outputs when defining your Halide function like below:
int main(int arg, char ** argv)
{
Halide::ImageParam img(Halide::type_of<float>(), 3);
Halide::Func f;
Halide::Var x, y, c;
f(x, y, c) = Halide::pow(img(x,y,c), 2.f);
// You need the following
f.set_stride(0, f.output_buffer().extent(2));
f.set_stride(1, f.output_buffer().extent(0) * f.output_buffer().extent(2));
img.set_stride(0, img.extent(2));
img.set_stride(1, img.extent(2) *img.extent(0));
// <- up to here
std::vector<Halide::Argument> arguments = { img };
f.compile_to_file("function", arguments);
return 0;
}
then your code should run.

Why is cv::Mat::data always pointing to a uchar?

I try to read a NEF file using LibRaw and then put it in a cv::Mat. The NEF file stores data as 12bit, this means I need 16 bit, so I ought to use CV_16UC4 like this:
Mat img1(height, width, CV_16UC4);
Libraw stores data as ushort*[4], so I thought that this should work:
for (i = 0; i < iwidth*height; i++) {
img1.data[4*i+1] = Processor.imgdata.image[i][0];
img1.data[4*i+2] = Processor.imgdata.image[i][1];
img1.data[4*i+3] = Processor.imgdata.image[i][2];
img1.data[4*i+4] = Processor.imgdata.image[i][3];
}
I also get a build error that data may be lost since a ushort to uchar conversion is going to take place, which makes sense, but still, how do I put data bigger than uchar in the data?
If you need pointer to raw data of specific type, using cv::Mat::ptr() is the best practice:
ushort* ptr = img1.ptr<ushort>();
for (i = 0; i < iwidth*height; i++) {
ptr[4*i+1] = Processor.imgdata.image[i][0];
ptr[4*i+2] = Processor.imgdata.image[i][1];
ptr[4*i+3] = Processor.imgdata.image[i][2];
ptr[4*i+4] = Processor.imgdata.image[i][3];
}
Please see documentation.
cv::Mat::data uses uchar in order avoid being a template class. In order to fill it with other image data you'll need to cast the data pointer. In your case try something like this:
Mat img1(height, width, CV_16UC4);
ushort * data = reinterpret_cast< ushort* >( img1.data );
for (i = 0; i < iwidth*height; i++) {
...
}
Alternatively, instead of changing the data pointer img1.data directly in your for-loop, you could consider using
the templated pixel access function cv::Mat::at<T>()
img1.at<Vec4w>(y,x) = reinterpret_cast<Vec4w>(Processor.imgdata.image[i])
use the specialized class Mat4w img(height, width) and then operator(y,x)
img1(y,x) = reinterpret_cast<Vec4w>(Processor.imgdata.image[i])
Mat.data looks like a uchar, but actually it contains all the ushort data in the memory. You can simply copy the memory to your ushort array, like this:
memcpy(your_array, img.data, your_array_size);

C++ - Convert uint8_t* image data to double** image data

I am working on a C++ function (inside my iOS app) where I have image data in the form uint8_t*.
I obtained the image data using the code using the CVPixelBufferGetBaseAddress() method of the iOS SDK:
uint8_t *bPixels = (uint8_t *)CVPixelBufferGetBaseAddress(imageBuffer);
I have another function (from a third part source) that does some of the image processing functions I would like to use on my image data, but the input for the image data for these functions is double**.
Does anyone have any idea how to go about converting this?
What other information can I provide?
The constructor prototype for the class that use double** look like:
Image(double **iPixels, unsigned int iWidth, unsigned int iHeight);
Your uint8_t *bPixels seems to hold image data as 1-dimensional continuous array of height*width lenght. So to access pixel in the x-th row and y-th column you have to write bPixels[x*width+y].
Image() seems to work on 2-dimensional arrays. To access pixel like above you would have to write iPixels[x][y].
So you need to copy your existing 1-dimensional array to a 2-dimensional:
double **mypixels = new double* [height];
for (int x=0; x<height; x++)
{
mypixels[x] = new double [width];
for (int y=0; y<width; y++)
mypixels[x][y] = bPixels[x*width+y]; // attention here, maybe normalization is necessary
// e.g. mypixels[x][y] = bPixels[x*width+y] / 255.0
}
Because your 1-dimensional array has pixel of type uint8_t and the 2-dimensional one pixel of type double, you must allocate new memory. Otherwise, if both would have same pixel type, the more elegant solution (a simple map) would be:
uint8_t **mypixels = new uint8_t* [height];
for (int x=0; x<height; x++)
mypixels[x] = bPixels+x*width;
Attention: beside the problem of eventually necessary normalization, there is also a problem with the indices-compatibility! My examples assume that the 1-dimensional array is stored row-by-row and that the functions working on 2-dimensional index with [x][y] (that means first-row-then-column). The declaration of Image() however, could lead to the conclusion that it needs its arrays to be indexed with [y][x] maybe.
I'm going to take a giant bunch of guesses here in hopes that this will lead you towards getting at the documentation and answering back. If there's no further documentation, well, here's a starting point.
Guess 1) The Image constructor requires a doubly dimensioned array where each component is an R,G,B,Alpha channel in that order. So iPixels[0] is the red data, iPixels[1] is the green data, etc.
Guess 2) Because it's not integer data, the values range from 0 to 1.
Guess 3) All of this must be pre-allocated.
Guess 4) Image data is row-major
Guess 5) Source data is BRGA
So with that in mind, starting with bPixels
double *redData = new double[width*height];
double *greenData = new double[width*height];
double *blueData = new double[width*height];
double *alphaData = new double[width*height];
double **iPixels = new double*[4];
iPixels[0] = redData;
iPixels[1] = greenData;
iPixels[2] = blueData;
iPixels[3] = alphaData;
for(int y = 0;y < height;y++)
{
for(int x = 0;x < width;x++)
{
int alpha = bPixels[(y*width + x)*4 + 3];
int red = bPixels[(y*width +x)*4 + 2];
int green = bPixels[(y*width + x)*4 + 1];
int blue = bPixels[(y*width + x)*4];
redData[y*width + x] = red/255.0;
greenData[y*width + x] = green/255.0;
blueData[y*width + x] = blue/255.0;
alphaData[y*width + x] = alpha/255.0;
}
}
Image newImage(iPixels,width,height);
some of the things that can go wrong.
Source is not BGRA but RGBA, which will make the colors all wrong.
Not row major or destination is not in slices which will make things look all screwed up and/or seg-fault

QImage (or images generally) conversion to 3 1D arrays for RGB

A function that I am trying to conform to requires three 1-Dimensional arrays of type double[19200]. The following arrays are RGB arrays such that:
double r[19200]; // r
double g[19200]; // g
double b[19200]; // b
So far, I can extract pixel information from a QImage and populate the above arrays.
The problem is with testing. I don't know how to do the inverse: given the three 1-Dimensional arrays how do I create a new QImage from this data?
I would like to verify that I am indeed getting the correct values. (Things like column vs. row major order is giving me doubts). As a result, I am trying to construct an image a QImage from these three 1-D Dimensional arrays.
I don't really understand why you're having a problem if you managed to do it one way. The process is essentially the same:
for (int x=0; x<w; x++)
for (int y=0; y<h; y++)
image.setPixel(x,y, convertToRGB(r[x*w+y], ...);
Where convertToRGB is the inverse transform of what you to to convert and RGB value to your float values, supposing the image has dimension w*h. If you discover this is the wrong row-major/column major variant, just inverse it.
Since you gave no info about how you do the color space conversion, and we don't know if it's row-major or column-major either, can't help you much more than that.
Well it looks like QImage supports a couple of ways to load from pixel arrays.
QImage(const uchar *data, int width, int height, Format format)
bool QImage::loadFromData(const uchar *buf, int len, const char *format=0)
Using the first example, if you have the arrays you mention, then you will likely want to use the format QImage::Format_RGB888 (from qimage.h).
You will need to know the width and height yourself.
Finally you will want to repack your arrays into a single uchar* array
uchar* rgb_array = new uchar[19200+19200+19200];
for( int i = 0, j = 0; j < 19200; ++j )
{
// here we convert from the double range 0..1 to the integer range 0..255
rgb_array[i++] = r[j] * 255;
rgb_array[i++] = g[j] * 255;
rgb_array[i++] = b[j] * 255;
}
{
QImage my_image( rgb_array, width, height, QImage::Format_RGB888 );
// do stuff with my_image...
}
delete[] rgb_array; // note you need to hold onto this array while the image still exists