I have the following code that uses pointer arithmetic and would like to replace it using std::span (or, I suppose, gsl::span). The code iterates over a number of pixels, each represented by 4 contiguous bytes, and updates their blue and green colours.
auto* row = (uint8_t*)buffer->data;
for (auto y = 0; y < buffer->height; ++y) {
auto* pixel = (uint32_t*)row;
for (auto x = 0; x < buffer->width; ++x) {
auto blue = x + blueOffset;
auto green = y + greenOffset;
*pixel++ = ((green << 8) | blue);
}
row += buffer->pitch;
}
buffer->data is a void* returned from a call to Windows VirtualAlloc(...) function.
How can this code be written to use safe, modern C++, such as std::span, as suggested by the C++ Core Guidelines and Moderns C++?
This compiles with each of C++20, gsl and gsl-lite. As you didn't provide a reproducible example I didn't test or benchmark this solution.
auto const my_span = span{
reinterpret_cast<uint8_t *>(buffer->data),
buffer->height * buffer->pitch};
constexpr auto size_ratio = sizeof(uint32_t) / sizeof(uint8_t); // 4
auto const uint8_width = size_ratio * buffer->width;
auto row_offset = 0UL;
auto row = my_span.subspan(row_offset, buffer->width);
for (auto y = 0; y < buffer->height; ++y) {
auto const pixels = span{
reinterpret_cast<uint32_t *>(row.data()),
buffer->width};
auto pixel = pixels.begin();
for (auto x = 0; x < buffer->width; ++x) {
auto const blue = x + blueOffset;
auto const green = y + greenOffset;
*pixel++ = ((green << 8) | blue);
}
row_offset += buffer->pitch;
row = my_span.subspan(row_offset, uint8_width);
}
Obviously using spans here doesn't necessarily make the code easier to read due to interpreting the same memory both as uint8_t and uint32_t. The spans should still give more safety.
The code could be made easier to read by providing more members or member functions in the struct which is pointed at by buffer. E.g. member functions could provide you with the needed subspans or at least with uint8_width. As this wasn't asked, I didn't touch that struct here. It might be given by a library, but one could still write a wrapper in that case.
Related
I am currently working on a project that manipulates images. To speed up the process (and increase my knowledge), I decided to write some of the basic functions using SIMD instructions.
The code using for loops is
int idx;
uint16_t* A, B, C;
float gAlpha = 0.8;
float alpha = 0.2;
for (size_t rw = 0; rw < height; rw++) {
for (size_t cl = 0; cl < width; cl++) {
idx = rw * width + height;
C[idx] = static_cast<uint16_t>(gAlpha * static_cast<float>(A[idx]) + alpha * static_cast<float>(B[idx]));
}
}
}
This loop is probably not perfect but it makes its job perfectly and my unit test gives me the expected results.
As I said, I am trying to convert these loops using SIMD intrinsic. This is my working code and, as you will see, it is not very pretty... We do have access to intrinsic up to AVX2.
size_t n_pixels = height * width;
for (size_t px = 0; px < n_pixels; px += 8) {
__m128i xlo = _mm_unpacklo_epi16(_mm_load_si128((__m128i*)&A[px]), _mm_set1_epi16(0));
__m128i xhi = _mm_unpackhi_epi16(_mm_load_si128((__m128i*)&A[px]), _mm_set1_epi16(0));
__m128 ylo = _mm_cvtepi32_ps(xlo);
__m128 yhi = _mm_cvtepi32_ps(xhi);
__m256 pxMinFl = _mm256_castps128_ps256(ylo);
pxMinFl = _mm256_insertf128_ps(pxMinFl, yhi, 1);
xlo = _mm_unpacklo_epi16(_mm_load_si128((__m128i*)&B[px]), _mm_set1_epi16(0));
xhi = _mm_unpackhi_epi16(_mm_load_si128((__m128i*)&B[px]), _mm_set1_epi16(0));
ylo = _mm_cvtepi32_ps(xlo);
yhi = _mm_cvtepi32_ps(xhi);
__m256 pxMaxFl = _mm256_castps128_ps256(ylo);
pxMaxFl = _mm256_insertf128_ps(pxMaxFl, yhi, 1);
__m256 avGain1 = _mm256_set1_ps(gAlpha);
__m256 avGain2 = _mm256_set1_ps(alpha);
__m256 prodUp = _mm256_mul_ps(prodUp, avGain1);
__m256 prodBt = _mm256_mul_ps(prodBt, avGain2);
__m256 pxOutFl = _mm256_add_ps(prodUp, prodBt);
__m128 ylo_ps = _mm256_castps256_ps128(pxOutFl);
__m128 yhi_ps = _mm256_extractf128_ps(pxOutFl, 1);
__m128i xlo_ep = _mm_cvtps_epi32(ylo_ps);
__m128i xhi_ep = _mm_cvtps_epi32(yhi_ps); <- POINT 1
int* xl = reinterpret_cast<int*>(&xlo_ep); <- POINT 2
for (int i=0; i < 8; i++) { <- POINT 2
C[px + i] = static_cast<uint16_t>(xl[i]); <- POINT 2
}
}
There are probably tons of optimization that could be done on this code but I have checked that the output of pxOutFl corresponds to the expected value. Where is start to look like black magic to me is when I looked at how I had to save the data back into the output array C. First of all, the code doesn't work if I comment the line at POINT 1 even if, as you can read, I don't use the variable. Secondly, I would have guessed that there is a better solution than the trick I used to store the data back into the uint16_t array (POINT 2) but I can't find one that is working.
Could someone point me into the correct direction? What am I missing? How could I improve this code?
Thanks in advance!
PS: We use the Intel compiler 2017 for the parallel studio professional edition 2117 on Linux (Fedora 25).
You can re-write all of POINT 2 as:
_mm_storeu_si128((__m128i *)&C[px], xlo_ep);
Also note that all instances of _mm_load_si128 should probably be _mm_loadu_si128, since you don't seem to be guaranteeing alignment anywhere.
I want to realize the function of fill-light by use OpenCV, but There have some problem. Black part of pics is too dark, Photos become blurred, i don't know how to Optimization code。that my code:
V, value, 0~100, increase the amplitude of the brightness.
S,Scope, 0~255, dark is all less than S.
increase exposure to light dark photos increment, unchanged, so to see more details of the dark.
m_imgOriginal: original image ,type:Mat
m_imgNew: new image , clone from m_imgOriginal ,type:Mat
int OpenCVClass::AddExposure(int v, int s)
{
int new_r = v*m_mean_val.val[0] / 150;
int new_g = v*m_mean_val.val[1] / 150;
int new_b = v*m_mean_val.val[2] / 150;
for (int y = 0; y < m_imgOriginal.rows; y++)
{
auto ptr = m_imgOriginal.ptr<uchar>(y);
auto qtr = m_imgNew.ptr<uchar>(y);
for (int x = 0; x < m_imgOriginal.cols; x++)
{
int mean = (ptr[0] + ptr[1] + ptr[2]) / 3;
if (mean <= s)
{
int r = ptr[0] + new_r;
qtr[0] = r>255 ? 255 : r;
int g = ptr[1] + new_g;
qtr[1] = g>255 ? 255 : g;
int b = ptr[2] + new_b;
qtr[2] = b>255 ? 255 : b;
int newMean = (qtr[0] + qtr[1] + qtr[2]) / 3;
if (newMean > s)
{
int nr = ptr[0] + (s - mean) ;
int ng = ptr[1] + (s - mean) ;
int nb = ptr[2] + (s - mean) ;
qtr[0] = nr>255 ? 255 : nr;
qtr[1] = ng>255 ? 255 : ng;
qtr[2] = nb>255 ? 255 : nb;
}
}
else
{
qtr[0] = ptr[0];
qtr[1] = ptr[1];
qtr[2] = ptr[2];
}
ptr += 3;
qtr += 3;
}
RenderBuffer(m_imgNew, m_displayBuffer);
}
return 0;
}
Optimization before
Optimization after
First, I would suggest to calculate a luminance value for each pixel, when testing agains 's'. I mean calculate 'mean' a different way (see this link on how to calculate luminance):
http://www.niwa.nu/2013/05/math-behind-colorspace-conversions-rgb-hsl/
Second, you are dealing with an 8 bit per channel image, don't expect near-or-perfect dark pixels to have any extra detail when you make them "brighter", they will just become grey or whiter.
Third, when "adding" brightness, I suggest using the HSL representation of pixel color values and increasing the luminance. In pseudocode:
1) Convert pixel color from RGB to HSL.
2) Increase luminance (or 'lightness').
3) Convert back pixel color to RGB.
Below is my program. I am trying to apply grayscale filter using bitmapdata class in visual c++. I am getting AccessViolationException at 11, tagged by the comment. I have tried using CLR:Safe and CLR:pure but no use. In c# this would be solved by using unsafe block. Any suggestions? None of the other solutions on related questions worked.
Bitmap^ bmp = gcnew Bitmap(pictureBox1->Image);
BitmapData^ data = bmp->LockBits(Rectangle(0,0,bmp->Width,bmp->Height), ImageLockMode::ReadWrite, PixelFormat::Format24bppRgb);
int blue=0, green=0, red=0;
System::IntPtr s = data->Scan0;
int* P = (int*)(void*)s;
for (int i =0; i<bmp->Height;i++)
{
for (int j = 0; j < bmp->Width*3; j++)
{
blue = (int)P[0]; //access violation exception
green =(int )P[1];
red = (int)P[2];
int avg = (int)((blue + green + red) / 3);
P[0] = avg;
P[1] = avg;
P[2] = avg;
P +=3;
}
}
bmp->UnlockBits(data);
pictureBox1->Image = bmp;
You are using an int* when you should be using a byte*. Your pixels are three bytes each, one byte per channel. Your int is (likely) 4 bytes, so p[0] returns an entire pixel plus on byte past it. This is why you get an access violation; you are overrunning the bounds of the image buffer.
When you increment a pointer, you are adding sizeof *p bytes to it. In this case, P += 3 increments the pointer P by 12 bytes. Much too much, and you'll never be able to read a single pixel (or channel) of a 24bpp image with an int*. You are also assuming that your stride is Width * 3, which may or may not be correct (bitmaps are 4 byte aligned.)
Byte* base = (Byte*)data->Scan0;
int stride = data->Stride;
for(int y = 0; y < data->Height; ++y) {
Byte* src = base + y * stride;
for(int x = 0; x < data->Width; ++x, src += 3) {
// bitmaps are stored in BGR order (though not really important here).
// I'm assuming a 24bpp bitmap.
Byte b = src[0];
Byte g = src[1];
Byte r = src[2];
int average = (r + g + b) / 3;
src[0] = src[1] = src[2] = (Byte)average;
}
}
I am trying to parallelize a convolution filter using C++Amp. I would like the following function to start working (I don't know how to do it properly):
float* pixel_color[] = new float [16];
concurrency::array_view<float, 2> pixels(4, 4, pixel_array), taps(4, 4, myTap4Kernel_array);
concurrency::array_view<float, 1> pixel(16, pixel_color); // I don't know which data structure to use here
parallel_for_each(
pixels.extent, [=](concurrency::index<2> idx) restrict(amp)
{
int row=idx[0];
int col=idx[1];
pixels(row, col) = taps(row, col) * pixels(row, col);
pixel[0] += pixels(row, col);
});
pixel_color.synchronize();
pixels_.at<Pixel>(j, i) = pixel_color
}
The main problem is that I don't know how to use the pixel structure properly (which concurrent data structure to use here as I don't need all 16 elements). And I don't know if I can safely add the values this way.
The following code doesn't work, it does not add appropriate values to pixel[0].
I also would like to define
concurrency::array_view<float, 2> pixels(4, 4, pixel_array), taps(4, 4, myTap4Kernel_array);
outside the method (for example in the header file) and initialize it in the costructor or other function (as this is a bottle-neck and takes a lot of time copying the data between CPU and GPU). Does anybody know how to do this?
You're no the right track but doing in place manipulations of arrays on a GPU is tricky as you cannot guarantee the order in which different elements are updated.
Here's an example of something very similar. The ApplyColorSimplifierTiledHelper method contains an AMP restricted parallel_for_each that calls SimplifyIndexTiled for each index in the 2D array. SimplifyIndexTiled calculates a new value for each pixel in destFrame based on the value of the pixels surrounding the corresponding pixel in srcFrame. This solves the race condition issue present in your code.
This code comes from the Codeplex site for the C++ AMP book. The Cartoonizer case study includes several examples of these sorts of image processing problems implemented in C++ AMP using; arrays, textures, tiled/untiled and multi-GPU. The C++ AMP book discusses the implementation in some detail.
void ApplyColorSimplifierTiledHelper(const array<ArgbPackedPixel, 2>& srcFrame,
array<ArgbPackedPixel, 2>& destFrame, UINT neighborWindow)
{
const float_3 W(ImageUtils::W);
assert(neighborWindow <= FrameProcessorAmp::MaxNeighborWindow);
tiled_extent<FrameProcessorAmp::TileSize, FrameProcessorAmp::TileSize>
computeDomain = GetTiledExtent(srcFrame.extent);
parallel_for_each(computeDomain, [=, &srcFrame, &destFrame]
(tiled_index<FrameProcessorAmp::TileSize, FrameProcessorAmp::TileSize> idx)
restrict(amp)
{
SimplifyIndexTiled(srcFrame, destFrame, idx, neighborWindow, W);
});
}
void SimplifyIndex(const array<ArgbPackedPixel, 2>& srcFrame, array<ArgbPackedPixel,
2>& destFrame, index<2> idx,
UINT neighborWindow, const float_3& W) restrict(amp)
{
const int shift = neighborWindow / 2;
float sum = 0;
float_3 partialSum;
const float standardDeviation = 0.025f;
const float k = -0.5f / (standardDeviation * standardDeviation);
const int idxY = idx[0] + shift; // Corrected index for border offset.
const int idxX = idx[1] + shift;
const int y_start = idxY - shift;
const int y_end = idxY + shift;
const int x_start = idxX - shift;
const int x_end = idxX + shift;
RgbPixel orgClr = UnpackPixel(srcFrame(idxY, idxX));
for (int y = y_start; y <= y_end; ++y)
for (int x = x_start; x <= x_end; ++x)
{
if (x != idxX || y != idxY) // don't apply filter to the requested index, only to the neighbors
{
RgbPixel clr = UnpackPixel(srcFrame(y, x));
float distance = ImageUtils::GetDistance(orgClr, clr, W);
float value = concurrency::fast_math::pow(float(M_E), k * distance * distance);
sum += value;
partialSum.r += clr.r * value;
partialSum.g += clr.g * value;
partialSum.b += clr.b * value;
}
}
RgbPixel newClr;
newClr.r = static_cast<UINT>(clamp(partialSum.r / sum, 0.0f, 255.0f));
newClr.g = static_cast<UINT>(clamp(partialSum.g / sum, 0.0f, 255.0f));
newClr.b = static_cast<UINT>(clamp(partialSum.b / sum, 0.0f, 255.0f));
destFrame(idxY, idxX) = PackPixel(newClr);
}
The code uses ArgbPackedPixel, which is simply a mechanism for packing 8-bit RGB values into an unsigned long as C++ AMP does not support char. If your problem is small enough to fit into a texture then you may want to look at using this instead of an array as the pack/unpack is implemented in hardware on the GPU so is effectively "free", here you have to pay for it with additional compute. There is also an example of this implementation on CodePlex.
typedef unsigned long ArgbPackedPixel;
struct RgbPixel
{
unsigned int r;
unsigned int g;
unsigned int b;
};
const int fixedAlpha = 0xFF;
inline ArgbPackedPixel PackPixel(const RgbPixel& rgb) restrict(amp)
{
return (rgb.b | (rgb.g << 8) | (rgb.r << 16) | (fixedAlpha << 24));
}
inline RgbPixel UnpackPixel(const ArgbPackedPixel& packedArgb) restrict(amp)
{
RgbPixel rgb;
rgb.b = packedArgb & 0xFF;
rgb.g = (packedArgb & 0xFF00) >> 8;
rgb.r = (packedArgb & 0xFF0000) >> 16;
return rgb;
}
First of all, I'm programming in the kernel context so no existing libraries exist. In fact this code is going to go into a library of my own.
Two questions, one more important than the other:
As the title suggests, how can I efficiently render a 24-bpp image onto a 32-bpp device, assuming that I have the address of the frame buffer?
Currently I have this code:
void BitmapImage::Render24(uint16_t x, uint16_t y, void (*r)(uint16_t, uint16_t, uint32_t))
{
uint32_t imght = Math::AbsoluteValue(this->DIB->GetBitmapHeight());
uint64_t ptr = (uint64_t)this->ActualBMP + this->Header->BitmapArrayOffset;
uint64_t rowsize = ((this->DIB->GetBitsPerPixel() * this->DIB->GetBitmapWidth() + 31) / 32) * 4;
uint64_t oposx = x;
uint64_t posx = oposx;
uint64_t posy = y + (this->DIB->Type == InfoHeaderV1 && this->DIB->GetBitmapHeight() < 0 ? 0 : this->DIB->GetBitmapHeight());
for(uint32_t d = 0; d < imght; d++)
{
for(uint32_t w = 0; w < rowsize / (this->DIB->GetBitsPerPixel() / 8); w++)
{
r(posx, posy, (*((uint32_t*)ptr) & 0xFFFFFF));
ptr += this->DIB->GetBitsPerPixel() / 8;
posx++;
}
posx = oposx;
posy--;
}
}
r is a function pointer to a PutPixel-esque thing that accepts x, y, and colour parameters.
Obviously this code is terribly slow, since plotting pixels one at a time is never a good idea.
For my 32-bpp rendering code (which I also have a question about, more on that later) I can easily Memory::Copy() the bitmap array (I'm loading bmp files here) to the frame buffer.
However, how do I do this with 24bpp images? On a 24bpp display this would be fine but I'm working with a 32bpp one.
One solution I can think of right now is to create another bitmap array which essentially contains values of 0x00(colour) and the use that to draw to the screen -- I don't think this is very good though, so I'm looking for a better alternative.
Next question:
2. Given, for obvious reasons, one cannot simply Memory::Copy() the entire array at once onto the frame buffer, the next best thing would be to copy them row by row.
Is there a better way?
Basically something like this:
for (uint32_t l = 0; l < h; ++l) // l line index in pixels
{
// srcPitch is distance between lines in bytes
char* srcLine = (char*)srcBuffer + l * srcPitch;
unsigned* trgLine = ((unsigned*)trgBuffer) + l * trgPitch;
for (uint32_t c = 0; c < w; ++c) // c is column index in pixels
{
// build target pixel. arrange indexes to fit your render target (0, 1, 2)
++(*trgLine) = (srcLine[0] << 16) | (srcLine[1] << 8)
| srcLine[2] | (0xff << 24);
srcLine += 3;
}
}
A few notes:
- better to write to a different buffer than the render buffer so the image is displayed at once.
- using functions for pixel placement like you did is very (very very) slow.