For some reason on machines that only conditionally (D3DPTEXTURECAPS_NONPOW2CONDITIONAL) supports the "non power of two" (NONPOW2) textures I'm getting this kind of image distortion:
(On machines that fully supports NONPOW2 textures everything works fine.)
This is a 1280x720 resolution RGB24 (D3DFMT_X8R8G8B8).
I know there are four rules when dealing with conditionally supported textures:
1.The texture addressing mode for the texture stage is set to D3DTADDRESS_CLAMP.
mpDevice->SetSamplerState(0, D3DSAMP_ADDRESSU, D3DTADDRESS_CLAMP);
mpDevice->SetSamplerState(0, D3DSAMP_ADDRESSV, D3DTADDRESS_CLAMP);
mpDevice->SetSamplerState(0, D3DSAMP_ADDRESSW, D3DTADDRESS_CLAMP);
2.Texture wrapping for the texture stage is disabled (D3DRS_WRAP n set to 0).
mpDevice->SetRenderState(D3DRS_WRAP0, 0);
3.Mipmapping is not in use (use magnification filter only).
mpDevice->SetSamplerState(0, D3DSAMP_MIPFILTER, D3DTEXF_NONE);
4.Texture formats must not be D3DFMT_DXT1 through D3DFMT_DXT5.
I'm using D3DFMT_X8R8G8B8...
Texture is created like this:
mpDevice->CreateTexture(width, height, 1, D3DUSAGE_DYNAMIC, D3DFMT_X8R8G8B8, D3DPOOL_DEFAULT, &pTexture, nullptr);
And updated like this:
D3DLOCKED_RECT lockedRect;
pTexture->LockRect(0, &lockedRect, nullptr, D3DLOCK_DISCARD);
// const IppiSize roiSize = { width, height };
// ippiCopy_8u_C3AC4R((Ipp8u*) pPixels, width * 3, (Ipp8u*) lockedRect.pBits, width * 4, roiSize); // RGB to RGBA
// ippiCopy_8u_AC4C3R((Ipp8u*) pPixels, width * 4, (Ipp8u*) lockedRect.pBits, width * 3, roiSize); // RGBA to RGB
// TEMP.
unsigned char* pDst = (unsigned char*) lockedRect.pBits;
const unsigned char* pSrc = pPixels;
for (int k = 0; k < width; k++)
{
for (int j = 0; j < height; j++)
{
pDst[0] = pSrc[0];
pDst[1] = pSrc[1];
pDst[2] = pSrc[2];
pDst[3] = 255;
pDst += 4;
pSrc += 3;
}
}
pTexture->UnlockRect(0);
Does anyone had any similar problem? Or have any idea on what is going on?
Maybe there's something wrong with the RGB data itself?
Thanks.
Related
I need to create a bitmap from an array of pixels for a raycaster I'm working on in Direct2D. However, I'm having trouble understanding how to use the CreateBitmap function. Specifically, I'm not sure what the srcData parameter is supposed to be. I'm pretty sure/hoping it's a pointer to an array of pixels, but I'm not sure how to set up that array. What kind of array is it supposed to be? What data type? Etc.
Here's what I've tried:
int width = 400, height = 400;
D2D1::ColorF * arr = (D2D1::ColorF*)calloc(width * height * 4, sizeof(D2D1::ColorF));
for (int i = 0; i < width * height * 4; i++) { arr[i] = D2D1::ColorF(0.0f, 1.0f, 0.0f); }
// Create the bitmap and draw it on the screen
ID2D1Bitmap * bmp;
HRESULT hr;
hr = renderTarget->CreateBitmap(
D2D1::SizeU(width, height),
arr,
width * sizeof(int) * 4,
D2D1::BitmapProperties(),
&bmp);
if (hr != S_OK) { return; } // I've tested and found that hr does not equal S_OK
// Draw the bitmap...
What should the second and third lines look like? Is there anything else I'm doing incorrectly?
Syntax:
HRESULT CreateBitmap(
D2D1_SIZE_U size,
const void *srcData,
UINT32 pitch,
const D2D1_BITMAP_PROPERTIES & bitmapProperties,
ID2D1Bitmap **bitmap
);
Your code:
hr = renderTarget->CreateBitmap(
D2D1::SizeU(width, height),
arr, // <<--- Wrong, see (a) below
width * sizeof(int) * 4, // <<--- Close but wrong, see (b) below
D2D1::BitmapProperties(), // <<--- Wrong, see (c) below
&bmp);
(a) - you are supposed to provide an array of pixel data here, where the format depends on format of the bitmap. Note that this is optional an d you can create a bitmap without initialization. The pixels are not D2D1::ColorF exactly. They could be 4 byte RGBA data if you request respective bitmap format, see (c) below.
(b) - this is distance between rows in bytes, if your pixels are supposed to be 32-bit values you would normally want Width * 4 here
(c) - this requests DXGI_FORMAT_UNKNOWN D2D1_ALPHA_MODE_UNKNOWN and results in bitmap creation error. You need a real format here such as DXGI_FORMAT_B8G8R8A8_UNORM (see Pixel Formats and also Supported Pixel Formats and Alpha Modes)
The first link above shows how exactly bytes in memory map to pixel colors, and you are supposed to prepare your data respectively.
UPD
With DXGI_FORMAT_B8G8R8A8_UNORM your initialization structure is this:
UINT8* Data = malloc(Height * Width * 4);
for(UINT Y = 0; Y < Height; Y++)
for(UINT X = 0; X < Width; X++)
{
UINT8* PixelData = Data + ((Y * Width) + X) * 4;
PixelData[0] = unsigned integer blue in range 0..255;
PixelData[1] = unsigned integer red in range 0..255;
PixelData[2] = unsigned integer green in range 0..255;
PixelData[3] = 255;
}
I am trying to read the pixel data from a populated UTexture2D in an Unreal Engine C++ project. Before I post the question here, I tried to use the method described in this link: https://answers.unrealengine.com/questions/25594/accessing-pixel-values-of-texture2d.html. However, it doesn't work for me. All pixel values I got from the texture are some garbage data.
I just want to get the depth values from the SceneCapture2D and a post-processing material that contains SceneTexture: Depth node. I need the depth values available in C++ so that I can do further processing with OpenCV. In Directx11, staging texture can be used for CPU read, but in the unreal engine, I don't know how to create a 'staging texture' like Dx11 has. I can't get the correct pixel values from the current method which makes me think I may try to access a no-CPU-readable texture.
Here is my experimental code for reading data back from an RGB UTexture2D.
Initialize the RGB Texture:
VideoTextureColor= UTexture2D::CreateTransient(640, 480, PF_B8G8R8A8);
VideoTextureColor->UpdateResource();
VideoUpdateTextureRegionColor = new FUpdateTextureRegion2D(0, 0, 0, 0, 640, 480);
ColorRegionData = new FUpdateTextureRegionsData;
PixelDepthData.Init(FColor(0, 0, 0, 255), 640 * 480);
// Populate the texture with blue color
for (int i = 0; i < 640; i++) {
for (int j = 0; j < 480; j++) {
int idx = j * 640 + i;
PixelDepthData[idx].B = 255;
PixelDepthData[idx].G = 0;
PixelDepthData[idx].R = 0;
PixelDepthData[idx].A = 255;
}
}
UpdateTextureRegions(
VideoTextureColor,
(int32)0,
(uint32)1,
VideoUpdateTextureRegionColor,
(uint32)(4 * 640),
(uint32)4,
(uint8*)PixelDepthData.GetData(),
false,
ColorRegionData
);
Then, update read its value back to the PixelDepthData (TArray type) array and update this texture with values storing in the PixelDepthData, which is its old value.
UpdateTextureRegions(
VideoTextureColor,
(int32)0,
(uint32)1,
VideoUpdateTextureRegionColor,
(uint32)(4 * 640),
(uint32)4,
(uint8*)PixelDepthData.GetData(),
false,
ColorRegionData
);
ENQUEUE_UNIQUE_RENDER_COMMAND_ONEPARAMETER(
FRealSenseDelegator,
ARealSenseDelegator*, RealSenseDelegator, this,
{
FColor* tmpImageDataPtr = static_cast<FColor*>((RealSenseDelegator->VideoTextureColor)->PlatformData->Mips[0].BulkData.Lock(LOCK_READ_ONLY));
for (uint32 j = 0; j < 480; j++) {
for (uint32 i = 0; i < 640; i++) {
uint32 idx = j * 640 + i;
RealSenseDelegator->PixelDepthData[idx] = tmpImageDataPtr[idx];
RealSenseDelegator->PixelDepthData[idx].A = 255;
}
}
(RealSenseDelegator->VideoTextureColor)->PlatformData->Mips[0].BulkData.Unlock();
}
);
All I got is a white color texture instead of a blue color texture in the visualization scene.
Does anyone know how to read the data of the UTexture2D Object?
I figured that out. You have to get the UTexture2D's RHI texture reference first, and then use RHILockTexture2D to read it's data, and you have to do it in the RenderThread. The following code just an example:
FTexture2DResource* uTex2DRes = (FTexture2DResource*)(RealSenseDelegator->VideoTexturePixelDepth)->Resource;
float* cpuDataPtr = (float*)RHILockTexture2D(
uTex2DRes->GetTexture2DRHI(),
0,
RLM_ReadOnly,
destStride,
false);
for (uint32 j = 0; j < 480; j++) {
for (uint32 i = 0; i < 640; i++) {
uint32 idx = j * 640 + i;
// TODO Read the pixel data right here
}
}
RHIUnlockTexture2D(uTex2DRes->GetTexture2DRHI(), 0, false);
To do this in the Render Thread, you have to use the Macro such as ENQUEUE_UNIQUE_RENDER_COMMAND_ONEPARAMETER // If you just one to pass one parameter to the render thread, use this one.+
So I'm trying to create some simple terrain using a bmp and I don't know how to access the pixel color to get it's RGB value to use as the height. I understand the concept, just not how to put it into practice. This is the code I have so far. Any help is very much appreciated!
CUSTOMVERTEX vecArray[256][256];
m_pSurface = nullptr;
D3DXIMAGE_INFO imageInfo;
ZeroMemory(&imageInfo, sizeof(D3DXIMAGE_INFO));
HRESULT hr = D3DXGetImageInfoFromFile(L"heightmap.bmp", &imageInfo);
_pDevice->CreateOffscreenPlainSurface(imageInfo.Width, imageInfo.Height, D3DFMT_X8R8G8B8, D3DPOOL_SCRATCH, &m_pSurface, 0);
hr = D3DXLoadSurfaceFromFile(m_pSurface, 0, 0, L"heightmap.bmp", 0, D3DX_FILTER_NONE, 0, &imageInfo);
D3DLOCKED_RECT lockRect;
ZeroMemory(&lockRect, sizeof(D3DLOCKED_RECT));
m_pSurface->LockRect(&lockRect, 0, D3DLOCK_READONLY);
int iNumPixels = imageInfo.Width * imageInfo.Height;
int iPixelsWidth = imageInfo.Width;
int iPixelsHeight = imageInfo.Height;
for (int i = 0; i < iPixelsWidth; ++i) // HORIZONTAL ROWS
{
for (int j = 0; j < iPixelsHeight; ++j) // VERTICAL ROWS
{
vecArray[i][j].x = (float)i;
vecArray[i][j].y = (float)j;
vecArray[i][j].z = ???? // Get Height from bmp
vecArray[i][j].color = D3DCOLOR_XRGB(255, 255, 255);
}
}
m_pSurface->UnlockRect();
The struct is define as:
struct CUSTOMVERTEX
{
FLOAT x, y, z; // The untransformed, 3D position for the vertex
DWORD color;
};
Once you have the pixels mapped into a buffer, interpolate the color value as an integer (range 0 to 255) to your height range. If you are using a 3-channel image (rgb) you will need to first average the channels to one value (ie. (r + g + b) / 3). That will give you a single value for the pixel.
See "linear interpolation" to convert from pixel range to height range. If you're asking how to access the pixel values themselves, that depends on the method used to load the bitmap.
I'm trying to get the Kinect depth camera pixels to overlay onto the RGB camera. I am using the C++ Kinect 1.0 SDK with an Xbox Kinect, OpenCV and trying to use the new "NuiImageGetColorPixelCoordinateFrameFromDepthPixelFrameAtResolution" method.
I have watched the image render itself in slow motion and looks as if pixels are being drawn multiple times in the one frame. It first draws itself from the top and left borders, then it gets to a point (you can see a 45 degree angle in there) where it starts drawing weird.
I have been trying to base my code off of the C# code written by Adam Smith at the MSDN forums but no dice. I have stripped out the overlay stuff and just want to draw the depth normalized depth pixels where it "should" be in the RGB image.
The image on the left is what I'm getting when trying to fit the depth image to RGB space, and the image on the right is the "raw" depth image as I like to see it. I was hoping this my method would create a similar image to the one on the right with slight distortions.
This is the code and object definitions that I have at the moment:
// From initialization
INuiSensor *m_pNuiInstance;
NUI_IMAGE_RESOLUTION m_nuiResolution = NUI_IMAGE_RESOLUTION_640x480;
HANDLE m_pDepthStreamHandle;
IplImage *m_pIplDepthFrame;
IplImage *m_pIplFittedDepthFrame;
m_pIplDepthFrame = cvCreateImage(cvSize(640, 480), 8, 1);
m_pIplFittedDepthFrame = cvCreateImage(cvSize(640, 480), 8, 1);
// Method
IplImage *Kinect::GetRGBFittedDepthFrame() {
static long *pMappedBits = NULL;
if (!pMappedBits) {
pMappedBits = new long[640*480*2];
}
NUI_IMAGE_FRAME pNuiFrame;
NUI_LOCKED_RECT lockedRect;
HRESULT hr = m_pNuiInstance->NuiImageStreamGetNextFrame(m_pDepthStreamHandle, 0, &pNuiFrame);
if (FAILED(hr)) {
// return the older frame
return m_pIplFittedDepthFrame;
}
bool hasPlayerData = HasSkeletalEngine(m_pNuiInstance);
INuiFrameTexture *pTexture = pNuiFrame.pFrameTexture;
pTexture->LockRect(0, &lockedRect, NULL, 0);
if (lockedRect.Pitch != 0) {
cvZero(m_pIplFittedDepthFrame);
hr = m_pNuiInstance->NuiImageGetColorPixelCoordinateFrameFromDepthPixelFrameAtResolution(
m_nuiResolution,
NUI_IMAGE_RESOLUTION_640x480,
640 * 480, /* size is previous */ (unsigned short*) lockedRect.pBits,
(640 * 480) * 2, /* size is previous */ pMappedBits);
if (FAILED(hr)) {
return m_pIplFittedDepthFrame;
}
for (int i = 0; i < lockedRect.size; i++) {
unsigned char* pBuf = (unsigned char*) lockedRect.pBits + i;
unsigned short* pBufS = (unsigned short*) pBuf;
unsigned short depth = hasPlayerData ? ((*pBufS) & 0xfff8) >> 3 : ((*pBufS) & 0xffff);
unsigned char intensity = depth > 0 ? 255 - (unsigned char) (256 * depth / 0x0fff) : 0;
long
x = pMappedBits[i], // tried with *(pMappedBits + (i * 2)),
y = pMappedBits[i + 1]; // tried with *(pMappedBits + (i * 2) + 1);
if (x >= 0 && x < m_pIplFittedDepthFrame->width && y >= 0 && y < m_pIplFittedDepthFrame->height) {
m_pIplFittedDepthFrame->imageData[x + y * m_pIplFittedDepthFrame->widthStep] = intensity;
}
}
}
pTexture->UnlockRect(0);
m_pNuiInstance->NuiImageStreamReleaseFrame(m_pDepthStreamHandle, &pNuiFrame);
return(m_pIplFittedDepthFrame);
}
Thanks
I have found that the problem was that the loop,
for (int i = 0; i < lockedRect.size; i++) {
// code
}
was iterating on a per-byte basis, not on a per-short (2 bytes) basis. Since lockedRect.size returns the number of bytes the fix was simply changing the increment to i += 2, even better would be changing it to sizeof(short), like so,
for (int i = 0; i < lockedRect.size; i += sizeof(short)) {
// code
}
I need to create a CImage from a byte array (actually, its an array of unsigned char, but I can cast to whatever form is necessary). The byte array is in the form "RGBRGBRGB...". The new image needs to contain a copy of the image bytes, rather than using the memory of the byte array itself.
I have tried many different ways of achieving this -- including going through various HBITMAP creation functions, trying to use BitBlt -- and nothing so far has worked.
To test whether the function works, it should pass this test:
BYTE* imgBits;
int width;
int height;
int Bpp; // BYTES per pixel (e.g. 3)
getImage(&imgBits, &width, &height, &Bpp); // get the image bits
// This is the magic function I need!!!
CImage img = createCImage(imgBits, width, height, Bpp);
// Test the image
BYTE* data = img.GetBits(); // data should now have the same data as imgBits
All implementations of createCImage() so far have ended up with data pointing to an empty (zero filled) array.
CImage supports DIBs quite neatly and has a SetPixel() method so you could presumably do something like this (uncompiled, untested code ahead!):
CImage img;
img.Create(width, height, 24 /* bpp */, 0 /* No alpha channel */);
int nPixel = 0;
for(int row = 0; row < height; row++)
{
for(int col = 0; col < width; col++)
{
BYTE r = imgBits[nPixel++];
BYTE g = imgBits[nPixel++];
BYTE b = imgBits[nPixel++];
img.SetPixel(row, col, RGB(r, g, b));
}
}
Maybe not the most efficient method but I should think it is the simplest approach.
Use memcpy to copy the data, then SetDIBits or SetDIBitsToDevice depending on what you need to do. Take care though, the scanlines of the raw image data are aligned on 4-byte boundaries (IIRC, it's been a few years since I did this) so the data you get back from GetDIBits will never be exactly the same as the original data (well it might, depending on the image size).
So most likely you will need to memcpy scanline by scanline.
Thanks everyone, I managed to solve it in the end with your help. It mainly involved #tinman and #Roel's suggestion to use SetDIBitsToDevice(), but it involved a bit of extra bit-twiddling and memory management, so I thought I'd share my end-point here.
In the code below, I assume that width, height and Bpp (Bytes per pixel) are set, and that data is a pointer to the array of RGB pixel values.
// Create the header info
bmInfohdr.biSize = sizeof(BITMAPINFOHEADER);
bmInfohdr.biWidth = width;
bmInfohdr.biHeight = -height;
bmInfohdr.biPlanes = 1;
bmInfohdr.biBitCount = Bpp*8;
bmInfohdr.biCompression = BI_RGB;
bmInfohdr.biSizeImage = width*height*Bpp;
bmInfohdr.biXPelsPerMeter = 0;
bmInfohdr.biYPelsPerMeter = 0;
bmInfohdr.biClrUsed = 0;
bmInfohdr.biClrImportant = 0;
BITMAPINFO bmInfo;
bmInfo.bmiHeader = bmInfohdr;
bmInfo.bmiColors[0].rgbBlue=255;
// Allocate some memory and some pointers
unsigned char * p24Img = new unsigned char[width*height*3];
BYTE *pTemp,*ptr;
pTemp=(BYTE*)data;
ptr=p24Img;
// Convert image from RGB to BGR
for (DWORD index = 0; index < width*height ; index++)
{
unsigned char r = *(pTemp++);
unsigned char g = *(pTemp++);
unsigned char b = *(pTemp++);
*(ptr++) = b;
*(ptr++) = g;
*(ptr++) = r;
}
// Create the CImage
CImage im;
im.Create(width, height, 24, NULL);
HDC dc = im.GetDC();
SetDIBitsToDevice(dc, 0,0,width,height,0,0, 0, height, p24Img, &bmInfo, DIB_RGB_COLORS);
im.ReleaseDC();
delete[] p24Img;
Here is a simpler solution. You can use GetPixelAddress(...) instead of all this BITMAPHEADERINFO and SedDIBitsToDevice. Another problem I have solved was with 8-bit images, which need to have the color table defined.
CImage outImage;
outImage.Create(width, height, channelCount * 8);
int lineSize = width * channelCount;
if (channelCount == 1)
{
// Define the color table
RGBQUAD* tab = new RGBQUAD[256];
for (int i = 0; i < 256; ++i)
{
tab[i].rgbRed = i;
tab[i].rgbGreen = i;
tab[i].rgbBlue = i;
tab[i].rgbReserved = 0;
}
outImage.SetColorTable(0, 256, tab);
delete[] tab;
}
// Copy pixel values
// Warining: does not convert from RGB to BGR
for ( int i = 0; i < height; i++ )
{
void* dst = outImage.GetPixelAddress(0, i);
const void* src = /* put the pointer to the i'th source row here */;
memcpy(dst, src, lineSize);
}