How to downsample a not-power-of-2 texture in UnrealEngine? - c++

I am rendering the Viewport with a resolution of something like 1920x1080 multiplied by a Oversampling value like 4. Now i need to downsample from the rendered Resolution 7680‬x4320 back to the 1920x1080.
Are there any functions in Unreal I could use for that ? Or any Library (windows only) which handle this nicely ?
Or what would be a propper way of writing this my own ?
We tried to implement a downsampling but it only works if SnapshotScale is 2, when its higher than 2 it doesn't seem to have an effect regarding image quality.
UTexture2D* AAVESnapShotManager::DownsampleTexture(UTexture2D* Texture)
{
UTexture2D* Result = UTexture2D::CreateTransient(RenderSettings.imageWidth, RenderSettings.imageHeight, PF_B8G8R8A8);
void* TextureDataVoid = Texture->PlatformData->Mips[0].BulkData.Lock(LOCK_READ_ONLY);
void* ResultDataVoid = Result->PlatformData->Mips[0].BulkData.Lock(LOCK_READ_WRITE);
FColor* TextureData = (FColor*)TextureDataVoid;
FColor* ResultData = (FColor*)ResultDataVoid;
int32 WindowSize = RenderSettings.resolutionScale / 2;
for (int x = 0; x < Result->GetSizeX(); ++x)
{
for (int y = 0; y < Result->GetSizeY(); ++y)
{
const uint32 ResultIndex = y * Result->GetSizeX() + x;
uint32_t R = 0, G = 0, B = 0, A = 0;
int32 Samples = 0;
for (int32 dx = -WindowSize; dx < WindowSize; ++dx)
{
for (int32 dy = -WindowSize; dy < WindowSize; ++dy)
{
int32 PosX = (x * RenderSettings.resolutionScale + dx);
int32 PosY = (y * RenderSettings.resolutionScale + dy);
if (PosX < 0 || PosX >= Texture->GetSizeX() || PosY < 0 || PosY >= Texture->GetSizeY())
{
continue;
}
size_t TextureIndex = PosY * Texture->GetSizeX() + PosX;
FColor& Color = TextureData[TextureIndex];
R += Color.R;
G += Color.G;
B += Color.B;
A += Color.A;
++Samples;
}
}
ResultData[ResultIndex] = FColor(R / Samples, G / Samples, B / Samples, A / Samples);
}
}
Texture->PlatformData->Mips[0].BulkData.Unlock();
Result->PlatformData->Mips[0].BulkData.Unlock();
Result->UpdateResource();
return Result;
}
I expect a high quality oversampled Texture output, working with any positive int value in SnapshotScale.

I have a suggestion. It's not really direct, but it involves no writing of image filtering or importing of libraries.
Make an unlit Material with nodes TextureObject->TextureSample-> connect to Emissive.
Use the texture you start with in your function to populate the Texture Object on a Material Instance Dynamic of the material.
Use the "Draw Material to Render Target" function to draw the Material Instance Dynamic to a Render Target that is pre-set with your target resolution.

Related

Low framerate with only map and minimap drawing (SFML)

I'm working on a small "game" like project as a practice, and I've managed to get my framerate down to not even 3 FPS. While the only thing that is being drawn is screen filling tiles and a minimap.
Now I've found that the problem is with the minimap, without it caps at 60 FPS. But unfortunately I'm not experienced enough to find out what the real problem is with it...
My draw function:
void StateIngame::draw()
{
m_gui.removeAllWidgets();
m_window.setView(m_view);
// Frame counter
float ct = m_clock.restart().asSeconds();
float fps = 1.f / ct;
m_time = ct;
char c[10];
sprintf(c, "%f", fps);
std::string fpsStr(c);
sf::String str(fpsStr);
auto fpsText = tgui::Label::create();
fpsText->setText(str);
fpsText->setTextSize(16);
fpsText->setPosition(15, 15);
m_gui.add(fpsText);
//////////////
// Draw map //
//////////////
int camOffsetX, camOffsetY;
int tileSize = m_map.getTileSize();
Tile tile;
sf::IntRect bounds = m_camera.getTileBounds(tileSize);
camOffsetX = m_camera.getTileOffset(tileSize).x;
camOffsetY = m_camera.getTileOffset(tileSize).y;
// Loop and draw each tile
// x and y = counters, tileX and tileY is the coordinates of the tile being drawn
for (int y = 0, tileY = bounds.top; y < bounds.height; y++, tileY++)
{
for (int x = 0, tileX = bounds.left; x < bounds.width; x++, tileX++)
{
try {
// Normal view
m_window.setView(m_view);
tile = m_map.getTile(tileX, tileY);
tile.render((x * tileSize) - camOffsetX, (y * tileSize) - camOffsetY, &m_window);
} catch (const std::out_of_range& oor)
{}
}
}
bounds = sf::IntRect(bounds.left - (bounds.width * 2), bounds.top - (bounds.height * 2), bounds.width * 4, bounds.height * 4);
for (int y = 0, tileY = bounds.top; y < bounds.height; y++, tileY++)
{
for (int x = 0, tileX = bounds.left; x < bounds.width; x++, tileX++)
{
try {
// Mini map
m_window.setView(m_minimap);
tile = m_map.getTile(tileX, tileY);
sf::RectangleShape miniTile(sf::Vector2f(4, 4));
miniTile.setFillColor(tile.m_color);
miniTile.setPosition((x * (tileSize / 4)), (y * (tileSize / 4)));
m_window.draw(miniTile);
} catch (const std::out_of_range& oor)
{}
}
}
// Gui
m_window.setView(m_view);
m_gui.draw();
}
The Tile class has a variable of type sf::Color which is set during map generating. This color is then used to draw the minimap instead of the 16x16 texture that is used for the map itself.
So when I leave out the minimap drawing, and only draw the map itself, it's more fluid than I could wish for...
Any help is appreciated!
You are generating the view completly new for every frame. Do this once on startup should be enought.

Alpha tested text rendering of signed distance field creates wiggly edges

I am trying to render the text in my application using a glyph atlas represented as a signed distance field texture, which means a texture which stores the distance to the nearest outline in each pixel.
This distance field texture is generated from the original binary glyph atlas texture (0 - outside of glyph outline, 1 - inside glyph outline) with this algorithm, which is searching an incremental radius around every pixel in the texture until a pixel with the opposite state is found, then stores the distance at which that pixel was found. Later, all distances are mapped between 0 and 1.
//set size of signed distance field atlas
atlas.width = binaryAtlasWidth* pDistanceFieldResolution;
atlas.height = binaryAtlasHeight * pDistanceFieldResolution;
const unsigned int atlasPixelCount = (atlas.width * atlas.height);
atlas.buffer.resize(atlasPixelCount);
//temporary buffer for the distances of each pixel
std::vector<float> unmappedBuffer;
unmappedBuffer.resize(atlasPixelCount);
//support multisampling
unsigned int samplesPerOutPixel = ceil(1.0 / pDistanceFieldResolution);
//for mapping the values between 0 and 1 later
float maxDistance = 0.0;
float minDistance = 0.0;
for (unsigned int outPixel = 0; outPixel < atlasPixelCount; ++outPixel) {
//coordinate of the input sample
unsigned int outPixelX = outPixel%atlas.width;
unsigned int outPixelY = outPixel/atlas.width;
float distanceSum = 0.0f;
for (unsigned int sampleY = 0; sampleY < samplesPerOutPixel; ++sampleY) {
for (unsigned int sampleX = 0; sampleX < samplesPerOutPixel; ++sampleX) {
glm::uvec2 sampleCoord = glm::uvec2(outPixelX * samplesPerOutPixel+ sampleX, outPixelY * samplesPerOutPixel+ sampleY);
unsigned int samplePos = sampleCoord.x + sampleCoord.y*binaryAtlasWidth;
unsigned char sampleVal = buffer[samplePos];
//inital distance is maximum search radius(outside of glyph)
float dist = spread;
int found = 0;
unsigned int rad = 0;
while(!found && (rad*(!sampleVal)) < spread_pixels) {
//if sampleVal is 1(inside), search until found
float radius = (float)rad + 1.0f;
unsigned int compareCount = round(2.0f*radius*M_PI);
float step = 1.0 / (float)compareCount;
for (unsigned int t = 0; t < compareCount && !found; ++t) {
float theta = step*(float)t*360.0f;
glm::vec2 compareLocalCoord = glm::vec2(std::cos(theta), std::sin(theta))*radius;
glm::uvec2 compareCoord = sampleCoord + glm::uvec2(compareLocalCoord);
int comparePos = compareCoord.x + compareCoord.y*binaryAtlasWidth;
if (compareCoord.x >= 0 && compareCoord.x < binaryAtlasWidth&& compareCoord.y >= 0 && compareCoord.y < binaryAtlasHeight) {
unsigned char compareVal = buffer[comparePos];
if (compareVal != sampleVal ) {
float distance = sqrt(pow(compareLocalCoord.x, 2) + pow(compareLocalCoord.y, 2));
found = 1;
dist = std::min(distance * (1 - (sampleVal * 2)) , dist) ;
}
}
}
++rad;
}
distanceSum += dist;
}
}
float avgDistance = distanceSum / (float)(samplesPerOutPixel*samplesPerOutPixel);
printf("pixel %i of %i has %f distance\n", outPixel, atlasPixelCount, avgDistance);
unmappedBuffer[outPixel] = avgDistance;
maxDistance = std::max(maxDistance, avgDistance);
minDistance = std::min(minDistance, avgDistance);
}
minDistance *= -1.0;
float diff = maxDistance + minDistance;
//map all values between 0 and 255
for(unsigned int p = 0; p < atlasPixelCount; ++p) {
float toMap = unmappedBuffer[p];
float mappedDistance = 1.0f - (toMap + minDistance) / diff;
atlas.buffer[p] = mappedDistance * 255;
}
this algorithm creates these results:
266 x 183 input texture
SDF result without downsampling (still 266 x 183)
SDF result with downsampling (106 x 73)
Render results with alpha testing on(pass when alpha is greater than 0.5):
no downsampling, nearest filtering
downsampled, nearest filtering
no downsampling, linear filtering
downsampled, linear filtering
I mean, I am getting there, but actually I expected accurate edges as shown in valves paper. What am i missing for accurate edges?
PS: my fragment shader currently only uses the distance texture value as an alpha value. (color = vec4(1, 1, 1, distance);)

Pixels in YUV image

I am using opencv to achieve object tracking. I read that YUV image is better option to use than RGB image. My problem is that I fail to understand about the YUV format although i spend much time read notes. Y is the brightness which i believe is calculated from the combination of R, G, B component.
My main problem is how can I access and manipulate the pixels in YUV image format. In RGB format its easy to access the component and therefore change it using simple operatin like
src.at<Vec3b>(j,i).val[0] = 0; for example
But this is not the case in YUV. I need help in accessing and changing the pixel values in YUV image. For example if pixel in RGB is red, then I want to only keep the corresponding pixel in YUV and the rest is removed. Please help me with this.
I would suggest operating on your image in HSV or LAB rather than RGB.
The raw image from the camera will be in YCbCr (sometimes called YUV, which I think is incorrect, but I may be wrong), and laid out in a way that resembles something like YUYV (repeating), so if you can convert directly from that to HSV, you will avoid additional copy and conversion operations which will save you some time. That may only matter to you if you're processing video or batches of images however.
Here's some C++ code for converting between YCbCr and RGB (one uses integer math, the other floating point):
Colour::bgr Colour::YCbCr::toBgrInt() const
{
int c0 = 22987;
int c1 = -11698;
int c2 = -5636;
int c3 = 29049;
int y = this->y;
int cb = this->cb - 128;
int cr = this->cr - 128;
int b = y + (((c3 * cb) + (1 << 13)) >> 14);
int g = y + (((c2 * cb + c1 * cr) + (1 << 13)) >> 14);
int r = y + (((c0 * cr) + (1 << 13)) >> 14);
if (r < 0)
r = 0;
else if (r > 255)
r = 255;
if (g < 0)
g = 0;
else if (g > 255)
g = 255;
if (b < 0)
b = 0;
else if (b > 255)
b = 255;
return Colour::bgr(b, g, r);
}
Colour::bgr Colour::YCbCr::toBgrFloat() const
{
float y = this->y;
float cb = this->cb;
float cr = this->cr;
int r = y + 1.40200 * (cr - 0x80);
int g = y - 0.34414 * (cb - 0x80) - 0.71414 * (cr - 0x80);
int b = y + 1.77200 * (cb - 0x80);
if (r < 0)
r = 0;
else if (r > 255)
r = 255;
if (g < 0)
g = 0;
else if (g > 255)
g = 255;
if (b < 0)
b = 0;
else if (b > 255)
b = 255;
return Colour::bgr(b, g, r);
}
And a conversion from BGR to HSV:
Colour::hsv Colour::bgr2hsv(bgr const& in)
{
Colour::hsv out;
int const hstep = 255 / 3; // Hue step size between red -> green -> blue
int min = in.r < in.g ? in.r : in.g;
min = min < in.b ? min : in.b;
int max = in.r > in.g ? in.r : in.g;
max = max > in.b ? max : in.b;
out.v = max; // v
int chroma = max - min;
if (max > 0)
{
out.s = 255 * chroma / max; // s
}
else
{
// r = g = b = 0 // s = 0, v is undefined
out.s = 0;
out.h = 0;
out.v = 0; // it's now undefined
return out;
}
if (chroma == 0)
{
out.h = 0;
return out;
}
const int chroma2 = chroma * 2;
int offset;
int diff;
if (in.r == max)
{
offset = 3 * hstep;
diff = in.g - in.b;
}
else if (in.g == max)
{
offset = hstep;
diff = in.b - in.r;
}
else
{
offset = 2 * hstep;
diff = in.r - in.g;
}
int h = offset + (diff * (hstep + 1)) / chroma2;
// Rotate such that red has hue 0
if (h >= 255)
h -= 255;
assert(h >= 0 && h < 256);
out.h = h;
return out;
Unfortunately I do not have code to do this in one step.
You can also use the built-in OpenCV functions for colour conversion.
cvtColor(img, img, CV_BGR2HSV);
Also the U and V components are calculated as linear combinations of RGB values. Then it means, that different intensities of red (R,0,0) are mapped to some (y*R + a,u*R + b, v*R + c), which again means that to detect "red" in YUV one can calculate if the distance of the pixel to that line determined by y,u,v,a,b,c (some of which are redundant) is close to zero. That's achievable with a single dot product. Then set the remaining pixels to the (0,128,128) in YUV space (I think that's R=0,G=0,B=0 in almost all varieties of YCrCb, YUV and such).
There are several YUV formats, but the common ones keep Y at the same resolution as the original image, but U and V are half size, and are saved as separate or interlaced planes/channels after the single channel Y image buffer.
This allows you to efficiently access Y as a 1-channel 8-bit greyscale image.
Access and manipulate pixels does not know the colorformat so the same code applies for color components Y U and V. If you need to access in RGB mode, best is probably calling cv::cvtColor for your region of interest first.

OgreBullet incorrect HeightmapCollisionShape shape scale?

I am trying to load a HeightmapTerrainShape in OgreBullet by (mostly) using the demo code, but my terrain mesh is offset from the HeightmapTerrainShape. I have no clue why this is happening. This is my code:
void TerrainLoader::setTerrainPhysics(Ogre::Image *imgPtr)
{
unsigned page_size = terrainGroup->getTerrainSize();
Ogre::Vector3 terrainScale(4096 / (page_size-1), 600, 4096 / (page_size-1));
float *heights = new float[page_size*page_size];
for(unsigned y = 0; y < page_size; ++y)
{
for(unsigned x = 0; x < page_size; ++x)
{
Ogre::ColourValue color = imgPtr->getColourAt(x, y, 0);
heights[x + y * page_size] = color.r;
}
}
OgreBulletCollisions::HeightmapCollisionShape *terrainShape = new OgreBulletCollisions::HeightmapCollisionShape(
page_size,
page_size,
terrainScale,
heights,
true
);
OgreBulletDynamics::RigidBody *terrainBody = new OgreBulletDynamics::RigidBody(
"Terrain",
OgreInit::level->physicsManager->getWorld()
);
imgPtr = NULL;
Ogre::Vector3 terrainShiftPos(terrainScale.x/(page_size-1), 0, terrainScale.z/(page_size-1));
terrainShiftPos.y = terrainScale.y / 2 * terrainScale.y;
Ogre::SceneNode *pTerrainNode = OgreInit::sceneManager->getRootSceneNode()->createChildSceneNode();
terrainBody->setStaticShape(pTerrainNode, terrainShape, 0.0f, 0.8f, terrainShiftPos);
//terrainBody->setPosition(terrainBody->getWorldPosition()-Ogre::Vector3(0.005, 0, 0.005));
OgreInit::level->physicsManager->addBody(terrainBody);
OgreInit::level->physicsManager->addShape(terrainShape);
}
This is what it looks like with the debug drawer turned on:
My world is 4096*600*4096 in size, and each chunk is 64*600*64
heights[x + y * page_size] = color.r;
This Line gives you negative values. If you combine negative terrain height values with ogre bullet terrain, you get a wrong bounding box conversation.
You need to use the intervall 0-1 for height values.
Had the same problem with perlin noise filter that gives you values from -1 to 1.

Quick code to resize DIB image and maintain good img quality

There is many algorithms to do image resizing - lancorz, bicubic, bilinear, e.g. But most of them are pretty complex and therefore consume too much CPU.
What I need is fast relatively simple C++ code to resize images with acceptable quality.
Here is an example of what I'm currently doing:
for (int y = 0; y < height; y ++)
{
int srcY1Coord = int((double)(y * srcHeight) / height);
int srcY2Coord = min(srcHeight - 1, max(srcY1Coord, int((double)((y + 1) * srcHeight) / height) - 1));
for (int x = 0; x < width; x ++)
{
int srcX1Coord = int((double)(x * srcWidth) / width);
int srcX2Coord = min(srcWidth - 1, max(srcX1Coord, int((double)((x + 1) * srcWidth) / width) - 1));
int srcPixelsCount = (srcX2Coord - srcX1Coord + 1) * (srcY2Coord - srcY1Coord + 1);
RGB32 color32;
UINT32 r(0), g(0), b(0), a(0);
for (int xSrc = srcX1Coord; xSrc <= srcX2Coord; xSrc ++)
for (int ySrc = srcY1Coord; ySrc <= srcY2Coord; ySrc ++)
{
RGB32 curSrcColor32 = pSrcDIB->GetDIBPixel(xSrc, ySrc);
r += curSrcColor32.r; g += curSrcColor32.g; b += curSrcColor32.b; a += curSrcColor32.alpha;
}
color32.r = BYTE(r / srcPixelsCount); color32.g = BYTE(g / srcPixelsCount); color32.b = BYTE(b / srcPixelsCount); color32.alpha = BYTE(a / srcPixelsCount);
SetDIBPixel(x, y, color32);
}
}
The code above is fast enough, but the quality is not ok on scaling pictures up.
Therefore, possibly someone already has fast and good C++ code sample for scaling DIBs?
Note: I was using StretchDIBits before - it was super-slow when was needed to downsize 10000x10000 picture down to 100x100 size, my code is much, much faster, I just want to have a bit higher quality
P.S. I'm using my own SetPixel/GetPixel functions, to work directly with data array and fast, that's not device context!
Why are you doing it on the CPU? Using GDI, there's a good chance of some hardware acceleration. Use StretchBlt and SetStretchBltMode.
In pseudocode:
create source dc and destination dc using CreateCompatibleDC
create source and destination bitmaps
SelectObject source bitmap into source DC and dest bitmap into dest DC
SetStretchBltMode
StretchBlt
release DCs
Allright, here is the answer, had to do it myself... It works perfectly well for scaling pictures up (for scaling down my initial code works perfectly well too). Hope someone will find a good use for it, it's fast enough and produced very good picture quality.
for (int y = 0; y < height; y ++)
{
double srcY1Coord = (y * srcHeight) / (double)height;
int srcY1CoordInt = (int)(srcY1Coord);
double srcY2Coord = ((y + 1) * srcHeight) / (double)height - 0.00000000001;
int srcY2CoordInt = min(maxSrcYcoord, (int)(srcY2Coord));
double yMultiplierForFirstCoord = (0.5 * (1 - (srcY1Coord - srcY1CoordInt)));
double yMultiplierForLastCoord = (0.5 * (srcY2Coord - srcY2CoordInt));
for (int x = 0; x < width; x ++)
{
double srcX1Coord = (x * srcWidth) / (double)width;
int srcX1CoordInt = (int)(srcX1Coord);
double srcX2Coord = ((x + 1) * srcWidth) / (double)width - 0.00000000001;
int srcX2CoordInt = min(maxSrcXcoord, (int)(srcX2Coord));
RGB32 color32;
ASSERT(srcX1Coord < srcWidth && srcY1Coord < srcHeight);
double r(0), g(0), b(0), a(0), multiplier(0);
for (int xSrc = srcX1CoordInt; xSrc <= srcX2CoordInt; xSrc ++)
for (int ySrc = srcY1CoordInt; ySrc <= srcY2CoordInt; ySrc ++)
{
RGB32 curSrcColor32 = pSrcDIB->GetDIBPixel(xSrc, ySrc);
double xMultiplier = xSrc < srcX1Coord ? (0.5 * (1 - (srcX1Coord - srcX1CoordInt))) : (xSrc >= srcX2Coord ? (0.5 * (srcX2Coord - srcX2CoordInt)) : 0.5);
double yMultiplier = ySrc < srcY1Coord ? yMultiplierForFirstCoord : (ySrc >= srcY2Coord ? yMultiplierForLastCoord : 0.5);
double curPixelMultiplier = xMultiplier + yMultiplier;
if (curPixelMultiplier > 0)
{
r += (curSrcColor32.r * curPixelMultiplier); g += (curSrcColor32.g * curPixelMultiplier); b += (curSrcColor32.b * curPixelMultiplier); a += (curSrcColor32.alpha * curPixelMultiplier);
multiplier += curPixelMultiplier;
}
}
color32.r = BYTE(r / multiplier); color32.g = BYTE(g / multiplier); color32.b = BYTE(b / multiplier); color32.alpha = BYTE(a / multiplier);
SetDIBPixel(x, y, color32);
}
}
P.S. Please don't ask why I’m not using StretchDIBits - leave comments for these who understand that not always system api is available or acceptable.
Again, why do it on the CPU? Why not use OpenGL / DirectX and fragment shaders? In pseudocode:
upload source texture (cache it if it's to be reused)
create destination texture
use shader program
render quad
download output texture
where shader program is the filtering method you're using. The GPU is much better at processing pixels than CPU/GetPixel/SetPixel.
You could probably find fragment shaders for lots of different filtering methods on the web - GPU Gems is a good place to start.