With little experience in color spaces, I used the following code to convert BGR image (array of unsigned characters where each channel ranges from 0 to 255) to lab color space:
double F(double input) // function f(...), which is used for defining L, a and b changes within [4/29,1]
{
if (input > 0.008856)
return (pow(input, 0.333333333)); // maximum 1
else
return ((841/108)*input + 4/29); //841/108 = 29*29/36*16
}
// RGB to XYZ
void RGBtoXYZ(uchar R, uchar G, uchar B, double &X, double &Y, double &Z)
{
// RGB Working Space: sRGB
// Reference White: D65
X = 0.412453*R + 0.357580*G + 0.189423*B; // maximum value = 0.959456 * 255 = 244.66128
Y = 0.212671*R + 0.715160*G + 0.072169*B; // maximum value = 1 * 255 = 255
Z = 0.019334*R + 0.119193*G + 0.950227*B; // maximum value = 1.088754 * 255 = 277.63227
}
// XYZ to CIELab
void XYZtoLab(double X, double Y, double Z, double &L, double &a, double &b)
{
const double Xo = 244.66128; // reference white
const double Yo = 255.0;
const double Zo = 277.63227;
L = 116 * F(Y / Yo) - 16; // maximum L = 100
a = 500 * (F(X / Xo) - F(Y / Yo)); // maximum
b = 200 * (F(Y / Yo) - F(Z / Zo));
}
// RGB to CIELab
void RGBtoLab(double R, double G, double B, double &L, double &a, double &b)
{
double X, Y, Z;
RGBtoXYZ(R, G, B, X, Y, Z);
XYZtoLab(X, Y, Z, L, a, b);
}
I have re-converted the resulting lab image to BGR (using cvtcolor) to display it using OpenCV using the following code (I don't want to do the conversion using OpenCV, I have just used it to display the results. Basically I wanted to understand how color space conversion works):
// Lchannel, Achannel, Bchannel are arrays of type double
cv::Mat temp64bitL(height, width, CV_64FC1, Lchannel);
cv::Mat temp32bitL;
temp64bitL.convertTo(temp32bitL, CV_32F);
cv::Mat temp64bitA(height, width, CV_64FC1, Achannel);
cv::Mat temp32bitA;
temp64bitA.convertTo(temp32bitA, CV_32F);
cv::Mat temp64bitB(height, width, CV_64FC1, Bchannel);
cv::Mat temp32bitB;
temp64bitB.convertTo(temp32bitB, CV_32F);
cv::Mat chan[3] = {
temp32bitL, temp32bitA, temp32bitB
};
cv::Mat merged;
cv::merge(chan, 3, merged);
cv::Mat BGRImage;
cv::cvtColor(merged, BGRImage, CV_Lab2BGR, 3);
However, the computed image is different from the original image. is that due to a problem in the code?
Your code has a bug in double F(double input). It does not work as intended because of the integer division you have. You might be willing to change the function to read something like below. Note the double castings to make the divisions work in the floating-point domain, and the use of cbrt instead of pow.
#include <cmath>
double F(double input) // function f(...), which is used for defining L, a and b
// changes within [4/29,1]
{
if (input > 0.008856)
return std::cbrt(input); // maximum 1 --- prefer cbrt to pow for cubic root
else
return ((double(841) / 108) * input +
double(4) / 29); // 841/108 = 29*29/36*16
}
Then, another problem could be the reference values you are using for the XYZ space. We have the below reference values, coming from D65 / CIE-1931:
double Xo = 95.047;
double Yo = 100;
double Zo = 108.883;
Then, our RGBtoXYZ conversion was working like this:
template <class float_t> struct Convert<XYZ<float_t>> {
template <class real_t> static XYZ<float_t> from(const RGB<real_t> &rhs) {
// Assume RGB has the type invariance satisfied, i.e., channels \in [0,255]
float_t var_R = float_t(rhs.comp1()) / 255;
float_t var_G = float_t(rhs.comp2()) / 255;
float_t var_B = float_t(rhs.comp3()) / 255;
var_R = (var_R > 0.04045) ? std::pow((var_R + 0.055) / 1.055, 2.4)
: var_R / 12.92;
var_G = (var_G > 0.04045) ? std::pow((var_G + 0.055) / 1.055, 2.4)
: var_G / 12.92;
var_B = (var_B > 0.04045) ? std::pow((var_B + 0.055) / 1.055, 2.4)
: var_B / 12.92;
var_R *= 100;
var_G *= 100;
var_B *= 100;
return XYZ<float_t>{var_R * float_t(0.4124) + var_G * float_t(0.3576) +
var_B * float_t(0.1805),
var_R * float_t(0.2126) + var_G * float_t(0.7152) +
var_B * float_t(0.0722),
var_R * float_t(0.0193) + var_G * float_t(0.1192) +
var_B * float_t(0.9505)};
}
};
where RGB was assumed to have its channels inside the valid range, as stated in the comment. Then, the XYZtoLAB function we have is the same except for the cbrt and reference value changes.
EDIT. Above numbers are obtained from EasyRGB's Math page. You can find the conversion from sRGB to XYZ and XYZ to Lab on the page, with a table of XYZ reference values. What we used was the set for "Daylight, sRGB, Adobe RGB."
Related
I tried to convert panorama to tiny planet using C++ and OpenCV but the image result is noisy. I am not really sure which part I did wrong. I think it has something to do with color.
I tried to convert panorama to tiny planet using C++ and OpenCV but the image result is noisy. I am not really sure which part I did wrong. I think it has something to do with color.
Tutorial I referred to
http://codeofthedamned.com/index.php/the-little-planet-effect
Panorama source
Tiny image result
#import <opencv2/opencv.hpp>
#import <opencv2/imgcodecs/ios.h>
#import "OpenCVWrapper.h"
using namespace cv;
#implementation OpenCVWrapper
+ (UIImage*)createTinyPlanetFromImage: (UIImage*)image {
Mat pano;
UIImageToMat(image, pano);
Mat grayMat;
RenderProjection(pano, 1000.0, grayMat);
return MatToUIImage(grayMat);
}
void RenderProjection(Mat &pano, long len, Mat &output) {
const double k_pi = 3.1415926535897932384626433832795;
const double k_pi_inverse = 0.31830988618379067153776752674503;
output.create(len, len, CV_16UC3);
long half_len = len / 2;
cv::Size sz = pano.size();
for (long indexX = 0; indexX < len; ++indexX) {
for (long indexY = 0; indexY < len; ++indexY) {
double sphereX = (indexX - half_len) * 10.0 / len;
double sphereY = (indexY - half_len) * 10.0 / len;
double Qx, Qy, Qz;
if (GetIntersection(sphereX, sphereY, Qx, Qy, Qz)) {
double theta = std::acos(Qz);
double phi = std::atan2(Qy, Qx) + k_pi;
theta = theta * k_pi_inverse;
phi = phi * (0.5 * k_pi_inverse);
double Sx = min(sz.width -2.0, sz.width * phi);
double Sy = min(sz.height-2.0, sz.height * theta);
output.at<Vec3s>(int(indexY), int(indexX)) = BilinearSample(pano, Sx, Sy);
}
}
}
}
bool GetIntersection(double u, double v, double &x, double &y, double &z) {
double Nx = 0.0;
double Ny = 0.0;
double Nz = 1.0;
double dir_x = u - Nx;
double dir_y = v - Ny;
double dir_z = -1.0 - Nz;
double a = (dir_x * dir_x) + (dir_y * dir_y) + (dir_z * dir_z);
double b = (dir_x * Nx) + (dir_y * Ny) + (dir_z * Nz);
b *= 2;
double d = b * b;
double q = -0.5 * (b - std::sqrt(d));
double t = q / a;
x = (dir_x * t) + Nx;
y = (dir_y * t) + Ny;
z = (dir_z * t) + Nz;
return true;
}
Vec3s BilinearSample(Mat &image, double x, double y) {
Vec3s c00 = image.at<Vec3s>(int(y), int(x));
Vec3s c01 = image.at<Vec3s>(int(y), int(x) + 1);
Vec3s c10 = image.at<Vec3s>(int(y) + 1, int(x));
Vec3s c11 = image.at<Vec3s>(int(y) + 1, int(x) + 1);
double X0 = x - floor(x);
double X1 = 1.0 - X0;
double Y0 = y - floor(y);
double Y1 = 1.0 - Y0;
double w00 = X0 * Y0;
double w01 = X1 * Y0;
double w10 = X0 * Y1;
double w11 = X1 * Y1;
short r = short(c00[2] * w00 + c01[2] * w01
+ c10[2] * w10 + c11[2] * w11);
short g = short(c00[1] * w00 + c01[1] * w01
+ c10[1] * w10 + c11[1] * w11);
short b = short(c00[0] * w00 + c01[0] * w01
+ c10[0] * w10 + c11[0] * w11);
return make_BGR(b, g, r);
}
Vec3s make_BGR(short blue, short green, short red) {
Vec3s result;
result[0] = blue;
result[1] = green;
result[2] = red;
return result;
}
#end
There problem solved when I replaced UIImageToMat(image, pano); and MatToUIImage(grayMat); with this code and we can remove this header #import <opencv2/imgcodecs/ios.h>
static void UIImageToMat2(UIImage *image, cv::Mat &mat) {
assert(image.size.width > 0 && image.size.height > 0);
assert(image.CGImage != nil || image.CIImage != nil);
// Create a pixel buffer.
NSInteger width = image.size.width;
NSInteger height = image.size.height;
cv::Mat mat8uc4 = cv::Mat((int)height, (int)width, CV_8UC4);
// Draw all pixels to the buffer.
CGColorSpaceRef colorSpace = CGColorSpaceCreateDeviceRGB();
if (image.CGImage) {
// Render with using Core Graphics.
CGContextRef contextRef = CGBitmapContextCreate(mat8uc4.data, mat8uc4.cols, mat8uc4.rows, 8, mat8uc4.step, colorSpace, kCGImageAlphaPremultipliedLast | kCGBitmapByteOrderDefault);
CGContextDrawImage(contextRef, CGRectMake(0, 0, width, height), image.CGImage);
CGContextRelease(contextRef);
} else {
// Render with using Core Image.
static CIContext* context = nil; // I do not like this declaration contains 'static'. But it is for performance.
if (!context) {
context = [CIContext contextWithOptions:#{ kCIContextUseSoftwareRenderer: #NO }];
}
CGRect bounds = CGRectMake(0, 0, width, height);
[context render:image.CIImage toBitmap:mat8uc4.data rowBytes:mat8uc4.step bounds:bounds format:kCIFormatRGBA8 colorSpace:colorSpace];
}
CGColorSpaceRelease(colorSpace);
// Adjust byte order of pixel.
cv::Mat mat8uc3 = cv::Mat((int)width, (int)height, CV_8UC3);
cv::cvtColor(mat8uc4, mat8uc3, cv::COLOR_RGBA2BGR);
mat = mat8uc3;
}
and
static UIImage *MatToUIImage2(cv::Mat &mat) {
// Create a pixel buffer.
assert(mat.elemSize() == 1 || mat.elemSize() == 3);
cv::Mat matrgb;
if (mat.elemSize() == 1) {
cv::cvtColor(mat, matrgb, cv::COLOR_GRAY2RGB);
} else if (mat.elemSize() == 3) {
cv::cvtColor(mat, matrgb, cv::COLOR_BGR2RGB);
}
// Change a image format.
NSData *data = [NSData dataWithBytes:matrgb.data length:(matrgb.elemSize() * matrgb.total())];
CGColorSpaceRef colorSpace;
if (matrgb.elemSize() == 1) {
colorSpace = CGColorSpaceCreateDeviceGray();
} else {
colorSpace = CGColorSpaceCreateDeviceRGB();
}
CGDataProviderRef provider = CGDataProviderCreateWithCFData((__bridge CFDataRef)data);
CGImageRef imageRef = CGImageCreate(matrgb.cols, matrgb.rows, 8, 8 * matrgb.elemSize(), matrgb.step.p[0], colorSpace, kCGImageAlphaNone|kCGBitmapByteOrderDefault, provider, NULL, false, kCGRenderingIntentDefault);
UIImage *image = [UIImage imageWithCGImage:imageRef];
CGImageRelease(imageRef);
CGDataProviderRelease(provider);
CGColorSpaceRelease(colorSpace);
return image;
}
I have programmed a simple dragon curve fractal. It seems to work for the most part, but there is an odd logical error that shifts the rotation of certain lines by one pixel. This wouldn't normally be an issue, but after a few generations, at the right size, the fractal begins to look wonky.
I am using open cv in c++ to generate it, but I'm pretty sure it's a logical error rather than a display error. I have printed the values to the console multiple times and seen for myself that there is a one-digit difference between values that are intended to be the exact same - meaning a line may have a y of 200 at one end and 201 at another.
Here is the full code:
#include<iostream>
#include<cmath>
#include<opencv2/opencv.hpp>
const int width=500;
const int height=500;
const double PI=std::atan(1)*4.0;
struct point{
double x;
double y;
point(double x_,double y_){
x=x_;
y=y_;
}};
cv::Mat img(width,height,CV_8UC3,cv::Scalar(255,255,255));
double deg_to_rad(double degrees){return degrees*PI/180;}
point rotate(int degree, int centx, int centy, int ll) {
double radians = deg_to_rad(degree);
return point(centx + (ll * std::cos(radians)), centy + (ll * std::sin(radians)));
}
void generate(point & r, std::vector < point > & verticies, int rotation = 90) {
int curRotation = 90;
bool start = true;
point center = r;
point rot(0, 0);
std::vector<point> verticiesc(verticies);
for (point i: verticiesc) {
double dx = center.x - i.x;
double dy = center.y - i.y;
//distance from centre
int ll = std::sqrt(dx * dx + dy * dy);
//angle from centre
curRotation = std::atan2(dy, dx) * 180 / PI;
//add 90 degrees of rotation
rot = rotate(curRotation + rotation, center.x, center.y, ll);
verticies.push_back(rot);
//endpoint, where the next centre will be
if (start) {
r = rot;
start = false;
}
}
}
void gen(int gens, int bwidth = 1) {
int ll = 7;
std::vector < point > verticies = {
point(width / 2, height / 2 - ll),
point(width / 2, height / 2)
};
point rot(width / 2, height / 2);
for (int i = 0; i < gens; i++) {
generate(rot, verticies);
}
//draw lines
for (int i = 0; i < verticies.size(); i += 2) {
cv::line(img, cv::Point(verticies[i].x, verticies[i].y), cv::Point(verticies[i + 1].x, verticies[i + 1].y), cv::Scalar(0, 0, 0), 1, 8);
}
}
int main() {
gen(10);
cv::imshow("", img);
cv::waitKey(0);
return 0;
}
First, you use int to store point coordinates - that's a bad idea - you lose all accuracy of point position. Use double or float.
Second, your method for drawing fractals is not too stable numericly. You'd better store original shape and all rotation/translation/scale that indicate where and how to draw scaled copies of the original shape.
Also, I believe this is a bug:
for(point i: verices)
{
...
vertices.push_back(rot);
...
}
Changing size of vertices while inside such a for-loop might cause a crash or UB.
Turns out it was to do with floating-point precision. I changed
x=x_;
y=y_;
to
x=std::round(x_);
y=std::round(y_);
and it works.
I’m using a modified version of a gauss-newton method to refine a pose estimate using OpenCV. The unmodified code can be found here: http://people.rennes.inria.fr/Eric.Marchand/pose-estimation/tutorial-pose-gauss-newton-opencv.html
The details of this approach are outlined in the corresponding paper:
Marchand, Eric, Hideaki Uchiyama, and Fabien Spindler. "Pose
estimation for augmented reality: a hands-on survey." IEEE
transactions on visualization and computer graphics 22.12 (2016):
2633-2651.
A PDF can be found here: https://hal.inria.fr/hal-01246370/document
The part that is relevant (Pages 4 and 5) are screencapped below:
Here is what I have done. First, I’ve (hopefully) “corrected” some errors: (a) dt and dR can be passed by reference to exponential_map() (even though cv::Mat is essentially a pointer). (b) The last entry of each 2x6 Jacobian matrix, J.at<double>(i*2+1,5), was -x[i].y but should be -x[i].x. (c) I’ve also tried using a different formula for the projection. Specifically, one that includes the focal length and principal point:
xq.at<double>(i*2,0) = cx + fx * cX.at<double>(0,0) / cX.at<double>(2,0);
xq.at<double>(i*2+1,0) = cy + fy * cX.at<double>(1,0) / cX.at<double>(2,0);
Here is the relevant code I am using, in its entirety (control starts at optimizePose3()):
void exponential_map(const cv::Mat &v, cv::Mat &dt, cv::Mat &dR)
{
double vx = v.at<double>(0,0);
double vy = v.at<double>(1,0);
double vz = v.at<double>(2,0);
double vtux = v.at<double>(3,0);
double vtuy = v.at<double>(4,0);
double vtuz = v.at<double>(5,0);
cv::Mat tu = (cv::Mat_<double>(3,1) << vtux, vtuy, vtuz); // theta u
cv::Rodrigues(tu, dR);
double theta = sqrt(tu.dot(tu));
double sinc = (fabs(theta) < 1.0e-8) ? 1.0 : sin(theta) / theta;
double mcosc = (fabs(theta) < 2.5e-4) ? 0.5 : (1.-cos(theta)) / theta / theta;
double msinc = (fabs(theta) < 2.5e-4) ? (1./6.) : (1.-sin(theta)/theta) / theta / theta;
dt.at<double>(0,0) = vx*(sinc + vtux*vtux*msinc)
+ vy*(vtux*vtuy*msinc - vtuz*mcosc)
+ vz*(vtux*vtuz*msinc + vtuy*mcosc);
dt.at<double>(1,0) = vx*(vtux*vtuy*msinc + vtuz*mcosc)
+ vy*(sinc + vtuy*vtuy*msinc)
+ vz*(vtuy*vtuz*msinc - vtux*mcosc);
dt.at<double>(2,0) = vx*(vtux*vtuz*msinc - vtuy*mcosc)
+ vy*(vtuy*vtuz*msinc + vtux*mcosc)
+ vz*(sinc + vtuz*vtuz*msinc);
}
void optimizePose3(const PoseEstimation &pose,
std::vector<FeatureMatch> &feature_matches,
PoseEstimation &optimized_pose) {
//Set camera parameters
double fx = camera_matrix.at<double>(0, 0); //Focal length
double fy = camera_matrix.at<double>(1, 1);
double cx = camera_matrix.at<double>(0, 2); //Principal point
double cy = camera_matrix.at<double>(1, 2);
auto inlier_matches = getInliers(pose, feature_matches);
std::vector<cv::Point3d> wX;
std::vector<cv::Point2d> x;
const unsigned int npoints = inlier_matches.size();
cv::Mat J(2*npoints, 6, CV_64F);
double lambda = 0.25;
cv::Mat xq(npoints*2, 1, CV_64F);
cv::Mat xn(npoints*2, 1, CV_64F);
double residual=0, residual_prev;
cv::Mat Jp;
for(auto i = 0u; i < npoints; i++) {
//Model points
const cv::Point2d &M = inlier_matches[i].model_point();
wX.emplace_back(M.x, M.y, 0.0);
//Imaged points
const cv::Point2d &I = inlier_matches[i].image_point();
xn.at<double>(i*2,0) = I.x; // x
xn.at<double>(i*2+1,0) = I.y; // y
x.push_back(I);
}
//Initial estimation
cv::Mat cRw = pose.rotation_matrix;
cv::Mat ctw = pose.translation_vector;
int nIters = 0;
// Iterative Gauss-Newton minimization loop
do {
for (auto i = 0u; i < npoints; i++) {
cv::Mat cX = cRw * cv::Mat(wX[i]) + ctw; // Update cX, cY, cZ
// Update x(q)
//xq.at<double>(i*2,0) = cX.at<double>(0,0) / cX.at<double>(2,0); // x(q) = cX/cZ
//xq.at<double>(i*2+1,0) = cX.at<double>(1,0) / cX.at<double>(2,0); // y(q) = cY/cZ
xq.at<double>(i*2,0) = cx + fx * cX.at<double>(0,0) / cX.at<double>(2,0);
xq.at<double>(i*2+1,0) = cy + fy * cX.at<double>(1,0) / cX.at<double>(2,0);
// Update J using equation (11)
J.at<double>(i*2,0) = -1 / cX.at<double>(2,0); // -1/cZ
J.at<double>(i*2,1) = 0;
J.at<double>(i*2,2) = x[i].x / cX.at<double>(2,0); // x/cZ
J.at<double>(i*2,3) = x[i].x * x[i].y; // xy
J.at<double>(i*2,4) = -(1 + x[i].x * x[i].x); // -(1+x^2)
J.at<double>(i*2,5) = x[i].y; // y
J.at<double>(i*2+1,0) = 0;
J.at<double>(i*2+1,1) = -1 / cX.at<double>(2,0); // -1/cZ
J.at<double>(i*2+1,2) = x[i].y / cX.at<double>(2,0); // y/cZ
J.at<double>(i*2+1,3) = 1 + x[i].y * x[i].y; // 1+y^2
J.at<double>(i*2+1,4) = -x[i].x * x[i].y; // -xy
J.at<double>(i*2+1,5) = -x[i].x; // -x
}
cv::Mat e_q = xq - xn; // Equation (7)
cv::Mat Jp = J.inv(cv::DECOMP_SVD); // Compute pseudo inverse of the Jacobian
cv::Mat dq = -lambda * Jp * e_q; // Equation (10)
cv::Mat dctw(3, 1, CV_64F), dcRw(3, 3, CV_64F);
exponential_map(dq, dctw, dcRw);
cRw = dcRw.t() * cRw; // Update the pose
ctw = dcRw.t() * (ctw - dctw);
residual_prev = residual; // Memorize previous residual
residual = e_q.dot(e_q); // Compute the actual residual
std::cout << "residual_prev: " << residual_prev << std::endl;
std::cout << "residual: " << residual << std::endl << std::endl;
nIters++;
} while (fabs(residual - residual_prev) > 0);
//} while (nIters < 30);
optimized_pose.rotation_matrix = cRw;
optimized_pose.translation_vector = ctw;
cv::Rodrigues(optimized_pose.rotation_matrix, optimized_pose.rotation_vector);
}
Even when I use the functions as given, it does not produce the correct results. My initial pose estimate is very close to optimal, but when I try run the program, the method takes a very long time to converge - and when it does, the results are very wrong. I’m not sure what could be wrong and I’m out of ideas. I’m confident my inliers are actually inliers (they were chosen using an M-estimator). I’ve compared the results from exponential map with those from other implementations, and they seem to agree.
So, where is the error in this gauss-newton implementation for pose optimization? I’ve tried to make things as easy as possible for anyone willing to lend a hand. Let me know if there is anymore information I can provide. Any help would be greatly appreciated. Thanks.
Edit: 2019/05/13
There is now solvePnPRefineVVS function in OpenCV.
Also, you should use x and y calculated from the current estimated pose instead.
In the cited paper, they expressed the measurements x in the normalized camera frame (at z=1).
When working with real data, you have:
(u,v): 2D image coordinates (e.g. keypoints, corner locations, etc.)
K: the intrinsic parameters (obtained after calibrating the camera)
D: the distortion coefficients (obtained after calibrating the camera)
To compute the 2D image coordinates in the normalized camera frame, you can use in OpenCV the function cv::undistortPoints() (link to my answer about cv::projectPoints() and cv::undistortPoints()).
When there is no distortion, the computation (also called "reverse perspective transformation") is:
x = (u - cx) / fx
y = (v - cy) / fy
I am trying to implement a bilinear interpolation function, but for some reason I am getting bad output. I cant seem to figure out what's wrong, any help getting on the right track will be appreciated.
double lerp(double c1, double c2, double v1, double v2, double x)
{
if( (v1==v2) ) return c1;
double inc = ((c2-c1)/(v2 - v1)) * (x - v1);
double val = c1 + inc;
return val;
};
void bilinearInterpolate(int width, int height)
{
// if the current size is the same, do nothing
if(width == GetWidth() && height == GetHeight())
return;
//Create a new image
std::unique_ptr<Image2D> image(new Image2D(width, height));
// x and y ratios
double rx = (double)(GetWidth()) / (double)(image->GetWidth()); // oldWidth / newWidth
double ry = (double)(GetHeight()) / (double)(image->GetHeight()); // oldWidth / newWidth
// loop through destination image
for(int y=0; y<height; ++y)
{
for(int x=0; x<width; ++x)
{
double sx = x * rx;
double sy = y * ry;
uint xl = std::floor(sx);
uint xr = std::floor(sx + 1);
uint yt = std::floor(sy);
uint yb = std::floor(sy + 1);
for (uint d = 0; d < image->GetDepth(); ++d)
{
uchar tl = GetData(xl, yt, d);
uchar tr = GetData(xr, yt, d);
uchar bl = GetData(xl, yb, d);
uchar br = GetData(xr, yb, d);
double t = lerp(tl, tr, xl, xr, sx);
double b = lerp(bl, br, xl, xr, sx);
double m = lerp(t, b, yt, yb, sy);
uchar val = std::floor(m + 0.5);
image->SetData(x,y,d,val);
}
}
}
//Cleanup
mWidth = width; mHeight = height;
std::swap(image->mData, mData);
}
Input Image (4 pixels wide and high)
My Output
Expected Output (Photoshop's Bilinear Interpolation)
Photoshop's algorithm assumes that each source pixel's color is in the center of the pixel, while your algorithm assumes that the color is in its topleft. This causes your results to be shifted half a pixel up and left compared to Photoshop.
Another way to look at it is that your algorithm maps the x coordinate range (0, srcWidth) to (0, dstWidth), while Photoshop maps (-0.5, srcWidth-0.5) to (-0.5, dstWidth-0.5), and the same in y coordinate.
Instead of:
double sx = x * rx;
double sy = y * ry;
You can use:
double sx = (x + 0.5) * rx - 0.5;
double sy = (y + 0.5) * ry - 0.5;
to get similar results. Note that this can give you a negative value for sx and sy.
If I have a texture, is it then possible to generate a normal-map for this texture, so it can be used for bump-mapping?
Or how are normal maps usually made?
Yes. Well, sort of. Normal maps can be accurately made from height-maps. Generally, you can also put a regular texture through and get decent results as well. Keep in mind there are other methods of making a normal map, such as taking a high-resolution model, making it low resolution, then doing ray casting to see what the normal should be for the low-resolution model to simulate the higher one.
For height-map to normal-map, you can use the Sobel Operator. This operator can be run in the x-direction, telling you the x-component of the normal, and then the y-direction, telling you the y-component. You can calculate z with 1.0 / strength where strength is the emphasis or "deepness" of the normal map. Then, take that x, y, and z, throw them into a vector, normalize it, and you have your normal at that point. Encode it into the pixel and you're done.
Here's some older incomplete-code that demonstrates this:
// pretend types, something like this
struct pixel
{
uint8_t red;
uint8_t green;
uint8_t blue;
};
struct vector3d; // a 3-vector with doubles
struct texture; // a 2d array of pixels
// determine intensity of pixel, from 0 - 1
const double intensity(const pixel& pPixel)
{
const double r = static_cast<double>(pPixel.red);
const double g = static_cast<double>(pPixel.green);
const double b = static_cast<double>(pPixel.blue);
const double average = (r + g + b) / 3.0;
return average / 255.0;
}
const int clamp(int pX, int pMax)
{
if (pX > pMax)
{
return pMax;
}
else if (pX < 0)
{
return 0;
}
else
{
return pX;
}
}
// transform -1 - 1 to 0 - 255
const uint8_t map_component(double pX)
{
return (pX + 1.0) * (255.0 / 2.0);
}
texture normal_from_height(const texture& pTexture, double pStrength = 2.0)
{
// assume square texture, not necessarily true in real code
texture result(pTexture.size(), pTexture.size());
const int textureSize = static_cast<int>(pTexture.size());
for (size_t row = 0; row < textureSize; ++row)
{
for (size_t column = 0; column < textureSize; ++column)
{
// surrounding pixels
const pixel topLeft = pTexture(clamp(row - 1, textureSize), clamp(column - 1, textureSize));
const pixel top = pTexture(clamp(row - 1, textureSize), clamp(column, textureSize));
const pixel topRight = pTexture(clamp(row - 1, textureSize), clamp(column + 1, textureSize));
const pixel right = pTexture(clamp(row, textureSize), clamp(column + 1, textureSize));
const pixel bottomRight = pTexture(clamp(row + 1, textureSize), clamp(column + 1, textureSize));
const pixel bottom = pTexture(clamp(row + 1, textureSize), clamp(column, textureSize));
const pixel bottomLeft = pTexture(clamp(row + 1, textureSize), clamp(column - 1, textureSize));
const pixel left = pTexture(clamp(row, textureSize), clamp(column - 1, textureSize));
// their intensities
const double tl = intensity(topLeft);
const double t = intensity(top);
const double tr = intensity(topRight);
const double r = intensity(right);
const double br = intensity(bottomRight);
const double b = intensity(bottom);
const double bl = intensity(bottomLeft);
const double l = intensity(left);
// sobel filter
const double dX = (tr + 2.0 * r + br) - (tl + 2.0 * l + bl);
const double dY = (bl + 2.0 * b + br) - (tl + 2.0 * t + tr);
const double dZ = 1.0 / pStrength;
math::vector3d v(dX, dY, dZ);
v.normalize();
// convert to rgb
result(row, column) = pixel(map_component(v.x), map_component(v.y), map_component(v.z));
}
}
return result;
}
There's probably many ways to generate a Normal map, but like others said, you can do it from a Height Map, and 3d packages like XSI/3dsmax/Blender/any of them can output one for you as an image.
You can then output and RGB image with the Nvidia plugin for photoshop, an algorithm to convert it or you might be able to output it directly from those 3d packages with 3rd party plugins.
Be aware that in some case, you might need to invert channels (R, G or B) from the generated normal map.
Here's some resources link with examples and more complete explanation:
http://developer.nvidia.com/object/photoshop_dds_plugins.html
http://en.wikipedia.org/wiki/Normal_mapping
http://www.vrgeo.org/fileadmin/VRGeo/Bilder/VRGeo_Papers/jgt2002normalmaps.pdf
I don't think normal maps are generated from a texture. they are generated from a model.
just as texturing allows you to define complex colour detail with minimal polys (as opposed to just using millions of ploys and just vertex colours to define the colour on your mesh)
A normal map allows you to define complex normal detail with minimal polys.
I believe normal maps are usually generated from a higher res mesh, and then is used with a low res mesh.
I'm sure 3D tools, such as 3ds max or maya, as well as more specific tools will do this for you. unlike textures, I don't think they are usually done by hand.
but they are generated from the mesh, not the texture.
I suggest starting with OpenCV, due to its richness in algorithms. Here's one I wrote that iteratively blurs the normal map and weights those to the overall value, essentially creating more of a topological map.
#define ROW_PTR(img, y) ((uchar*)((img).data + (img).step * y))
cv::Mat normalMap(const cv::Mat& bwTexture, double pStrength)
{
// assume square texture, not necessarily true in real code
int scale = 1.0;
int delta = 127;
cv::Mat sobelZ, sobelX, sobelY;
cv::Sobel(bwTexture, sobelX, CV_8U, 1, 0, 13, scale, delta, cv::BORDER_DEFAULT);
cv::Sobel(bwTexture, sobelY, CV_8U, 0, 1, 13, scale, delta, cv::BORDER_DEFAULT);
sobelZ = cv::Mat(bwTexture.rows, bwTexture.cols, CV_8UC1);
for(int y=0; y<bwTexture.rows; y++) {
const uchar *sobelXPtr = ROW_PTR(sobelX, y);
const uchar *sobelYPtr = ROW_PTR(sobelY, y);
uchar *sobelZPtr = ROW_PTR(sobelZ, y);
for(int x=0; x<bwTexture.cols; x++) {
double Gx = double(sobelXPtr[x]) / 255.0;
double Gy = double(sobelYPtr[x]) / 255.0;
double Gz = pStrength * sqrt(Gx * Gx + Gy * Gy);
uchar value = uchar(Gz * 255.0);
sobelZPtr[x] = value;
}
}
std::vector<cv::Mat>planes;
planes.push_back(sobelX);
planes.push_back(sobelY);
planes.push_back(sobelZ);
cv::Mat normalMap;
cv::merge(planes, normalMap);
cv::Mat originalNormalMap = normalMap.clone();
cv::Mat normalMapBlurred;
for (int i=0; i<3; i++) {
cv::GaussianBlur(normalMap, normalMapBlurred, cv::Size(13, 13), 5, 5);
addWeighted(normalMap, 0.4, normalMapBlurred, 0.6, 0, normalMap);
}
addWeighted(originalNormalMap, 0.3, normalMapBlurred, 0.7, 0, normalMap);
return normalMap;
}