I'm looking for some pointers on how to sample an image using OpenGL at a list of specified locations. Any links to tutorial or examples similar to the problem below?
At the moment we have a code that calculates the 'output intensity' at a list of specified locations x1,y1, x2,y2, ..., xn,yn applying a Lanczos2 filter to an input image. The amount of locations at the moment is 20 (which is actually the list of phosphene locations in a visual prosthesis) but it will eventually increase up to 256 and GPU processing will certainly accelerate things. The list of locations can't be hardcoded.
So far I have seen how to implement a median filter and alike, but in my case there is no need to compute the convolution with the filter kernel at every image pixel, just at the locations specified.
Handle those values (intesity) in a second texture that has a sample or not bit.
If you use OpenGL, you'll be able to define the ROI (region of interest), the portion of an image to which you want to apply edits or processing, as you describe.
If you go that route, this is how you calculate the median in a pixel neighborhood radius of your choosing using OpenGL ES 2.0/3.0:
kernel vec4 medianUnsharpKernel(sampler u) {
vec4 pixel = unpremultiply(sample(u, samplerCoord(u)));
vec2 xy = destCoord();
int radius = 3;
int bounds = (radius - 1) / 2;
vec4 sum = vec4(0.0);
for (int i = (0 - bounds); i <= bounds; i++)
{
for (int j = (0 - bounds); j <= bounds; j++ )
{
sum += unpremultiply(sample(u, samplerTransform(u, vec2(xy + vec2(i, j)))));
}
}
vec4 mean = vec4(sum / vec4(pow(float(radius), 2.0)));
float mean_avg = float(mean);
float comp_avg = 0.0;
vec4 comp = vec4(0.0);
vec4 median = mean;
for (int i = (0 - bounds); i <= bounds; i++)
{
for (int j = (0 - bounds); j <= bounds; j++ )
{
comp = unpremultiply(sample(u, samplerTransform(u, vec2(xy + vec2(i, j)))));
comp_avg = float(comp);
median = (comp_avg < mean_avg) ? max(median, comp) : median;
}
}
return premultiply(vec4(vec3(abs(pixel.rgb - median.rgb)), 1.0));
}
A brief description of the steps
1. Calculate the mean of the values of the pixels surrounding the source pixel in a 3x3 neighborhood;
2. Find the maximum pixel value of all pixels in the same neighborhood that are less than the mean.
3. [OPTIONAL] Subtract the median pixel value from the source pixel value for edge detection.
If you're using the median value for edge detection, there are a couple of ways to modify the above code for better results, namely, hybrid median filtering and truncated media filtering (a substitute and a better 'mode' filtering). If you're interested, please ask.
Related
I am having a procedural terrain generation application. Now i want to generate textures for the terrain based on height.
Say i have got 5 textures for different height levels now for every pixel i calculate the the position of it on the mesh then get its height and then decide which texture to sample from.
Note texture is always a square.
In code it will be something like:
for (int i = 0; i < resolution; i++) {
for (int j = 0; j < resolution; j++) {
tex[i * resolution* 3 + j * 3 + 0] = SampleTextureR(i, j);
tex[i * resolution* 3 + j * 3 + 1] = SampleTextureG(i, j);
tex[i * resolution* 3 + j * 3 + 2] = SampleTextureB(i, j);
}
}
Now SampleTextureR(i, j) is just like:
for(TextureData* t : txtures){
if(t.heightl > GetMeshElevation(i, j) && t.heightg < GetMeshElevation(i, j))
return t.sampleR(i, j);
}
return 0;
GetMeshElevation returns height of mesh at a point. t.sampleR() returns unsigned char value of texture's red pixels at (i, j).
heightl is minimum height of the texture
heightg is maximum height of the texture
Now the problem is this this is very slow method. How can i make this fast enough to be done in realtime so that the changes to heightl or heghtg is immediately reflected. the heightl and heightg are for each texture.
These textures can be upto 4K 4096X4096
Use a varying variable between your vertex and fragment shader. A single float value should suffice, since you're only interested in the height coordinate.
Other than that, introduce 5 uniform varaiables for your textures in the fragment shader and do the calculations on the GPU.
In more detail:
For each fragment you get in the fragment shader the interpolated height value of the current mesh. Depending on the height value you simply select the sample from the desired texture and put that color out.
I want to make a mosaic photo with different window-size (that has been determined by user). This is just like a first draft of the code but I have problems to get the pixels and calculating averages. Then put the avarage value in each pixel and continue to the end. Even I get error to converting them of diffrent types: (Also the other part manufacturers a gray-scale image)
p.s: sorry that I am in the very first steps of learning image processing.
''' void CImageProcessingDoc::OnProcessMosaic()
{
if (m_pImage) {
DlgMosaicOption dlg;
if (dlg.DoModal() == IDOK) {
DWORD dwWindowSize = dlg.m_dwWindowSize;
DWORD width = m_pImage->GetWidth();
DWORD height = m_pImage->GetHeight();
RGBQUAD color;
RGBQUAD newcolor;
float X_step = width / dwWindowSize;
float Y_step = height / dwWindowSize;
int avg, pixel;
for (DWORD y = 0; y < dwWindowSize; y++) {
for (DWORD x = 0; x < dwWindowSize; x++) {
color = m_pImage->GetPixelColor(x, y);
(RGBQUAD) pixel = m_pImage->GetPixelColor(x, y);
avR += (int)(color.red(pixel);
avG += (int)(color.green(pixel);
avB += (int)(color.blue(pixel);
newcolor.rgbBlue = (BYTE)RGB2GRAY(color.rgbRed, color.rgbGreen, color.rgbBlue);
newcolor.rgbGreen = (BYTE)RGB2GRAY(color.rgbRed, color.rgbGreen, color.rgbBlue);
newcolor.rgbRed = (BYTE)RGB2GRAY(color.rgbRed, color.rgbGreen, color.rgbBlue);
m_pImage->SetPixelColor(x, y, newcolor);
}
}
}
}
} '''
Could anyone please help me to understand the problem?
I think you are mixing up spatial, spectral and temporal average here.
Spatial average
This is the operation of computing average of pixels over an area.
You have to compute eR = 1/N * (P0.R + P1.R + P2.R + P3.R + ...), eG = 1/N * (P0.G + P1.G + ...), eB = 1/N * (P0.B + P1.B + ...)
You'll get a pixel with as many color as there was in the input picture, but with limited spatial frequency, a picture computed like this will appear blurred, with no details
Spectral average
This is the operation of computing average of the components (spectrum) of each pixels.
You have to compute e = 1/3 * (P0.R + P0.G + P0.B)
You'll get a monochromic picture with the exact same spatial frequency as the initial picture.
Temporal average
While you haven't talked about it, this is for reference. The idea is to compute the average of each pixel, and each component for N pictures in a temporal sequence
This gives a kind of motion blurred picture.
Answer
If I understand your question correctly, you want spectral average to convert a RGB to the average grey value taken that grey = (R+G+B)/3.
Thus, you pixel loop should look like this:
for (DWORD y = 0; y < dwWindowSize; y++) {
for (DWORD x = 0; x < dwWindowSize; x++) {
color = m_pImage->GetPixelColor(x, y);
float avg = (color.rgbRed + color.rgbGreen + color.rgbBlue) / 3.f;
m_pImage->SetPixelColor(x, y, RGBQUAD(avg, avg, avg, 1.0f));
}
}
Please notice that converting non linear RGB (usually called sRGB) to luminance using the average is a poor formula for RGB to grayscale conversion. You should read about RGB to Lab* conversion (you are interested in L part only) or at least RGB to YUV (you are interested to Y part only).
If your question is about resizing the input picture, then you are not using the appropriate algorithm, what you want is called resampling.
I'm working on project, where I need to visualize spectral analysis to set some precise parameters. Now I'm with conversion of bins to screen space, because in linear space, magnitudes in lower frequencies are squashed together. Here's my code in c++:
float windowSize = 640;
float windowHeight = 480;
for (size_t i = 0; i < bins; i++)
{
float m = audioIn.getSpectrum.at(i)*windowHeight;
float pos = i;
drawLine(vec2(pos, 0), vec2(pos, m));
}
I was trying to compute pos by using different approaches, but failed miserably. I'm missing crucial knowledge about logarithms I guess.
DISCLAIMER: this is for personal art project, not homework assignment.
Typically spectrographs are displayed on a base 10 logarithmic scale.
Assuming bins in your case go from 0 Hz to nyquist Hz you might try something like this (for 44.1kHz audio):
float nyquist = 22050.0;
float logMax = log10(nyquist);
float log = log10((float)i * nyquist / (float)bins);
float pos = log / logMax * windowSize;
I am trying to create a very simple C++ program that given an argument in range [0-100] applies a low-pass filter to a grayscale image that should "compress" it proprotionally to the value of the given argument.
I am using the FFTW library.
I have some doubts about how I define the frequency threshold, cut. Is there any more effective way to define such value?
//fftw_complex *fft
//double[] magnitude
// . . .
int percent = 100;
if (percent < 0 || percent > 100) {
cerr << "Compression rate must be a value between 0 and 100." << endl;
return -1;
}
double cut =(double)(w*h) * ((double)percent / (double)100);
for (i = 0; i < (w * h); i++) {
magnitude[i] = sqrt(pow(fft[i][0], 2.0) + pow(fft[i][1], 2.0));
if (magnitude[i] < cut) {
fft[i][0] = 0.0;
fft[i][1] = 0.0;
}
}
Update1:
I've changed my code to this, but again I'm not sure this is a proper way to filter frequencies. The image is surely compressed, but non-square images are messed up and setting compression to 100% isn't the real maximum compression available (I can go up to ~140%).
Here you can find an image of what I see now.
int cX = w/2;
int cY = h/2;
cout<<"TEST "<<((double)percent/(double)100)*h<<endl;
for(i = 0; i<(w*h);i++){
int row = i/s;
int col = i%s;
int distance = sqrt((col-cX)*(col-cX)+(row-cY)*(row-cY));
if(distance<((double)percent/(double)100)*min(cX,cY)){
fft[i][0] = 0.0;
fft[i][1] = 0.0;
}
}
This is not a low-pass filter at all. A low-pass filter passes low frequencies, i.e. it removes fine details (blurring). You obviously need a 2D FFT for that.
This code just removes random bits, essentially.
[edit]
The new code looks a lot more like a low-pass filter. The 141% setting is expected: the diagonal of a square is sqrt(2)=1.41 times its side. Converting an index into a row/column pair should use the image width, not some random unexplained s.
I don't know where your zero frequency is located. That should be easy to spot (largest value) but it might be in (0,0) instead of (w/2,h/2)
I want to smooth a histogram.
Therefore I tried to smooth the internal matrix of cvHistogram.
typedef struct CvHistogram
{
int type;
CvArr* bins;
float thresh[CV_MAX_DIM][2]; /* for uniform histograms */
float** thresh2; /* for non-uniform histograms */
CvMatND mat; /* embedded matrix header for array histograms */
}
I tried to smooth the matrix like this:
cvCalcHist( planes, hist, 0, 0 ); // Compute histogram
(...)
// smooth histogram with Gaussian Filter
cvSmooth( hist->mat, hist_img, CV_GAUSSIAN, 3, 3, 0, 0 );
Unfortunately, this is not working because cvSmooth needs a CvMat as input instead of a CvMatND. I couldn't transform CvMatND into CvMat (CvMatND is 2-dim in my case).
Is there anybody who can help me? Thanks.
You can use the same basic algorithm used for Mean filter, just calculating the average.
for(int i = 1; i < NBins - 1; ++i)
{
hist[i] = (hist[i - 1] + hist[i] + hist[i + 1]) / 3;
}
Optionally you can use a slightly more flexible algorithm allowing you to easily change the window size.
int winSize = 5;
int winMidSize = winSize / 2;
for(int i = winMidSize; i < NBins - winMidSize; ++i)
{
float mean = 0;
for(int j = i - winMidSize; j <= (i + winMidSize); ++j)
{
mean += hist[j];
}
hist[i] = mean / winSize;
}
But bear in mind that this is just one simple technique.
If you really want to do it using OpenCv tools, I recommend you access the openCv forum: http://tech.groups.yahoo.com/group/OpenCV/join
You can dramatically change the "smoothness" of a histogram by changing the number of bins you use. A good rule of thumb is to have sqrt(n) bins if you have n data points. You might try applying this heuristic to your histogram and see if you get a better result.