This question already has answers here:
How to render ocean wave using opengl in 3D? [closed]
(2 answers)
Closed 7 years ago.
I have absolutely no idea how to render water sources (ocean, lake, etc). It's like every tutorial I come across assumes I have the basic knowledge in this subject, and therefore speaks abstractly about the issue, but I don't.
My goal is to have a height based water level in my terrain.
I can't find any good article that will help me get started.
The question is quite broad. I'd split it up into separate components and get each working in turn. Hopefully this will help narrow down what those might be, unfortunately I can only offer the higher level discussion you aren't directly after.
The wave simulation (geometry and animation):
A procedural method will give a fixed height for a position and time based on some noise function.
A very basic idea is y = sin(x) + cos(z). Some more elaborate examples are in GPUGems.
Just like in the image, you can render geometry by creating a grid, sampling heights (y) at the grid x,y positions and connecting those points with triangles.
If you explicitly store all the heights in a 2D array, you can create some pretty decent looking waves and ripples. The idea here is to update height based on the neighbouring heights, using a few simple rules. For example, each height moves towards the average neighbouring height but also tends towards the equilibrium height equals zero. For this to work well, heights will need a velocity value to give the water momentum.
I found some examples of this kind of dynamic water here:
height_v[i][j] += ((height_west+ height_east + height_south + height_north)/4 - height[i][j]);
height_v[i][j] *= damping;
height[i][j] += height_v[i][j];
Rendering:
Using alpha transparency is a great first step for water. I'd start here until your simulation is running OK. The primary effect you'll want is reflection, so I'll just cover that. Further on you'll want to scale the reflection value using the Fresnel ratio. You may want an absorption effect (like fog) underwater based on distance (see Beer's law, essentially exp(-distance * density)). Getting really fancy, you might want to render the underneath parts of the water with refraction. But back to reflections...
Probably the simplest way to render a planar reflection is stencil reflections, where you'd draw the scene from underneath the water and use the stencil buffer to only affect pixels where you've previously drawn water.
An example is here.
However, this method doesn't work when you have a bumpy surface and the reflection rays are perturbed.
Rather than render the underwater reflection view directly to the screen, you can render it to a texture. Then you have the colour information for the reflection when you render the water. The tricky part is working out where in the texture to sample after calculating the reflection vector.
An example is here.
This uses textures but just for a perfectly planar reflection.
See also: How do I draw a mirror mirroring something in OpenGL?
Related
I am looking forward to implement a model reconstruction of RGB-D images. Preferred on mobile phones. For that I read, it is all done with an TSDF-representation. I read a lot of papers now over hierarchical structures and other ideas to speed this up, but my problem is, that I still hat no clue how to actually implement this representation.
If I have a volume grid of size n, so n x n x n and I want to store in each voxel the signed distance, weight and color information. My only guess is, that I have to build a discrete set of points, for each voxel position. And with GLSL "paint" all these points and calculate the nearest distance. But that don't seem quite good or efficient to calculate this n^3 times.
How can I imagine to implement such a TSDF-representation?
The problem is, my only idea is to render the voxel grid to store in the data of signed distances. But for each depth map I have to render again all voxels and calculate all distances. Is there any way to render it the other way around?
So can't I render the points of the depth map and store informations in the voxel grid?
How is the actual state of art to render such a signed distance representation in an efficient way?
You are on the right track, it's an ambitious project but very cool if you can do it.
First, it's worth getting a good idea how these things work. The original paper identifying a TSDF was by Curless and Levoy and is reasonably approachable - a copy is here . There are many later variations but this is the starting point.
Second, you will need to create nxnxn storage as you have said. This very quickly gets big. For example, if you want 400x400x400 voxels with RGB data and floating point values for distance and weight then that will be 768MB of GPU memory - you may want to check how much GPU memory you have available on a mobile device. Yup, I said GPU because...
While you can implement toy solutions on CPU, you really need to get serious about GPU programming if you want to have any kind of performance. I built an early version on an Intel i7 CPU laptop. Admittedly I spent no time optimising it but it was taking tens of seconds to integrate a single depth image. If you want to get real time (30Hz) then you'll need some GPU programming.
Now you have your TSFD data representation, each frame you need to do this:
1. Work out where the camera is with respect to the TSDF in world coordinates.
Usually you assume that you are the origin at time t=0 and then measure your relative translation and rotation with regard to the previous frame. The most common way to do this is with an algorithm called iterative closest point (ICP) You could implement this yourself or use a library like PCL though I'm not sure if they have a mobile version. I'd suggest you get started without this by just keeping your camera and scene stationary and build up to movement later.
2. Integrate the depth image you have into the TSDF This means updating the TSDF with the next depth image. You don't throw away the information you have to date, you merge the new information with the old.
You do this by iterating over every voxel in your TSDF and :
a) Work out the distance from the voxel centre to your camera
b) Project the point into your depth camera's image plane to get a pixel coordinate (using the extrinsic camera position obtained above and the camera intrinsic parameters which are easily searchable for Kinect)
c) Lookup up the depth in your depth map at that pixel coordinate
d) Project this point back into space using the pixel x and y coords plus depth and your camera properties to get a 3D point corresponding to that depth
e) Update the value for the current voxel's distance with the value distance_from_step_d - distance_from_step_a (update is usually a weighted average of the existing value plus the new value).
You can use a similar approach for the voxel colour.
Once you have integrated all of your depth maps into the TSDF, you can visualise the result by either raytracing it or extracting the iso surface (3D mesh) and viewing it in another package.
A really helpful paper that will get you there is here. This is a lab report by some students who actually implemented Kinect Fusion for themselves on a PC. It's pretty much a step by step guide though you'll still have to learn CUDA or similar to implement it
You can also check out my source code on GitHub for ideas though all normal disclaimers about fitness for purpose apply.
Good luck!
After I posted my other answer I thought of another approach which seems to match the second part of your question but it definitely is not reconstruction and doesn't involve using a TSDF. It's actually a visualisation but it is a lot simpler :)
Each frame you get an RGB and a Depth image. Assuming that these images are registered, that is the pixel at (x,y) in the RGB image represents the same pixel as that at (x,y) in the depth image then you can create a dense point cloud coloured using the RGB data. To do this you would:
For every pixel in the depth map
a) Use the camera's intrinsic matrix (K), the pixel coordinates and the depth value in the map at that point to project the point into a 3D point in camera coordinates
b) Associate the RGB value at the same pixel with that point in space
So now you have an array of (probably 640x480) structures like {x,y,z,r,g,b}
You can render these using on GLES just by creating a set of vertices and rendering points. There's a discussion on how to do this here
With this approach you throw away the data every frame and redo from scratch. Importantly, you don't get a reconstructed surface, and you don't use a TSDF. You can get pretty results but it's not reconstruction.
I have a function for generating a 3d-matrix with grey values (char values from 0 to 255). Now I want to generate a 3d-object out of this matrix, e.g. I want to display these values as a 3d-object (in cpp). What is the best way to do that platform-independent and as fast as possible?
I have already read a bit about using OGL, but then I run in the following problem: The matrix can contain up to $4\cdot10^9$ values. When I want to load the complete matrix into the RAM, it will collapse. So a direct draw from the matrix is impossible. Furthermore I only found functions for drawing 2d-images in OGL. Is there a way to draw 3d-pixels in OGL? Or should I rather use another approach?
I do not need a moving functionality (at least not at the moment), I just want to display the data.
Edit 2: For narrowing the question in: Is there a way to draw pixels in 3d-space with OGL taken from a 3d-matrix? I did not find a suitable function, I only found 2d-functions.
What you're looking to do is called volume rendering. There are various techniques to achieve it, and ultimately it depends on what you want it to look like.
There is no simple way to do this either. You can't just draw 3d pixels. You can draw using GL_POINTS and have each transformed point raster to 1 pixel, but this is probably completely unsatisfactory for you because it will only draw a some pixels to the screen (you wont see anything on big resolutions).
A general solution would be to just render a cube using normal triangles, for each point. Sort it back to front if you need alpha blending. If you want a more specific answer you will need to narrow your request. Ray tracing also has merits in volume rendering. Learn more on volume rendering.
I'm quite a beginner with c++, especially graphically related.
I would like to make an animated background for my graphicsview which looks kind of like this:
Gradient Field Airflow
The picture represents the turbulence of an airflow over an object.
The colors must be based on a matrix of values.
I can only find how to do single-direction gradients with QT.
How do I set this up? How do I get two-directional gradients?
/*edit
It has been pointed out well that technically speaking this is not a gradient, but an color interpolation on a 2d array of nodes.
*/
Well you have not provided the input data so no one knows what you really want to achieve !
if you have the flow trajectories and mass
Then you can use some particle system + heavy blurring/smoothing filtering to achieve this. For any known point along the trajectory plot a dithered circle with color depend on the mass/temp/velocity... and color scale. It should be solid in the middle and transparent on the edges. After rendering just blur/smooth the image few times and that should be it. The less points used the bigger the circles must be to cover the area nicely also can do it in multi pass and change the points coordinates randomly to improve randomness in the image...
if you have field strength/speed/temp or what ever grid values
Then it is similar to #1 also you can instead of particle system do the rendering via QUADs/Squares. The 2D linear gradient is called Bilinear Filtering
c00 -- x --> c01
|
|
y c(x,y)
|
|
V
c10 c11
where:
c00,c01,c10,c11 are corner colors
c(x,y) is color on x,y position inside square
x,y are in range <0,1> for simplicity (but you can use any with appropriate equations scaling)
Bilinear interpolation is 3x linear interpolation:
c0=c(x,0)=c00+((c01-c00)*x)
c1=c(x,1)=c10+((c11-c10)*x)
c(x,y) =c0 +((c1 -c0 )*y)
so render all pixels of the square with above computed colors and that is what you seek. This kind of filtering usually produce artifacts on the edges between squares or on diagonals to avoid that use non linear filtering or blur/smooth the final image
There is a tutorial on gradients in Qt: http://qt-project.org/doc/qt-4.8/demos-gradients.html and a class: http://harmattan-dev.nokia.com/docs/library/html/qt4/qgradient.html I have never used other than linear gradients and according to the docs, it seems there are only three basic types of gradients available in Qt: linear, radial and conical. If you cannot compose your desired gradient using these three types, then I am afraid you will need to program your image pixels by yourself. Not to forget, it might be worthy to explore if OpenGL somehow could help. Qt has some classes using OpenGL but I am not familiar with them to provide more advice.
I've been following the GPU Gems 3 tutorial on how to blur based on camera movement. However I'm wanting to implement a blur based on object movement too. The solution is presented in the article (see quote below), however I'm curious as to how exactly to implement this.
At the moment I'm multiplying the object's matrix by the view-projection, then separately again for the previous-view-projection and then passing them into the pixel shader to calculate the velocity instead of just the view-projections.
If that is in fact the correct method, then why am I not simply able to pass in the model-view-projection? I would have assumed they would be the same value?
GPU Gems 3 Motion Blur
To generate a velocity texture for rigid dynamic objects, transform the object by using the current frame's view-projection matrix and the last frame's view-projection matrix, and then compute the difference in viewport positions the same way as for the post-processing pass. This velocity should be computed per-pixel by passing both transformed positions into the pixel shader and computing the velocity there.
Check out my research I did a few months ago on this topic: https://slu-files.s3.us-east-1.amazonaws.com/Fragment_shader_dynamic_blur.pdf
(source: stevenlu.net)
(source: stevenlu.net)
Sadly I did not implement textured objects when producing this material, but do use your imagination. I am working on a game engine so when that finally sees the light of day in the form of a game, you can be sure that I'll come and post breadcrumbs here.
It primarily addresses how to implement this effect in 2D, and in cases where objects do not overlap. There is not really a good way to handle using a fragment shader to "sweep" samples in order to generate "accurate" blur. While the effect approaches pixel-perfection as the sample count is cranked up, the geometry that must be generated to cover the sweep area has to be manually assembled using some "ugly" techniques.
In full 3D it's a rather difficult problem to know which pixels a dynamic object will sweep over during the course of a frame. Even with static geometry and a moving camera the solution proposed by the GPU Gems article is incorrect when moving past things quickly because it is unable to address that issue of requiring blending of the area swept out by something moving...
That said, if this approximation which neglects the sweep is sufficient (and it may be) then what you can do to extend to dynamic objects is to take their motion into account. You'll need to work out the details of course but look at lines 2 and 5 in the second code block on the article you linked: They are the current and previous screen space "positions" of the pixel. You simply have to somehow pass in the matrices that will allow you to compute the previous position of each pixel, taking into account the dynamic motion of your object.
It shouldn't be too bad. In the pass where you render your dynamic object you send in an extra matrix that represents its motion over the last frame.
Update:
I found that this paper describes an elegant and performant approach that provides somewhat high quality physically correct blurring for a 3D pipeline. It'll be hard to do much better than this within the constraint of rendering the full scene no more than one time for performance reasons.
I noticed with some of the examples the quality of the velocity buffer could be better. for example a rotating wheel should have some curves in the velocity space. I believe if they can be set properly (may require custom fragment shaders to render the velocity out...) they will look intuitively correct like the spinning cube seen above from my 2D exploration into dynamic motion blurring.
As seen in the image
I draw set of contours (polygons) as GL_LINE_STRIP.
Now I want to select curve(polygon) under the mouse to delete,move..etc in 3D .
I am wondering which method to use:
1.use OpenGL picking and selection. ( glRenderMode(GL_SELECT) )
2.use manual collision detection , by using a pick-ray and check whether the ray is inside each polygon.
I strongly recommend against GL_SELECT. This method is very old and absent in new GL versions, and you're likely to get problems with modern graphics cards. Don't expect it to be supported by hardware - probably you'd encounter a software (driver) fallback for this mode on many GPUs, provided it would work at all. Use at your own risk :)
Let me provide you with an alternative.
For solid, big objects, there's an old, good approach of selection by:
enabling and setting the scissor test to a 1x1 window at the cursor position
drawing the screen with no lighting, texturing and multisampling, assigning an unique solid colour for every "important" entity - this colour will become the object ID for picking
calling glReadPixels and retrieving the colour, which would then serve to identify the picked object
clearing the buffers, resetting the scissor to the normal size and drawing the scene normally.
This gives you a very reliable "per-object" picking method. Also, drawing and clearing only 1 pixel with minimal per-pixel operation won't really hurt your performance, unless you are short on vertex processing power (unlikely, I think) or have really a lot of objects and are likely to get CPU-bound on the number of draw calls (but then again, I believe it's possible to optimize this away to a single draw call if you could pass the colour as per-pixel data).
The colour in RGB is 3 unsigned bytes, but it should be possible to additionally use the alpha channel of the framebuffer for the last byte, so you'd get 4 bytes in total - enough to store any 32-bit pointer to the object as the colour.
Alternatively, you can create a dedicated framebuffer object with a specific pixel format (like GL_R32UI, or even GL_RG32UI if you need 64 bits) for that.
The above is a nice and quick alternative (both in terms of reliability and in implementation time) for the strict geometric approach.
I found that on new GPUs, the GL_SELECT mode is extremely slow. I played with a few different ways of fixing the problem.
The first was to do a CPU collision test, which worked, but wasn't as fast as I would have liked. It definitely slows down when you are casting rays into the screen (using gluUnproject) and then trying to find which object the mouse is colliding with. The only way I got satisfactory speeds was to use an octree to reduce the number of collision tests down and then do a bounding box collision test - however, this resulted in a method that was not pixel perfect.
The method I settled on was to first find all the objects under the mouse (using gluUnproject and bounding box collision tests) which is usually very fast. I then rendered each of the objects that have potentially collided with the mouse in the backbuffer as a different color. I then used glReadPixel to get the color under the mouse, and map that back to the object. glReadPixel is a slow call, since it has to read from the frame buffer. However, it is done once per frame, which ends up taking a negligible amount of time. You can speed it up by rendering to a PBO if you'd like.
Giawa
umanga, Cant see how to reply inline... maybe I should sign up :)
First of all I must apologize for giving you the wrong algo - i did the back face culling one. But the one you need is very similar which is why I got confused... d'oh.
Get the camera position to mouse vector as said before.
For each contour, loop through all the coords in pairs (0-1, 1-2, 2-3, ... n-0) in it and make a vec out of them as before. I.e. walk the contour.
Now do the cross prod of those two (contour edge to mouse vec) instead of between pairs like I said before, do that for all the pairs and vector add them all up.
At the end find the magnitude of the resulting vector. If the result is zero (taking into account rounding errors) then your outside the shape - regardless of facing. If your interested in facing then instead of the mag you can do that dot prod with the mouse vector to find the facing and test the sign +/-.
It works because the algo finds the amount of distance from the vector line to each point in turn. As you sum them up and you are outside then they all cancel out because the contour is closed. If your inside then they all sum up. Its actually Gauss's Law of electromagnetic fields in physics...
See:http://en.wikipedia.org/wiki/Gauss%27s_law and note "the right-hand side of the equation is the total charge enclosed by S divided by the electric constant" noting the word "enclosed" - i.e. zero means not enclosed.
You can still do that optimization with the bounding boxes for speed.
In the past I've used GL_SELECT to determine which object(s) contributed the pixel(s) of interest and then used computational geometry to get an accurate intersection with the object(s) if required.
Do you expect to select by clicking the contour (on the edge) or the interior of the polygon? Your second approach sounds like you want clicks in the interior to select the tightest containing polygon. I don't think that GL_SELECT after rendering GL_LINE_STRIP is going to make the interior responsive to clicks.
If this was a true contour plot (from the image I don't think it is, edges appear to intersect) then a much simpler algorithm would be available.
You cant use select if you stay with the lines because you would have to click on the line pixels rendered not the space inside the lines bounding them which I read as what you wish to do.
You can use Kos's answer but in order to render the space you need to solid fill it which would involve converting all of your contours to convex types which is painful. So I think that would work sometimes and give the wrong answer in some cases unless you did that.
What you need to do is use the CPU. You have the view extents from the viewport and the perspective matrix. With the mouse coord, generate the view to mouse pointer vector. You also have all the coords of the contours.
Take the first coord of the first contour and make a vector to the second coord. Make a vector out of them. Take 3rd coord and make a vector from 2 to 3 and repeat all the way around your contour and finally make the last one from coord n back to 0 again. For each pair in sequence find the cross product and sum up all the results. When you have that final summation vector keep hold of that and do a dot product with the mouse pointer direction vector. If its +ve then the mouse is inside the contour, if its -ve then its not and if 0 then I guess the plane of the contour and the mouse direction are parallel.
Do that for each contour and then you will know which of them are spiked by your mouse. Its up to you which one you want to pick from that set. Highest Z ?
It sounds like a lot of work but its not too bad and will give the right answer. You might like to additionally keep bounding boxes of all your contours then you can early out the ones off of the mouse vector by doing the same math as for the full vector but only on the 4 sides and if its not inside then the contour cannot be either.
The first is easy to implement and widely used.