Short Version
How can I draw short text labels in an OpenGL mapping application without having to manually recompute coordinates as the user zooms in and out?
Long Version
I have an OpenGL-based mapping application where I need to be able to draw data sets with up to about 250k points. Each point can have a short text label, usally about 4 or 5 characters long.
Currently, I do this using a single textue containing all the characters. For each point, I define a quad for each character in its label. So a point with the label "Fred" would have four quads associated with it, and each quad uses texture coordinates into that single texture to draw its corresponding character.
When I draw the map, I draw the map points themselves in map coordinates (e.g., longitude/latitude). Then I compute the position of each point in screen coordinates and update the four corner points for each of that point's label quads, again in screen coordinates. (For instance, if I determine the point is drawn at screen point 100, 150, I could set the quad for the first character in the point's label to be the rectangle starting with left-top point of 105, 155 and having a width of 6 pixels and a height of 12 pixels, as appropriate for the particular character. Then the second character might start at 120, 155, and so on.) Then once all these label character quads are positioned correctly, I draw them using an orthogonal screen projection.
The problem is that the process of updating all of those character quad coordinates is slow, taking about half a second for a particular test data set with 150k points (meaning that, since each label is about four characters long, there are about 150k * [ 4 characters per point] * [ 4 coordinate pairs per character] coordinate pairs that need to be set on each update.
If the map application didn't involve zooming, I would not need to recompute all these coordinates on each refresh. I could just compute the label coordinates once and then simply shift my viewing rectangle to show the right area. But with zooming, I can't see how to make it work without doing coordniate computation, because otherwise the characters will grow huge as you zoom in and tiny as you zoom out.
What I want (and what I understand OpenGL doesn't provide) is a way to tell OpenGL that a quad should be drawn in a fixed screen-coordinate rectangle, but that the top-left position of that rectangle should be a fixed distance from a given point in map coordinate space. So I want both a primitive hierarchy (a given map point is that parent of its label character quads) and the ability to mix two different coordinate systems within this hierarchy.
I'm trying to understand whether there is some magic transformation matrix I can set that will do all this form me, but I can't see how to do it.
The other alternative I've considered is using a shader on each point to handle computing the label character quad coordinates for that point. I haven't worked with shaders before, and I'm just trying to understand (a) if it's possible to use shaders to do this, and (b) whether computing all those points in shader code actually buys me anything over computing them myself. (By the way, I have confirmed that the big bottleneck is computing the quad coordinates, not in uploading the updated coordinates to the GPU. The latter takes a bit of time, but it's the computation, the sheer number of coordinates being updated, that takes up the bulk of that half second.)
(Of course, the other other alternative is to be smarter about which labels need to be drawn in a given view in the first place. But for now I'd like to concentrate on the solution assuming all labels need to be drawn.)
So the basic problem ("because otherwise the characters will grow huge as you zoom in and tiny as you zoom out") is that you are doing calculations in map coordinates rather than screen coordinates? And if you did it in screen coords, this would require more computations? Obviously, any rendering needs to translate from map coordinates to screen coordinates. The problem seems to be that you are translating from map to screen too late. Therefore, rather than doing a single map-to-screen for each point, and then working in screen coords, you are working mostly in map coords, and then translating per-character to screen coords at the very end. And the slow part is that you are working in screen coords, then having to manually translate back to map coords just to tell OpenGL the map coords, and it will convert those back to screen coords! Is that a fair assessment of your problem?
The solution therefore is to push that transformation earlier in your pipeline. However, I can see why it is tricky, because at first glance, OpenGL seems want to do everything in "world coordinates" (for you, map coords), but not in screen coords.
Firstly, I am wondering why you are doing separate coordinate calculations for each character. What font rendering system are you using? Something like FreeType will automatically generate a bitmap image of an entire string, and doesn't require you to work per-character [edit: this isn't quite true; see comments]. You definitely shouldn't need to calculate the map coordinate (or even screen coordinate) for every character. Calculate the screen coordinate for the top-left corner of the label, and have your font rendering system produce the bitmap of the entire label in one go. That should speed things up about fourfold (since you assume 4 characters per label).
Now as for working in screen coords, it may be helpful to learn a bit about shaders. The more you learn about OpenGL, the more you learn that really it isn't a 3D rendering engine at all. It's just a 2D graphics library with some very fast matrix primitives built-in. OpenGL actually works, at the lowest level, in screen coordinates (not pixel coordinates -- it works in normalized screen space, I think from memory from -1 to 1 in both the X and Y axis). The only reason it "feels" like you're working in world coordinates is because of these matrices you have set up.
So I think the reason why you are working in map coords all the way until the end is because it's easiest: OpenGL naturally does the map-to-screen transform for you (using the matrices). You have to change that, because you want to work in screen coords yourself, and therefore you need to make the transformation a long time before OpenGL gets its hands on your data. So when you go to draw a label, you should manually apply the map-to-screen transformation matrix on each point, as follows:
You have a particular point (which needs a label drawn) in map coords.
Apply the map-to-screen matrix to convert the point to screen coords. This probably means multiplying the point by the MODELVIEW and PROJECTION matrices, using the same algorithm that OpenGL does when it's rendering a vertex. So you could either glGet the GL_MODELVIEW_MATRIX and GL_PROJECTION_MATRIX to extract OpenGL's current matrices, or you could manually keep around a copy of the matrix yourself.
Now you have the map label in screen coords, compute the position of the label's text. This is simply adding 5 pixels in the X and Y axis, as you said above. However, remember that you aren't in pixel space, but normalised screen space, so you are working in percentages (add 0.05 units, would add 5% of the screen space, for example). It's probably better not to think in pixels, because then your application will scale to match the resolution. But if you really want to think in pixels, you will have to calculate the pixels-to-units based on the resolution.
Use glPushMatrix to save the current matrix, then glLoadIdentity to set the current matrix to the identity -- tell OpenGL not to transform your vertices. (I think you will have to do this for both the PROJECTION and MODELVIEW matrices.)
Draw your label, in screen coordinates.
So you don't really need to write a shader. You could certainly do this in a shader, and it would certainly make step 2 faster (no need to write your own software matrix multiply code; multiplying matrices on the GPU is extremely fast). But that would be a later optimisation, and a lot of work. I think the above steps will help you work in screen coordinates and avoid having to waste a lot of time just to give OpenGL map coordinates.
Side comment on:
"""
generate a bitmap image of an entire string, and doesn't require you to work per-character
...
Calculate the screen coordinate for the top-left corner of the label, and have your font rendering system produce the bitmap of the entire label in one go. That should speed things up about fourfold (since you assume 4 characters per label).
"""
Freetype or no, you could certainly compute a bitmap image for each label, rather than each character, but that would require one of:
storing thousands of different textures, one for each label
It seems like a bad idea to store that many textures, but maybe it's not.
or
rendering each label, for each point, at each screen update.
this would certainly be too slow.
Just to follow up on the resolution:
I didn't really solve this problem, but I ended up being smarter about when I draw labels in the first place. I was able to quickly determine whether I was about to draw too many characters (i.e., so many characters that on a typical screen with a typical density of points the labels would be too close together to read in a useful way) and then I simply don't label at all. With drawing up to about 5000 characters at a time there isn't a noticeable slowdown recomputing the character coordinates as described above.
Related
Is there a way to set a transformation for NDC to window, but separately specify the clipping region so it matches the actual window size?
Background: I have a bunch of openGL code that renders a 2D map to a window. It's a lot of complex code, because I use both the GPU and the CPU to draw on the map, so it's important that I keep to a consistent coordinate system in both places. To keep that simple, I use glViewport(0,0,mapSizeX, mapSizeY), and now map coordinates correspond well to pixel coordinates in the frame buffer, exactly what I need. I can use GLSL to draw some of the map, call glReadPixels and use the CPU to draw on top of that, and glDrawPixels to send that back to the frame buffer, all of that using the same coordinate system. Finally I use GLSL to draw a few final things over that (that I don't want zoomed). That all works, except...
The window isn't the same size as the map, and glViewport doesn't just set up the transformation. It also sets up clipping. So now when I go to draw a few last items, and the window is larger than the map, things I draw near the top of the screen get clipped away. Is there a workaround?
glViewport doesn't just set up the transformation. It also sets up clipping.
No, it just sets up the transformation. By the time the NDC-to-window space transform happens, clipping has already been done. That happened immediately after vertex processing; your vertex shader (or whatever you're doing to transform vertices) handled that based on how it transformed vertices into clip-space.
You should use the viewport to set up how you want the NDC box to visibly appear in the window. Your VS needs to handle the transformation into the clipping area. So it effectively decides how much of the world gets put into the NDC box that things get clipped to.
Basically, you have map space (the coordinates used by your map) and clip-space (the coordinates after vertex transformations). And you have some notion of which part of the map you want to actually draw to the window. You need to transform the region of your map that you want to see such that the corners of this region appear in the corners of the clipping box (for orthographic projections, this is typically [-1, 1]).
In compatibility OpenGL, this might be defined by using glOrtho for othographic projections to transform from you. In a proper vertex shader, you'll need to provide an appropriate orthographic matrix.
This is NOT about how to create and render curves (e.g. Bezeier, NURBS, etc) BUT about how to be able to interactively 'pick' such curves by mouse click when the mouse cursor is hovering over ANY part of such a curve.
After doing the required computing I render a curve, in either 2D or 3D, by subdividing it into lots of individual smaller line segments. The more such line segments there are the smoother and more realistic the curve looks.
I deliberately create each of these GL_LINE segments as separate entities (each being assigned their own unique ID number). These line segments belong to the Curve 'object' (which also has it's own unique ID). Then, using using Ray Casting I can enable mouse-line collsion detection and know when an individual line segment has been 'hit' - and highlight it (e.g. temporarily change its color).
BUT at the same time also highlight all the other line segments that belong to the Curve - and so give appearance of the whole curve being selected.
THE PROBLEM is that because each curve is made up of not just the 'core' control points, which effectively define the curve, but also the thousands of points that effectively draw the curve there is quickly a very noticable slowing down of graphics performance.
I am aware that I could more efficiently instead compute the all the subdivision points and use LINE_STRIP instead to render the curve as one graphical object? BUT then that will NOT allow me to use the ray casting technique described to be be able hover the mouse cursor over any part of the curve and 'select' the curve object.
So....how can I more efficiently 'pick' curves in OpenGL?
You got 2 obvious options for this:
use ID buffer
when you render your curve you assign color to the RGBA frame buffer. So simply also assign ID of renderd curve to separate buffer (of the same resolution as view) and then simply pick the pixel position under mouse from this buffer to see exactly which curve or object you select.
This is pixel perfect and O(1) super fast ... However if your objects are too thin you might have problems using it for mouse picking from single pixel so test up to some distance from mouse (you can glReadPixels rectangle around mouse) end return either all the IDs present or just the most often one.
See this:
OpenGL 3D-raypicking with high poly meshes
Do not forget to clear the ID buffer before rendering and if you got too many objects that will not fit into 8bit stencil use different buffer... In case of 2D you can use depth buffer, or you can render to RGBA framebufer in 2 passes first the ID then read into CPU side memory and ten normal render. You can also render to texture ...
compute distance to curve
its doable for example see:
Is it possible to express âtâ variable from Cubic Bezier Curve equation?
as you can see its possible to use this also for the rendering itself (no line approximation just the "perfect" curve) and even with speed ...
So simply compute the distance of mouse to each of your curves and remember the closest one. If the distance is bigger than some threshold/distance no selection occurs.
I have some 64x64 sprites which work fine (no flicker/shuffling) when I move my camera normally. But as soon as I change the camera.zoom (was supposed to be a major mechanic in my game) level away from 1f the sprites flicker every time you move.
For example changing to 0.8f:
Left flicker:
One keypress later: (Right flicker)
So when you move around it's really distracting for gameplay when the map is flickering... (however slightly)
The zoom is a flat 0.8f and I'm currently using camera.translate to move around, I've tried casting to (int) and it still flickered... My Texture/sprite is using Nearest filtering.
I understand zooming may change/pixelate my tiles but why do they flicker?
Edit
For reference here is my tilesheet:
It's because of the nearest filtering. Depending on amount of zoom, certain lines of artwork pixels will straddle lines of screen pixels so they get drawn one pixel wider than other lines. As the camera moves, the rounding works out differently on each frame of animation so that different lines are drawn wider on each frame.
If you aren't going for a retro low-res aesthetic, you could use linear filtering with mip maps (MipMapLinearLinear or MipMapLinearNearest). Then start with larger resolution art. The first looks better if you are smoothly transitioning between zoom levels, with a possible performance impact.
Otherwise, you could round the camera's position to an exact multiple of the size of a pixel in world units. Then the same enlarged columns will always correspond with the same screen pixels, which would cut down on perceived flickering considerably. You said you were casting the camera translation to an int, but this requires the art to be scaled such that one pixel of art exactly corresponds with one pixel of screen.
This doesn't fix the other problem, that certain lines of pixels are drawn wider so they appear to have greater visual weight than similar nearby lines, as can be seen in both your screenshots. Maybe one way to get round that would be to do the zoom a secondary step, so you can control the appearance with an upscaling shader. Draw your scene to a frame buffer that is sized so one pixel of texture corresponds to one world pixel (with no zoom), and also lock your camera to integer locations. Then draw the frame buffer's contents to the screen, and do your zooming at this stage. Use a specialized upscaling shader for drawing the frame buffer texture to the screen to minimize blurriness and avoid nearest filtering artifacts. There are various shaders for this purpose that you can find by searching online. Many have been developed for use with emulators.
I need to have a 2D layer in my OpenGL application.I have implemented it first using a typical ortho projection like this:
Mat4 ortho =Glm.ortho(0,viewWidth , 0 ,viewHeight);
The 2d worked fine except the fact that when running in different screen sizes the 2d shapes are scaled relatively to a new aspect.That is not what I want (opposite to what usually people need). I need the 2d shapes to get stretched or squeezed according to the new screen size.
I tried not to use the ortho matrix but just an identity.This one works but in such a case I have to use numbers in range 0 -1 to manipulate the objects in the visible frustum area.And I need to use numbers in regular (not normalized ) ranges.So it is sort of forcing me to get back to ortho projection which is problematic because of what already said.
So the question is how do I transform 2d object without perspective staying in the world coordinates system.
UPDATE:
The best example is 2D layers in Adobe AfterEffects. If one changes composition dimension ,2d layers don't get scaled according to new dimensions.That is what I am after.
It's tricky to know how to answer this, because to some degree your requirements are mutually exclusive. You don't want normalised coordinates, you want to use screen coordinates. But by definition, screen coordinates are defined in pixels, and pixels are usually square... So I think you need some form of normalised coordinates, albeit maybe uniformly scaled.
Perhaps what you want is to fix the ratio for width and height in your ortho. That would allow you to address the screen in some kind of pseudo-pixel unit, where one axis is "real" pixels, but the other can be stretched. So instead of height, pass 3/4 of the width for a 4:3 display, or 9/16ths on a 16:9, etc. This will be in units of pixels if the display is the "right" dimension, but will stretch in one dimension only if it's not.
You may need to switch which dimension is "real" pixels depending on the ratio being less or greater than your "optimal" ratio, but it's tricky to know what you're really shooting for here.
So I'm rendering this diagram each frame:
https://dl.dropbox.com/u/44766482/diagramm.png
Basically, each second it moves everything one pixel to the left and every frame it updates the rightmost pixel column with current data. So a lot of changes are made.
It is completely constructed from GL_LINES, always from bottom to top.
However those black missing columns are not intentional at all, it's just the rasterizer not picking them up.
I'm using integers for positions and bytes for colors, the projection matrix is exactly 1:1; translating by 1 means moving 1 pixel. Orthogonal.
So my problem is, how to get rid of the black lines? I suppose I could write the data to texture, but that seems expensive. Currently I use a VBO.
Render you columns as quads instead with a width of 1 pixel, the rasterization rules of OpenGL will make sure you have no holes this way.
Realize the question is already closed, but you can also get the effect you want by drawing your lines centered at 0.5. A pixel's CENTER is at 0.5, and drawing a line there will always be picked up by the rasterizer in the right place.