I'm writing some code that uses SDL2 to display an image with moving markers layered on it, and I think I'd like to use the new (?) 2D hardware accelerated rendering. As I understand it, I have to load an image and convert it to a texture -- but what's the difference? Searching for 'image texture 2d sdl' only gets me tutorials on how to load textures and I'm looking for more of the background rather than the how-to.
So, some questions:
What's a texture versus an image? Aren't they the same thing?
Am I correct in assuming that I need to load the static background image as a texture if I want hardware accelerated rendering? In fact, it sounds like all the bits need to be textures for this to work.
Speaking of OpenGL, are SDL textures actually OpenGL textures?
I'm writing the main app for a single-purpose machine with limited resources (dual core ARM CPU, dual core Mali 400 GPU, 4GB RAM: Olimex A20 LIME2). All I need to do is render an 480x800 (yes, portrait layout) image and put markers on it. I expect the markers to have a single opaque and two transparency layers, to be updated at around 15 fps, and I expect about 125 of them, tops. Is it worth my while to use 2D hardware acceleration or should I just do it in software?
To understand the basics of textures, I advise you to have a look at a simpler library's documentation. Here, the term pixmap is used in the same way as SDL's texture. Essentially, those are already converted and uploaded into your GPU's memory, which makes operations quite a bit faster, but also more complex to deal with.
OpenGL textures are another beast, but we could basically say that they are the same, that is, images in video memory. When binding a texture in OpenGL, you need to upload it to the GPU memory, which is somewhat similar to this texture transformation.
At 125 objects, I think considering using the 2D acceleration becomes worth the hassle, especially if you have to move them around. If this is just a static image, I guess you could go for the regular image route.
As a general rule, I encourage you to use 2D acceleration (or just acceleration, for that matter) whenever possible, if only for the battery improvements. With that said, if the images are static, the outcome will exactly be the same, maybe just slightly different code-path wise. As such, I suppose you could load the static background image just as a regular image without any downsides (note that I am not a SDL professional, so this mixed approach might not work here, but it is worth trying since it will work on most 2D toolkits).
I hope I answered all of your questions :)
Related
I've been learning OpenGL, and as I sit trying to write my VBOs, PBOs, VAOs, textures, quads, bindings, fragment shaders, vertex shaders, and a whole suite of other modern abstractions upon abstractions built after decades of evolution, I wonder: Isn't the display nothing but a large block of memory?
I've heard of tales, that in the "good ol' days" (such as the Commodore 64), all you had to do was assign a value to an arbitrary byte in memory, and the screen would change a pixel. Extremely simple and elegant. In the modern day, this has changed with layers upon layers of abstractions and safeguards, such that changing a pixel on your display is several hundred feet away.
This begs the question, is it possible in the modern day to just "update a pixel of the screen"? Is it possible to write my own graphics driver or something, where I can send commands to some C wrapper which interfaces with the GPU to change those pixels? This is an extremely broad question, but I'm curious. The answer I'm looking for to this question would provide a rough outline of what you'd have to do in order to be able to arbitrarily get some C code to set a pixel on the screen, as well as a rough outline of why OpenGL has progressed the way it has - what problems did VBOs, PBOs, VAOs, bindings, shaders, etc. solve, and how we got to where we are today.
Isn't the display nothing but a large block of memory?
Yes, it is called a framebuffer.
I've heard of tales, that in the "good ol' days" (such as the Commodore 64)
Your current PC works like that right when you power it up! If you use the CPU to write into video memory, that is called a software renderer.
In the modern day, this has changed with layers upon layers of abstractions and safeguards, such that changing a pixel on your display is several hundred feet away.
No, they are not abstractions/safeguards for "changing pixels". Nowadays software renderers are not used anymore. Instead, you have to tell the GPU (which is another computer on its own) how to draw. That "talk" is what the APIs (like OpenGL) do for you.
Now, the GPUs are meant to be fast at drawing, and that requires specialized code and data structures. Those are all the things you mention: VBOs, PBOs, VAOs, shaders, etc. (in OpenGL parlance). There is no way around that, because GPUs are different hardware.
is it possible in the modern day to just "update a pixel of the screen"?
Yes, but that will end up being drawn somehow by the GPU, even if it looks to you like a memory write.
Is it possible to write my own graphics driver or something, where I can send commands to some C wrapper which interfaces with the GPU to change those pixels?
Yes, but that "C wrapper" is the graphics driver. A graphics driver for a modern GPU is very complex.
what you'd have to do in order to be able to arbitrarily get some C code to set a pixel on the screen
You cannot write a "C program" to write to a graphical screen because the C standard does not concern itself with graphical displays.
So it depends on your operating system, your hardware, whether you want 2D or 3D acceleration support, the API you choose...
as well as a rough outline of why OpenGL has progressed the way it has - what problems did VBOs, PBOs, VAOs, bindings, shaders, etc. solve, and how we got to where we are today.
See above.
You can make your own frame buffer - that is just an integer array - and do rasterization on it, then use for example the Windows GDI function SetBitmapBits() to draw it to the display in one go. The final draw-to-display command depends on the operating system.
How you do the rasterization on your framebuffer is completely up to you. You can use the CPU to draw individual pixels or rasterize lines and triangles, see for example this demo of my old CPU graphics engine using Windows GDI: https://youtu.be/GFzisvhtRS4.
Using the CPU is fine as long as you do not rasterize large datasets. From my experience, the limit to real-time 60fps rendering on the CPU is ~50k lines per frame.
If you want to rasterize really large datasets, you have to use a GPU in some way. Since the framebuffer is just an integer array, you can transfer it to/from the GPU using OpenCL or CUDA and on the GPU - if your dataset happens to already be in video memory - do all the rasterization extremely fast in parallel. For this you will need an additional z-buffer to decide which pixels to overdraw by occluding geometries. This way you can rasterize approximately 30 Million lines per frame at 60fps. This demo is rendered on the GPU in real time using OpenCL: https://youtu.be/lDsz2maaZEo
Is it possible in the modern day to just "update a pixel of the screen"?
Yes. In Windows for example, you can use SetPixel() to draw a pixel or BitBlt() to draw in bulk. See this Q/A
This works fine, but this means you're using the CPU for rendering and you'll find the GPU is much more effective for this task, especially if you require decent framerate and non-trivial graphics. The reason there's these "whole suite of other modern abstractions upon abstractions" is to serve as an interface to the GPU since it has an independent set of memory and totally different execution model. Other GPU libraries (OpenCL, DirectX, Vulkan, etc) all have the same kind of abstractions.
I've glossed over many nuances but I hope the point gets across.
I have used OpenGL pretty exclusively for all my rendering, to the point that I'm unaware of any other way to write pixels to a window unfortunately.
This is a problem because my current project is a work tool that emulates an LCD display (pixel perfect, 2D, very few pixels are touched each frame, all 'drawing' can be done with memcpy() to a pixel buffer) and I feel that OpenGL might be too heavy for this, but I could absolutely be wrong in that assumption.
My goal is to borrow as little CPU time as possible. What's the best way to draw pixels to a window, in this limited way, on a modern typical machine running windows 10 circa 2019? Is OpenGL suited for this type of rendering, or should I adopt another rendering method in this case? And if so, what would that method be?
edit:
I should also mention, OpenGL can be used right away for me. If rendering textured triangles with an optimal setup is the fastest method, then I can already do that. Anything that just acts as an API over OpenGL or DirectX will likely be worse in my case.
edit2:
After some research, and thanks to the comments, I think I may just use OpenGL with Pixel Buffer Objects to optimize pixel uploads and keep rendering inexpensive.
I was moving from SDL to SDL2, and was confused of the 'render & texture' system introduced.
Back in SDL, the most frequent operation was creating Surface's and BlitSurface them onto screen. Now there seems a trend of using renderers and textures. However, this is extremely slow (in terms of coding overhead) from my point. Why can't I just load_BMP and BlitSurface as before? What good can be introduced from this whole window-renderer-texture thing?
I have read a couple threads What is a SDL renderer? but still a little confusing.
So..
Will the old Surface way work in SDL2?
What is the point in Renderer & Texture? (could be about hardware-accelaration according to a little googling, but not sure what that means)
You might want to take a look at the migration guide for SDL2, it provides information on the new way of dealing with 2D graphics.
The point of using textures instead of surfaces is that textures works on the GPU and get loaded into video memory and surfaces works in system memory with the CPU and since GPUs are much better suited than CPU for handling graphics it will be faster. Also, the renderer hides the underlying system used (it could be D3D, OpenGL, or something else).
You can still load surfaces, but you'll have to convert them to textures before rendering them or use the SDL_UpdateWindowSurface and SDL_GetWindowSurface functions; the latter link includes an example on how to use them.
As for the SDL2 approach being slow, as you assert, I don't agree with you, you set up the window and renderer once, load your textures much like you loaded surfaces, copy them to the renderer instead of blitting and finally use SDL_RenderPresent instead of SDL_Flip. Not that different really :)
But, take a look at the migration guide, it has the information you're looking for.
If we want to make an application like MS Paint, should we use OpenGL for render graphics?
I want to mention about performance if using traditional GDI vs. OpenGL.
And if there are exist some better libs for this purpose, please see me one.
GDI, X11, OpenGL... are rendering APIs, i.e. you usually don't use them for image manipulation (you can do this, but it requires some precautions).
In a drawing application like MS Paint, if it's pixel based, you'll normally manipulate some picture buffer with customary code, or a special image manipulation library, then send the full buffer to the rendering API.
If your data model consists of strokes and individual shapes, i.e. vector graphics, then OpenGL makes a quite good backend. However it may be worth looking into some other API for vector graphics, like OpenVG (which in its current implementations sits on top of OpenGL, but native implementations operating directly on the GPU may come).
In your usage scenario you'll not run into any performance problems on current computers, so don't choose your API from that criteria. OpenGL is definitely faster than GDI when it comes to texturing, alpha blending, etc. However depending on system and GPU pure GDI may outperform OpenGL for so simple things like drawing an arc or filling a complex self intersecting polygon with complex winding rules.
There is no good reason not to use OpenGL for this. Except maybe if you have years of experience with GDI but don't know a single thing about OpenGL.
On the other hand, OpenGL may very well be superior in many cases. Compositing layers or adjusting hue/saturation/brightness/contrast in a GLSL shader will be several orders of magnitude faster (in fact, pretty much "instantly") if there is a reasonably new card in the computer. Stroking a freedraw path with a "fuzzy" pen (i.e. blending a sprite with alpha transparency over and over again) will be orders of magnitude faster. On images with somewhat reasonable dimensions, most filter kernels should run close to realtime. Rescaling with bilinear filtering runs in hardware.
Such things won't matter on a 512x512 image, as pretty much everything is instantaneous at such resolutions, but on a typical 4096x3072 (or larger) image from your digital camera, it may be very noticeable, especially if you have 4-6 layers.
I don't know whether this is the right forum. Anyway here is the question. In one of our application we display medical images and on top of them some algorithm generated bitmap. The real bitmap is a 16bit gray scale bitmap. From this we generate a color bitmap based on a look up table for eg
(0-100)->green
(100-200)->blue
(200>above)->red
The display is working well and good with small images 256x256. But when the display area becomes big say 1024x1024 the gray scale to color bitmap conversion takes a while and the interactions are not smooth any more. In the recent times I have heard a lot about general purpose GPU programming. In our deployment we have high end (Nvidia QuadroFX) graphics card.
Our application is built using .Net/C# if requiured I can add little bit of C++/CLI too.
Now my question is can this bitmap conversion be offloaded to the graphics processor? Where should I look for further reading?
Yes -- and since you're (apparently) displaying the bitmap, you don't need to go the GPGPU route (e.g., OpenCL or CUDA). You can use a programmable shader for it -- and if I understand what you're saying, it'll be a pretty straightforward one at that.
As far as how to write the shader, it will depend (mostly) on how you're doing the rest of your drawing. Just for an obvious example, if you're already using WPF for your drawing, you'll probably want to use an HLSL shader (WPF supports pixel shaders fairly directly).
It's probably also worth noting that if you had to support older hardware, a table lookup like this is something you could actually manage pretty easily on the GPU, even without programmable shaders. As long as you only need to support recent hardware, a shader will probably be simpler though.