Is it possible yet to draw CUDA/OPENCL results directly to the screen with any existing API (opengl, directx, something else)? Skipping the typical drawing a textured quad method.
Even with registering resources and using modern CUDA interop methods, we still have to march through entire rendering pipelines just to render an array of colors. For applications like mine where every ms counts, this is a problem.
There's no way to draw directly on the screen with OpenCL or CUDA.
It is a solvable problem, but as far as I know, NVIDIA has not provided the needed APIs because they would be very complicated both to implement and to use, and the performance benefits would be limited at best.
The two main issues are:
1) the differing layouts of the buffers used for rendering (i.e. you'd have to use surface load/store functionality - a mapping into CUDA's address space is not suitable for graphics because the pitch-linear layout has poor performance in that context) and
2) the platform-specific details of incorporating your CUDA/OpenCL output into the presentation model (be it the desktop or a page-flipped full-screen experience, like a Direct3D game, or incorporating your app's output into the desktop). Bear in mind that most desktops these days are themselves page-flipped, so scribbling on the front buffer is frowned upon in any case.
I very much doubt that there is any performance lost in drawing pixels using a textured quad but you can draw pixels directly on the framebuffer with glDrawPixels.
Related
I've been learning OpenGL, and as I sit trying to write my VBOs, PBOs, VAOs, textures, quads, bindings, fragment shaders, vertex shaders, and a whole suite of other modern abstractions upon abstractions built after decades of evolution, I wonder: Isn't the display nothing but a large block of memory?
I've heard of tales, that in the "good ol' days" (such as the Commodore 64), all you had to do was assign a value to an arbitrary byte in memory, and the screen would change a pixel. Extremely simple and elegant. In the modern day, this has changed with layers upon layers of abstractions and safeguards, such that changing a pixel on your display is several hundred feet away.
This begs the question, is it possible in the modern day to just "update a pixel of the screen"? Is it possible to write my own graphics driver or something, where I can send commands to some C wrapper which interfaces with the GPU to change those pixels? This is an extremely broad question, but I'm curious. The answer I'm looking for to this question would provide a rough outline of what you'd have to do in order to be able to arbitrarily get some C code to set a pixel on the screen, as well as a rough outline of why OpenGL has progressed the way it has - what problems did VBOs, PBOs, VAOs, bindings, shaders, etc. solve, and how we got to where we are today.
Isn't the display nothing but a large block of memory?
Yes, it is called a framebuffer.
I've heard of tales, that in the "good ol' days" (such as the Commodore 64)
Your current PC works like that right when you power it up! If you use the CPU to write into video memory, that is called a software renderer.
In the modern day, this has changed with layers upon layers of abstractions and safeguards, such that changing a pixel on your display is several hundred feet away.
No, they are not abstractions/safeguards for "changing pixels". Nowadays software renderers are not used anymore. Instead, you have to tell the GPU (which is another computer on its own) how to draw. That "talk" is what the APIs (like OpenGL) do for you.
Now, the GPUs are meant to be fast at drawing, and that requires specialized code and data structures. Those are all the things you mention: VBOs, PBOs, VAOs, shaders, etc. (in OpenGL parlance). There is no way around that, because GPUs are different hardware.
is it possible in the modern day to just "update a pixel of the screen"?
Yes, but that will end up being drawn somehow by the GPU, even if it looks to you like a memory write.
Is it possible to write my own graphics driver or something, where I can send commands to some C wrapper which interfaces with the GPU to change those pixels?
Yes, but that "C wrapper" is the graphics driver. A graphics driver for a modern GPU is very complex.
what you'd have to do in order to be able to arbitrarily get some C code to set a pixel on the screen
You cannot write a "C program" to write to a graphical screen because the C standard does not concern itself with graphical displays.
So it depends on your operating system, your hardware, whether you want 2D or 3D acceleration support, the API you choose...
as well as a rough outline of why OpenGL has progressed the way it has - what problems did VBOs, PBOs, VAOs, bindings, shaders, etc. solve, and how we got to where we are today.
See above.
You can make your own frame buffer - that is just an integer array - and do rasterization on it, then use for example the Windows GDI function SetBitmapBits() to draw it to the display in one go. The final draw-to-display command depends on the operating system.
How you do the rasterization on your framebuffer is completely up to you. You can use the CPU to draw individual pixels or rasterize lines and triangles, see for example this demo of my old CPU graphics engine using Windows GDI: https://youtu.be/GFzisvhtRS4.
Using the CPU is fine as long as you do not rasterize large datasets. From my experience, the limit to real-time 60fps rendering on the CPU is ~50k lines per frame.
If you want to rasterize really large datasets, you have to use a GPU in some way. Since the framebuffer is just an integer array, you can transfer it to/from the GPU using OpenCL or CUDA and on the GPU - if your dataset happens to already be in video memory - do all the rasterization extremely fast in parallel. For this you will need an additional z-buffer to decide which pixels to overdraw by occluding geometries. This way you can rasterize approximately 30 Million lines per frame at 60fps. This demo is rendered on the GPU in real time using OpenCL: https://youtu.be/lDsz2maaZEo
Is it possible in the modern day to just "update a pixel of the screen"?
Yes. In Windows for example, you can use SetPixel() to draw a pixel or BitBlt() to draw in bulk. See this Q/A
This works fine, but this means you're using the CPU for rendering and you'll find the GPU is much more effective for this task, especially if you require decent framerate and non-trivial graphics. The reason there's these "whole suite of other modern abstractions upon abstractions" is to serve as an interface to the GPU since it has an independent set of memory and totally different execution model. Other GPU libraries (OpenCL, DirectX, Vulkan, etc) all have the same kind of abstractions.
I've glossed over many nuances but I hope the point gets across.
I have used OpenGL pretty exclusively for all my rendering, to the point that I'm unaware of any other way to write pixels to a window unfortunately.
This is a problem because my current project is a work tool that emulates an LCD display (pixel perfect, 2D, very few pixels are touched each frame, all 'drawing' can be done with memcpy() to a pixel buffer) and I feel that OpenGL might be too heavy for this, but I could absolutely be wrong in that assumption.
My goal is to borrow as little CPU time as possible. What's the best way to draw pixels to a window, in this limited way, on a modern typical machine running windows 10 circa 2019? Is OpenGL suited for this type of rendering, or should I adopt another rendering method in this case? And if so, what would that method be?
edit:
I should also mention, OpenGL can be used right away for me. If rendering textured triangles with an optimal setup is the fastest method, then I can already do that. Anything that just acts as an API over OpenGL or DirectX will likely be worse in my case.
edit2:
After some research, and thanks to the comments, I think I may just use OpenGL with Pixel Buffer Objects to optimize pixel uploads and keep rendering inexpensive.
If we want to make an application like MS Paint, should we use OpenGL for render graphics?
I want to mention about performance if using traditional GDI vs. OpenGL.
And if there are exist some better libs for this purpose, please see me one.
GDI, X11, OpenGL... are rendering APIs, i.e. you usually don't use them for image manipulation (you can do this, but it requires some precautions).
In a drawing application like MS Paint, if it's pixel based, you'll normally manipulate some picture buffer with customary code, or a special image manipulation library, then send the full buffer to the rendering API.
If your data model consists of strokes and individual shapes, i.e. vector graphics, then OpenGL makes a quite good backend. However it may be worth looking into some other API for vector graphics, like OpenVG (which in its current implementations sits on top of OpenGL, but native implementations operating directly on the GPU may come).
In your usage scenario you'll not run into any performance problems on current computers, so don't choose your API from that criteria. OpenGL is definitely faster than GDI when it comes to texturing, alpha blending, etc. However depending on system and GPU pure GDI may outperform OpenGL for so simple things like drawing an arc or filling a complex self intersecting polygon with complex winding rules.
There is no good reason not to use OpenGL for this. Except maybe if you have years of experience with GDI but don't know a single thing about OpenGL.
On the other hand, OpenGL may very well be superior in many cases. Compositing layers or adjusting hue/saturation/brightness/contrast in a GLSL shader will be several orders of magnitude faster (in fact, pretty much "instantly") if there is a reasonably new card in the computer. Stroking a freedraw path with a "fuzzy" pen (i.e. blending a sprite with alpha transparency over and over again) will be orders of magnitude faster. On images with somewhat reasonable dimensions, most filter kernels should run close to realtime. Rescaling with bilinear filtering runs in hardware.
Such things won't matter on a 512x512 image, as pretty much everything is instantaneous at such resolutions, but on a typical 4096x3072 (or larger) image from your digital camera, it may be very noticeable, especially if you have 4-6 layers.
I just started with OpenGL programming and I am building a clock application. I want it to look something simple like this: http://i.stack.imgur.com/E73ap.jpg
However, my application looks very "un-anti-aliased" : http://i.stack.imgur.com/LUx2v.png
I tried the GL_SMOOTH_POLYGON method mentioned in the Red Book. However that doesn't seem to do a thing.
I am working on a laptop with Intel integrated graphics. The card doesn't support things like GL_ARB_multisample.
What are my options at this point to my app look anti-aliased?
Intel integrated videocards are notorious for their lack of support for OpenGL antialiasing. You can work around that, however.
First option: Manual supersampling
Make a texture 2x times as big as the screen. Render your scene to the texture via FBO, then render the texture at half size so it fills the screen, with bilinear interpolation. Can be very slow (in complex scenes) due to the 4x increase in pixels to draw.
Will result in weak antialiasing (so I don't recommend it for desktop software like your clock). See for yourself:
Second option: (advanced)
Use a shader to perform Morphological Antialiasing. This is a new technique and I don't know how easy it is to implement. It's used by some advanced games.
Third option:
Use textures and bilinear interpolation to your advantage by emulating OpenGL's primitives via textures. The technique is described here.
Fourth option:
Use a separate texture for every element of your clock.
For example, for your hour-arrow, don't use a flat black GL_POLYGON shaped like your arrow. Instead, use a rotated GL_QUAD, textured with a hour-arrow image drawn in an image program. Then bilinear interpolation will take care of antialiasing it as you rotate it.
This option would take the least effort and looks very well.
Fifth option:
Use a library that supports software rendering -
Qt
Cairo
Windows GDI+
WPF
XRender
etc
Such libraries contain their own algorithms for antialiased rendering, so they don't depend on your videocard for antialiasing. The advantages are:
Will render the same on every platform. (this is not guaranteed with OpenGL in various cases - for example, the thick diagonal "tick" lines in your screenshot are rendered as parallelograms, rather than rectangles)
Has a big bunch of convenient drawing functions ("drawArc", "drawText", "drawConcavePolygon", and those will support gradients and borders. also you get things like an Image class.)
Some, like Qt, will provide much more desktop-app type functionality. This can be very useful even for a clock app. For example:
in an OpenGL app you'd probably loop every 20msec and re-render the clock, and not even think twice. This would hog unnecessary CPU cycles, and wake up the CPU on a laptop, depleting the battery. By contrast, Qt is very intelligent about when it must redraw parts of your clock (e.g., when the right half of the clock stops being covered by a window, or when your clock moves the minute-arrow one step).
once you get to implementing, e.g. a tray icon, or a settings dialog, for your clock, a library like Qt can make it a snap. It's nice to use the same library for everything.
The disadvantage is much worse performance, but that doesn't matter at all for a clock app, and it turns around when you take into account the intelligent-redrawing functionality I mentioned.
For something like a clock app, the fifth option is very much recommended. OpenGL is mainly useful for games, 3D software and intense graphical stuff like music visualizers. For desktop apps, it's too low-level and the implementations differ too much.
Draw it into a framebuffer object at twice (or more) the final resolution and then use that image as a texture for a single quad drawn in the actual window.
I want to render font in a way that takes account of the current transforms and similar settings, especially the projection transform and viewport.
I'm thinking that the best way to do that is to have an off screen surface to render the text to, and then render that surface where I really want the text.
However I'm not certain on a number of aspects of this solution.
Is this the best way to go about it at all?
Are there far better free font renderers around that id be better off spending my time with that allow such things. I see alot of people complaining about the d3dx font interfaces for various reasons, but never a link to a better unicode capable renderer...?
Is there any advantage to useing certain surface formats and/or surface sizes (eg always using the smallest possible rather than some standard large one, which requires the extra step of trying to work the size out...)
Yeah, render to texture and then drawing a textured quad to orient and position the text is going to be the easiest way to realize this functionality.
As for D3DX text renderers, it really depends on which SDK you are using. DirectWrite (only for Windows 7 and Vista) will provide a higher quality text rendering approach for applications that need high quality text rendering in a manner that is interoperable with Direct3D.
You can of course do your own rasterization. There are font rasterization engines out there that are open source that could be repurposed for this need, but we're talking tons of coding here for a benefit that may not be distinguishable enough to warrant the development expense.
Having said that, there's a completely new alternative available to you with Direct3D and shaders, provided that you have access to the glyph outlines as curve data. The idea is to use the shader to rasterize the text and store the curve definitions in the vertex stream and associated textures. Try looking at this paper, which describes the technique.