I am learning how to use VBOs and, as the book says,
"...you can free up CPU memory by moving vertex data to the OpenGL
memory on the GPU."
Well, just exactly what can a GPU handle in this regard? Is it acceptable to assume that the "OpenGL memory" can store the vertex data for millions of polygons? What about the GPU in a mobile device?
While developers are used to having a frame of reference for memory restrictions on a CPU, learning OpenGL is partly challenging because I don't know much about GPUs and what to expect from their hardware. So when I read a vague statement like the above, it makes me nervous.
OpenGL has an abstract device and memory model. And technically in the world of OpenGL there is not CPU and GPU memory, but client and server memory. OpenGL buffer objects live on the server side. Server, that simply means everything the OpenGL driver abstracts away. And the OpenGL driver is perfectly allowed to swap out data from the GPU to the CPU if the GPU memory, which acts like a cache, is not sufficient. Hence what your book states:
"...you can free up CPU memory by moving vertex data to the OpenGL memory on the GPU."
Is not entirely correct, as the data in a OpenGL buffer object may very well reside in CPU memory.
There are minimal requirements in spec, but in general, the amount of GPU memory is quite broadly available information which you certainly noticed when buying your PC (overhyped by sellers). However, as #datenwolf said, you can't really know where the data actually is; all that matters is that you can destroy your temporary buffers.
You should take capabilities of the targetted hardware into account regardless of the technology used.
Related
In an Nvidia developer blog: An Even Easier Introduction to CUDA the writer explains:
To compute on the GPU, I need to allocate memory accessible by the
GPU. Unified Memory in CUDA makes this easy by providing a single
memory space accessible by all GPUs and CPUs in your system. To
allocate data in unified memory, call cudaMallocManaged(), which
returns a pointer that you can access from host (CPU) code or device
(GPU) code.
I found this both interesting (since it seems potentially convenient) and confusing:
returns a pointer that you can access from host (CPU) code or device
(GPU) code.
For this to be true, it seems like cudaMallocManaged() must be syncing 2 buffers across VRAM and RAM. Is this the case? Or is my understanding lacking?
In my work so far with GPU acceleration on top of the WebGL abstraction layer via GPU.js, I learned the distinct performance difference between passing VRAM based buffers (textures in WebGL) from kernel to kernel (keeping the buffer on the GPU, highly performant) and retrieving the buffer value outside of the kernels to access it in RAM through JavaScript (pulling the buffer off the GPU, taking a performance hit since buffers in VRAM on the GPU don't magically move to RAM).
Forgive my highly abstracted understanding / description of the topic, since I know most CUDA / C++ devs have a much more granular understanding of the process.
So is cudaMallocManaged() creating synchronized buffers in both RAM
and VRAM for convenience of the developer?
If so, wouldn't doing so come with an unnecessary cost in cases where
we might never need to touch that buffer with the CPU?
Does the compiler perhaps just check if we ever reference that buffer
from CPU and never create the CPU side of the synced buffer if it's
not needed?
Or do I have it all wrong? Are we not even talking VRAM? How does
this work?
So is cudaMallocManaged() creating synchronized buffers in both RAM and VRAM for convenience of the developer?
Yes, more or less. The "synchronization" is referred to in the managed memory model as migration of data. Virtual address carveouts are made for all visible processors, and the data is migrated (i.e. moved to, and provided a physical allocation for) the processor that attempts to access it.
If so, wouldn't doing so come with an unnecessary cost in cases where we might never need to touch that buffer with the CPU?
If you never need to touch the buffer on the CPU, then what will happen is that the VA carveout will be made in the CPU VA space, but no physical allocation will be made for it. When the GPU attempts to actually access the data, it will cause the allocation to "appear" and use up GPU memory. Although there are "costs" to be sure, there is no usage of CPU (physical) memory in this case. Furthermore, once instantiated in GPU memory, there should be no ongoing additional cost for the GPU to access it; it should run at "full" speed. The instantiation/migration process is a complex one, and what I am describing here is what I would consider the "principal" modality or behavior. There are many factors that could affect this.
Does the compiler perhaps just check if we ever reference that buffer from CPU and never create the CPU side of the synced buffer if it's not needed?
No, this is managed by the runtime, not compile time.
Or do I have it all wrong? Are we not even talking VRAM? How does this work?
No you don't have it all wrong. Yes we are talking about VRAM.
The blog you reference barely touches on managed memory, which is a fairly involved subject. There are numerous online resources to learn more about it. You might want to review some of them. here is one. There are good GTC presentations on managed memory, including here. There is also an entire section of the CUDA programming guide covering managed memory.
I can use glTexImage2D or glBufferData to send some data to the gpu memory. Let's assume that I request driver to send more data to the gpu but the gpu memory is already full. I probably get GL_OUT_OF_MEMORY. What might happen with a rendering thread ? What are possible scenarios ? Is it possible that a rendering thread will be terminated ?
It depends on the actual OpenGL implementation. But the most likely scenario is, that you'll just encounter a serious performance drop, but things will just keep working.
OpenGL uses an abstract memory model, for which actual implementation threat the GPU's own memory as a cache. In fact for most OpenGL implementation when you load texture data it doesn't even go directly to the GPU at first. Only when it's actually required for rendering it gets loaded into the GPU RAM. If there are more textures in use than fit into GPU RAM, textures are swapped in and out from GPU RAM as needed to complete the rendering.
Older GPU generations required for a texture to completely fit into their RAM. The GPUs that came out after 2012 actually can access texture subsets from host memory as required thereby lifting that limit. In fact you're sooner running into maximum texture dimension limits, rather than memory limits (BT;DT).
Of course other, less well developed OpenGL implementations may bail out with an out of memory error. But at least for AMD and NVidia that's not an issue.
I'm making a 2D game with OpenGL. Something I'm concerned with is texture memory consumption. 2D games use a few orders of magnitude more texture memory than 3D games, most of that coming from animation frames and backgrounds.
Obviously there's a limit to how much texture memory a program can allocate, but what determines that limit? Does the limit come from available general memory to the program, or is it limited by how much video memory is available on the GPU? Can it use swap space at all? What happens when video memory is exhausted?
OpenGL's memory model is very abstract. Up to version including 2.1 there were two kinds of memory fast "server" memory and slow "client" memory. But there are no limits that could be queried in any way. When a OpenGL implementation runs out of "server" (=GPU) memory it may start swapping or just report "out of memory" errors.
OpenGL-3 did away (mostly, OpenGL-4 finished that job) with the two different kinds of memory. There's just "memory" and the limits are quite arbitrary and depend on the OpenGL implementation (= GPU + driver) the program is running on. All OpenGL implementations are perfectly capable of swapping out textures not used in a while. So the only situation where you would run into a out of memory situation would be the attempt to create a very large texture. The more recent GPUs are in fact capable of swapping in and out parts of textures on a as-needed base. Things will get slow, but keep working.
Obviously there's a limit to how much texture memory a program can allocate, but what determines that limit?
The details of the OpenGL implementation of the system and depending on that the amount of memory installed in that system.
I'm asking this question because I don't want to spend time writing some code that duplicates functionalities of the OpenGL drivers.
Can the OpenGL driver/server hold more data than the video card? Say, I have enough video RAM to hold 10 textures. Can I ask OpenGL to allocate 15 textures without getting an GL_OUT_OF_MEMORY error?
If I can rely on the driver to cleverly send the textures/buffers/objects from the 'normal' RAM to the video RAM when needed then I don't really need to Gen/Delete these objects myself. I become limited by the 'normal' RAM which is often plentiful when compared to the video RAM.
The approach "memory is abundant so I don't need to delete" is bad, and the approach "memory is abundant, so I'll never get out of memory errors" is flawed.
OpenGL memory management is obscure, both for technical reasons (see t.niese's comment above) and for ideological reasons ("you don't need to know, you don't want to know"). Though there exist vendor extensions (such as ATI_meminfo) that let you query some non-authorative numbers (non-authorative insofar as they could change the next millisecond, and they do not take effects like fragmentation into account).
Generally, for the most part, your assumption that you can use more memory than there is GPU memory is correct.
However, you are not usually not able to use all available memory. More likely, there is a limit well below "all available RAM" due to constraints on what memory regions (and how large regions) the driver can allocate, lock, and DMA to/from. And even though you can normally use more memory than will fit on the GPU (even if you used it exclusively), this does not mean careless allocations can't and won't eventually fail.
Usually, but not necessarily, you consume as much system memory as GPU memory, too (without knowing, the driver does that secretly). Since the driver swaps resources in and out as needed, it needs to maintain a copy. Sometimes, it is necessary to keep 2 or 3 copies (e.g. when streaming or for ARB_copy_buffer operations). Sometimes, mapping a buffer object is yet another copy in a specially allocated block, and sometimes you're allowed to write straight into the driver's memory.
On the other hand, PCIe 2.0 (and PCIe 3.0 even more so) is fast enough to stream vertices from main memory, so you do not even strictly need GPU memory (other than a small buffer). Some drivers will stream dynamic geometry right away from system memory.
Some GPUs do not even have separate system and GPU memory (Intel Sandy Bridge or AMD Fusion).
Also, you should note that deleting objects does not necessarily delete them (at least not immediately). Usually, with very few exceptions, deleting an OpenGL object is merely a tentative delete which prevents you from further referencing the object. The driver will keep the object valid for as long as it needs to.
On the other hand, you really should delete what you do not need any more, and you should delete early. For example, you should delete a shader immediately after attaching it to the program object. This ensures that you do not leak resources, and it is guaranteed to work. Deleting and re-specifying the in-use vertex or pixel buffer when streaming (by calling glBufferData(... NULL); is a well-known idiom. This only affects your view of the object, and it allows the driver to continue using the old object in parallel for as long as it needs to.
Some additional information to my comment that did not fit in there.
There are different reasons why this is not part of OpenGL.
It isn't an easy task for the system/driver to guess which resources are and will be required. The driver for sure could create an internal heuristic if resource will be required often or rarely (like CPU does for if statements and doing pre executing code certain code parts on that guess). But the GPU will not know (without knowing the application code) what resource will be required next. It even has no knowledge where the geometry is places in the scene (because you do this with you model and view martix you pass to your shader yourself)
If you e.g. have a game where you can walk through a scene, you normally won't render the parts that are out of the view. So the GPU could think that these resources are not required anymore, but if you turn around then all this textures and geometry is required again and needs to be moved from system memory to gpu memory, which could result in really bad performance. But the Game Engine itself has, because of the use of octrees (or similar techniques) and the possible paths that can be walked, an in deep knowledge about the scene and which resource could be removed from the GPU and which one could be move to the GPU while playing and where it would be necessary to display a loading screen.
If you look at the evolution of OpenGL and which features become deprecated you will see that they go to the direction to remove everything except the really required features that can be done best by the graphic card, driver and system. Everything else is up to the user to implement on it's own to get the best performance. (you e.g. create your projection matrix yourself to pass it to the shader, so OpenGl even does not know where the object is placed in the scene).
Here's my TL;DR answer, I recommend reading Daemon's and t.niese's answers as well:
Can the OpenGL driver/server hold more data than the video card?
Yes
Say, I have enough video RAM to hold 10 textures. Can I ask OpenGL to allocate 15 textures without getting an GL_OUT_OF_MEMORY error?
Yes. Depending on the driver / GPU combination it might even be possible to allocate a single texture that exceeds the GPU's memory, and actually use it for rendering. At my current occupation I exploit that fact to extract slices of arbitrary orientation and geometry from large volumetric datasets, using shaders to apply filters on the voxel data in situ. Works well, but doesn't work for interactive frame rates.
is it possible to learn the allocated memory range of an OpenGL context? Supposedly this memory range should then be accessed with mmap() from another process. Can this technique work, or are there fundamental problems with it?
Update We're using a GNU/Linux system with a modern X11 installation and can pick the video card manufacturer whose drivers support such a trick.
Well, there are innumerable reasons why it won't work.
First, the "allocated memory range of an OpenGL context" is always changing. OpenGL contexts allocate new memory and deallocate it as it decides to.
Second, I would not trust an OpenGL driver to survive under memory mapped conditions like this. Multiple OpenGL contexts can coexist, but only because they all know about each other and the driver can therefore compensate for them. It is highly unlikely that a context can assimilate changes made by another context.
Third, GPUs often work with graphics memory. Even if you can use mmap on GPU memory (which itself is unlikely), you're probably going to lose a lot of performance when you do. And GPU memory gets shuffled around a lot more than CPU memory.
You appear to be trying to do IPC-based graphics. Your best bet would be to have the graphics system be its own process that you communicate with via IPC methods, rather than trying to talk to OpenGL via IPC.
Depends on the OS and driver. It's possible with an X-server. Although a combination of the X server, display driver and openGL means it could move the memory for a particular object around on the card when it draws it.
An easier way is probably to use an openGL pixel/vertex buffer and get the buffer pointer
is it possible to learn the allocated memory range of an OpenGL context?
I think you're asking for accessing the memory where a OpenGL context keeps its objects and the render output.
No. The OpenGL context is an abstract construct and have it's memory on an entirely different machine and/or architecture.
In addition to that there is no standard or even common memory layout for the contents of an OpenGL context. If you're interested in the rendering outcome only, you could tap the framebuffer device (/dev/fb…), though the performance will be inferior to just read back the framebuffer contents with glReadPixels. Similar goes for tapping the PCI memory range, which is virtually the same as tapping the framebuffer device.