I need some help with OpenGL-Vulkan Memory Exchange.
I've already found this topic
How to render to OpenGL from Vulkan?
But it is not quite what I need. I dont want Vulkan to allocate and export memory. I want to import OpenGL memory to Vulkan, create Vulkan Image and bind it to the imported memory.
My question is, is it actually possible to get 'HANDLE' (in terms of WinOS) that can be used with vk::ImportMemoryWin32HandleInfoKHR.
As far as I can tell, there is no OpenGL extension yet, which would allow to do this. It also kind of makes sense, since memory semantics of OpenGL allocated objects are very vague and the data may actually be all over the place. When you create a texture, buffer, etc. in OpenGL it's completely open, when, how and where the thing will get its memory allocated eventually.
This is very different in Vulkan, where memory management is explicit, and once created you have "perfect" knowledge about it. Which is, why it's possible to simply "import" that memory into an OpenGL object; as far as the OpenGL driver goes, it's just another way to get to the memory, only that this way around it doesn't have to concern itself with the dirty details.
In the end it doesn't make a practical difference if you allocate the memory with Vulkan or OpenGL. Just allocate with Vulkan, then import into OpenGL. You can still write to the memory from OpenGL, i.e. also use it as a renderbuffer or texture for framebuffer attachment.
Related
The OpenGL tradition is to let the user manipulate OpenGL objects using an unsigned int handle. Why not just give a pointer instead? What are the advantages of unique IDs over pointers?
TL;DR: OpenGL IDs don't map bijectively to memory locations. A single OpenGL ID may refer to multiple memory locations at the same time. Also OpenGL has been designed to work for distributed rendering architectures (like X11) as well, and given an indirect context programs running on different machines may use the same OpenGL context.
OpenGL has been designed as an architecture and display system agnostic API. When OpenGL was first developed this happened in light of client-server display architectures (like X11). If you look into the OpenGL specification, even of modern OpenGL-4 it refers to clients and servers.
However in a client/server architectures pointers make no sense. For one the address space of the server is not accessible to the clients without jumping some hoops. And even if you set up a shared memory mapping, the addresses of objects are not the same for client and server. Add to this that on architectures like X11 a single indirect OpenGL context can be used by multiple clients, that may even run on different machines. Pointers simply don't work for that.
Last but not least the OpenGL object model is highly abstract and the OpenGL drawing model is asynchonous Say I do the following:
id = glGenTextures(1)
glBindTexture(id)
glTexStorage(…)
glTexSubImage(image_a)
draw_something()
glTexSubImage(image_b)
draw_someting_b()
When the end of this little snippet has reached, actually nothing at all may have been drawn yet, because no synchronization point has been reached (glFinish, glReadPixels, a buffer swap). Note the two calls to glTexSubImage, which happen on the same id. When the pixels are finally put to the framebuffer, there two different images to be sourced from a single texture ID, because OpenGL guarantees you, that things will appear as if things were drawn synchronously. So at the end of a drawing batch a single object ID may refer to a whole collection of different data sets with different locations in memory.
My first consideration - having pointers would make programmers wonder if they can operate with them in a pointer-arithmetic way, e.g. by pointing to a middle of a texture to update it or something like that. Maybe even more crazy things, such as patching shaders code on-the-fly. That all sounds like a whole new cool degree of freedom, unless you think of additional complications caused by tampering with highly efficient and optimized GPU "black-box" way of operation.
For example - consider inner workings of GPU memory allocation. Just like with OS - pointers you get from OS are not the real "physical" ones, OS memory manager can move things around behind the scenes while keeping the pointers the same (f.e. swapping to HDD). In that case IDs are just the same - GPU can optimize and pack entities with even more freedom, while keeping the nice facade of them being available at 1-2-3.
Another example - OpenGL is not actually the same across manufacturers. In fact OpenGL is just a description of API, where each vendor can make his own implementation the way it works best for him. For example there's no rule on hot to store texture mipmaps, aligned, or interleaved or whatever. Having pointers to a texture would lure developers into tampering with mipmaps, which would cause a lot of trouble to support various implementations or force all the implementations to become strictly unified, which again is a bad idea for performance.
The OpenGL device (GPU) may have its own memory with its own address space, independent of the host (CPU) memory system. (Think of a discrete video card with its own onboard RAM.) The host can't (directly) access that memory, so it's not possible to have a pointer to it.
It's best to think of the GPU as a whole separate computer; it's actually possible to do OpenGL over a network, with a program running on one computer rendering graphics on the video card in another. When you set up your textures and buffers, you're basically uploading data to the GL device for its own internal use.
I'm asking this question because I don't want to spend time writing some code that duplicates functionalities of the OpenGL drivers.
Can the OpenGL driver/server hold more data than the video card? Say, I have enough video RAM to hold 10 textures. Can I ask OpenGL to allocate 15 textures without getting an GL_OUT_OF_MEMORY error?
If I can rely on the driver to cleverly send the textures/buffers/objects from the 'normal' RAM to the video RAM when needed then I don't really need to Gen/Delete these objects myself. I become limited by the 'normal' RAM which is often plentiful when compared to the video RAM.
The approach "memory is abundant so I don't need to delete" is bad, and the approach "memory is abundant, so I'll never get out of memory errors" is flawed.
OpenGL memory management is obscure, both for technical reasons (see t.niese's comment above) and for ideological reasons ("you don't need to know, you don't want to know"). Though there exist vendor extensions (such as ATI_meminfo) that let you query some non-authorative numbers (non-authorative insofar as they could change the next millisecond, and they do not take effects like fragmentation into account).
Generally, for the most part, your assumption that you can use more memory than there is GPU memory is correct.
However, you are not usually not able to use all available memory. More likely, there is a limit well below "all available RAM" due to constraints on what memory regions (and how large regions) the driver can allocate, lock, and DMA to/from. And even though you can normally use more memory than will fit on the GPU (even if you used it exclusively), this does not mean careless allocations can't and won't eventually fail.
Usually, but not necessarily, you consume as much system memory as GPU memory, too (without knowing, the driver does that secretly). Since the driver swaps resources in and out as needed, it needs to maintain a copy. Sometimes, it is necessary to keep 2 or 3 copies (e.g. when streaming or for ARB_copy_buffer operations). Sometimes, mapping a buffer object is yet another copy in a specially allocated block, and sometimes you're allowed to write straight into the driver's memory.
On the other hand, PCIe 2.0 (and PCIe 3.0 even more so) is fast enough to stream vertices from main memory, so you do not even strictly need GPU memory (other than a small buffer). Some drivers will stream dynamic geometry right away from system memory.
Some GPUs do not even have separate system and GPU memory (Intel Sandy Bridge or AMD Fusion).
Also, you should note that deleting objects does not necessarily delete them (at least not immediately). Usually, with very few exceptions, deleting an OpenGL object is merely a tentative delete which prevents you from further referencing the object. The driver will keep the object valid for as long as it needs to.
On the other hand, you really should delete what you do not need any more, and you should delete early. For example, you should delete a shader immediately after attaching it to the program object. This ensures that you do not leak resources, and it is guaranteed to work. Deleting and re-specifying the in-use vertex or pixel buffer when streaming (by calling glBufferData(... NULL); is a well-known idiom. This only affects your view of the object, and it allows the driver to continue using the old object in parallel for as long as it needs to.
Some additional information to my comment that did not fit in there.
There are different reasons why this is not part of OpenGL.
It isn't an easy task for the system/driver to guess which resources are and will be required. The driver for sure could create an internal heuristic if resource will be required often or rarely (like CPU does for if statements and doing pre executing code certain code parts on that guess). But the GPU will not know (without knowing the application code) what resource will be required next. It even has no knowledge where the geometry is places in the scene (because you do this with you model and view martix you pass to your shader yourself)
If you e.g. have a game where you can walk through a scene, you normally won't render the parts that are out of the view. So the GPU could think that these resources are not required anymore, but if you turn around then all this textures and geometry is required again and needs to be moved from system memory to gpu memory, which could result in really bad performance. But the Game Engine itself has, because of the use of octrees (or similar techniques) and the possible paths that can be walked, an in deep knowledge about the scene and which resource could be removed from the GPU and which one could be move to the GPU while playing and where it would be necessary to display a loading screen.
If you look at the evolution of OpenGL and which features become deprecated you will see that they go to the direction to remove everything except the really required features that can be done best by the graphic card, driver and system. Everything else is up to the user to implement on it's own to get the best performance. (you e.g. create your projection matrix yourself to pass it to the shader, so OpenGl even does not know where the object is placed in the scene).
Here's my TL;DR answer, I recommend reading Daemon's and t.niese's answers as well:
Can the OpenGL driver/server hold more data than the video card?
Yes
Say, I have enough video RAM to hold 10 textures. Can I ask OpenGL to allocate 15 textures without getting an GL_OUT_OF_MEMORY error?
Yes. Depending on the driver / GPU combination it might even be possible to allocate a single texture that exceeds the GPU's memory, and actually use it for rendering. At my current occupation I exploit that fact to extract slices of arbitrary orientation and geometry from large volumetric datasets, using shaders to apply filters on the voxel data in situ. Works well, but doesn't work for interactive frame rates.
So far I know about immediate, display list, vertex buffer and vertex buffer object rendering. Which is the fastest? Which OpenGL version does each require? What should I use?
The best (and pretty much only) method of rendering now is to use general purpose buffers, AKA Vertex Buffer Objects. They are in core from 2.1, if I'm correct, but generally appeared as an extension in 1.5 (as ARB_vertex_buffer_object). They have hardware support, which means they can be and probably will be stored directly in GPU memory.
When you load data to them, you specify the suggested usage. You can read more about it in glBufferData manual. For example, GL_STATIC_DRAW is something very similar to static display list. This allows your graphics card to optimize access to them.
Modern (read: non-ancient) hardware really dislikes immediate mode. I've seen a nearly 2-order-of-magnitude performance improvement by replacing immediate mode with vertex arrays.
OpenGL 3 and above support only buffer objects, all other rendering modes are deprecated.
Display lists are a serious pain to use correctly, and not worth it on non-ancient hardware.
To summarize: if you use OpenGL 3+, you have to use (V)BOs. If you target OpenGL 2, use vertex arrays or VBOs as appropriate.
is it possible to learn the allocated memory range of an OpenGL context? Supposedly this memory range should then be accessed with mmap() from another process. Can this technique work, or are there fundamental problems with it?
Update We're using a GNU/Linux system with a modern X11 installation and can pick the video card manufacturer whose drivers support such a trick.
Well, there are innumerable reasons why it won't work.
First, the "allocated memory range of an OpenGL context" is always changing. OpenGL contexts allocate new memory and deallocate it as it decides to.
Second, I would not trust an OpenGL driver to survive under memory mapped conditions like this. Multiple OpenGL contexts can coexist, but only because they all know about each other and the driver can therefore compensate for them. It is highly unlikely that a context can assimilate changes made by another context.
Third, GPUs often work with graphics memory. Even if you can use mmap on GPU memory (which itself is unlikely), you're probably going to lose a lot of performance when you do. And GPU memory gets shuffled around a lot more than CPU memory.
You appear to be trying to do IPC-based graphics. Your best bet would be to have the graphics system be its own process that you communicate with via IPC methods, rather than trying to talk to OpenGL via IPC.
Depends on the OS and driver. It's possible with an X-server. Although a combination of the X server, display driver and openGL means it could move the memory for a particular object around on the card when it draws it.
An easier way is probably to use an openGL pixel/vertex buffer and get the buffer pointer
is it possible to learn the allocated memory range of an OpenGL context?
I think you're asking for accessing the memory where a OpenGL context keeps its objects and the render output.
No. The OpenGL context is an abstract construct and have it's memory on an entirely different machine and/or architecture.
In addition to that there is no standard or even common memory layout for the contents of an OpenGL context. If you're interested in the rendering outcome only, you could tap the framebuffer device (/dev/fb…), though the performance will be inferior to just read back the framebuffer contents with glReadPixels. Similar goes for tapping the PCI memory range, which is virtually the same as tapping the framebuffer device.
How to get total memory in bytes used by OpenGL in C++?
I'm building an OpenGL application and the total memory used seems to be rising, I can get the info about the total memory used by variables & objects created by myself but can't guarantee how much memory OpenGL is using for its variables & objects & textures, etc. So is it possible to get the total memory in bytes used by OpenGL in C++?
In general, you don't. OpenGL is ultimately a hardware abstraction. And OpenGL simply doesn't provide a way to get that sort of information.
There are vendor-specific extensions that will give you ways to ask, though what you get back depends on the architecture. AMD hardware provides the ATI_meminfo extension. It breaks memory down into types of objects: buffer objects, textures, and renderbuffers.
NVIDIA provides the experimental extension NVX_gpu_memory_info. There's no information in the registry about how to use it, so I can't link you to anything.
In any case, the most effective way to know what the GPU is using is to just keep track of it yourself. Always use internal image formats with sizes; this means you can compute a pretty good estimate of how much memory a texture takes up. The same goes for buffer objects and so forth.
You won't get exact numbers, as padding, alignment, and the like can confound you. But you'll get something pretty decent.