Is there a way to get the right values for a VkMemoryRequirements structure without having to allocate a buffer first and without using vkGetBufferMemoryRequirements?
Is it supported/compliant?
Motivation - Short version
I have an application that does the following, and everything works as expected.
VkMemoryRequirements memReq;
vkGetBufferMemoryRequirements(application.shell->device, uniformBuffer, &memReq);
int memType = application.shell->findMemoryType(memReq.memoryTypeBits, VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT | VK_MEMORY_PROPERTY_HOST_COHERENT_BIT);
Internally, findMemoryType loops over the memory types and checks that they have the required property flags.
If I replace the call to vkGetMemoryRequirements with hardcoded values (which are not portable, specific to my system and obtained through debugging), everything still works and I don't get any validation errors.
VkMemoryRequirements memReq = { 768, 256, 1665 };
int memType = application.shell->findMemoryType(memReq.memoryTypeBits, VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT | VK_MEMORY_PROPERTY_HOST_COHERENT_BIT);
The above code is IMHO neat because enables to pre-allocate memory before you actually need it.
Motivation - Long version
In Vulkan you create buffers which initially are not backed by device memory and at a later stage you allocate the memory and bind it to the buffer using vkBindBufferMemory:
VkResult vkBindBufferMemory(
VkDevice device,
VkBuffer buffer,
VkDeviceMemory memory,
VkDeviceSize memoryOffset);
Its Vulkan spec states that:
memory must have been allocated using one of the memory types allowed
in the memoryTypeBits member of the VkMemoryRequirements structure
returned from a call to vkGetBufferMemoryRequirements with buffer
Which implies that before allocating the memory for a buffer, you should have already created the buffer.
I have a feeling that in some circumstances it would be useful to pre-allocate a chunk of memory before you actually need it; in most of the OpenGL flavors I have experience with this was not possible, but Vulkan should not suffer from this limitation, right?
Is there a (more or less automatic) way to get the memory requirements before creating the first buffer?
Is it supported/compliant?
Obviously, when you do allocate the memory for the first buffer you can allocate a little more so that when you need a second buffer you can bind it to another range in the same chunk. But my understanding is that, to comply with the spec, you would still need to call vkGetBufferMemoryRequirements on the second buffer, even if it is exactly the same type and the same size as the first one.
This question already recognizes that the answer is "no"; you just seem to want to do an end-run around what you already know. Which you can't.
The code you showed with the hard-coded values works because you already know the answer. It's not that Vulkan requires you to ask the question; Vulkan requires you to provide buffers that use the answer.
However, since "the answer" is implementation-specific, it changes depending on the hardware. It could change when you install a new driver. Indeed, it could change even depending on which extensions or Vulkan features you activate when creating the VkDevice.
That having been said:
Which implies that before allocating the memory for a buffer, you should have already created the buffer.
Incorrect. It requires that you have the answer and have selected memory and byte offsets appropriate to that answer. But Vulkan is specifically loose about what "the answer" actually means.
Vulkan has specific guarantees in place which allow you to know the answer for a particular buffer/image without necessarily having asked about that specific VkBuffer/Image object. The details are kind of complicated, but for buffers they are pretty lax.
The basic idea is that you can create a test VkBuffer/Image and ask about its memory properties. You can then use that answer to know what the properties of the buffers you intend to use which are "similar" to that. At the very least, Vulkan guarantees that two identical buffer/images (formats, usage flags, sizes, etc) will always produce identical memory properties.
But Vulkan also offers a few other guarantees. There are basically 3 things that the memory properties tell you:
The memory types that this object can be bound to.
The alignment requirement for the offset for the memory object.
The byte size the object will take up in memory.
For the size, you get only the most basic guarantee: equivalent buffer/images will produce equivalent sizes.
For the alignment, images are as strict as sizes: only equivalent images are guaranteed to produce equivalent alignment. But for buffers, things are more lax. If the test buffer differs only in usage flags, and the final buffer uses a subset of the usage flags, then the alignment for the final buffer will not be more restrictive than the test buffer. So you can use the alignment from the test buffer.
For the memory types, things are even more loose. For images, the only things that matter are:
Tiling
Certain memory flags (sparse/split-instance binding)
Whether the image format is color or depth/stencil
If the image format is depth/stencil, then the formats must match
External memory
Transient allocation usage
If all of these are the same for two VkImage objects, then the standard guarantees that all such images will support the same set of memory types.
For buffers, things are even more lax. For non-sparse buffers, if your test buffer differs from the final buffer only by usage flags, then if the final one has a subset of the usage flags of the test buffer, then the set of memory types it supports must include all of the ones from the test buffer. The final buffer could support more, but it must support at least those of such a test buffer.
Oh, and linear images and buffers must always be able to be used in at least one mappable, coherent memory type. Of course, this requires that you have created a valid VkDevice/Image with those usage and flags fields, so if the device doesn't allow (for example) linear images to be used as textures, then that gets stopped well before asking about memory properties.
Related
A recommended approach for memory management in Vulkan is sub-allocation of buffers, for instance see the image below.
I'm trying to implement "the good" approach. I have a system in place that can tell me where within a Memory Allocation is available, so I can bind a sub area of a single large buffer.
However, I can't find the mechanism to do this, or am just misunderstanding what is happening, as the bind functions take a buffer as input, and an offset. I can't see how to specify the size of the binding other than through the existing buffer.
So I have a few questions I guess:
are the dotted rectangles in the image below just bindings, or are they additional buffers?
if they are bindings, how do I tell Vulkan (ideally using VMA) to use that subsection of the buffer?
if they are additional buffers, how do I create them?
if neither, what are they?
I have read up on a few custom allocators, but they seem to follow the "bad" approach, returning offsets into large allocations for binding, so still plenty of buffers but lower allocation counts.
To be clear, I am not using custom allocator callbacks other than through VMA; the "system" to which I refer above sits on top of the VMA calls.
Any pointers much appreciated!
are the dotted rectangles in the image below just bindings, or are they additional buffers?
They represent the actual data. So the "Index" block is the range of storage that contains vertex indices.
if they are bindings, how do I tell Vulkan (ideally using VMA) to use that subsection of the buffer?
That depends on the particular nature of how you're using that VkBuffer as a resource. Generally speaking, every function that uses a VkBuffer as a resource takes a byte offset that represents where to start reading from. Many such functions also take a size which coupled with the offset represents the full quantity of data that can be read through that particular resource.
For example, vkCmdBindVertexBuffers takes an array of VkBuffers, and for each VkBuffer it also takes a byte offset that represents the starting point for that vertex buffer. VkDescriptorBufferInfo, the structure that represents a buffer used by a descriptor, takes a VkBuffer, a byte offset, and a size.
The vertex buffer (and index buffer) bindings don't have a size, but they don't need one. Their effective size is defined by the rendering command used with them (and the index data being read by it). If you render using 100 32-bit indices, then the expectation is that the index buffer's size, minus the starting offset, should be at least 400 bytes. And if it isn't, UB results.
I know this function create a "buffer." But what exactly is a buffer? Is it a COM object in memory? If it is, then in my understanding, this function takes in a descriptor and some initial data to create this COM object in memory, and then set the ID3D11Buffer pointer pointed by the input ID3D11Buffer** to the interface in the newly created COM object. Once the COM object is created, the initializing data is not needed any more and we can delete them. And once we call ID3DBuffer::Release(), the underline COM object will be destroyed. Is my understanding correct?
CreateBuffer returns a COM interface object ID3D11Buffer*. As long as it has a non-zero reference count (it starts at 1; each call to AddRef adds 1, each call to Release subtracts 1) then whatever resources it controls are active.
As to where exactly the resources are allocated, it really depends. You may find this article interesting as it covers different ways Direct3D allocates resources.
UPDATE: You should also read this Microsoft Docs introduction to the subset of COM used by DirectX.
In the general case, a buffer is a continuous, managed, area of memory.
Memory is a large set of addresses of read/writable elements (one element per address, of course), say 230 addresses of elements of 8-bit makes a 1GiB memory.
If there is only a single program and it uses these addresses statically (e.g. addresses from 0x1000 to 0x2000 are used to store the list of items) then memory doesn't need to be managed and in this context a buffer is just a continuous range of addresses.
However, if there are multiple programs or a program memory usage is dynamic (e.g. it depends on how many items it's been asked to read from input) then memory must be managed.
You must keep track of which ranges are already in use and which are not. So a buffer becomes a continuous range of addresses with their attributes (e.g. if it's in use or not).
The attributes of a buffer can vary a lot between the different memory allocators, in general, we say that a buffer is managed because we let the memory allocator handle it: find a suitable free range, mark it used, tell it if it can move the buffer aftward, mark it free when where are finished.
This is true for every memory that is shared, so it is certainly true for the main memory (RAM) and the graphic memory.
This is the memory inside the graphic card, that is accessed just like the main memory (from the CPU point of view).
What CreateBuffer return is a COM object in the main memory that contains the metadata necessary to handle the buffer just allocated.
It doesn't contain the buffer itself because this COM object is always in memory while the buffer usually is not (it is in the graphic memory).
CreateBuffer asks the graphic driver to find a suitable range of free addresses, in the memory asked, and fill in some metadata.
Before the CPU can access the main memory it is necessary to set up some metadata tables (the page tables) as part of its protection mechanism.
This is also true if the CPU needs to access the graphic memory (with possibly a few extra steps, for managing the MMIO if necessary).
The GPU also has page tables, so if the main memory has to be accessed by the GPU these page tables must also have to be created.
You see that it's important to know how the buffer will be used.
Another thing to consider is that the GPUs use highly optimized memory format - for example, the buffer used for a surface can be pictured as a rectangular area of memory.
The same is true for the buffer used by a texture.
However the twos are stored differently: the surface is stored linearly, each row after another, while the texture buffer is tiled (it's like it's made of many, say, 16x16 surfaces stored linearly one after the other).
This makes sampling and filtering faster.
Also, some GPU may need to have texture images on a specific area of memory, vertex buffer in another and so on.
So it's important to give the graphic driver all the information it needs to make the best choice when allocating a buffer.
Once the buffer has been found, the driver (or the D3D runtime) will initialize the buffer if requested.
It can do this by copying the data or by aliasing through the page tables (if the pitch allows for it) and eventually using some form of Copy-On-Write.
However it does that, the source data are not needed anymore (see this).
The COM object returned by CreateBuffer is a convenient proxy, then it is disposed of, thanks to the usual come AddRef/Release mechanism, it also asks the graphic driver to deallocate the buffer.
Can I use a VBO which I initialise like this:
GLuint bufferID;
glGenBuffers(1,&BufferID);
glBindBuffer(GL_ARRAY_BUFFER,bufferID);
glBufferData(GL_ARRAY_BUFFER,nBytes,indexData,GL_DYNAMIC_DRAW);
as an index buffer, like this:
glBindBuffer(GL_ELEMENT_ARRAY_BUFFER,bufferID);
/* ... set up vertex attributes, NOT using bufferID in the process ... */
glDrawElements(...);
I would like to use the buffer mostly as an attribute buffer and occasionally as an index buffer (but never at the same time).
There is nothing in the GL which prevents you from doing such things, your code above is legal GL. You can bind every buffer to every buffer binding target (you can even bind the same buffer to different targets at the same time, so it is even OK if attributes and index data come from the same buffer). However, the GL implementation might do some optimizations based on the observed behavior of the application, so you might end up with sub-optimal performance if you suddenly change the usage of an existing buffer object with such an approach, or use it for two things at once.
Update
The ARB_vertex_buffer_object extension spec, which introduced the concept of buffer objects to OpenGL, mentions this topic in the "Issues" section:
Should this extension include support for allowing vertex indices to be stored in buffer objects?
RESOLVED: YES. It is easily and cleanly added with just the
addition of a binding point for the index buffer object. Since
our approach of overloading pointers works for any pointer in GL,
no additional APIs need be defined, unlike in the various
*_element_array extensions.
Note that it is expected that implementations may have different
memory type requirements for efficient storage of indices and
vertices. For example, some systems may prefer indices in AGP
memory and vertices in video memory, or vice versa; or, on
systems where DMA of index data is not supported, index data must
be stored in (cacheable) system memory for acceptable
performance. As a result, applications are strongly urged to
put their models' vertex and index data in separate buffers, to
assist drivers in choosing the most efficient locations.
The reasoning that some implementations might prefer to keep index buffers in system RAM seems quite outdated, though.
While completely legal, it's sometimes discouraged to have attribute data and index data in the same buffer. I suspect that this is mostly based on a paragraph in the spec document (e.g. page 49 of the OpenGL 3.3 spec, at the end of the section "2.9.7 Array Indices in Buffer Objects"):
In some cases performance will be optimized by storing indices and array data in separate buffer objects, and by creating those buffer objects with the corresponding binding points.
While it seems plausible that it could be harmful to performance, I would be very interested to see benchmark results on actual platforms showing it. Attribute data and index data are used at the same time, and with the same access operations (CPU write, or blit from temporary storage, for filling the buffer with data, GPU read during rendering). So I can't think of a very good reason why they would need to be treated differently.
The only difference I can think of is that the index data is always read sequentially, while the attribute data is read out of order during indexed rendering. So it might be possible to apply different caching attributes for performance tuning the access in both cases.
Suppose I map a buffer, with
map_ptr = glMapBuffer (..) (The target shouldn't matter, but let's say its GL_TEXTURE_BUFFER)
Next I upload texture data with:
glTexImage2D(..., map_ptr), passing map_ptr as my texture data. (I don't have a GL_PIXEL_UNPACK_BUFFER bound)
Semantically, this involves copying the data from the buffer's data store to the texture object's data store, and the operation can be accomplished with a GPU DMA copy.
But what actually happens? Is the data copied entirely on the GPU, or does the CPU read and cache the mapped memory, and then write back to GPU at a separate GPU memory location? I.e. is the copy asynchronous, or does the CPU synchronously coordinate the copy, utilizing CPU cycles?
Is the answer to that implementation dependent? Does it depend on whether the OpenGL driver is intelligent enough to recognize the data pointer passed to glTexImage2D a GPU memory mapped pointer, and that a round-trip to the CPU is unnecessary? If so, how common is this feature in prevalent drivers today?
Also, what about the behaviour for an OpenCL buffer whose memory was mapped, i.e:
map_ptr = clEnqueueMapBuffer(..) (OpenCL buffer mapped memory)
and map_ptr was passed to glTexImage2D?
What you do there is simply undefined behavior as per the spec.
Pointer values returned by MapBufferRange may not be passed as parameter
values to GL commands. For example, they may not be used to specify array
pointers, or to specify or query pixel or texture image data; such
actions produce undefined results, although implementations may not
check for such behavior for performance reasons.
Let me quote from the GL_ARB_vertex_buffer_object extension spec, which originally introduced buffer objects and mapping operations (emphasis mine):
Are any GL commands disallowed when at least one buffer object is mapped?
RESOLVED: NO. In general, applications may use whatever GL
commands they wish when a buffer is mapped. However, several
other restrictions on the application do apply: the
application must not attempt to source data out of, or sink
data into, a currently mapped buffer. Furthermore, the
application may not use the pointer returned by Map as an
argument to a GL command.
(Note that this last restriction is unlikely to be enforced in
practice, but it violates reasonable expectations about how
the extension should be used, and it doesn't seem to be a very
interesting usage model anyhow. Maps are for the user, not
for the GL.)
I have a need to stream a texture (essentially a camera feed).
With object streaming, the following scenarios seem to be arise:
Is the new object's data store larger, smaller or same size as the old one?
Subset of or whole texture being updated?
Are we streaming a buffer object or texture object (any difference?)
Here are the following approaches I have come across:
Allocate object data store (either BufferData for buffers or TexImage2D for textures) and then each frame, update subset of data with BufferSubData or TexSubImage2D
Nullify/invalidate the object after the last call (eg. draw) that uses the object either with:
Nullify: glTexSubImage2D( ..., NULL), glBufferSubData( ..., NULL)
Invalidate: glBufferInvalidate(), glMapBufferRange with the GL_MAP_INVALIDATE_BUFFER_BIT, glDeleteTextures ?
Simpliy reinvoke BufferData or TexImage2D with the new data
Manually implement object multi-buffering / buffer ping-ponging.
Most immediately, my problem scenario is: entire texture being replaced with new one of same size. How do I implement this? Will (1) implicitly synchronize ? Does (2) avoid the synchronization? Will (3) synchronize or will a new data store for the object be allocated, where our update can be uploaded without waiting for all drawing using the old object state to finish? This passage from the Red Book V4.3 makes be believe so:
Data can also be copied between buffer objects using the
glCopyBufferSubData() function. Rather than assembling chunks of data
in one large buffer object using glBufferSubData(), it is possible to
upload the data into separate buffers using glBufferData() and then
copy from those buffers into the larger buffer using
glCopyBufferSubData(). Depending on the OpenGL implementation, it may
be able to overlap these copies because each time you call
glBufferData() on a buffer object, it invalidates whatever contents
may have been there before. Therefore, OpenGL can sometimes just
allocate a whole new data store for your data, even though a copy
operation from the previous store has not completed yet. It will then
release the old storage at a later opportunity.
But if so, why the need for (2)[nullify/invalidates]?
Also, please discuss the above approaches, and others, and their effectiveness for the various scenarios, while keeping in mind atleast the following issues:
Whether implicit synchronization to object (ie. synchronizing our update with OpenGL's usage) occurs
Memory usage
Speed
I've read http://www.opengl.org/wiki/Buffer_Object_Streaming but it doesn't offer conclusive information.
Let me try to answer at least a few of the questions you raised.
The scenarios you talk about can have a great impact on the performance on the different approaches, especially when considering the first point about the dynamic size of the buffer. In your scenario of video streaming, the size will rarely change, so a more expensive "re-configuration" of the data structures you use might be possible. If the size changes every frame or every few frames, this is typically not feasable. However, if a resonable maximum size limit can be enforced, just using buffers/textures with the maximum size might be a good strategy. Neither with buffers nor with textures you have to use all the space there is (although there are some smaller issues when you do this with texures, like wrap modes).
3.Are we streaming a buffer object or texture object (any difference?)
Well, the only way to efficiently stream image data to or from the GL is to use pixel buffer objects (PBOs). So you always have to deal with buffer objects in the first place, no matter if vertex data, image data or whatever data is to be tranfered. The buffer is just the source for some glTex*Image() call in the texture case, and of course you'll need a texture object for that.
Let's come to your approaches:
In approach (1), you use the "Sub" variant of the update commands. In that case, (parts of or the whole) storage of the existing object is updated. This is likely to trigger an implicit synchronziation ifold data is still in use. The GL has basically only two options: wait for all operations (potentially) depending on that data to complete, or make an intermediate copy of the new data and let the client go on. Both options are not good from a performance point of view.
In approach (2), you have some misconception. The "Sub" variants of the update commands will never invalidate/orphan your buffers. The "non-sub" glBufferData() will create a completely new storage for the object, and using it with NULL as data pointer will leave that storage unintialized. Internally, the GL implementation might re-use some memory which was in use for earlier buffer storage. So if you do this scheme, there is some probablity that you effectively end up using a ring-buffer of the same memory areas if you always use the same buffer size.
The other methods for invalidation you mentiond allow you to also invalidate parts of the buffer and also a more fine-grained control of what is happening.
Approach (3) is basically the same as (2) with the glBufferData() oprhaning, but you just specify the new data directly at this stage.
Approach (4) is the one I actually would recommend, as it is the one which gives the application the most control over what is happening, without having to relies on the GL implementation's specific internal workings.
Without taking synchronization into account, the "sub" variant of the update commands is
more efficient, even if the whole data storage is to be changed, not just some part. That is because the "non-sub" variants of the commands basically recreate the storage and introduce some overhead with this. With manually managing the ring buffers, you can avoid any of that overhead, and you don't have to rely in the GL to be clever, by just using the "sub" variants of the updates functions. At the same time, you can avoid implicit synchroniztion by only updating buffers which aren't in use by th GL any more. This scheme can also nicely be extenden into a multi-threaded scenario. You can have one (or several) extra threads with separate (but shared) GL contexts to fill the buffers for you, and just passing the buffer handlings to the draw thread as soon as the update is complete. You can also just map the buffers in the draw thread and let the be filled by worker threads (wihtout the need for additional GL contexts at all).
OpenGL 4.4 introduced GL_ARB_buffer_storage and with it came the GL_MAP_PERSISTEN_BIT for glMapBufferRange. That will allow you to keep all of the buffers mapped while they are used by the GL - so it allows you to avoid the overhead of mapping the buffers into the address space again and again. You then will have no implicit synchronzation at all - but you have to synchronize the operations manually. OpenGL's synchronization objects (see GL_ARB_sync) might help you with that, but the main burden on synchronization is on your applications logic itself. When streaming videos to the GL, just avoid re-using the buffer which was the source for the glTexSubImage() call immediately and try to delay its re-use as long as possible. You are of course also trading throughput for latency. If you need to minimize latency, you might to have to tweak this logic a bit.
Comparing the approaches for "memory usage" is really hard. There are a lot of of implementation specific details to consider here. A GL implementation might keep some old buffer memories around for some time to fullfill recreation requests of the same size. Also, an GL implementation might make shadow copies of any data at any time. The approaches which don't orphan and recreate storages all the time in principle expose more control of the memory which is in use.
"Speed" itself is also not a very useful metric. You basically have to balance throughput and latency here, according to the requirements of your application.