In all of the create info structs (vk*CreateInfo) in the new Vulkan API, there is ALWAYS a .sType member. Why is this there if the value can only be one thing? Also the Vulkan specification is very explicit that you can only use vk*CreateInfo structs as parameters for their corresponding vkCreate* function. It seems a little redundant. I can see that if the driver was passing this struct straight to the GPU, you might need to have it (I did notice it is always the first member). But this seems like a really bad idea for the app to do it because if the driver did it, apps would be much less error prone, and prepending an int to a struct doesn't seems like an extremely computational inefficient operation. I just don't see why it exists.
TL;DR
Why do the vk*CreateInfo structs have the .sType member?
They have one so that the pNext field actually works.
Yes, the API takes a struct with a proper C type, so both the caller and the receiver agree on what type that struct is. But especially nowadays, many such structs have linked lists of structures that provide additional information to the implementation. These extension structures (though many are core in Vulkan 1.1/2) are just like all other structures, with their own sType field.
These fields are crucial because the linked lists are built with pNext pointers... which are void*s. They have no set type. The way the implementation determines what a non-NULL pNext pointer points to is by examining the first 4 bytes stored there. This is the sType field; it allows the implementation to know what type to cast the pointer to.
Of course, the primary struct that an API takes doesn't strictly need an sType field, since its type is part of the API itself. However, there is a hypothetical reason to do so (it hasn't panned out in Vulkan releases).
A later version of Vulkan could expand on the creation of, for example, command buffer pools. But how would it do that? Well, they could add a whole new entrypoint: vkCreateCommandPool2. But this function would have almost the exact same signature as vkCreateCommandPool; the only difference is that they take different pCreateInfo structures.
So instead, all you have to do is declare a VkCommandPoolCreateInfo2 structure. And then declare that vkCreateCommandPool can take either one. How would the implementation tell which one you passed in?
Because the first 4 bytes of any such structure is sType. They can test that value. If the value is VK_STRUCTURE_TYPE_COMMAND_POOL_CREATE_INFO, then it's the old structure. If it's VK_STRUCTURE_TYPE_COMMAND_POOL_CREATE_INFO_2, then it's the new one.
Of course, as previously stated, this hasn't panned out; post-1.0 Vulkan versions opted to incorporate extension structs rather than replacing existing ones. But the option is there.
Related
So basically I'm trying to compare two VkPhysicalDeviceFeatures from Vulkan, one from the VkPhysicalDevice I'm looking at, and the other that corresponds to a set of features I actually require. a VkPhysicalDeviceFeatures struct only contains VkBool32 members (which are typedefs of uint32_t), but each minor version of vulkan can add an unknown number of these features. What I would like to do is simply compare the members of each struct against each other, but not for equality, more of a logical comparison. If the corresponding member in the physical device struct is false, but my struct has true for that member, then the comparison should return false.
The only way I can think of doing this is something like what this answer posted:
bool hasRequiredFeatures(VkPhysicalDevice physical_device,
VkPhysicalDeviceFeatures required_features) {
VkPhysicalDeviceFeatures physical_device_features = getSupportedFeatures(physical_device);
std::size_t struct_length = sizeof(VkPhysicalDeviceFeatures) / sizeof(VkBool32);
auto physical_device_features_bool_ptr = reinterpret_cast<VkBool32*>(&physical_device_features);
auto required_features_bool_ptr = reinterpret_cast<VkBool32*>(&required_features);
for(std::size_t i = 0; i < struct_length; ++i){
if(physical_device_features_bool_ptr[i] == VK_FALSE && required_features_bool_ptr[i] == VK_TRUE){
return false;
}
}
return true;
}
This does what I want (though it would be nice to have a way to see which specific member by name failed the comparison, but that I guess that is impossible with out reflection) but I don't think C++ guarantees strict alignment like this? Is there a cross platform way for me to accomplish this?
There's a big problem with your approach that has less to do with C++ and more to do with Vulkan. Specifically:
minor version of vulkan can add an unknown number of these structs
This tells me that you intend to apply such technology to VkPhysicalDeviceFeatures2, as well as any feature struct which can appear in its pNext chain. Hence the "an unknown number of these structs".
Well, here's the thing: VkPhysicalDeviceFeatures2 is not just a bunch of VkBools. It is an extensible Vulkan struct, which means that it starts with the sType and pNext fields common to such structures. So are all of the post-1.0 Vulkan device feature structs.
Code that can do what you've stated with post-1.0 feature structs must be able to take an entire pNext chain of feature structs and test them. And in order to do that, you have to know what they are. There's no way to query, from just a pointer to arbitrary data, that this data contains X number of VkBools.
To make this work, you will need to be able to map an sType value to the size of that structure. And therefore, it cannot be automatically extensible (and no, C++ reflection can't fix that; it cannot know what struct a void *pNext points to); this will require some level of manual upkeep.
Fortunately, the Vulkan XML specification description files clearly specify which structs can exist in a particular pNext chain. So you can write a tool to load the XML, find VkPhysicalDeviceFeatures2, and process all of the structs appearing in its pNext chain (this part is easier said than done, since the XML format was only built to be processed by Khronos's own tools) to find which structs are available, and generate the C++ information you need. I'm sure you can write such a thing relatively easily in any scripting language.
But since you've got the struct definitions in (quasi-reasonable) XML, and you've got this tool that's going to be generating some C++ code anyway... you can just generate the actual comparison logic. That is, instead of hand-writing the member-to-member comparisons, just generate the member-to-member comparisons. You can even get the member names that don't match.
If you're going to process arbitrary pNext-chain features, then you're going to need a generator tool of some kind. And if you're going to need a generator tool anyway, just use it to solve the whole problem.
Now, it's important to realize that the generation code for a hypothetical hasRequiredFeatures implementation is going to have to be semi-complicated. If you're allowing a full pNext chain of structs, then you need to build your own corresponding chain of equivalent structs to use to query from Vulkan. And that's not exactly trivial.
You will need to iterate through the pNext chain and examine the sType field of each struct. But of course, pNext is a void*, so you're going to have to lie/cheat C++'s rules a bit to read the sType field. Basically, you'll have to reinterpret_cast the void* to an VkStructureType*, read the value, compare it against all the possibilities you're working with, and proceed from there. You should skip any sType that you don't know about, which will require doing more C++ trickery.
But you're using a low-level API; breaking C++'s rules is just a thing you'll have to get used to down here.
For each such struct, you need to allocate a matching struct, fill in its sType appropriately, and then add it to the pNext chain you're building.
Once you've built all of these, you can make your Vulkan call, do your comparisons, collect the data, and lastly delete the entire chain of structs.
If your goal really is to stick with just VkPhysicalDeviceFeatures and not the extensible structs, and you just want a C++-portable way to compare such structs, then just memcpy them into a VkBool array and compare the two arrays for mismatches. The two types are trivially copyable, so it's not prima facie illegitmate to do this.
This code has not been compiled or tested.
bool hasRequiredFeatures(VkPhysicalDevice physical_device,
VkPhysicalDeviceFeatures required_features)
{
constexpr auto feature_count = sizeof(VkPhysicalDeviceFeatures) / sizeof(VkBool32);
using FeatureArray = std::array<VkBool, feature_count>;
auto physical_device_features = getSupportedFeatures(physical_device);
FeatureArray required;
memcpy(&required, &required_features, sizeof(FeatureArray));
FeatureArray retrieved;
memcpy(&retrieved, &physical_device_features, sizeof(FeatureArray));
bool did_mismatch = false;
for(auto it_pair = std::mismatch(required.begin(), required.end(), retrieved.begin());
it_pair.first != required.end();
it_pair = std::mismatch(it_pair.first, required.end(), it_pair.second))
{
did_mismatch = true
auto mismatch_index = it_pair.first - required.begin();
//Do something with mismatch_index
}
return did_mismatch;
}
I am taking my first look into the Windows API and upon encountering WNDCLASSX I couldn't help wondering why its member, cbSize, existed. The description of cbSize, per the MSDN is: The size, in bytes, of this structure. Set this member to sizeof(WNDCLASSEX). Be sure to set this member before calling the GetClassInfoEx function. This describes it, and hints at its purpose, but I don't undestand the necessity.
My question is this: Why would a struct ever need to store its own size? Wouldn't any function handling the struct have access to that information using sizeof?
Later versions of the Windows API may add new fields to the struct. If the struct has a size, then older code can call the API function, which only copies the fields that the old code is aware of.
I am trying to write a persistent datastructure in C++ , however I feel that I should be able to make it binary compatible with various other implementations of my datastructure readers, and hence, my current idea is to declare datastructure in the native memory without any abstraction.
For example, I would specify a linear block of memory as a datastructure (using new keyword) and then describe what the first byte means, what the second byte means and so on. I know I can do this using struct but then, the datastructure would be bound to one language and other languages will have to then use this structure. Also, the implementation might then change from compiler to compiler. I would instead like it as a memory standard.
Is what I am trying to do somewhat sensible? Or I am trying to over-simplify things and should really proceed with a struct data structure? Now onto the C++ part, if you believe that I should be using a struct data structure, then what are the disadvantages of using a full-fledged class?
(I am using a class anyway to wrap around the memory structure and provide functions to it since the datastructure is anyway persistent.)
EDIT
As justin as suggested, I do not need any such advanced interface wrapper around the memory structure, so my last point about class wrapper is not stated properly. What I mean is I would like to have a class interface for the memory representation, it does not necessarily have to be a wrapper.
Several file formats I have read/worked with do exactly that -- define a memory standard or layout, then typically back it up with a demonstration in C-like pseudo-structure. Sometimes they will provide struct or class representations, and some are completely abstracted by a library. Of course, these formats go on to document all fields, their sizes, the endianness of the data and so on.
I figure endian related issues, padding, complexity (e.g. introduced by variations in the data structures) and proper versioning are the biggest sources of errors. Another issue I find is the use of data structures of yesteryear and inconsistency of data structures used to represent similar functionalities -- You may receive a spec, and realize it contains several different string representations -- all of which are archaic, and somebody has to go on to support all of these (bidirectionally).
Proceeding that route:
You should not commit to a binary representation (or compilable program) if you don't want to support it (and attempts of long-lived formats fail/stumble along the way, as platforms and toolsets change). Just commit to a formal memory standard at first, then build on top of that with tests and example input files to verify the representation is properly serialized and deserialized correctly. A very basic test suite will help ensure your model is portable on all the systems you need, and can point out potential pitfalls or platform specific considerations you may not have been aware of.
If you really want to provide a compilable representation, I'd stick with a very compliant struct representation -- clients can take that (in memory) representation and turn it into any C++ abstraction/representation they like. That is to say, a serialized representation should probably not reflect that of a representation in memory, apart from trivially simple representations and the intermediate storage of such a representation (flattened and packed structs).
One of the important parts is that you should have tests which confirm your in memory object graph which you create with these structs are forward and backwards serializable and de-serializable, and support proper versioning -- so it often takes a bit of work to make a complex serialized representation compatible. So you see this approach just introduces one abstraction layer on top of another. In this regard, you may want give C++ abstraction the ability to create itself from the packed in memory representation, and to ensure that that representation can also correctly populate the packed structure without data loss.
Beyond that, is there any need to have a more advanced interface? If there is, then you may want to provide that information.
So yes, the memory standard is the part that you must get correct and stable, and to which all implementations should refer to and test against -- regardless of platform/architecture differences. IOW, you're on the right track ;)
In C++ there's no practical difference between struct and class (besides the default accessibility being public in struct). Traditionally, struct is used when a type only has (public) member variables and no member functions but this is only a convention, not a rule enforced by the compiler.
I'd certainly use a struct/class to describe the data. If someone wants to write a reader of your data structure, they can either import your header file or implement the data structure in their language of choice - in most programming languages this should be pretty simple.
I recommend you start your structure something like this:
typedef struct
{
int Version; // struct layout version
int ByteSize; // byte size of structure for validation
...
} MYDATA;
This way when your data structure is being passed around, your code can verify that the allocated structure size matches with how many bytes you'd expect for a given version of your structure. You could then easily introduce new versions of your structure by simply updating the version field and checking for the new size.
When you save your data to disk, make sure that you write it out field-by-field, rather than through a single write (using a pointer and sizeof() to ensure that other languages won't have to deal with potential padding that your C++ compiler may decide to put in. It's possible to manually lay out fields in the structure so that there's no padding but you have to be very, very careful while doing that and it's easy to make mistakes.
I have a problem with detours. Detours, as you all know, can only move among 5 bytes of space (i.e a 'jmp' call and a 4 byte address). Because of this it is impossible to have the 'hook' function in a class (a method), you cannot supply the 'this' pointer because there is simply not enough space (here's the problem more thoroughly explained). So I've been brainstorming all day for a solution, and now I want your thoughts on the subject so I don't begin a 3-5 day project without knowing if it would be possible or not.
I had 3 goals initially, I wanted the 'hook' functions to be class methods, I wanted the whole approach to be object-oriented (no static functions or global objects) and, the worst/hardest part, to be completely dynamic. This is my (in theory) solution; with assembly one can modify functions at runtime (a perfect example is any detouring method). So since I can modify functions dynamically, shouldn't I also be able to create them dynamically? For example; I allocate memory for, let's say ~30 bytes (through malloc/new). Wouldn't it be possible to just replace all bytes with binary numbers corresponding to different assembly operators (like 0xE9 is 'jmp') and then call the address directly (since it would contain a function)?
NOTE: I know on beforehand the return value, and all the arguments to all functions that I want to detour, and since I'm using GCC, the thiscall convention is practically identical to the _cdecl one.
So this is my thought/soon-to-be implementation; I create a 'Function' class. This constructor takes a variadic amount of arguments (except the first argument, which describes the return value of the target function).
Each argument is a description of the arguments the hook will receive (the size, and whether it is a pointer or not). So let's say I want to create a Function class for a int * RandomClass::IntCheckNum(short arg1);. Then I would just have to do like this:Function func(Type(4, true), Type(4, true), Type(2, false));. Where 'Type' is defined as Type(uint size, bool pointer). Then through assembly I could dynamically create the function (note: this would all be using _cdecl calling convention) since I can calculate the number of arguments and total size.
EDIT: With the example, Type(4, true) is the return value (int*), the scondType(4, true) is the RandomClass 'this' pointer and Type(2, false) describes the first argument (short arg1).
With this implementation I could easily have class methods as callbacks, but it would require an extensive amount of assembly code (which I'm not even especially experienced at).
In the end, the only non-dynamic thing would be the methods in my callback class (which also would require pre and post callbacks).
So I wanted to know; is this possible? How much work would it require, and am I way over my head here?
EDIT: I'm sorry if I presented everything a bit fuzzy, but if there is something you want more thoroughly explained, do ask!
EDIT2: I'd also like to know, if I can find the hex values for all assembly operators somewhere? A list would help a ton! And/or if it is possible to somehow 'save' the asm(""); code at a memory address (which I highly doubt).
What you describe is usually called "thunking", and is quite commonly implemented. Historically, the most common purpose has been mapping between 16-bit and 32-bit code (by autogenerating a new 32-bit function that calls an existing 16-bit one or vice versa). I believe some C++ compilers generate similar functions to adjust base class pointers to subclass pointers in multiple inheritance, also.
It certainly seems like a viable solution to your problem, and I don't foresee any huge issues. Just make sure you allocate the memory with any flags needed in your operating system to make sure the memory is executable (most modern OSs give out non-executable memory by default).
You may find this link helpful, particularly if working in Win32: http://www.codeproject.com/Articles/16785/Thunking-in-Win32-Simplifying-Callbacks-to-Non-sta
Regarding finding the hex values of assembly operations, the best reference I know of is the Appendix to the manual of the NASM assembler (and I don't just say that because I helped write it). There's a copy available here: http://www.posix.nl/linuxassembly/nasmdochtml/nasmdoca.html
As discusses in The c++ Programming Language 3rd Edition in section 12.2.5, type fields tend to create code that is less versatile, error-prone, less intuitive, and less maintainable than the equivalent code that uses virtual functions and polymorphism.
As a short example, here is how a type field would be used:
void print(const Shape &s)
{
switch(s.type)
{
case Shape::TRIANGE:
cout << "Triangle" << endl;
case Shape::SQUARE:
cout << "Square" << endl;
default:
cout << "None" << endl;
}
}
Clearly, this is a nightmare as adding a new type of shape to this and a dozen similar functions would be error-prone and taxing.
Despite these shortcomings and those described in TC++PL, are there any examples where such an implementation (using a type field) is a better solution than utilizing the language features of virtual functions? Or should this practice be black listed as pure evil?
Realistic examples would be preferred over contrived ones, but I'd still be interested in contrived examples. Also, have you ever seen this in production code (even though virtual functions would have been easier)?
When you "know" you have a very specific, small, constant set of types, it can be easier to hardcode them like this. Of course, constants aren't and variables don't, so at some point you might have to rewrite the whole thing anyway.
This is, more or less, the technique used for discriminated unions in several of Alexandrescu's articles.
For example, if I was implementing a JSON library, I'd know each Value can only be an Object, Array, String, Integer, Boolean, or Null—the spec doesn't allow any others.
A type enum can be serialized via memcpy, a v-table can't. A similar feature is that corruption of a type enum value is easy to handle, corruption of the v-table pointer means instant badness. There's no portable way to even test a v-table pointer for validity, calling dynamic_cast or typeinfo to do RTTI checks on an invalid object is undefined behavior.
For example, one instance where I choose to use a type hierarchy with static dispatch controlled by a discriminator and not dynamic dispatch is when passing a pointer to a structure through a Windows message queue. This gives me some protection against other software that may have allocated broadcast messages from a range I'm using (it's supposed to be reserved for app-local messages, do not pass GO if you think that rule is actually respected).
The following guideline is from Clean Code by Robert C. Martin.
"My general rule for switch statements is that they can be tolerated if they appear only once, are used to create polymorphic objects, and are hidden behind an inheritance relationship so that the rest of the system can't see them".
The rationale is this: if you expose type fields to the rest of your code, you'll get multiple instances of the above switch statement. That is a clear violation of DRY. When you add a type, all these switches need to change (or, even worse, they become inconsistent without breaking your build).
My take is: It depends.
A parameterized Factory Method design pattern relies on this technique.
class Creator {
public:
virtual Product* Create(ProductId);
};
Product* Creator::Create (ProductId id) {
if (id == MINE) return new MyProduct;
if (id == YOURS) return new YourProduct;
// repeat for remaining products...
return 0;
}
So, is this bad. I don't think so as we do not have any other alternative at this stage. This is a place where it is absolutely necessary as it involves creation of an object. The type of the object is yet to be known.
The example in OP is however an example which sure needs refactoring. Here we are already dealing with an existing object/type (passed as argument to function).
As Herb Sutter mentions -
"Switch off: Avoid switching on the
type of an object to customize
behavior. Use templates and virtual
functions to let types (not their
calling code) decide their behavior."
Aren't there costs associated to virtual functions and polymorphism? Like maintaining a vtable per class, increase of each class object size by 4 byptes, runtime slowness (I have never measured it though) for resolving the virtual function appropriately. So, for simple situations, using a type field appears acceptable.
I think that if the type corresponds precisely to the implied classes then type is wrong. Where it gets complicated is where the type does not quite match or its not so cut and dried.
Taking your example what if type was Red, Green, Blue. Those are types of shapes. You could even have a color class as a mixin; but its probably too much.
I am thinking of using a type field to solve the problem of vector slicing. That is, I want a vector of hierarchical objects. For example I want my vector to be a vector of shapes, but I want to store circles, rectangles, triangles etc.
You can't do that in the most obvious simple way because of slicing. So the normal solution is to have a vector of pointers or smart pointers instead. But I think there are cases where using a type field will be a simpler solution, (avoids new/delete or alternative lifecycle techniques).
The best example I can think of (and the one I've run into before), is when your set of types is fixed and the set of functions you want to do (that depend on those types) is fluid. That way, when you add a new function, you modify a single place (adding a single switch) rather than adding a new base virtual function with the real implementation scattered all across the classes in your type hierarchy.
I don't know of any realistic examples. Contrived ones would depend on there being some good reason that virtual methods cannot be used.