I need to craft a packet that has a header, a trailer, and a variable length payload field. So far I have been using a vector for the payload so my struct is set up like this:
struct a_struct{
hdr a_hdr;
vector<unsigned int> a_vector;
tr a_tr;
};
When I try to access members of the vector I get a seg fault and a sizeof of an entire structs give me 32 (after I've added about 100 elements to the vector.
Is this a good approach? What is better?
I found this post
Variable Sized Struct C++
He was using a char array, and I'm using a vector though.
Even though the vector type is inlined in the struct, the only member that is in the vector is likely a pointer. Adding members to the vector won't increase the size of the vector type itself but the memory that it points to. That's why you won't ever see the size of the struct increase in memory and hence you get a seg fault.
Usually when people want to make a variable sized struct, they do so by adding an array as the last member of the struct and setting it's length to 1. They then will allocate extra memory for the structure that is actually required by sizeof() in order to "expand" the structure. This is almost always accompanied by an extra member in the struct detailing the size of the expanded array.
The reason for using 1 is thoroughly documented on Raymond's blog
http://blogs.msdn.com/oldnewthing/archive/2004/08/26/220873.aspx
The solution in the other SO answer is c-specific, and relies on the peculiarities of c arrays - and even in c, sizeof() won't help you find the "true" size of a variable size struct. Essentially, it's cheating, and it's a kind of cheating that isn't necessary in C++.
What you are doing is fine. To avoid seg faults, access the vector as you would any other vector in C++:
a_struct a;
for(int i = 0; i < 100; ++i) a.a_vector.push_back(i);
cout << a.a_vector[22] << endl; // Prints 22
i saw this implementation in boost..it looks really neat...to have a variable
length payload....
class msg_hdr_t
{
public:
std::size_t len; // Message length
unsigned int priority;// Message priority
//!Returns the data buffer associated with this this message
void * data(){ return this+1; } //
};
this may be totally un-related to the question, but i wanted to share the info
Related
I guess I'm still not understanding the limitations of C++ containers and arrays. According to this post and this It is impossible to store items of dynamic size in an STL vector.
However with the following code I can dynamically re-size an element of a vector with the results one would expect if it was ok to have items of varying and changing size in a vector.
string test = "TEST";
vector<string> studentsV;
for (int i = 0; i < 5; ++i)
{
studentsV.push_back(test);
}
studentsV[2].resize(100);
for (string s : studentsV)
{
cout << s << "end" << endl;
}
Result:
TESTend
TESTend
TEST
end
TESTend
TESTend
I can re-size the string element to any size, and it works fine. I can also do the same with a regular C-style array. So, what is the difference between the above posts and what I am doing, and can you give an example of what "dynamic item size" really means, because apparently I am not understanding.
A std::string uses dynamic memory to increase the size of the string being stored. This is not what those articles are talking about.
What they mean, is that sizeof(std::string) is constant. The actual object representing a std::string will always have the same size, but it might do additional allocations in another part of memory.
A std::vector is really just a friendly wrapper around a dynamically-sized array. The definition of an array in C or C++ is a contiguous block of memory where all elements are of equal size.
can you give an example of what "dynamic item size" really means, because apparently I am not understanding.
This is the core of your question.
Namely: if all C++ classes (even ones that manage dynamic memory as part of their implementations) have a fixed and known footprint size via sizeof()...just what sort of thing is it that you can't put in a std::vector?
Since something like a std::string and a std::bitset are classes of different sizes, you couldn't have a vector of [string string bitset string bitset string]. But the type system already wouldn't let you do that. So that can't be what they're talking about.
They're just saying there's no hook for supporting structures like this from the C world:
struct packetheader {
int id;
int filename_len;
};
struct packet {
struct packetheader h;
char filename[1];
};
You couldn't make a std::vector<packet> and expect to find some parameter to push_back letting you specify a per-item size. You'd lose any data you'd allocated outside of the structure boundary.
So to use something like that, you'd have to do std::vector<packet*> and store pointers.
The size of std::string is not dynamic. std::string is probably implemented with a pointer to a dynamically allocated memory. This makes sizeof(std::string) static and possibly different from the size of the actual string.
I need to use a function (part of an API) which stores some requested data into a dynamic sized struct using call by reference. The struct is defined as follows - it concerns access control lists of either posix or NFS4 version, but that is just the use case, I guess.
typedef struct my_struct
{
unsigned int len; /* total length of the struct in bytes */
... /* some other static sized fields */
unsigned int version; /* 2 different versions are possible */
unsigned int amount; /* number of entries that follow */
union {
entry_v1_t entry_v1[1];
entry_v2_t entry_v2[1];
};
} my_struct_t;
There are 2 versions of the entries and I know which one I will obtain (v1). Both entry_v1_t and entry_v2_t are fixed (but different) sized structs just containing integers (so I guess they are not worth being explained here). Now I need to use an existing function to fill my structure with the information I need using Call by Reference, the signature is as follows, including the comments - I don't have access to the implementation:
int get_information(char *pathname, void *ptr);
/* The ptr parameter must point to a buffer mapped by the my_struct
* structure. The first four bytes of the buffer must contain its total size.
*/
So the point is, that I must allocate memory for that struct but don't know for how much entries (and, as consequence, the total size) I must allocate. Have you ever dealt with such a situation?
Under Windows API there are many such functions, you normally call them with some NULL pointer to get size of the buffer, then call again with allocated buffer. In case during next call size of buffer have changed function returns error and you need allocate again. So you do it in a while loop till function returns with success.
So your get_information must implement somehow such mechanisms, either it returns false if buffer is to small or returns its correct size if ptr is NULL. But that is just my guess.
OK I thing I figured out how it works. Thanks for your ideas and notes. I declared a my_struct pointer and allocated minimum space for the fixed sized fields (5) before the dynamic array => 5 * sizeof(unsigned int). Invoking get_information with that pointer returns -1 and sets errno = 28 and strerror(errno) = "No space left on device".
But, it sets the my_struct->len field to the required size and that seems to be the answer to my question - how should you know? No I can invoke get_information initially with the minimum space and figure out how much I need to allocate, and afterwards call it again with the right sized memory allocated to get the information successfully.
The loop solution seems to make sense anyway and would have been my next try - since there are usually just a few entries in that dynamic array.
Thank you.
Okay so you have and array A[]... that is passed to you in some function say with the following function prototype:
void foo(int A[]);
Okay, as you know it's kind of hard to find the size of that array without knowing some sort of ending variable or knowing the size already...
Well here is the deal though. I have seem some people figure it out on a challenge problem, and I don't understand how they did it. I wasn't able to see their source code of course, that is why I am here asking.
Does anyone know how it would even be remotely possible to find the size of that array?? Maybe something like what the free() function does in C??
What do you think of this??
template<typename E, int size>
int ArrLength(E(&)[size]){return size;}
void main()
{
int arr[17];
int sizeofArray = ArrLength(arr);
}
The signature of that function is not that of a function taking an array, but rather a pointer to int. You cannot obtain the size of the array within the function, and will have to pass it as an extra argument to the function.
If you are allowed to change the signature of the function there are different alternatives:
C/C++ (simple):
void f( int *data, int size ); // function
f( array, sizeof array/sizeof array[0] ); // caller code
C++:
template <int N>
void f( int (&array)[N] ); // Inside f, size N embedded in type
f( array ); // caller code
C++ (though a dispatch):
template <int N>
void f( int (&array)[N] ) { // Dispatcher
f( array, N );
}
void f( int *array, int size ); // Actual function, as per option 1
f( array ); // Compiler processes the type as per 2
You cannot do that. Either you have a convention to signal the end of the array (e.g. that it is made of non-zero integers followed by a 0), or you transmit the size of the array (usually as an additional argument).
If you use the Boehm garbage collector (which has a lot of benefit, in particular you allocate with GC_malloc and friends but you don't care about free-ing memory explicitly), you could use the GC_size function to give you the size of a GC_malloc-ed memory zone, but standard malloc don't have this feature.
You're asking what we think of the following code:
template<typename E, int size>
int ArrLength(E(&)[size]){return size;}
void main()
{
int arr[17];
int sizeofArray = ArrLength(arr);
}
Well, void main has never been standard, neither in C nor in C++.
It's int main.
Regarding the ArrLength function, a proper implementation does not work for local types in C++98. It does work for local types by C++11 rules. But in C++11 you can write just end(a) - begin(a).
The implementation you show is not proper: it should absolutely not have int template argument. Make that a ptrdiff_t. For example, in 64-bit Windows the type int is still 32-bit.
Finally, as general advice:
Use std::vector and std::array.
One relevant benefit of this approach is that it avoid throwing away the size information, i.e. it avoids creating the problem you're asking about. There are also many other advantages. So, try it.
The first element could be a count, or the last element could be a sentinel. That's about all I can think of that could work portably.
In new code, for container-agnostic code prefer passing two iterators (or pointers in C) as a much better solution than just passing a raw array. For container-specific code use the C++ containers like vector.
No you can't. Your prototype is equivalent to
void foo(int * A);
there is obviously no size information. Also implementation dependent tricks can't help:
the array variable can be allocated on the stack or be static, so there is no information provided by malloc or friends
if allocated on the heap, a user of that function is not forced to call it with the first element of an allocation.
e.g the following are valid
int B[22];
foo(B);
int * A = new int[33];
foo(A + 25);
This is something that I would not suggest doing, however if you know the address of the beginning of the array and the address of the next variable/structure defined, you could subtract the address. Probably not a good idea though.
Probably an array allocated at compile time has information on its size in the debug information of the executable. Moreover one could search in the code for all the address corresponding to compile time allocated variables and assume the size of the array is minus the difference between its starting address and the next closest starting address of any variable.
For a dinamically allocated variable it should be possible to get its size from the heap data structures.
It is hacky and system dependant, but it is still a possible solution.
One estimate is as follows: if you have for instance an array of ints but know that they are between (stupid example) 0..80000, the first array element that's either negative or larger than 80000 is potentially right past the end of the array.
This can sometimes work because the memory right past the end of the array (I'm assuming it was dynamically allocated) won't have been initialized by the program (and thus might contain garbage values), but might still be part of the allocated pages, depending on the size of the array. In other cases it will crash or fail to provide meaningful output.
All of the other answers are probably better, i.e. you either have to pass the length of the array or terminate it with a special byte sequence.
The following method is not portable, but it works for me in VS2005:
int getSizeOfArray( int* ptr )
{
int size = 0;
void* ptrToStruct = ptr;
long adr = (long)ptrToStruct;
adr = adr - 0x10;
void* ptrToSize = (void*)adr;
size = *(int*)ptrToSize;
size /= sizeof(int);
return size;
}
This is entirely dependent of the memory model of your compiler and system so, again, it is not portable. I bet there are equivalent methods for other platforms. I would never use this in a production environment, merely stating this as an alternative.
You can use this: int n = sizeof(A) / sizeof(A[0]);
I have a struc like this:
struct process {int PID;int myMemory[];};
however, when I try to use it
process p;
int memory[2];
p.myMemory = memory;
I get an criptic error from eclipse saying int[0] is not compatible with int[2];
what am i doing wrong?
Thanks!
Don't use static arrays, malloc, or even new if you're using C++. Use std::vector which will ensure correct memory management.
#include <vector>
struct Process {
int pid;
std::vector<int> myMemory;
};
Process p;
p.reserve(2); // allocates enough space on the heap to store 2 ints
p.myMemory.push_back( 4815 ); // add an index-zero element of 4815
p.myMemory.push_back( 162342 ); // add an index-one element of 162342
I might also suggest creating a constructor so that pid does not initially have an undefined value:
struct Process {
Process() : pid(-1), myMemory() {
}
int pid;
std::vector<int> myMemory;
};
I think you should declare myMemory as an int* then malloc() when you know the size of it. After this it can be used like a normal array. Int[0] seems to mean "array with no dimension specified".
EXAMPLE:
int *a; // suppose you'd like to have an array with user specified length
// get dimension (int d)
a = (int *) malloc(d * sizeof(int));
// now you can forget a is a pointer:
a[0] = 5;
a[2] = 1;
free((void *) a); // don't forget this!
All these answers about vector or whatever are confused :) using a dynamically allocated pointer opens up a memory management problem, using vector opens up a performance problem as well as making the data type a non-POD and also preventing memcpy() working.
The right answer is to use
Array<int,2>
where Array is a template the C++ committee didn't bother to put in C++99 but which is in C++0x (although I'm not sure of the name). This is an inline (no memory management or performance issues) first class array which is a wrapper around a C array. I guess Boost has something already.
In C++, array definition is almost equal to pointer constants, meaning that their address cannot be changed, while the values which they point to can be changed. That said, you cannot copy elements of an array into another by the assignment operator. You have to go through the arrays and copy the elements one by one and check for the boundary conditions yourself.
The syntax ...
struct process {int PID;int myMemory[];};
... is not valid C++, but it may be accepted by some compilers as a language extension. In particular, as I recall g++ accepts it. It's in support for the C "struct hack", which is unnecessary in C++.
In C++, if you want a variable length array in a struct, use std::vector or some other array-like class, like
#include <vector>
struct Process
{
int pid;
std::vector<int> memory;
};
By the way, it's a good idea to reserve use of UPPERCASE IDENTIFIERS for macros, so as to reduce the probability of name collisions with macros, and not make people reading the code deaf (it's shouting).
Cheers & hth.,
You cannot make the array (defined using []) to point to another array. Because the array identifier is a const pointer. You can change the value pointed by the pointer but you cannot change the pointer itself. Think of "int array[]" as "int* const array".
The only time you can do that is during initialization.
// OK
int array[] = {1, 2, 3};
// NOT OK
int array[];
array = [1, 2, 3]; // this is no good.
int x[] is normally understood as int * x.
In this case, it is not, so if you want a vector of integers of an undetermined number of positions, change your declaration to:
struct process {int PID;int * myMemory;};
You should change your initialization to:
int memory[2];
p.myMemory = new int[ 10 ];
I have the following struct:
struct Datastore_T
{
Partition_Datastores_T cmtDatastores; // bytes 0 to 499
Partition_Datastores_T cdhDatastores; // bytes 500 to 999
Partition_Datastores_T gncDatastores; // bytes 1000 to 1499
Partition_Datastores_T inpDatastores; // bytes 1500 1999
Partition_Datastores_T outDatastores; // bytes 2000 to 2499
Partition_Datastores_T tmlDatastores; // bytes 2500 to 2999
Partition_Datastores_T sm_Datastores; // bytes 3000 to 3499
};
I want to set a char* to point to a struct of this type like so:
struct Datastore_T datastores;
// Elided: datastores is initialized with data here
char* DatastoreStartAddr = (char*)&datastores;
memset(DatastoreStartAddr, 0, 3500);
The problem I have is that DatastoreStartAddr always has a value of zero when it should point to the struct that has been initialized with data.
What am I doing wrong?
Edit: What I mean by zero is that the "values" in the structure are all zeros even after I initialize the structure. The address is not zero, it is the values in the struct that are zero.
Edit: I think I am asking the question wrong. Let's start over. If I have a struct that is initialized with data, and another object maintains a field member that is a pointer to that struct, if the struct is changed directly:
struct Datastore_T datastores;
char* DatastoreStartAddr = (char*)&datastores;
datastores.cmtDatastores.u16Region[0] = Scheduler.GetMinorFrameCount(); // byte 40,41
datastores.cmtDatastores.u16Region[1] = Scheduler.GetMajorFrameCount(); // byte 42,43
Shouldn't I be able to access these changes using the DatastoreStartAddr pointer?
EDIT: The following code tries to read the data set in datastores, but using the pointer to the struct:
CMT_UINT8_Tdef PayLoadBuffer[1500]= {NULL};
int TDIS = 0;
int DIS = 0;
int DSA = 0;
//copy DataStore info using address and size offsets
if ((PayLoadBuffer + TDIS + DIS) < IndvDEMMax)
{
memcpy((PayLoadBuffer + TDIS), Datastores+DSA, DIS);
TDIS += DIS;
}
In the memcpy((PayLoadBuffer + TDIS), Datastores+DSA, DIS) line, Datastores should point to structure and attempts to access an offset in that structure. But since the value is always zero, it copies zero in the PayLoadBuffer.
I don't know why you are getting an address of zero, but I would guess the code you don't show has something to do with it. Some other points:
Consider using an array of Partition_Datastores_T inside your struct
Do not use magic numbers for struct sizes, you want sizeof(Datastore_T )
There is no need for the intermediate char*
Edit: Bobby, to answer your supplementary question - yes you should be able to access it through a pointer, but not through a char * (without jumping through some hoops). You want:
struct Datastore_T datastores;
struct Datastore_T * DatastoreStartAddr = &datastores;
and when you use that pointer:
DatastoreStartAddr->cmtDatastores.u16Region[0] = Scheduler.GetMinorFrameCount();
Please note the use of the -> operator.
i just tested your code and it is not zero. Try to post bigger piece
You are doing it in the wrong order - should be like this
struct Datastore_T datastores;
char* DatastoreStartAddr = (char*)&datastores;
memset(DatastoreStartAddr, 0, sizeof(Datastore_T));
// Elided: datastores is initialized with data here
Now datastores is still initialized. And like everyone else says you might want this instead
struct Datastore_T * DatastoreStartAddr = datastores;
memset((void *)DatastoreStartAddr, 0, sizeof(Datastore_T));
I'm assuming that the code you are not showing is correct. It might be wise to show it to us for scrutiny. Likely, what I say below is not the problem at all.
Because of the cast, it might be that you're having aliasing issues here. If you have set aggressive compiler optimization flags (e.g. -fno-strict-aliasing on gcc), the compiler would assume that those two pointers can never refer to the same thing, because they have different types. Then either or both of the representations might be cached in different CPU registers, so updates to one would never be reflected in the other.
Again, this is a long shot. Considering the size of your struct (I didn't see that when I started answering your duplicate question), it is very unlikely that it would reside anywhere else but in main memory. But you could try turn down your compiler optimizations and see if it makes a difference.
At what point is your structure's values zero? If it's before the cast and memset(), the problem is with your initialization. If it is after the cast and memset(), then the values in your structure are zero because memset() overwrote with 0's the values you had initialized it with. The values in datastores should also be zero after the memset().