Aligning memory address of specific pragma packed struct member? - c++

I have a struct with data members with types of different lengths (some ints, some words, some just single bytes). The struct is packed (using #pragma pack(1)).
My question is: Is it possible to have one of the struct's data members memory address be aligned in memory (for example, have the address be divisable by 16)?
I know it is possible to align the whole struct (or in other words, its first member) using attribute. I want to know if it's possible to align a specific member and not just the first one.
Basically I need the struct to be packed (every member follows the previous one without padding) and I need that the address of one of the members, not the first one, be divisable by 16.

There doesn't seem to be any possibility to align structures with an offset. You must create the offset (padding) by yourself.
typedef struct __attribute__((packed)) special_align {
char misc[3];
int align_me;
};
typedef struct __attribute__((packed, aligned(16))) pad_container {
char padding[13]; //16-3
struct special_align data;
};
Of course accessing the struct will become more cumbersome.
If you don't want to calculate by hand how much padding you need, you can use the offsetof() macro. In the example offsetof(struct special_align,align_me) will return 3, that is to be substracted from the align size. (of course if the offset is greater than align, you must use a corresponding pad size multiplier. `
pad_size = (sizeof(special_align) * align_size - offsetof(special_align,align_me)) % align_size;

#pragma pack(1) is nasty -- once it is set for a given structure, it will apply to every member of that structure. None of alignas, __attribute__((packed, aligned(8))), std::aligned_storage, std::aligned_union, #pragma pack() constructs inside that structure will affect the alignment, the original packing will be maintained for the whole structure (an intermediate #pragma pack() will apply to any inner structure that follows, but the outer structure will be packed with its original packing).
So the only solution is to pack your structure manually, by adding padding bytes where necessary.
I'm using something like this, it triggers a compiler error and also shows what the (incorrect) alignment is:
template<int s> struct OffsetIs;
template<int N>
struct CheckAlignment {
static constexpr size_t ok = sizeof(OffsetIs<N>);
};
template<>
struct CheckAlignment<0> {
static constexpr bool ok = true;
};
#pragma pack(1)
struct MyStruct
{
char a;
//char pad[3]; // uncomment to make it compile
long x;
};
#pragma pack()
static_assert(CheckAlignment<offsetof(MyStruct, x) % 4>::ok, "Alignment of x is wrong");

Related

Why is this struct not the size I expect?

I am taking binary input from a file to a buffer vector then casting the pointer of that buffer to be my struct type.
The goal is for the data to populate the struct perfectly.
I know the size of all the various fields and the order they're going to come in.
As a result my struct needs to be tightly packed and be 42 bytes long.
My issue is that it is coming out at 44 bytes long when I test it.
Also, the first value lines up. After that, the data is incorrect.
Here's the struct:
#pragma pack(push, 1)
struct myStruct
{
uint8_t ID;
uint32_t size: 24;
uint16_t value;
char name[12];
char description[4];
char shoppingList[14];
char otherValue[6];
};
#pragma pack(pop)
Also, the first value lines up. After that, the data is incorrect.
uint32_t size: 24;
If you want to guarantee portably that this is three bytes with no padding before the next member, you're going to need to use a byte buffer and do the conversions yourself.
#pragma pack is an extension, and the packing of bitfield members is anyway implementation-defined.
FWIW both GCC and CLANG do seem to do what you want in this case, but unless it's defined by a platform ABI depending on this is still brittle.

cache alignment of a struct with declspec(align())

I know a little bit what is cache alignment and what will happen when we used it.
#define CACHE_LINE 32
#define CACHE_ALIGN __declspec(align(CACHE_LINE))
struct CACHE_ALIGN S1 { // cache align all instances of S1
int a, b, c, d;
};
struct s1;
for example, in the above example, s1 is 32-byte cache aligned but I can't get why the following code defined like this:
struct __declspec(align(CACHE_ALIGN)) CUSTINFO {
// Force each structure to be in a different cache line.
DWORD m_customer_id; // Mostly read-only
wchar_t m_name[100]; // Mostly read-only
// Force the following members to be in a different cache line.
__declspec(align(CACHE_ALIGN))
int m_balance; // Read-write
FILETIME m_last_order_date; // Read-write
};
Why two times __declspec(align(CACHE_ALIGN)) is used in the struct? Why should even align our structure? can someone explain these two code snippets in detail?

How do I create a "spacer" in a C++ class memory structure?

The issue
In a low level bare-metal embedded context, I would like to create a blank space in the memory, within a C++ structure and without any name, to forbid the user to access such memory location.
Right now, I have achieved it by putting an ugly uint32_t :96; bitfield which will conveniently take the place of three words, but it will raise a warning from GCC (Bitfield too large to fit in uint32_t), which is pretty legitimate.
While it works fine, it is not very clean when you want to distribute a library with several hundreds of those warnings...
How do I do that properly?
Why is there an issue in the first place?
The project I'm working on consists of defining the memory structure of different peripherals of a whole microcontroller line (STMicroelectronics STM32). To do so, the result is a class which contains a union of several structures which define all registers, depending on the targeted microcontroller.
One simple example for a pretty simple peripheral is the following: a General Purpose Input/Output (GPIO)
union
{
struct
{
GPIO_MAP0_MODER;
GPIO_MAP0_OTYPER;
GPIO_MAP0_OSPEEDR;
GPIO_MAP0_PUPDR;
GPIO_MAP0_IDR;
GPIO_MAP0_ODR;
GPIO_MAP0_BSRR;
GPIO_MAP0_LCKR;
GPIO_MAP0_AFR;
GPIO_MAP0_BRR;
GPIO_MAP0_ASCR;
};
struct
{
GPIO_MAP1_CRL;
GPIO_MAP1_CRH;
GPIO_MAP1_IDR;
GPIO_MAP1_ODR;
GPIO_MAP1_BSRR;
GPIO_MAP1_BRR;
GPIO_MAP1_LCKR;
uint32_t :32;
GPIO_MAP1_AFRL;
GPIO_MAP1_AFRH;
uint32_t :64;
};
struct
{
uint32_t :192;
GPIO_MAP2_BSRRL;
GPIO_MAP2_BSRRH;
uint32_t :160;
};
};
Where all GPIO_MAPx_YYY is a macro, defined either as uint32_t :32 or the register type (a dedicated structure).
Here you see the uint32_t :192; which works well, but it triggers a warning.
What I've considered so far:
I might have replaced it by several uint32_t :32; (6 here), but I have some extreme cases where I have uint32_t :1344; (42) (among others). So I would rather not add about one hundred lines on top of 8k others, even though the structure generation is scripted.
The exact warning message is something like:
width of 'sool::ll::GPIO::<anonymous union>::<anonymous struct>::<anonymous>' exceeds its type (I just love how shady it is).
I would rather not solve this by simply removing the warning, but the use of
#pragma GCC diagnostic push
#pragma GCC diagnostic ignored "-WTheRightFlag"
/* My code */
#pragma GCC diagnostic pop
may be a solution... if I find TheRightFlag. However, as pointed out in this thread, gcc/cp/class.c with this sad code part:
warning_at (DECL_SOURCE_LOCATION (field), 0,
"width of %qD exceeds its type", field);
Which tells us that there is no -Wxxx flag to remove this warning...
How about a C++-ish way?
namespace GPIO {
static volatile uint32_t &MAP0_MODER = *reinterpret_cast<uint32_t*>(0x4000);
static volatile uint32_t &MAP0_OTYPER = *reinterpret_cast<uint32_t*>(0x4004);
}
int main() {
GPIO::MAP0_MODER = 42;
}
You get autocompletion because of the GPIO namespace, and there is no need for dummy padding. Even, it is more clear what's going on, as you can see the address of each register, you don't have to rely on the compiler's padding behavior at all.
Use multiple adjacent anonymous bitfields. So instead of:
uint32_t :160;
for example, you'd have:
uint32_t :32;
uint32_t :32;
uint32_t :32;
uint32_t :32;
uint32_t :32;
One for each register you want to be anonymous.
If you have large spaces to fill it may be clearer and less error prone to use macros to repeat the single 32 bit space. For example, given:
#define REPEAT_2(a) a a
#define REPEAT_4(a) REPEAT_2(a) REPEAT_2(a)
#define REPEAT_8(a) REPEAT_4(a) REPEAT_4(a)
#define REPEAT_16(a) REPEAT_8(a) REPEAT_8(a)
#define REPEAT_32(a) REPEAT_16(a) REPEAT_16(a)
Then a 1344 (42 * 32 bit) space can be added thus:
struct
{
...
REPEAT_32(uint32_t :32;)
REPEAT_8(uint32_t :32;)
REPEAT_2(uint32_t :32;)
...
};
In the embedded systems arena, you can model hardware either by using a structure or by defining pointers to the register addresses.
Modeling by structure is not recommended because the compiler is allowed to add padding between members for alignment purposes (although many compilers for embedded systems have a pragma for packing the structure).
Example:
uint16_t * const UART1 = (uint16_t *)(0x40000);
const unsigned int UART_STATUS_OFFSET = 1U;
const unsigned int UART_TRANSMIT_REGISTER = 2U;
uint16_t * const UART1_STATUS_REGISTER = (UART1 + UART_STATUS_OFFSET);
uint16_t * const UART1_TRANSMIT_REGISTER = (UART1 + UART_TRANSMIT_REGISTER);
You could also use the array notation:
uint16_t status = UART1[UART_STATUS_OFFSET];
If you must use the structure, IMHO, the best method to skip addresses would be to define a member and not access it:
struct UART1
{
uint16_t status;
uint16_t reserved1; // Transmit register
uint16_t receive_register;
};
In one of our projects we have both constants and structs from different vendors (vendor 1 uses constants while vendor 2 uses structures).
geza's right that you really don't want to be using classes for this.
But, if you were to insist, the best way to add an unused member of n bytes' width, is simply to do so:
char unused[n];
If you add an implementation-specific pragma to prevent the addition of arbitrary padding to the class's members, this can work.
For GNU C/C++ (gcc, clang, and others that support the same extensions), one of the valid places to put the attribute is:
#include <stddef.h>
#include <stdint.h>
#include <assert.h> // for C11 static_assert, so this is valid C as well as C++
struct __attribute__((packed)) GPIO {
volatile uint32_t a;
char unused[3];
volatile uint32_t b;
};
static_assert(offsetof(struct GPIO, b) == 7, "wrong GPIO struct layout");
(example on the Godbolt compiler explorer showing offsetof(GPIO, b) = 7 bytes.)
To expand on #Clifford's and #Adam Kotwasinski's answers:
#define REP10(a) a a a a a a a a a a
#define REP1034(a) REP10(REP10(REP10(a))) REP10(a a a) a a a a
struct foo {
int before;
REP1034(unsigned int :32;)
int after;
};
int main(void){
struct foo bar;
return 0;
}
To expand on Clifford's answer, you can always macro out the anonymous bitfields.
So instead of
uint32_t :160;
use
#define EMPTY_32_1 \
uint32_t :32
#define EMPTY_32_2 \
uint32_t :32; \ // I guess this also can be replaced with uint64_t :64
uint32_t :32
#define EMPTY_32_3 \
uint32_t :32; \
uint32_t :32; \
uint32_t :32
#define EMPTY_UINT32(N) EMPTY_32_ ## N
And then use it like
struct A {
EMPTY_UINT32(3);
/* which resolves to EMPTY_32_3, which then resolves to real declarations */
}
Unfortunately, you'll need as many EMPTY_32_X variants as many bytes you have :(
Still, it allows you to have single declarations in your struct.
To define a large spacer as groups of 32 bits.
#define M_32(x) M_2(M_16(x))
#define M_16(x) M_2(M_8(x))
#define M_8(x) M_2(M_4(x))
#define M_4(x) M_2(M_2(x))
#define M_2(x) x x
#define SPACER int : 32;
struct {
M_32(SPACER) M_8(SPACER) M_4(SPACER)
};
I think it would be beneficial to introduce some more structure; which may, in turn, solve the issue of spacers.
Name the variants
While flat namespaces are nice, the issue is that you end up with a motley collection of fields and no simple way of passing all related fields together. Furthermore, by using anonymous structs in an anonymous union you cannot pass references to the structs themselves, or use them as template parameters.
As a first step, I would, therefore, consider breaking out the struct:
// GpioMap0.h
#pragma once
// #includes
namespace Gpio {
struct Map0 {
GPIO_MAP0_MODER;
GPIO_MAP0_OTYPER;
GPIO_MAP0_OSPEEDR;
GPIO_MAP0_PUPDR;
GPIO_MAP0_IDR;
GPIO_MAP0_ODR;
GPIO_MAP0_BSRR;
GPIO_MAP0_LCKR;
GPIO_MAP0_AFR;
GPIO_MAP0_BRR;
GPIO_MAP0_ASCR;
};
} // namespace Gpio
// GpioMap1.h
#pragma once
// #includes
namespace Gpio {
struct Map1 {
// fields
};
} // namespace Gpio
// ... others headers ...
And finally, the global header:
// Gpio.h
#pragma once
#include "GpioMap0.h"
#include "GpioMap1.h"
// ... other headers ...
namespace Gpio {
union Gpio {
Map0 map0;
Map1 map1;
// ... others ...
};
} // namespace Gpio
Now, I can write a void special_map0(Gpio:: Map0 volatile& map);, as well as get a quick overview of all available architectures at a glance.
Simple Spacers
With the definition split in multiple headers, the headers are individually much more manageable.
Therefore, my initial approach to exactly meet your requirements would be to stick with repeated std::uint32_t:32;. Yes, it adds a few 100s lines to the existing 8k lines, but since each header is individually smaller, it may not be as bad.
If you are willing to consider more exotic solutions, though...
Introducing $.
It is a little-known fact that $ is a viable character for C++ identifiers; it's even a viable starting character (unlike digits).
A $ appearing in the source code would likely raise eyebrows, and $$$$ is definitely going to attract attention during code reviews. This is something that you can easily take advantage of:
#define GPIO_RESERVED(Index_, N_) std::uint32_t $$$$##Index_[N_];
struct Map3 {
GPIO_RESERVED(0, 6);
GPIO_MAP2_BSRRL;
GPIO_MAP2_BSRRH;
GPIO_RESERVED(1, 5);
};
You can even put together a simple "lint" as a pre-commit hook or in your CI which looks for $$$$ in the committed C++ code and reject such commits.
Although I agree structs should not be used for MCU I/O port access, original question can be answered this way:
struct __attribute__((packed)) test {
char member1;
char member2;
volatile struct __attribute__((packed))
{
private:
volatile char spacer_bytes[7];
} spacer;
char member3;
char member4;
};
You may need to replace __attribute__((packed)) with #pragma pack or similar depending on your compiler syntax.
Mixing private and public members in a struct normally results in that memory layout is no longer guaranteed by C++ standard.
However if all non-static members of a struct are private, it is still considered POD / standard layout, and so are structs that embed them.
For some reason gcc produces a warning if a member of an anonymous struct is private so I had to give it a name. Alternatively, wrapping it into yet another anonymous struct also gets rid of the warning (this may be a bug).
Note that spacer member is not itself private, so data can still be accessed this way:
(char*)(void*)&testobj.spacer;
However such an expression looks like an obvious hack, and hopefully would not be used without a really good reason, let alone as a mistake.
Anti-solution.
DO NOT DO THIS: Mix private and public fields.
Maybe a macro with a counter to generate uniqie variable names will be useful?
#define CONCAT_IMPL( x, y ) x##y
#define MACRO_CONCAT( x, y ) CONCAT_IMPL( x, y )
#define RESERVED MACRO_CONCAT(Reserved_var, __COUNTER__)
struct {
GPIO_MAP1_CRL;
GPIO_MAP1_CRH;
GPIO_MAP1_IDR;
GPIO_MAP1_ODR;
GPIO_MAP1_BSRR;
GPIO_MAP1_BRR;
GPIO_MAP1_LCKR;
private:
char RESERVED[4];
public:
GPIO_MAP1_AFRL;
GPIO_MAP1_AFRH;
private:
char RESERVED[8];
};

why size of structure does not change when use 24 bit integer

I am trying to port embed code in windows platform.
I have come across below problem I am posting a sample code here.
here even after I use int24 size remains 12 bytes in windows why ?
struct INT24
{
INT32 data : 24;
};
struct myStruct
{
INT32 a;
INT32 b;
INT24 c;
};
int _tmain(int argc, _TCHAR* argv[])
{
unsigned char myArr[11] = { 0x00,0x00,0x00,0x00, 0x00,0x00,0x00,0x00, 0xFF,0xFF,0xFF };
myStruct *p = (myStruct*)myArr;
cout << sizeof(*p);
}
There are two reasons, each of which would be enough by themselves.
Presumably the size of INT32 is 4 bytes. The size of INT24 is also 4 bytes, because it contains a INT32 bit field. Since, myStruct contains 3 members of size 4, its size must therefore be at least 12.
Presumably the alignment requirement of INT32 is 4. So, even if the size of INT24 were 3, the size of myStruct would still have to be 12, because it must have at least the alignment requirement of INT32 and therefore the size of myStruct must be padded to the nearest multiple of 4.
any way or workaround ?
This is implementation specific, but the following hack may work for some compilers/cpu combinations. See the manual of your compiler for the syntax for similar feature, and the manual for your target cpu whether it supports unaligned memory access. Also do realize that unaligned memory access does have a performance penalty.
#pragma pack(push, 1)
struct INT24
{
INT32 data : 24;
};
#pragma pack(pop)
#pragma pack(push, 1)
struct myStruct
{
INT32 a;
INT32 b;
INT24 c;
};
#pragma pack(pop)
Packing a bit field might not work the same in all compilers. Be sure to check how yours behaves.
I think that a standard compliant way would be to store char arrays of sizes 3 and 4, and whenever you need to read or write one of the integer, you'd have to std::memcpy the value. That would be a bit burdensome to implement and possibly also slower than the #pragma pack hack.
Sadly for you, the compiler in optimising the code for a particular architecture reserves the right to pad out the structure by inserting spaces between members and even at the end of the structure.
Using a bit field does not reduce the size of the struct; you still get the whole of the "fielded" type in the struct.
The standard guarantees that the address of the first member of a struct is the same as the address of the struct, unless it's a polymorphic type.
But all is not lost: you can rely on the fact that an array of char will always be contiguous and contain no packing.
If CHAR_BIT is defined as 8 on your system (it probably is), you can model an array of 24 bit types on an array of char. If it's not 8 then even this approach will not work: I'd then suggest resorting to inline assembly.

Can I define a struct whose objects will always be in separate cache lines

I know that you can align variables to a cache line by using for example attribute((align(64))) in gcc. However, I'm interested in aligning (or you could call it padding) at structure declaration time. So for example, for the following struct I want to ask the compiler to create necessary padding so that any object of this structure is always aligned with a cache line.
typedef struct
{
int a;
int b;
// I want the compiler to create a padding here for cache alignment
} my_type;
Yes. I can't remember where I got this code from. I think it might have been Herb Sutter's blog:
#define CACHE_LINE_SIZE 64 // Intel Core 2 cache line size.
template<typename T>
struct CacheLineStorage {
public:
[[ align(CACHE_LINE_SIZE) ]] T data;
private:
char pad[ CACHE_LINE_SIZE > sizeof(T)
? CACHE_LINE_SIZE - sizeof(T)
: 1 ];
};
This is straightforward. You probably just missed the "ed" in aligned.
typedef struct
{
int a __attribute__((aligned(64)));
int b;
} my_type;
The resulting struct will have a 56 byte padding after b if you create variables or an array of it.