The code allocating memory holding an image of width * height is as follows:
const size_t alignment = 64;
size_t space = width * height * bytes_per_pixel + alignment;
rawdata = new unsigned char[space];
//typedef unsigned long int uintptr_t
uintptr_t ptr = reinterpret_cast<uintptr_t>(rawdata);
uintptr_t aligned = (ptr - 1u + alignment) & -alignment;
data = reinterpret_cast<unsigned char *>(aligned);
It seems that the 64-bytes alignment was performed on rawdata(i.e. the initially allocated memory), which generated aligned memory pointed by data. But, what puzzled me was the line:
uintptr_t aligned = (ptr - 1u + alignment) & -alignment
Can anyone help me out?
That calculation makes sure the address is aligned to the amount given (which must be a power of 2). This means the lowest n bits must be zero when alignment is 2^n.
Let’s do it in binary. Let’s assume we get a random pointer aligned at 16 bytes while we want it to be aligned at 64 bytes and calculate. (This assumes two’s complement, which is not guaranteed, by the way, but is de facto standard):
address = ...1101010000
address - 1 -> ...1101001111
address + 64 -> ...1110001111
-alignment -> ...1111000000
address & -alignment -> ...1110000000
So in effect it finds the smallest value that is divisible by alignment due to -alignment having all bits zero below the alignment spot. It also assures it’s larger than original pointer by adding alignment-1, which is the negation of -alignment as bits, that is all top bits zero but lower ones one.
What if the address is already aligned? Then the calculation results in the original pointer since it has lowest bits zero, you make all lowest bits to one, then AND them away.
Related
I have two bytes read from a sensor over I2C, these are stored as unsigned char. I do know which is the most significant byte, and which is the least significant byte.
unsigned char yawMSB;
unsigned char yawLSB;
How would I go about converting these two bytes of data, into a single int?
I've had this implemented properly in C# using
BitConverter.ToInt16(new byte[] { yawLSB, yawMSB }, 0)
In a 16-bit integer, the top 8 bits are the most significant byte, and the low 8 bits are the least significant byte.
To make the most significant byte occupy the top bits, you need to shift its value up (by 8 bits), which is done with the left-shift bitwise operator <<.
Then to get the least significant byte you just add the low 8 bits using bitwise or |.
Put together it will be something like yawMSB << 8 | yawLSB.
I do know which is the most significant byte, and which is the least significant byte.
MSB means Most Significant byte.
LSB means Least Significant byte.
How would I go about converting these two bytes of data, into a single float?
You can then build a float whose value is:
const float yaw = yawMSB << 8 + yawLSB;
Actually, the value yawMSB << 8 + yawLSB is probably on a scale defined by your implementation. If it is true, and if it is a linear scale from 0 to MAX_YAW, you should define your value as:
const float yaw = float(yawMSB << 8 + yawLSB) / MAX_YAW; // gives yaw in [0.f, 1.f].
I found this piece of code:
void* aligned_malloc(size_t required_bytes, size_t alignment) {
int offset = alignment - 1;
void* P = (void * ) malloc(required_bytes + offset);
void* q = (void * ) (((size_t)(p) + offset) & ~(alignment - 1));
return q;
}
that is the implementation of aligned malloc in C++. Aligned malloc is a function that supports allocating memory such that the
memory address returned is divisible by a specific power of two.
Example:
align_malloc (1000, 128) will return a memory address that is a multiple of 128 and that points to memory of size 1000 bytes.
But I don't understand line 4. Why sum twice the offset?
Thanks
Why sum twice the offset?
offset isn't exactly being summed twice. First use of offset is for the size to allocate:
void* p = (void * ) malloc(required_bytes + offset);
Second time is for the alignment:
void* q = (void * ) (((size_t)(p) + offset) & ~(alignment - 1));
Explanation:
~(alignment - 1) is a negation of offset (remember, int offset = alignment - 1;) which gives you the mask you need to satisfy the alignment requested. Arithmetic-wise, adding the offset and doing bitwise and (&) with its negation gives you the address of the aligned pointer.
How does this arithmetic work? First, remember that the internal call to malloc() is for required_bytes + offset bytes. As in, not the alignment you asked for. For example, you wanted to allocate 10 bytes with alignment of 16 (so the desired behavior is to allocate the 10 bytes starting in an address that is divisible with 16). So this malloc() from above will give you 10+16-1=25 bytes. Not necessarily starting at the right address in terms of being divisible with 16). But then this 16-1 is 0x000F and its negation (~) is 0xFFF0. And now we apply the bitwise and like this: p + 15 & 0xFFF0 which will cause every pointer p to be a multiple of 16.
But wait, why add this offset of alignment - 1 in the first place? You do it because once you get the pointer p returned by malloc(), the one thing you cannot do -- do in order to find the nearest address which is a multiple of the alignment requested -- is look for it before p, as this could cross into an address space of something allocated before p. For this, you begin by adding alignment - 1, which, think about it, is exactly the maximum by which you'd have to advance to get your alignment.
* Thanks to user DevSolar for some additional phrasing.
Note 1: For this way to work the alignment must be a power of 2. This snippet does not enforce such a thing and so could cause unexpected behavior.
Note 2: An interesting question is how could you implement a free() version for such an allocation, with the return value from this function.
My machine is 64 bit. My code as below:
unsigned long long periodpackcount=*(mBuffer+offset)<<32|*(mBuffer+offset+1)<<24|* (mBuffer+offset+2)<<16|*(mBuffer+offset+3)<<8|*(mBuffer+offset+4);
mBuffer is unsigned char*. I want to get 5 bytes data and transform the data to host byte-order.
How can I avoid this warning ?
Sometimes it's best to break apart into a few lines in order to avoid issues. You have a 5 byte integer you want to read.
// Create the number to read into.
uint64_t number = 0; // uint64_t is in <stdint>
char *ptr = (char *)&number;
// Copy from the buffer. Plus 3 for leading 0 bits.
memcpy(ptr + 3, mBuffer + offset, 5);
// Reverse the byte order.
std::reverse(ptr, ptr + 8); // Can bit shift here instead
Probably not the best byte swap ever (bit shifting is faster). And my logic might be off for the offsetting, but something along those lines should work.
The other thing you may want to do is cast each byte before shifting since you're leaving it up to the compiler to determine the data type *(mBuffer + offset) is a character (I believe), so you may want to cast it to a larger type static_cast<uint64_t>(*(mBuffer + offset)) << 32 or something.
I have a question on problem 13.9 in the book, "cracking the coding interview".
The question is to write an aligned alloc and free function that supports allocating memory, and in the answer the code is given below:
void *aligned_malloc(size_t required_bytes, size_t alignment) {
void *p1;
void **p2;
int offset=alignment-1+sizeof（void*);
if((p1=(void*)malloc(required_bytes+offset))==NULL)
return NULL;
p2=(void**)(((size_t)(p1)+offset)&~(alignment-1)); //line 5
p2[-1]=p1; //line 6
return p2;
}
I am so confused with the line 5 and line 6. Why do you have to do an "and" since you have already add offset to p1? and what does [-1] mean? Thanks for the help in advance.
Your sample code is not complete. It allocates nothing. It is pretty obvious you are missing a malloc statement, which sets the p1 pointer. I don't have the book, but I think the complete code should goes along these lines:
void *aligned_malloc(size_t required_bytes, size_t alignment) {
void *p1;
void **p2;
int offset=alignment-1+sizeof（void*);
p1 = malloc(required_bytes + offset); // the line you are missing
p2=(void**)(((size_t)(p1)+offset)&~(alignment-1)); //line 5
p2[-1]=p1; //line 6
return p2;
}
So ... what does the code do?
The strategy is to malloc more space than what we need (into p1), and return a p2 pointer somewhere after the beginning of the buffer.
Since alignment is a power of two, in binary it has the form of 1 followed by zeros. e.g. if alignment is 32, it will be 00100000 in binary
(alignment-1) in binary format will turn the 1 into 0, and all the 0's after the 1 into 1. For example: (32-1) is 00011111
the ~ will reverse all the bits. That is: 11100000
now, p1 is a pointer to the buffer (remember, the buffer is larger by offset than what we need). we add offset to p1: p1+offset.
So now, (p1+offset) is greater than what we want to return. We'll nil all the insignificant bits by doing a bitwise and: (p1+offset) & ~(offset-1)
This is p2, the pointer that we want to return. Note that because its last 5 digits are zero it is 32 aligned, as requested.
p2 is what we'll return. However, we must be able to reach p1 when the user calls aligned_free. For this, note that we reserved location for one extra pointer when we calculated the offset (that's the sizeof(void*) in line 4.
so, we want to put p1 immediately before p2. This is p2[-1]. This is a little bit tricky calculation. Remember that p2 is defined as void**. One way to look at it is as array of void*. C array calculation say that p2[0] is exactly p2. p2[1] is p2 + sizeof(void*). In general p2[n] = p2 + nsizeof(void). The compiler also supports negative numbers, so p2[-1] is one void* (typically 4 bytes) before p2.
I'm going to guess that aligned_free is something like:
void aligned_free( void* p ) {
void* p1 = ((void**)p)[-1]; // get the pointer to the buffer we allocated
free( p1 );
}
p1 is the actual allocation. p2 is the pointer being returned, which references memory past the point of allocation and leaves enough space for both allignment AND storing the actual allocated pointer in the first place. when aligned_free() is called, p1 will be retrieved to do the "real" free().
Regarding the bit math, that gets a little more cumbersome, but it works.
p2=(void**)(((size_t)(p1)+offset)&~(alignment-1)); //line 5
Remember, p1 is the actual allocation reference. For kicks, lets assume the following, with 32bit pointers:
alignment = 64 bytes, 0x40
offset = 0x40-1+4 = 0x43
p1 = 0x20000110, a value returned from the stock malloc()
What is important is the original malloc() that allocates an additional 0x43 bytes of space above and beyond the original request. This is to ensure both the alignment math and the space for the 32bit pointer can be accounted for:
p2=(void**)(((size_t)(p1)+offset)&~(alignment-1)); //line 5
p2 = (0x20000110 + 0x43) &~ (0x0000003F)
p2 = 0x20000153 &~ 0x0000003F
p2 = 0x20000153 & 0xFFFFFFC0
p2 = 0x20000140
p2 is aligned on a 0x40 boundary (i.e. all bits in 0x3F are 0) and enough space is left behind to store the 4-byte pointer for the original allocation, referenced by p1.
It has been forever since i did alignment math, so if i f'ed up the bits, please someone correct this.
And by the way, this is not the only way to do this.
void* align_malloc(size_t size, size_t alignment)
{
// sanity check for size/alignment.
// Make sure alignment is power of 2 (alignment&(alignment-1) ==0)
// allocate enough buffer to accommodate alignment and metadata info
// We want to store an offset to address where CRT reserved memory to avoid leak
size_t total = size+alignment+sizeof(size_t);
void* crtAlloc = malloc(total);
crtAlloc += sizeof(size_t); // make sure we have enough buffer ahead to store metadata info
size_t crtArithmetic = reinterprete_cast<int>(crtAlloc); // treat as int for pointer arithmetic
void* pRet = crtAlloc + (alignment - (crtArithmetic%alignment));
size_t *pMetadata = reinterprete_cast<size_t*>(pRet);
pMetadata[-1] = pRet - crtArithmetic- sizeof(size_t);
return pRet;
}
As an example size = 20, alignement = 16 and crt malloc returned address 10. and assuming sizeof(size_t) as 4 byte
- total bytes to allocate = 20+16+4 = 40
- memory committed by crt = address 10 to 50
- first make space in front by adding sizeof(size_t) 4 bytes so you point at 14
- add offset to align which is 14 + (16-14%16) = 16
- move back sizeof(size_t) 4 bytes (i.e. 12) and treat that as size_t pointer and store offset 2 (=12-10) to point to crt malloc
start
Same way, align_free will cast void pointer to size_t pointer, move back one location, read value stored there and adjust offset to move to crt alloc beginning
void* align_free(void* ptr)
{
size_t* pMetadata = reinterprete_cast<size_t*> (ptr);
free(ptr-pMetadata[-1]);
}
Optimization: You don't need sizeof(size_t) extra allocation if your alignment was more than sizeof(size_t)
I have a pointer and I want to assign the bytes from x to (x+1) to a unsigned short. I was hoping that I could just
(unsigned short) pointer[x]
but the first byte seems to always be empty. But when I check the value of bytes x and x+1 individually the data is there.
Where have I noobed? (I just made it a verb)
unsigned short i = (pointer[x] << 8) + pointer[x+1]
Assuming you want x to be the high bits and x+1 to be the low bits. Actually, you should keep endianness in mind here too.
EDIT:
This may also work:
*(unsigned short *) pointer[x]
However, this will be affected by endianness, for example, in little endian the first byte will be the lower 8 bits, while x+1 will be the higher bytes.