Consider the following program compiled in VC++ 2010:
#pragma pack(push, 1) // 1, 2, 4, 8
struct str_test
{
unsigned int n;
unsigned short s;
unsigned char b[4];
};
#pragma pack(pop)
int main()
{
str_test str;
str.n = 0x01020304;
str.s = 0xa1a2;
str.b[0] = 0xf0;
str.b[1] = 0xf1;
str.b[2] = 0xf2;
str.b[3] = 0xf3;
unsigned char* p = (unsigned char*)&str;
std::cout << sizeof(str_test) << std::endl;
return 0;
}
I set breakpoint on return 0; line and look memory window in the debugger, starting from address p. I get the following results (sizeof and memory layout, depending on pack):
// 1 - 10 (pack, sizeof)
// 04 03 02 01 a2 a1 f0 f1 f2 f3
// 2 - 10
// 04 03 02 01 a2 a1 f0 f1 f2 f3
// 4 - 12
// 04 03 02 01 a2 a1 f0 f1 f2 f3
// 8 - 12
// 04 03 02 01 a2 a1 f0 f1 f2 f3
Two questions:
Why sizeof(str_test) is 12 for pack 8?
Why memory layout is the same and doesn't depend on the pack value?
Why sizeof(str_test) is 12 for pack 8?
From the MSDN docs:
The alignment of a member will be on a boundary that is either a
multiple of n or a multiple of the size of the member, whichever is
smaller.
In your case the largest member is 4bytes which is smaller than 8, so 4bytes will be used for alignement.
Why memory layout is the same and doesn't depend on the pack value?
The compiler isn't permited to reorder struct members, but can pad the members. In case of pack 8 it does the following;
#pragma pack(push, 8) // largest element is 4bytes so it will be used instead of 8
struct str_test
{
unsigned int n; // 4 bytes
unsigned short s; // 2 bytes
unsigned char b[4]; // 4 bytes
//2 bytes padding here;
};
#pragma pack(pop)
And hence sizeof(str_test) will be 12.
Well, it seems the compiler (MSVC2010) change padding location based on the type, in the case of unsigned char b[4]; it placed two bytes padding at the end of the structure. In your case the 2 bytes cc cc happens to be after the character array.
#pragma pack(push, 8) // largest element is 4bytes so it will be used instead of 8
struct str_test
{
unsigned int n; // 4 bytes
unsigned short s; // 2 bytes
//2 bytes padding here;
int; // 4 bytes
};
#pragma pack(pop)
What I did is change the last member from char[4] to int, and can be verified by subtracting the addresses of the last and first member in both cases 6 and 8 respectively.
Memory dump in the case of last member int as follows
04 03 02 01 a2 a1 cc cc f0 f1 f2 f3
Memory dump in the case of last member unsigned char[4] as follows
04 03 02 01 a2 a1 f0 f1 f2 f3 cc cc
Related
I've spotted a strange behavior in this small C program. I have 2 structures, both with padding bytes, but in different places.
The first structure has padding bytes with indices [1:3], and the output is expected: static variables are zeroed-out, so padding values are all 0, local variables on stack are left with garbage values in padding bytes. Example output:
Char is first, then int:
aa 60 8e ef ff ff ff ff
aa 00 00 00 ff ff ff ff
But in the second structure, something strange happens. Padding bytes in this structure are with indices [5:7], so I expected some garbage values in non-static variable, but every time the output is:
Int is first, then char:
ff ff ff ff aa 7f 00 00
ff ff ff ff aa 00 00 00
Why the padding is always 7f 00 00?
The complete program:
#include "stdio.h"
#include "stdint.h"
#include "stddef.h"
// 0 1 2 3 4 5 6 7
// |a|#|#|#|b|b|b|b|
typedef struct
{
uint8_t a;
uint32_t b;
} S1;
// 0 1 2 3 4 5 6 7
// |a|a|a|a|b|#|#|#|
typedef struct
{
uint32_t a;
uint8_t b;
} S2;
void print_bytes(void* mem, size_t num_bytes)
{
for (size_t i = 0; i < num_bytes; i++)
printf("%02x ", *((unsigned char*)mem + i));
putc('\n', stdout);
}
int main()
{
S1 var1 = { .a = 0xAA, .b = 0xFFFFFFFF };
static S1 var1_s = { .a = 0xAA, .b = 0xFFFFFFFF };
printf("Char is first, then int:\n");
print_bytes(&var1, sizeof(S1));
print_bytes(&var1_s, sizeof(S1));
S2 var2 = { .a = 0xFFFFFFFF, .b = 0xAA };
static S2 var2_s = { .a = 0xFFFFFFFF, .b = 0xAA };
printf("\nInt is first, then char:\n");
print_bytes(&var2, sizeof(S2));
print_bytes(&var2_s, sizeof(S2));
}
There's no problem if I run your program. Last bytes are random. It likely depends on your system.
I am preparing a minidriver to perform sign in smartcard using NCryptSignHash function of Microsoft CNG.
When I perform sign with an SECP521R1 EC key in smartcard it generates a sign data with length of 139 as ECC signed data format:
ECDSASignature ::= SEQUENCE {
r INTEGER,
s INTEGER
}
Sample signed data is
308188024201A2001E9C0151C55BCA188F201020A84180B339E61EDE61F6EAD0B277321CAB81C87DAFC2AC65D542D0D0B01C3C5E25E9209C47CFDDFD5BBCAFA0D2AF2E7FD86701024200C103E534BD1378D8B6F5652FB058F7D5045615DCD940462ED0F923073076EF581210D0DD95BF2891358F5F743DB2EC009A0608CEFAA9A40AF41718881D0A26A7F4
But when I perform Sign using MS_KEY_STORAGE_PROVIDER it generates a sign with length of 132 byte.
What is the procedure to reduce the sign data size from 139 to 132?
Your input is an X9.62 signature format which is a SEQUENCE containing two ASN.1 / DER encoded signatures. These integers are variable sized, signed, big endian numbers. They are encoded in the minimum number of bytes. This means that the size of the encoding can vary.
The 139 bytes is common because it assumes the maximum size of the encoding for r and s. These values are computed using modular arithmetic and they can therefore contain any number of bits, up to the number of bits of order n, which is the same as the key size, 521 bits.
The 132 bytes are specified by ISO/IEC 7816-8 / IEEE P1363 which is a standard that deals with signatures for smart cards. The signature consists of the concatenation of r and s, where r and s are encoded as the minimum number of bytes to display a value of the same size as the order, in bytes. The r and s are statically sized, unsigned, big endian numbers.
The calculation of the number of bytes of r or s is ceil((double) n / 8) or (n + 8 - 1) / 8 where 8 is the number of bits in a byte. So if the elliptic curve is 521 bits then the resulting size is 66 bytes, and together they therefore consume 132 bytes.
Now on to the decoding. There are multiple ways of handling this: perform a full ASN.1 parse, obtain the integers and then encode them back again in the ISO 7816-8 form is the most logical one.
However, you can also see that you could simply copy bytes as r and s will always be non-negative (and thus unsigned) and big endian. So you just need to compensate for the size. Otherwise the only hard part is to be able to decode the length of the components within the X9.62 structure.
Warning: code in C# instead of C++ as I expected the main .NET language; language not indicated in question when I wrote the main part of the answer.
class ConvertECDSASignature
{
private static int BYTE_SIZE_BITS = 8;
private static byte ASN1_SEQUENCE = 0x30;
private static byte ASN1_INTEGER = 0x02;
public static byte[] lightweightConvertSignatureFromX9_62ToISO7816_8(int orderInBits, byte[] x9_62)
{
int offset = 0;
if (x9_62[offset++] != ASN1_SEQUENCE)
{
throw new IllegalSignatureFormatException("Input is not a SEQUENCE");
}
int sequenceSize = parseLength(x9_62, offset, out offset);
int sequenceValueOffset = offset;
int nBytes = (orderInBits + BYTE_SIZE_BITS - 1) / BYTE_SIZE_BITS;
byte[] iso7816_8 = new byte[2 * nBytes];
// retrieve and copy r
if (x9_62[offset++] != ASN1_INTEGER)
{
throw new IllegalSignatureFormatException("Input is not an INTEGER");
}
int rSize = parseLength(x9_62, offset, out offset);
copyToStatic(x9_62, offset, rSize, iso7816_8, 0, nBytes);
offset += rSize;
// --- retrieve and copy s
if (x9_62[offset++] != ASN1_INTEGER)
{
throw new IllegalSignatureFormatException("Input is not an INTEGER");
}
int sSize = parseLength(x9_62, offset, out offset);
copyToStatic(x9_62, offset, sSize, iso7816_8, nBytes, nBytes);
offset += sSize;
if (offset != sequenceValueOffset + sequenceSize)
{
throw new IllegalSignatureFormatException("SEQUENCE is either too small or too large for the encoding of r and s");
}
return iso7816_8;
}
/**
* Copies an variable sized, signed, big endian number to an array as static sized, unsigned, big endian number.
* Assumes that the iso7816_8 buffer is zeroized from the iso7816_8Offset for nBytes.
*/
private static void copyToStatic(byte[] sint, int sintOffset, int sintSize, byte[] iso7816_8, int iso7816_8Offset, int nBytes)
{
// if the integer starts with zero, then skip it
if (sint[sintOffset] == 0x00)
{
sintOffset++;
sintSize--;
}
// after skipping the zero byte then the integer must fit
if (sintSize > nBytes)
{
throw new IllegalSignatureFormatException("Number format of r or s too large");
}
// copy it into the right place
Array.Copy(sint, sintOffset, iso7816_8, iso7816_8Offset + nBytes - sintSize, sintSize);
}
/*
* Standalone BER decoding of length value, up to 2^31 -1.
*/
private static int parseLength(byte[] input, int startOffset, out int offset)
{
offset = startOffset;
byte l1 = input[offset++];
// --- return value of single byte length encoding
if (l1 < 0x80)
{
return l1;
}
// otherwise the first byte of the length specifies the number of encoding bytes that follows
int end = offset + l1 & 0x7F;
uint result = 0;
// --- skip leftmost zero bytes (for BER)
while (offset < end)
{
if (input[offset] != 0x00)
{
break;
}
offset++;
}
// --- test against maximum value
if (end - offset > sizeof(uint))
{
throw new IllegalSignatureFormatException("Length of TLV is too large");
}
// --- parse multi byte length encoding
while (offset < end)
{
result = (result << BYTE_SIZE_BITS) ^ input[offset++];
}
// --- make sure that the uint isn't larger than an int can handle
if (result > Int32.MaxValue)
{
throw new IllegalSignatureFormatException("Length of TLV is too large");
}
// --- return multi byte length encoding
return (int) result;
}
}
Note that the code is somewhat permissive in the fact that it doesn't require the minimum length encoding for the SEQUENCE and INTEGER length encoding (which it should).
It also allows wrongly encoded INTEGER values that are unnecessarily left-padded with zero bytes.
Neither of these issues should break the security of the algorithm but other libraries may and should be less permissive.
What is the procedure to reduce the sign data size from 139 to 132?
You have an ASN.1 encoded signature (shown below). It is used by Java, OpenSSL and some other libraries. You need the signature in P1363 format, which is a concatenation of r || s, without the ASN.1 encoding. P1363 is used by Crypto++ and a few other libraries. (There's another common signature format, and that is OpenPGP).
For the concatenation of r || s, both r and s must be 66 bytes because of secp-521r1 field element size on an octet boundary. That means the procedure is, you have to strip the outer SEQUENCE, and then strip the two INTEGER, and then concatenate the values of the two integers.
Your formatted r || s signature using your sample data will be:
01 A2 00 1E ... 7F D8 67 01 || 00 C1 03 E5 ... 0A 26 A7 F4
Microsoft .Net 2.0 has ASN.1 classes that allow you to manipulate ASN.1 encoded data. See AsnEncodedData class.
$ echo 08188024201A2001E9C0151C55BCA188F201020A84180B339E61EDE61F6EAD0B277321CAB
81C87DAFC2AC65D542D0D0B01C3C5E25E9209C47CFDDFD5BBCAFA0D2AF2E7FD86701024200C103E5
34BD1378D8B6F5652FB058F7D5045615DCD940462ED0F923073076EF581210D0DD95BF2891358F5F
743DB2EC009A0608CEFAA9A40AF41718881D0A26A7F4 | xxd -r -p > signature.bin
$ dumpasn1 signature.bin
0 136: SEQUENCE {
3 66: INTEGER
: 01 A2 00 1E 9C 01 51 C5 5B CA 18 8F 20 10 20 A8
: 41 80 B3 39 E6 1E DE 61 F6 EA D0 B2 77 32 1C AB
: 81 C8 7D AF C2 AC 65 D5 42 D0 D0 B0 1C 3C 5E 25
: E9 20 9C 47 CF DD FD 5B BC AF A0 D2 AF 2E 7F D8
: 67 01
71 66: INTEGER
: 00 C1 03 E5 34 BD 13 78 D8 B6 F5 65 2F B0 58 F7
: D5 04 56 15 DC D9 40 46 2E D0 F9 23 07 30 76 EF
: 58 12 10 D0 DD 95 BF 28 91 35 8F 5F 74 3D B2 EC
: 00 9A 06 08 CE FA A9 A4 0A F4 17 18 88 1D 0A 26
: A7 F4
: }
0 warnings, 0 errors.
Another noteworthy item is, .Net uses the XML format detailed in RFC 3275, XML-Signature Syntax and Processing. It is a different format than ASN.1, P1363, OpenPGP, CNG and other libraries.
The ASN.1 to P1363 conversion is rather trivial. You can see an example using the Crypto++ library at ECDSA sign with BouncyCastle and verify with Crypto++.
You might find Cryptographic Interoperability: Digital Signatures on Code Project helpful.
The following apparently valid code produces a misaligned address runtime error using the UndefinedBehaviorSanitizer sanitiser.
#include <memory>
#include <functional>
struct A{
std::function<void()> data; // seems to occur only if data is a std::function
} ;
struct B{
char data; // occurs only if B contains a member variable
};
struct C:public virtual A,public B{
};
struct D:public virtual C{
};
void test(){
std::make_shared<D>();
}
int main(){
test();
return 0;
}
Compiling and executing on a macbook with
clang++ -fsanitize=undefined --std=c++11 ./test.cpp && ./a.out
produces the output
runtime error: constructor call on misaligned address 0x7fe584500028 for type 'C', which requires 16 byte alignment [...].
I would like to understand how and why the error occurs.
Since alignment of std::function<void()> is 16 and size is 48 lets simplify. This code has the same behavior but is easier to understand:
struct alignas(16) A
{ char data[48]; };
struct B
{ char data; };
struct C : public virtual A, public B
{};
struct D : public virtual C
{};
int main()
{
D();
}
We have the following alignments and sizes:
|__A__|__B__|__C__|__D__|
alignment (bytes): | 16 | 1 | 16 | 16 |
size (bytes): | 48 | 1 | 64 | 80 |
Now lets see how this looks like in memory. More explanation on that can be found in this great answer.
A: char[48] + no padding == 48B
B: char[1] + no padding == 1B
C: A* + B + A + 7 bytes of padding (align to 16) == 64B
D: C* + C + 8 bytes of padding (align to 16) == 80B
Now it is easy to see that the offset of C inside D is 8 bytes, but C is aligned to 16. Thus error, which is helpfully accompanied by this pseudo-graphic
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
^
Here each zero is 1 byte.
UPDATE:
Where and how to place padding is up to a C++ compiler. Standard does not specify it. It looks like with the size of padding it has, clang is unable to align everything in D. One way to mitigate the misalignment is to design your classes carefully so that they have the same alignment (e.g., 8 bytes).
I'm trying to align my structure and make it as small as possible using bit fields. I have to send this data back to a client, which will examine the fields to set a few data members.
The size of the structure is indeed the same, but when I set members it does not work at all.
Here's some example code:
#pragma pack(push, 1)
struct PW_INFO
{
char hash[16]; //Does not matter
uint32_t number; //Does not matter
uint32_t salt_id : 30; //Position: 0 bits
uint32_t enc_level : 7; //Position: 30 bits
uint32_t delta : 27; //Position: 37 bits
}; //Total size: 28 bytes
#pragma pack(pop)
void int64shrl(uint64_t& base, uint32_t to_shift, uint32_t position)
{
uint64_t res = static_cast<uint64_t>(to_shift);
res = Int64ShllMod32(res, position);
base |= res;
}
int32_t main()
{
std::cout << "Size of PW_INFO: " << sizeof(PW_INFO) << "\n"; //Returns 28 as expected (16 + sizeof(uint32_t) + 8)
PW_INFO pw = { "abc123", 0, 0, 0, 0 };
pw.enc_level = 105;
uint64_t base{ 0 };
&base; //debug purposes
int64shrl(base, 103, 30);
return 0;
}
Here's where it gets weird: setting the "salt_id" field (which is 30 bits into the bitfield) will yield the following result in memory:
0x003FFB8C 61 62 63 31 32 33 00 00 abc123..
0x003FFB94 00 00 00 00 00 00 00 00 ........
0x003FFB9C 00 00 00 00 00 00 00 00 ........
0x003FFBA4 69 00 00 00 i...
(Only the last 8 bytes are of concern since they represent the bit field.)
But, Int64ShllMod32 returns a correct result (the remote client undersands it perfectly):
0x003FFB7C 00 00 00 c0 19 00 00 00 ...À....
I'm guessing it has to do with alignment, if so how would I completely get rid of it? It seems even if the size is correct, it will try to align it (1 byte boundary as the #pragma directive suggests).
More information:
I use Visual Studio 2015 and its compiler.
I am not trying to write those in a different format, the reason I'm asking this is that I do NOT want to use my own format. They are reading from 64 bit bitfields everywhere, I don't have access to the source code but I see a lot of calls to Int64ShrlMod32 (from what I read, this is what the compiler produces when dealing with 8 byte structures).
The actual bitfield starts at "salt_id". 30 + 7 + 27 = 64 bits, I hope it is clearer now.
I have a variable
long long int alpha;
This alpha is basically 8 bytes
but I wish the byte size can be decided dynamically by input to the function.
So that it could be inserted to char* array.
If there is a function
int putInput(int sizeOfAlpha){
long long int alpha;
char* beta = (char*)malloc(128);
for(int i = 0 ; i < 128 ; i++){
... alpha calculation ...
beta[i*sizeOfAlpha] = alpha; // This is also wrong
}
}
Then the size of alpha has to be modified by sizeOfAlpha
For instance if sizeOfAlpha is 2 in decimal,
and if alpha is 0x00 00 00 00 00 00 04 20 in hex,
and if i is 0 ,
then beta[0] should be 04 and beta[1] should be 20 in hex
if alpha is 0x00 00 00 00 00 00 42 AB in hex,
and if i is 1 ,
then beta[2] should be 42 and beta[3] should be AB in hex
Can anyone help me with this?
Assuming alpha is unsigned :
std::vector<std::uint8_t> vec(8);
for(std::size_t j = (i + 1u) * sizeOfAlpha - 1u; sizeOfAlpha; --j, --sizeOfAlpha) {
vec[j] = alpha & 0xff;
alpha >>= 8;
}
Live on Coliru