I am trying to create a data packet, using memcpy. I expect to see the output in pOutBuffer, whose first four bytes will have 999, followed by 111 followed by 12; But currently i am getting some garbage.
The problem is that instead of copying the value, it copies the address, I think. How can i copy these values in to a contiguous memory so that i can write it to disk and can retrieve the data at the receiving end with my defined format?
Thanks.
#include "stdafx.h"
#include "windows.h"
typedef struct
{
int Begin;
int End;
int Size;
}PACKET;
void AddBuffer(PACKET* pPacket, BYTE* pOutBuffer)
{
memcpy(pOutBuffer, &pPacket->Begin, sizeof(int));
memcpy(pOutBuffer+sizeof(int), &pPacket->End, sizeof(int));
memcpy(pOutBuffer+sizeof(int)+sizeof(int), &pPacket->Size, sizeof(int));
}
int _tmain(int argc, _TCHAR* argv[])
{
PACKET* pPacket = new PACKET;
pPacket->Begin = 999;
pPacket->End = 111;
pPacket->Size = 12;
BYTE* pOutBuffer = new BYTE [pPacket->Size];
AddBuffer(pPacket, pOutBuffer);
//Write pOutBuffer on to the disk
//WriteFile(vhFileToWrite,(BYTE*)pOutBuffer,pPacket.Size,&vRetFileSize,NULL);
//Delete pOutBuffer
return 0;
}
Source sample has been updated. It now builds ok
Your code works correctly. On a little-endian machine with sizeof(int)==4, the number 999 will be stored as the four bytes 0xe7, 0x03, 0x00, 0x00.
You said you saw the character 'ç': That is because you are trying to view the array as a string, and ç has the character code 0xe7, which is indeed the first byte written. If you view it as an array (either using Visual Studio's memory view, or by typing pOutBuffer,12 in the watch window), you will see the correct byte values.
Related
I have a long array of char (coming from a raster file via GDAL), all composed of 0 and 1. To compact the data, I want to convert it to an array of bits (thus dividing the size by 8), 4 bytes at a time, writing the result to a different file. This is what I have come up with by now:
uint32_t bytes2bits(char b[33]) {
b[32] = 0;
return strtoul(b,0,2);
}
const char data[36] = "00000000000000000000000010000000101"; // 101 is to be ignored
char word[33];
strncpy(word,data,32);
uint32_t byte = bytes2bits(word);
printf("Data: %d\n",byte); // 128
The code is working, and the result is going to be written in a separate file. What I'd like to know is: can I do that without copying the characters to a new array?
EDIT: I'm using a const variable here just to make a minimal, reproducible example. In my program it's a char *, which is continually changing value inside a loop.
Yes, you can, as long as you can modify the source string (in your example code you can't because it is a constant, but I assume in reality you have the string in writable memory):
uint32_t bytes2bits(const char* b) {
return strtoul(b,0,2);
}
void compress (char* data) {
// You would need to make sure that the `data` argument always has
// at least 33 characters in length (the null terminator at the end
// of the original string counts)
char temp = data[32];
data[32] = 0;
uint32_t byte = bytes2bits(data);
data[32] = temp;
printf("Data: %d\n",byte); // 128
}
In this example by using char* as a buffer to store that long data there is not necessary to copy all parts into a temporary buffer to convert it to a long.
Just use a variable to step through the buffer by each 32 byte length period, but after the 32th byte there needs the 0 termination byte.
So your code would look like:
uint32_t bytes2bits(const char* b) {
return strtoul(b,0,2);
}
void compress (char* data) {
int dataLen = strlen(data);
int periodLen = 32;
char* periodStr;
char tmp;
int periodPos = periodLen+1;
uint32_t byte;
periodStr = data[0];
while(periodPos < dataLen)
{
tmp = data[periodPos];
data[periodPos] = 0;
byte = bytes2bits(periodStr);
printf("Data: %d\n",byte); // 128
data[periodPos] = tmp;
periodStr = data[periodPos];
periodPos += periodLen;
}
if(periodPos - periodLen <= dataLen)
{
byte = bytes2bits(periodStr);
printf("Data: %d\n",byte); // 128
}
}
Please than be careful to the last period, which could be smaller than 32 bytes.
const char data[36]
You are in violation of your contract with the compiler if you declare something as const and then modify it.
Generally speaking, the compiler won't let you modify it...so to even try to do so with a const declaration you'd have to cast it (but don't)
char *sneaky_ptr = (char*)data;
sneaky_ptr[0] = 'U'; /* the U is for "undefined behavior" */
See: Can we change the value of an object defined with const through pointers?
So if you wanted to do this, you'd have to be sure the data was legitimately non-const.
The right way to do this in modern C++ is by using std::string to hold your string and std::string_view to process parts of that string without copying it.
You can using string_view with that char array you have though. It's common to use it to modernize the classical null-terminated string const char*.
Good evening, I am attempting to read some binary information from a .img file. I can retrieve 16-bit numbers (uint16_t) from ntohs(), but when I try to retrieve from the same position using ntohl(), it gives me 0 instead.
Here are the critical pieces of my program.
#include <iostream>
#include <fcntl.h>
#include <sys/stat.h>
#include <sys/mman.h>
#include <arpa/inet.h>
#include <cmath>
int fd;
struct blockInfo {
long blockSize = 0;
long blockCount = 0;
long fatStart = 0;
long fatBlocks = 0;
long rootStart = 0;
long rootBlocks = 0;
long freeBlocks = 0;
long resBlocks = 0;
long alloBlocks = 0;
};
int main(int argc, char *argv[]) {
fd = open(argv[1], O_RDWR);
// Get file size
struct stat buf{};
stat(path, &buf);
size_t size = buf.st_size;
// A struct to hold data retrieved from a big endian image.
blockInfo info;
auto mapPointer = (char*) mmap(nullptr, size,
(PROT_READ | PROT_WRITE), MAP_PRIVATE, fd, 0);
info.blockSize = ntohs((uint16_t) mapPointer[12]);
long anotherBlockSize = ntohl((uint32_t) mapPointer[11]);
printf("%ld", info.blockSize); // == 512, correct
printf("%ld", anotherBlockSize); // == 0, what?
}
I understand that blockSize and anotherBlockSize are not supposed to be equal, but anotherBlockSize should be non-zero at the least, right?
Something else, I go to access data at ntohs(pointer[16]), which should return 2, but also returns 0. What is going on here? Any help would be appreciated.
No, anotherBlockSize will not necessarily be non-zero
info.blockSize = ntohs((uint16_t) mapPointer[12]);
This code reads a char from offset 12 relatively to mapPointer, casts it to uint16_t and applies ntohs() to it.
long anotherBlockSize = ntohl((uint32_t) mapPointer[11]);
This code reads a char from offset 11 relatively to mapPointer, casts it to uint32_t and applies ntohl() to it.
Obviously, you are reading non-overlapped data (different chars) from the mapped memory, so you should not expect blockSize and anotherBlockSize to be connected.
If you are trying to read the same memory in different ways (as uint32_t and uint16_t), you must do some pointer casting:
info.blockSize = ntohs( *((uint16_t*)&mapPointer[12]));
Note that such code will generally be platform dependent. Such cast working perfectly on x86 may fail on ARM.
auto mapPointer = (char*) ...
This declares mapPointer to be a char *.
... ntohl((uint32_t) mapPointer[11]);
Your obvious intent here is to use mapPointer to retrieve a 32 bit value, a four-byte value, from this location.
Unfortunately, because mapPointer is a plain, garden-variety char *, the expression mapPointer[11] evaluates to a single, lonely char value. One byte. That's what the code reads from the mmaped memory block, at the 11th offset from the start of the block. The (uint32_t) does not read an uint32_t from the address referenced mapPointer+11. mapPointer[11] reads a single char value from mapPointer+11, because mapPointer is a pointer to a char, converts it to a uint32_t, and feeds to to ntohl().
I am writing some data to a file. Occasionally, I want to write a block of data from memory, and then move the put pointer along either 1, 2 or 3 bytes to maintain a 4 byte data boundary format.
I could make a new block of data containing zeros and write this, but this seems unnecessary and clumsy. How can I move the put pointer along 1, 2 or 3 bytes?
I am not sure how to do this, because if I call seekp() surely I will move the pointer outside of the current file size? Whereas I assume ofstream.write() deals with this correctly? ie: It resizes the file somehow while writing data?
I am assuming you are doing something like, except instead of writing two bytes of data you want to write 4 bytes with some padding.
#include <fstream>
using namespace std;
struct data
{
char first;
char second;
};
int _tmain(int argc, _TCHAR* argv[])
{
ofstream outFile;
data data1;
data data2;
data1.first = 'a';
data1.second = 'b';
data2.first = 'c';
data2.second = 'd';
outFile.open("somefile.dat");
outFile.write(reinterpret_cast<char*>(&data1), sizeof(data));
outFile.write(reinterpret_cast<char*>(&data2), sizeof(data));
outFile.close();
return 0;
}
One option is to simply make the struct 4 bytes. This could have a disadvantage as it could increase memory footprint.
Using seekp probably is not a good option, I tried it and it sort of worked but not really.
outFile.write(reinterpret_cast<char*>(&data1), sizeof(data));
outFile.seekp(2, ios_base::cur);
outFile.write(reinterpret_cast<char*>(&data2), sizeof(data));
outFile.seekp(2, ios_base::cur);
This did succeed in adding padding after data1 but not data2. Moving the pointer past the just isn't a good idea as it doesn't change the file size. I tried writing 0 bytes after seekp but this didn't work either.
Honestly I would implement a helper function to provide this functionality. Seems much cleaner this way. Here is a simple example:
#include <fstream>
using namespace std;
struct data
{
char first;
char second;
};
void WriteWithPadding(ofstream* outFile, data d, int width);
int _tmain(int argc, _TCHAR* argv[])
{
ofstream* outFile = new ofstream();
data data1;
data data2;
data1.first = 'a';
data1.second = 'b';
data2.first = 'c';
data2.second = 'd';
outFile->open("somefile.dat");
WriteWithPadding(outFile, data1, 4);
WriteWithPadding(outFile, data1, 4);
outFile->close();
delete outFile;
return 0;
}
void WriteWithPadding(ofstream* outFile, data d, int width)
{
if (sizeof(d) > width)
throw;
width = width - sizeof(d); // width is now amount of padding required
outFile->write(reinterpret_cast<char*>(&d), sizeof(data));
// Add Padding
for (int i = 0; i < width; i++)
{
outFile->put(0);
}
}
Just to be pedantic, I assume you have opened your file with ios::binary, because you'll have issues if you haven't.
When writing a file, the file is only as large as the number of bytes you have written to your file. So if you write three bytes to the file, you will have a three-byte file.
To maintain a four-byte resolution, you must make sure to write four bytes at a time -- if you write a three-byte object, write an additional byte (zero?) to bring it up to four bytes.
Hope this helps.
Im trying to build modified example of usage of DES CBC EDE on openssl. Im using gcc version 4.4.6 (Buildroot 2012.02) compiling for arm (not my choise of compiler) on ubuntu 10.04. Here is a problem: when pointer declaration is uncommented - everything is normal. Test message deciphers. But when pointer declaration gets commented - test message shows only 2 first letters after deciphering. I just can not understand what could do a declaration of an unused pointer. Here is code:
#include <openssl/des.h>
#include <cstring>
#define BUFSIZE 512
using namespace std;
int main(int argc, char *argv[]) {
unsigned char in[BUFSIZE] = {};
unsigned char out[BUFSIZE] = {};
unsigned char back[BUFSIZE] = {};
unsigned char *strangePointer = &out[0]; // what is wrong with it?
int len;
DES_cblock key1, key2, key3;
DES_cblock ivsetup = {0xE1, 0xE2, 0xE3, 0xD4, 0xD5, 0xC6, 0xC7, 0xA8};
DES_cblock ivecLocal;
DES_key_schedule ks1, ks2, ks3;
const char* key = "0A0A0B0B0C0C0A0A0B0B0C0C";
memcpy(&key1,key,8);
memcpy(&key2,key + 8,8);
memcpy(&key2,key + 16,8);
DES_set_odd_parity(&key1);
DES_set_odd_parity(&key2);
DES_set_odd_parity(&key3);
DES_set_key((C_Block *)key1, &ks1);
DES_set_key((C_Block *)key2, &ks2);
DES_set_key((C_Block *)key3, &ks3);
const char* message = "Now is the time for all men to stand up and be counted";
/* 64 bytes of plaintext */
len = strlen(message);
memcpy(in,message,len);
printf("Plaintext: [%s]\n", in);
memcpy(ivecLocal, ivsetup, sizeof(ivsetup));
DES_ede3_cbc_encrypt(in, out, len, &ks1, &ks2, &ks3, &ivecLocal, DES_ENCRYPT);
int lenout = 0;
while(out[lenout] != '\0') ++lenout;
memcpy(ivecLocal, ivsetup, sizeof(ivsetup));
DES_ede3_cbc_encrypt(out, back, lenout, &ks1, &ks2, &ks3, &ivecLocal, DES_DECRYPT);
printf("Decrypted Text: [%s]\n", back);
return 0;
}
You have several things incorrect in your code. I'll start with the simple answer first:
Your third key isn't even populated with key data. It is random data on the local activation stack.
memcpy(&key1,key,8);
memcpy(&key2,key + 8,8);
memcpy(&key2,key + 16,8); // <<=== NOTE still key2
Oh, copy-paste, ye are a cruel and heartless wench. Anyway, by removing the variable you're commenting out, this key is moving further up (or down, depending on your implementation) the stack in memory, and the result is a different value . But the bottom line, you're using indeterminate data for your third key.
But that isn't the entire problem. That change in key exposes another issue, you're output length calculation, which is also flat-wrong. This:
int lenout = 0;
while(out[lenout] != '\0') ++lenout;
assumes the length of the output cipher can be found by searching for a 0-byte. The DES algorithm can easily emit such a byte anywhere in an encryption block. This is plain wrong. The size of the output of a DES_ede3_cbc_encrypt encryption operation is always a multiple of the block size, which for DES is 8 bytes (not coincidentally the size of a DES_cblock), The correct calculation of your output buffer size is:
int lenout = ((len + sizeof(DES_cblock) - 1)/sizeof(DES_cblock))*sizeof(DES_cblock);
which may seem like a helluva lot, but in the end all this does is round up to the nearest multiple of the block length. Further, you don't even need the input buffer. The API will do the calculation for you if you simply pass your input data message and its true length. But I leave that for you to finish (hint: just remove input and pass message).
the result is:
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <openssl/des.h>
#define BUFSIZE 512
int main(int argc, char *argv[])
{
unsigned char in[BUFSIZE] = {};
unsigned char out[BUFSIZE] = {};
unsigned char back[BUFSIZE] = {};
int len;
DES_cblock key1, key2, key3;
DES_cblock ivsetup = {0xE1, 0xE2, 0xE3, 0xD4, 0xD5, 0xC6, 0xC7, 0xA8};
DES_cblock ivecLocal;
DES_key_schedule ks1, ks2, ks3;
const char* key = "0A0A0B0B0C0C0A0A0B0B0C0C";
memcpy(&key1,key,8);
memcpy(&key2,key + 8,8);
memcpy(&key3,key + 16,8);
DES_set_odd_parity(&key1);
DES_set_odd_parity(&key2);
DES_set_odd_parity(&key3);
DES_set_key(&key1, &ks1);
DES_set_key(&key2, &ks2);
DES_set_key(&key3, &ks3);
const char* message = "Now is the time for all men to stand up and be counted";
len = strlen(message);
memcpy(in,message,len);
printf("Plaintext: [%s]\n", in);
memcpy(ivecLocal, ivsetup, sizeof(ivsetup));
DES_ede3_cbc_encrypt(in, out, len, &ks1, &ks2, &ks3, &ivecLocal, DES_ENCRYPT);
int lenout = ((len + sizeof(DES_cblock) - 1)/sizeof(DES_cblock))*sizeof(DES_cblock);
memcpy(ivecLocal, ivsetup, sizeof(ivsetup));
DES_ede3_cbc_encrypt(out, back, lenout, &ks1, &ks2, &ks3, &ivecLocal, DES_DECRYPT);
printf("Decrypted Text: [%s]\n", back);
return 0;
}
Output
Plaintext: [Now is the time for all men to stand up and be counted]
Decrypted Text: [Now is the time for all men to stand up and be counted]
You are most probably experiencing a buffer overflow. The declaration of len comes after back on the stack. If back overflows, then len will be overwritten. Adding another variable (strangePointer) in between adds a gap on the stack that stops len getting overwritten.
Probably increasing BUFSIZE should solve your issue.
Hello I have a chunk of memory (allocated with malloc()) that contains bits (bit literal), I'd like to read it as an array of char, or, better, I'd like to printout the ASCII value of 8 consecutively bits of the memory.
I have allocated he memory as char *, but I've not been able to take characters out in a better way than evaluating each bit, adding the value to a char and shifting left the value of the char, in a loop, but I was looking for a faster solution.
Thank you
What I've wrote for now is this:
for allocation:
char * bits = (char*) malloc(1);
for writing to mem:
ifstream cleartext;
cleartext.open(sometext);
while(cleartext.good())
{
c = cleartext.get();
for(int j = 0; j < 8; j++)
{ //set(index) and reset(index) set or reset the bit at bits[i]
(c & 0x80) ? (set(index)):(reset(index));//(*ptr++ = '1'):(*ptr++='0');
c = c << 1;
}..
}..
and until now I've not been able to get character back, I only get the bits printed out using:
printf("%s\n" bits);
An example of what I'm trying to do is:
input.txt contains the string "AAAB"
My program would have to write "AAAB" as "01000001010000010100000101000010" to memory
(it's the ASCII values in bit of AAAB that are 65656566 in bits)
Then I would like that it have a function to rewrite the content of the memory to a file.
So if memory contains again "01000001010000010100000101000010" it would write to the output file "AAAB".
int numBytes = 512;
char *pChar = (char *)malloc(numBytes);
for( int i = 0; i < numBytes; i++ ){
pChar[i] = '8';
}
Since this is C++, you can also use "new":
int numBytes = 512;
char *pChar = new char[numBytes];
for( int i = 0; i < numBytes; i++ ){
pChar[i] = '8';
}
If you want to visit every bit in the memory chunk, it looks like you need std::bitset.
char* pChunk = malloc( n );
// read in pChunk data
// iterate over all the bits.
for( int i = 0; i != n; ++i ){
std::bitset<8>& bits = *reinterpret_cast< std::bitset<8>* >( pByte );
for( int iBit = 0; iBit != 8; ++iBit ) {
std::cout << bits[i];
}
}
I'd like to printout the ASCII value of 8 consecutively bits of the memory.
The possible value for any bit is either 0 or 1. You probably want at least a byte.
char * bits = (char*) malloc(1);
Allocates 1 byte on the heap. A much more efficient and hassle-free thing would have been to create an object on the stack i.e.:
char bits; // a single character, has CHAR_BIT bits
ifstream cleartext;
cleartext.open(sometext);
The above doesn't write anything to mem. It tries to open a file in input mode.
It has ascii characters and common eof or \n, or things like this, the input would only be a textfile, so I think it should only contain ASCII characters, correct me if I'm wrong.
If your file only has ASCII data you don't have to worry. All you need to do is read in the file contents and write it out. The compiler manages how the data will be stored (i.e. which encoding to use for your characters and how to represent them in binary, the endianness of the system etc). The easiest way to read/write files will be:
// include these on as-needed basis
#include <algorithm>
#include <iostream>
#include <iterator>
#include <fstream>
using namespace std;
// ...
/* read from standard input and write to standard output */
copy((istream_iterator<char>(cin)), (istream_iterator<char>()),
(ostream_iterator<char>(cout)));
/*-------------------------------------------------------------*/
/* read from standard input and write to text file */
copy(istream_iterator<char>(cin), istream_iterator<char>(),
ostream_iterator<char>(ofstream("output.txt"), "\n") );
/*-------------------------------------------------------------*/
/* read from text file and write to text file */
copy(istream_iterator<char>(ifstream("input.txt")), istream_iterator<char>(),
ostream_iterator<char>(ofstream("output.txt"), "\n") );
/*-------------------------------------------------------------*/
The last remaining question is: Do you want to do something with the binary representation? If not, forget about it. Else, update your question one more time.
E.g: Processing the character array to encrypt it using a block cipher
/* a hash calculator */
struct hash_sha1 {
unsigned char operator()(unsigned char x) {
// process
return rc;
}
};
/* store house of characters, could've been a vector as well */
basic_string<unsigned char> line;
/* read from text file and write to a string of unsigned chars */
copy(istream_iterator<unsigned char>(ifstream("input.txt")),
istream_iterator<char>(),
back_inserter(line) );
/* Calculate a SHA-1 hash of the input */
basic_string<unsigned char> hashmsg;
transform(line.begin(), line.end(), back_inserter(hashmsg), hash_sha1());
Something like this?
char *buffer = (char*)malloc(42);
// ... put something into the buffer ...
printf("%c\n", buffer[0]);
But, since you're using C++, I wonder why you bother with malloc and such...
char* ptr = pAddressOfMemoryToRead;
while(ptr < pAddressOfMemoryToRead + blockLength)
{
char tmp = *ptr;
// temp now has the char from this spot in memory
ptr++;
}
Is this what you are trying to achieve:
char* p = (char*)malloc(10 * sizeof(char));
char* p1 = p;
memcpy(p,"abcdefghij", 10);
for(int i = 0; i < 10; ++i)
{
char c = *p1;
cout<<c<<" ";
++p1;
}
cout<<"\n";
free(p);
Can you please explain in more detail, perhaps including code? What you're saying makes no sense unless I'm completely misreading your question. Are you doing something like this?
char * chunk = (char *)malloc(256);
If so, you can access any character's worth of data by treating chunk as an array: chunk[5] gives you the 5th element, etc. Of course, these will be characters, which may be what you want, but I can't quite tell from your question... for instance, if chunk[5] is 65, when you print it like cout << chunk[5];, you'll get a letter 'A'.
However, you may be asking how to print out the actual number 65, in which case you want to do cout << int(chunk[5]);. Casting to int will make it print as an integer value instead of as a character. If you clarify your question, either I or someone else can help you further.
Are you asking how to copy the memory bytes of an arbitrary struct into a char* array? If so this should do the trick
SomeType t = GetSomeType();
char* ptr = malloc(sizeof(SomeType));
if ( !ptr ) {
// Handle no memory. Probably should just crash
}
memcpy(ptr,&t,sizeof(SomeType));
I'm not sure I entirely grok what you're trying to do, but a couple of suggestions:
1) use std::vector instead of malloc/free and new/delete. It's safer and doesn't have much overhead.
2) when processing, try doing chunks rather than bytes. Even though streams are buffered, it's usually more efficient grabbing a chunk at a time.
3) there's a lot of different ways to output bits, but again you don't want a stream output for each character. You might want to try something like the following:
void outputbits(char *dest, char source)
{
dest[8] = 0;
for(int i=0; i<8; ++i)
dest[i] = source & (1<<(7-i)) ? '1':'0';
}
Pass it a char[9] output buffer and a char input, and you get a printable bitstring back. Decent compilers produce OK output code for this... how much speed do you need?