So I have some data I convert from packet to string, in binary (datagram):
std::string Packet::packetToString()
{
//packing to one bitset
std::bitset<208> pak(std::string(std::bitset<2>(type).to_string() + std::bitset<64>(num1).to_string() + std::bitset<64>(num2).to_string() + std::bitset<64>(num3).to_string() + std::bitset<4>(state).to_string() + std::bitset<4>(id).to_string() + "000000"));
std::string temp;
std::bitset<8> tempBitset(0);
for (int i = pak.size() - 1; i >= 0; i--)
{
tempBitset[i % 8] = pak[i];
if (i % 8 == 0)
{
char t = static_cast<char> (tempBitset.to_ulong());
temp.push_back(t);
}
}
return temp;
}
Then I want to convert this string to char array (in this case char buffer[26];) and send it with SendTo("127.0.0.1", 1111, buffer, 26);
What's the problem:
Packet pak1(... data I input ... );
string packet;
packet = pak1.packetToString();
char buffer[26];
strcpy_s(buffer, packet.c_str());
Data send with this array seems to be erased in case 0x00(NULL) appears. This is caused by c_str() i guess. How can I deal with this? :)
strcpy() and strcpy_s() copy null terminated c-strings. So indeed, if there's any 0x00 char in the c_str() the copy will end.
Use std::copy() to copy the full data, regardless of the any 0x00 that you might encounter:
copy (packet.begin(), packet.end(), buffer); // hoping packet.size()<26
or with copy_n():
copy_n (packet.begin(), 26, buffer); // assuming that packet is at least 26 bytes
Related
I'm trying to write a program that parses ID3 tags, for educational purposes (so please explain in depth, as I'm trying to learn). So far I've had great success, but stuck on an encoding issue.
When reading the mp3 file, the default encoding for all text is ISO-8859-1. All header info (frame IDs etc) can be read in that encoding.
This is how I've done it:
ifstream mp3File("../myfile.mp3");
mp3File.read(mp3Header, 10); // char mp3Header[10];
// .... Parsing the header
// After reading the main header, we get into the individual frames.
// Read the first 10 bytes from buffer, get size and then read data
char encoding[1];
while(1){
char frameHeader[10] = {0};
mp3File.read(frameHeader, 10);
ID3Frame frame(frameHeader); // Parses frameHeader
if (frame.frameId[0] == 'T'){ // Text Information Frame
mp3File.read(encoding, 1); // Get encoding
if (encoding[0] == 1){
// We're dealing with UCS-2 encoded Unicode with BOM
char data[frame.size];
mp3File.read(data, frame.size);
}
}
}
This is bad code, because data is a char*, its' inside should look like this (converted undisplayable chars to int):
char = [0xFF, 0xFE, C, 0, r, 0, a, 0, z, 0, y, 0]
Two questions:
What are the first two bytes? - Answered.
How can I read wchar_t from my already open file? And then get back to reading the rest of it?
Edit Clarification: I'm not sure if this is the correct way to do it, but essentially what I wanted to do was.. Read the first 11 bytes to a char array (header+encoding), then the next 12 bytes to a wchar_t array (the name of the song), and then the next 10 bytes to a char array (the next header). Is that possible?
I figured out a decent solution: create a new wchar_t buffer and add the characters from the char array in pairs.
wchar_t* charToWChar(char* cArray, int len) {
char wideChar[2];
wchar_t wideCharW;
wchar_t *wArray = (wchar_t *) malloc(sizeof(wchar_t) * len / 2);
int counter = 0;
int endian = BIGENDIAN;
// Check endianness
if ((uint8_t) cArray[0] == 255 && (uint8_t) cArray[1] == 254)
endian = LITTLEENDIAN;
else if ((uint8_t) cArray[1] == 255 && (uint8_t) cArray[0] == 254)
endian = BIGENDIAN;
for (int j = 2; j < len; j+=2){
switch (endian){
case LITTLEENDIAN: {wideChar[0] = cArray[j]; wideChar[1] = cArray[j + 1];} break;
default:
case BIGENDIAN: {wideChar[1] = cArray[j]; wideChar[0] = cArray[j + 1];} break;
}
wideCharW = (uint16_t)((uint8_t)wideChar[1] << 8 | (uint8_t)wideChar[0]);
wArray[counter] = wideCharW;
counter++;
}
wArray[counter] = '\0';
return wArray;
}
Usage:
if (encoding[0] == 1){
// We're dealing with UCS-2 encoded Unicode with BOM
char data[frame.size];
mp3File.read(data, frame.size);
wcout << charToWChar(data, frame.size) << endl;
}
I have an identical encryption method on my client and my server:
QByteArray Server::encrypt(QString input){
std::string original = input.toStdString();
std::string encrypted = "";
std::string key = "key";
for (unsigned int temp = 0; temp < original.size(); temp++){
encrypted += original[temp] ^ ((atoi(key.c_str()) + temp) + 2) % 253;
}
QByteArray byteArray(encrypted.c_str(), encrypted.length());
return byteArray;
}
And I'm passing a QString with a length of 143 characters into the function. When I pass a QByteArray containing the encrypted text to my decrypt function (which simply reverses this XOR encryption) it will correctly decrypt the first 126 characters, but the remainder of the string will be incorrectly decrypted. If I look at the encrypted string, all of the characters after the 126th are ? instead of random characters as I would expect.
I'm not sure what's causing this. A QByteArray is an array of bytes, so the maximum value that one element can have is limited to the size of a byte, however I'm not sure why the number of elements seems to be limited to the size of a byte. Especially considering that QByteArray::size() returns an int.
When I debug the program and look at the contents of the QByteArray, elements 0-125 are okay, but the rest of the elements (the 127th element at location 126) are negative numbers - for example, instead of seeing 91 '[' as the value at a location, I see -79 / 177.
How do I go about fixing this issue, so I can run this encryption with input of a length greater than 126?
If the encryption algorithm produce the value 0 for a byte, the line
encrypted += original[temp] ^ ((atoi(key.c_str()) + temp) + 2) % 253;
won't do anything. Not only that, the line
QByteArray byteArray(encrypted.c_str(), encrypted.length());
will end up accessing out of bound memory.
Also, you have defined
std::string key = "key";
and then you are using atoi(key.c_str()). That will always return 0. Hope that's an oversight.
I suggest:
QByteArray Server::encrypt(QString input){
std::string original = input.toStdString();
size_t size = original.size();
std::vector<unsigned char> encrypted(size);
std::string key = "key";
for (unsigned int temp = 0; temp < size; temp++){
encrypted[temp] = original[temp] ^ ((atoi(key.c_str()) + temp) + 2) % 253;
}
QByteArray byteArray(encrypted.data(), size);
return byteArray;
}
Building on the solution that #R Sahu provided (it didn't quite work either but was a step in the right direction), I've managed to solve the problem. The reworked encryption method I've written is:
std::vector<unsigned char> encrypt(QString input){
std::string original = input.toStdString();
size_t size = original.size();
std::vector<unsigned char> encrypted(size);
std::string key = "key"; //integer in the actual implementation
for (unsigned int temp = 0; temp < original.size(); temp++){
encrypted[temp] = original[temp] ^ ((atoi(key.c_str()) + temp) + 2) % 253;
}
return encrypted;
}
Instead of dealing with QByteArrays, (which seem to have issues when dealing with unsigned values, despite the fact that they shouldn't) I simply return the unsigned char vector and keep it in this form. When I decrypt, I take an unsigned char vector instead of a QByteArray, and return a QString using this method to convert the unsigned chars into signed chars for display purposes:
std::string reinterpretedString(reinterpret_cast<char*>(unencryptVector.data())); //unsigned --> signed
QString output;
output = QString::fromStdString(reinterpretedString); //output is now decrypted and readable
And I use the reinterpret_cast<char*>() function whenever I have an unsigned char vector and I want to do something with it (unsigned char vectors are not common enough for functions to have overloaded implementations for them), such as store it in a char array in preparation for sending data through a socket.
Suppose I have a character array, char a[8] containing 10101010. If I store this data in a .txt file, this file has 8 bytes size. Now I am asking that how can I convert this data to binary format and save it in a file as 8 bits (and not 8 bytes) so that the file size is only 1 byte.
Also, Once I convert these 8 bytes to a single byte, Which File format should I save the output in? .txt or .dat or .bin?
I am working on Huffman Encoding of text files. I have already converted the text format into binary, i.e. 0's and 1's, but when I store this output data on a file, each digit(1 or 0) takes a byte instead of a bit. I want a solution such that each digit takes only a bit.
char buf[100];
void build_code(node n, char *s, int len)
{
static char *out = buf;
if (n->c) {
s[len] = 0;
strcpy(out, s);
code[n->c] = out;
out += len + 1;
return;
}
s[len] = '0'; build_code(n->left, s, len + 1);
s[len] = '1'; build_code(n->right, s, len + 1);
}
This is how I build up my code tree with help of a Huffman tree. And
void encode(const char *s, char *out)
{
while (*s)
{
strcpy(out, code[*s]);
out += strlen(code[*s++]);
}
}
This is how I Encode to get the final output.
Not entirely sure how you end up with a string representing the binary representation of a value,
but you can get an integer value from a string (in any base) using standard functions like std::strtoul.
That function provides an unsigned long value, since you know your value is within 0-255 range you can store it in an unsigned char:
unsigned char v=(unsigned char)(std::strtoul(binary_string_value.c_str(),0,2) & 0xff);
Writing it to disk, you can use ofstream to write
Which File format should I save the output in? .txt or .dat or .bin?
Keep in mind that the extension (the .txt, .dat or .bin) does not really mandate the format (i.e. the structure of the contents). The extension is a convention commonly used to indicate that you're using some well-known format (and in some OS/environments, it drives the configuration of which program can best handle that file). Since this is your file, it is up to you define the actual format... and to name the file with any extension (or even no extension) you like best (or in other words, any extension that best represent your contents) as long as it is meaningful to you and to those that are going to consume your files.
Edit: additional details
Assuming we have a buffer of some length where you're storing your string of '0' and '1'
int codeSize; // size of the code buffer
char *code; // code array/pointer
std::ofstream file; // File stream where we're writing to.
unsigned char *byteArray=new unsigned char[codeSize/8+(codeSize%8+=0)?1:0]
int bytes=0;
for(int i=8;i<codeSize;i+=8) {
std::string binstring(code[i-8],8); // create a temp string from the slice of the code
byteArray[bytes++]=(unsigned char)(std::strtoul(binstring.c_str(),0,2) & 0xff);
}
if(i>codeSize) {
// At this point, if there's a number of bits not multiple of 8,
// there are some bits that have not
// been writter. Not sure how you would like to handle it.
// One option is to assume that bits with 0 up to
// the next multiple of 8... but it all depends on what you're representing.
}
file.write(byteArray,bytes);
Function converting input 8 chars representing bit representation into one byte.
char BitsToByte( const char in[8] )
{
char ret = 0;
for( int i=0, pow=128;
i<8;
++i, pow/=2;
)
if( in[i] == '1' ) ret += pow;
return ret;
}
We iterate over array passed to function (of size 8 for obvious reasons) and based of content of it we increase our return value (first element in the array represents the oldest bit). pow is set to 128 because 2^(n-1)is value of n-th bit.
You can shift them into a byte pretty easily, like this:
byte x = (s[3] - '0') + ((s[2] - '0') << 1) + ((s[1] - '0') << 2) + ((s[0] - '0') << 3);
In my example, I only shifted a nibble, or 4-bits. You can expand the example to shift an entire byte. This solution will be faster than a loop.
One way:
/** Converts 8 bytes to 8 bits **/
unsigned char BinStrToNum(const char a[8])
{
return( ('1' == a[0]) ? 128 : 0
+ ('1' == a[1]) ? 64 : 0
+ ('1' == a[2]) ? 32 : 0
+ ('1' == a[3]) ? 16 : 0
+ ('1' == a[4]) ? 8 : 0
+ ('1' == a[5]) ? 4 : 0
+ ('1' == a[6]) ? 2 : 0
+ ('1' == a[7]) ? 1 : 0);
);
};
Save it in any of the formats you mentioned; or invent your own!
int main()
{
rCode=0;
char *a = "10101010";
unsigned char byte;
FILE *fp=NULL;
fp=fopen("data.xyz", "wb");
if(NULL==fp)
{
rCode=errno;
fprintf(stderr, "fopen() failed. errno:%d\n", errno);
goto CLEANUP;
}
byte=BinStrToNum(a);
fwrite(&byte, 1, 1, fp);
CLEANUP:
if(fp)
fclose(fp);
return(rCode);
}
I am reading through a buffer (char *) and i have a cursor, where i am tracking my starting position of the buffer, is there a way to copy characters 7-64 out of the buffer, or is my best bet to just loop the buffer from poistion x to position y?
The size of the destination buffer is the result of another function dynamically computed
Initializing this returns
variable-sized object 'version' may not be initialized
Relevant code parts:
int32_t size = this->getObjectSizeForMarker(cursor, length, buffer);
cursor = cursor + 8; //advance cursor past marker and size
char version[size] = this->getObjectForSizeAndCursor(size, cursor, buffer);
-
char* FileReader::getObjectForSizeAndCursor(int32_t size, int cursor, char *buffer) {
char destination[size];
memcpy(destination, buffer + cursor, size);
}
-
int32_t FileReader::getObjectSizeForMarker(int cursor, int eof, char * buffer) {
//skip the marker and read next 4 byes
cursor = cursor + 4; //skip marker and read 4
unsigned char *ptr = (unsigned char *)buffer + cursor;
int32_t objSize = (ptr[0] << 24) | (ptr[1] << 16) | (ptr[2] << 8) | ptr[3];
return objSize;
}
Move the pointer to buffer six units ahead (to get to the seventh index), and then memcpy 64-7 (57) bytes, e.g.:
const char *buffer = "foo bar baz...";
char destination[SOME_MAX_LENGTH];
memcpy(destination, buffer + 6, 64-7);
You may want to terminate the destination array so that you can work with it using standard C string functions. Note that we're adding the null character at the 58th index, after the 57 bytes that were copied over:
/* terminate the destination string at the 58th byte, if desired */
destination[64-7] = '\0';
If you need to work with a dynamically sized destination, use a pointer instead of an array:
const char *buffer = "foo bar baz...";
char *destination = NULL;
/* note we do not multiply by sizeof(char), which is unnecessary */
/* we should cast the result, if we're in C++ */
destination = (char *) malloc(58);
/* error checking */
if (!destination) {
fprintf(stderr, "ERROR: Could not allocate space for destination\n");
return EXIT_FAILURE;
}
/* copy bytes and terminate */
memcpy(destination, buffer + 6, 57);
*(destination + 57) = '\0';
...
/* don't forget to free malloc'ed variables at the end of your program, to prevent memory leaks */
free(destination);
Honestly, if you're in C++, you should really probably be using the C++ strings library and std::string class. Then you can call the substr substring method on your string instance to get the 57-character substring of interest. It would involve fewer headaches and less re-inventing the wheel.
But the above code should be useful for both C and C++ applications.
I receive values using winsock from another computer on the network. It is a TCP socket, with the 4 first bytes of the message carrying its size. The rest of the message is formatted by the server using protobuf (protocol buffers from google).
The problemn, I think, is that it seems that the values sent by the server are hex values sent as char (ie only 10 received for 0x10). To receive the values, I do this :
bytesreceived = recv(sock, buffer, msg_size, 0);
for (int i=0;i<bytesreceived;i++)
{
data_s << hex << buffer[i];
}
where data_s is a stringstream. Them I can use the ParseFromIstream(&data_s) method from protobuf and recover the information I want.
The problem that I have is that this is VERY VERY long (I got another implementation using QSock taht I can't use for my project but which is much faster, so there is no problem on the server side).
I tried many things that I took from here and everywhere on the internet (using Arrays of bytes, strings), but nothing works.
Do I have any other options ?
Thank you for your time and comments ;)
not sure if this will be of any use, but I've used a similar protocol before (first 4 bytes holds an int with the length, rest is encoded using protobuf) and to decode it I did something like this (probably not the most efficient solution due to appending to strings):
// Once I've got the first 4 bytes, cast it to an int:
int msgLen = ntohl(*reinterpret_cast<const int*>(buffer));
// Check I've got enough bytes for the message, if I have then
// just parse the buffer directly
MyProtobufObj obj;
if( bytesreceived >= msgLen+4 )
{
obj.ParseFromArray(buffer+4,msgLen);
}
else
{
// just keep appending buffer to an STL string until I have
// msgLen+4 bytes and then do
// obj.ParseFromString(myStlString)
}
I wouldn't use the stream operators. They're for formatted data and that's not what you want.
You can keep the values received in a std::vector with the char type (vector of bytes). That would essentially just be a dynamic array. If you want to continue using a string stream, you can use the stringstream::write function which takes a buffer and a length. You should have the buffer and number of bytes received from your call to recv.
If you want to use the vector method, you can use std::copy to make it easier.
#include <algorithm>
#include <iterator>
#include <vector>
char buf[256];
std::vector<char> bytes;
size_t n = recv(sock, buf, 256, 0);
std::copy(buf, buf + n, std::back_inserter(bytes));
Your question is kind of ambiguous. Let's follow your example. You receive 10 as characters and you want to retrieve this as a hex number.
Assuming recv will give you this character string, you can do this.
First of all make it null terminated:
bytesreceived[msg_size] = '\0';
then you can very easily read the value from this buffer using standard *scanf function for strings:
int hexValue;
sscanf(bytesreceived, "%x", &hexValue);
There you go!
Edit: If you receive the number in reverse order (so 01 for 10), probably your best shot is to convert it manually:
int hexValue = 0;
int positionValue = 1;
for (int i = 0; i < msg_size; ++i)
{
int digit = 0;
if (bytesreceived[i] >= '0' && bytesreceived[i] <= '9')
digit = bytesreceived[i]-'0';
else if (bytesreceived[i] >= 'a' && bytesreceived[i] <= 'f')
digit = bytesreceived[i]-'a';
else if (bytesreceived[i] >= 'A' && bytesreceived[i] <= 'F')
digit = bytesreceived[i]-'A';
else // Some kind of error!
return error;
hexValue += digit*positionValue;
positionValue *= 16;
}
This is just a clear example though. In reality you would do it with bit shifting for example rather than multiplying.
What data type is buffer?
The whole thing looks like a great big no-op, since operator<<(stringstream&, char) ignores the base specifier. The hex specifier only affects formatting of non-character integral types. For certain you don't want to be handing textual data to protobuf.
Just hand the buffer pointer to protobuf, you're done.
OK, a shot into the dark: Let's say your ingress stream is "71F4E81DA...", and you want to turn this into a byte stream { 0x71, 0xF4, 0xE8, ...}. Then we can just assemble the bytes from the character literals as follows, schematically:
char * p = getCurrentPointer();
while (chars_left() >= 2)
{
unsigned char b;
b = get_byte_value(*p++) << 8;
b += get_byte_value(*p++);
output_stream.insert(b);
}
Here we use a little helper function:
unsigned char get_byte_value(char c)
{
if ('0' <= c && c <= '9') return c - '0';
if ('A' <= c && c <= 'F') return 10 + c - 'A';
if ('a' <= c && c <= 'f') return 10 + c - 'a';
return 0; // error
}