The reader and writer
#include<string>
#include<fstream>
#include<memory>
class BinarySearchFile{
BinarySearchFile::BinarySearchFile(std::string file_name){
// concatenate extension to fileName
file_name += ".dat";
// form complete table data filename
data_file_name = file_name;
// create or reopen table data file for reading and writing
binary_search_file.open(data_file_name, std::ios::binary); // create file
if(!binary_search_file.is_open()){
binary_search_file.clear();
binary_search_file.open(data_file_name, std::ios::out | std::ios::binary);
binary_search_file.close();
binary_search_file.open(data_file_name), std::ios::out | std::ios::in | std::ios::binary | std::ios::ate;
}
std::fstream binary_search_file;
void BinarySearchFile::writeT(std::string attribute){
if(binary_search_file){
binary_search_file.write(reinterpret_cast<char *>(&attribute), attribute.length() * 2);
}
}
std::string BinarySearchFile::readT(long filePointerLocation, long sizeOfData)
{
if(binary_search_file){
std::string data;
data.resize(sizeOfData);
binary_search_file.seekp(filePointerLocation);
binary_search_file.seekg(filePointerLocation);
binary_search_file.read(&data[0], sizeOfData);
return data;
}
};
The reader call
while (true){
std::unique_ptr<BinarySearchFile> data_file(new BinarySearchFile("classroom.dat"));
std::string attribute_value = data_file->read_data(0, 20);
}
The writer call
data_file->write_data("packard ");
The writer writes a total of 50 bytes
"packard 101 500 "
The reader is to read the first 20 bytes and the result is "X packard X" where X represents some malformed bytes of data. Why is the data read back in x-number of bytes corrupt?
You can't simply write data out by casting it's address to a char* and hoping to get anything useful. You have to define the binary format you want to use, and implement it. In the case of std::string, this may mean outputing the length in some format, then the actual data. Or in the case where fixed length fields are needed, forcing the string (or a copy of the string) to that length using std::string::resize, then outputting that, using std::string::data() to get your char const*.
Reading will, of course, be similar. You'll read the data into a std::vector<char> (or for fixed length fields, a char[]), and parse it.
binary_search_file.write(reinterpret_cast<char *>(&attribute), attribute.length() * 2);
It is incorrect to cast std::string to char* if you need char* you must use attribute.c_str().
std::string apart from string pointer contains other data members, for example, allocator, your code will write all of that data to file. Also I don't see any reason to multiply string length by 2. +1 makes sense if you want to output terminating zero.
Related
We have a file which is present in a data store (S3) which contains data in the form of byte[] (uploaded using Java language).
Now when i download the file, the data i get is in the form of std::basic_streambuf (Ideally it should also be having bytes). Now i want to send this data to another API which takes uint8_t* as the input.
What is the way to do so? Is it making any sense to even do that?
I tried this:
// Assume streambuf is:
std::streambuf *buf;
std::stringstream ss;
ss << buf;
// Solution1
const std::string output1 = ss.str();
cout<<output1;
// This prints the whole data with some weird characters (i think weird characters are valid because data is in byte form). Upon converting output1 to uint8_t*, the final data contains only 20 characters/bytes.
// Solution2
uint8_t* finalString;
ss >> finalString;
cout<<finalString;
// This prints only initial 20 characters/bytes and the remaining part is skipped.
So with both Solution1 and Solution2, ultimate goal of getting uint8_t* of full data could not be achieved. What is the suggested way to do so?
You have to read your data out of the buffer (since the buffer itself can be streaming the data in as it's available). One possible implementation is something like this:
vector<uint8_t> bytes;
do {
bytes.push_back(buf->sgetc());
} while(buf->snextc() != EOF);
// your data is in bytes.data() of type uint8_t*
Of course if you know the number of bytes from the beginning instead of having to read the buffer to find out, simply pre-allocate the vector beforehand.
I have a char array[] and is like following:
// MessageBox
char xcode[] = "\x31\xc9\x64\x8b\x41\x30\x8b\x40\xc\x8b\x70\x14\xad\x96\xad\x8b\x58\x10\x8b\x53\x3c\x1\xda\x8b\x52\x78\x1\xda\x8b\x72\x20\x1\xde\x31\xc9\x41\xad\x1\xd8\x81\x38\x47\x65\x74\x50\x75\xf4\x81\x78\x4\x72\x6f\x63\x41\x75\xeb\x81\x78\x8\x64\x64\x72\x65\x75\xe2\x8b\x72\x24\x1\xde\x66\x8b\xc\x4e\x49\x8b\x72\x1c\x1\xde\x8b\x14\x8e\x1\xda\x31\xc9\x53\x52\x51\x68\x61\x72\x79\x41\x68\x4c\x69\x62\x72\x68\x4c\x6f\x61\x64\x54\x53\xff\xd2\x83\xc4\xc\x59\x50\x51\x66\xb9\x6c\x6c\x51\x68\x33\x32\x2e\x64\x68\x75\x73\x65\x72\x54\xff\xd0\x83\xc4\x10\x8b\x54\x24\x4\xb9\x6f\x78\x41\x0\x51\x68\x61\x67\x65\x42\x68\x4d\x65\x73\x73\x54\x50\xff\xd2\x83\xc4\x10\x68\x61\x62\x63\x64\x83\x6c\x24\x3\x64\x89\xe6\x31\xc9\x51\x56\x56\x51\xff\xd0";
Then i had inserted all this content of variable above into a file (file with UTF-8 format and content without the "") and tried load this way:
ifstream infile;
infile.open("shellcode.bin", std::ios::in | std::ios::binary);
infile.seekg(0, std::ios::end);
size_t file_size_in_byte = infile.tellg();
char* xcode = (char*)malloc(sizeof(char) * file_size_in_byte);
infile.seekg(0, std::ios::beg);
infile.read(xcode, file_size_in_byte);
printf("%s\n", xcode); // << prints content of xcode after load from file
if (infile.eof()) {
size_t bytes_really_read = infile.gcount();
}
else if (infile.fail()) {
}
infile.close();
I'm seeing some strange characters in end of text see:
What is need to fix it?
The issue is that the printf format specifier "%s" requires that the string is null-terminated. In your case, the null-terminator just happens to be after those characters you're seeing, but nothing guarantees where the null is unless you put one there.
Since you're using C++, one way to print the characters is to use the write() function available for streams:
#include <iostream>
//...
std::cout.write(xcode, file_size_in_bytes);
The overall point is this -- if you have a character array that is not null-terminated and contains data, you must either:
Put the null in the right place before using the array in functions that look for the null-terminator or
Use functions that state how many characters to process from the character array.
The answer above uses item 2.
I'm trying to split and combine a binary file, for a reason not related to this question i'm using protobuf to store the files char* in a protobuf byte type
The code to serialize char* looks like this:
char* buffer = new char[buffer_size];
ifstream fileStream;
fileStream.open(fileName,ios::in | ios::binary);
//stuff here
Data *data = new Data(); // Protobuf Constructor
fileStream.read(buffer, buffer_size);
data->set_payload(buffer);
data->set_piece_no(piece_no);
.proto file :
syntax = "proto3";
message Data {
int32 piece_no = 1;
bytes payload = 2;
}
Then i try to combine all the pieces like so :
ofstream fileOutput;
fileOutput.open("out.bin", ios::out | ios::binary);
fileOutput << data->payload();
But sadly this doesn't work and the binary file generated is substantially smaller than the original.
I then suspect that the bytes could have null characters \0, and as a result the bytes have actually been truncated.
To test out my hypothesis i do the following:
Data *data = new Header();
data->set_payload("hel\0lo");
data->set_piece_no(piece_no);
ofstream fileOutput;
fileOutput.open("out.bin",ios::out | ios::binary);
fileOutput << data->payload();
Opening the binary file in a text editor (vscode) shows the following:
hel
But the following code:
string data("hel\0lo",6);
ofstream fileOutput;
fileOutput.open("out.bin", ios::out | ios::binary);
fileOutput << data;
Shows the following:
hel?lo
How can i output exactly what i inputted into protobuf, without any truncation because of arbitrary null bytes ?
If you pass a string literal, then it will treat it as such and only read until the first null terminator.
Instead you can pass a std::string directly as in your last example.
See under "Singular String Fields (proto3)" in https://developers.google.com/protocol-buffers/docs/reference/cpp-generated#oneof
Im having problems writing string into a binary file. This is my code:
ofstream outfile("myfile.txt", ofstream::binary);
std::string text = "Text";
outfile.write((char*) &text, sizeof (string));
outfile.close();
Then, I try to read it,
char* buffer = (char*) malloc(sizeof(string));
ifstream infile("myfile.txt", ifstream::binary);
infile.read(buffer, sizeof (prueba));
std::string* elem = (string*) buffer;
cout << *elem;
infile.close();
I just cant get it to work. I am sorry, I am just desperate. Thank you!
To write a std::string to a binary file, you need to save the string length first:
std::string str("whatever");
size_t size=str.size();
outfile.write(&size,sizeof(size));
outfile.write(&str[0],size);
To read it in, reverse the process, resizing the string first so you will have enough space:
std::string str;
size_t size;
infile.read(&size, sizeof(size));
str.resize(size);
infile.read(&str[0], size);
Because strings have a variable size, unless you put that size in the file you will not be able to retrieve it correctly. You could rely on the '\0' marker that is guaranteed to be at the end of a c-string or the equivalent string::c_str() call, but that is not a good idea because
you have to read in the string character by character checking for the null
a std::string can legitimately contain a null byte (although it really shouldn't because calls to c_str() are then confusing).
the line
outfile.write((char*) &text, sizeof (string));
is not correct
sizeof(string) doesn't return the length of the string, it returns the sizeof the string type in bytes.
also do not cast text to char* using a C cast, you can get hold of the char* by using the appropriate member function text.c_str()
you can simply write
outfile << text;
instead.
Why are you using pointers to std::string class?
You should not use sizeof with std::string, as it returns the size of the std::string object, and not the real size of the string inside.
You should try:
string text = "Text";
outfile.write(text.c_str(), text.size());
or
outfile << text;
Should probably also use c_str() to get the char pointer too, instead of that straight crazy cast.
Your code is wrong wrong way you are using to write & read the file
and file extension error you are trying to read text file .txt
correct code
Write to file
std::string text = "Text";
ofstream outfile("myfile.dat", ofstream::binary | ios::out);
outfile.write(&text,sizeof (string));//can take type
outfile.write(&text,sizeof (text));//can take variable name
outfile.close();
reading file
char* buffer = (char*) malloc(sizeof(string));
ifstream infile("myfile.dat", ifstream::binary | ios::in);
infile.read(buffer, sizeof (prueba));
std::string* elem = (string*) buffer;
cout << *elem;
infile.close();
Try This it will work
I had the same problem. I found the perfect answer here: Write file in binary format
Key issues: use string::length to get the length of the string when writing out and use resize() before reading the string. And both for reading and writing, use mystring.c_str() instead the string itself.
Try this code snippet.
/* writing string into a binary file */
fstream ifs;
ifs.open ("c:/filename.exe", fstream::binary | fstream::in | fstream::out);
if (ifs.is_open())
{
ifs.write("string to binary", strlen("string to binary"));
ifs.close();
}
Here is a good example.
I'm new to C++, I have an image named "test.jpg", i convert it to base64 and decode it again like this:
std::ifstream inputFile;
inputFile.open("test.jpg",std::ios::binary);
std::filebuf* pbuf = inputFile.rdbuf();
inputFile.seekg (0, ios::end);
int length = inputFile.tellg();
// allocate memory to contain file data
char* buffer=new char[length];
// get file data
pbuf->sgetn (buffer,length);
inputFile.close();
CBase64 base64;
string encodedData = base64.base64_encode((unsigned char*)buffer,length);
delete[] buffer;
string decodedData = base64.base64_decode(encodedData);
ofstream outPutFile;
outPutFile.open("test2.jpg",ios::binary | ios::out);
outPutFile.write(decodedData.c_str(), decodedData.length());
outPutFile.close();
the "test2.jpg" has exact same size as "test.jpg"(the original file) but, i can't open it.
i couldn't find what is the problem.
i got it working. i just replaced:
outPutFile.open("test2.jpg",ios::binary | ios::out);
with
outPutFile.open("test2.jpg", ios_base::out | ios_base::binary);
std::string path = "file.txt";
std::string cfgString = "data";
std::ofstream output(path.c_str(), ios_base::out | std::ios::binary);
if (output.is_open()) {
output.write(cfgString.data(), cfgString.length());
}
output.close();
Apparently, there is no superficial problem with your file writing logic even though there are some irregularities. The biggest problem is in your main logic.
The program seems to be simple program of copying a file. What you are doing is reading a file, converting its data to base64 string and then again decoding the data to std::string. Now one small problem. Conversion of base64 string cannot be successfully done into a null terminated ANSI string for obvious reasons that any 0 in decoded binary data will terminate the string prematurely. Secondly you are opening a file in binary mode to write but trying to write std::string in the file. But that doesn't matter as you data has already been corrupted in your previous operation.
To solve this, you can simply use file copying example as this or make sure you write only binary data with care to your output file which means read in binary from input file and write to output file the same buffer. No base64 encoding decoding is required.
it looks like you forgot to write
inputFile.seekg (0, ios::beg);
after getting file length. it means you try to read from the end of the file instead of its beginning.