i want to be able to store data into a data.bin.gz using using zstr (a library that use zlib). I succeed to write into the file, but i cannot read it back. Here is a short example.
std::auto_ptr<std::ostream> ofs = std::auto_ptr<std::ostream>(new zstr::ofstream(fileName));
std::string str("hello world");
ofs.get()->write(str.c_str(), 11);
std::cout << "data sent: " << str << std::endl;
std::auto_ptr<std::istream> ifs = std::auto_ptr<std::istream>(new zstr::ifstream(fileName));
std::streamsize buffSize = 11;
char* buff = new char [11];
// fill buff to see if its content change
for (int i = 0; i < 11; i++) {
buff[i] = 'A';
}
ifs.get()->read(buff, buffSize);
std::cout << std::string(buff, buff+11) << std::endl;
delete [] buff;
i fill buff with some specfic content to see if it changes when reading the stream. but it does not change.
Here is a version that does approximately what you're asking for, but using standard file streams, not the non-standard zstr library which doesn't seem essential here:
#include <iostream>
#include <fstream>
#include <memory>
#include <string>
#include <vector>
using namespace std::string_literals;
int main()
{
constexpr auto fileName = "test.bin";
{
const auto str = "hello world"s;
auto ofs = std::ofstream( fileName, std::ios::binary );
ofs.write( str.data(), str.size() );
} // ofs is closed here by RAII
auto buff = std::vector<char>(100, 'A');
auto ifs = std::ifstream( fileName, std::ios::binary );
ifs.read(buff.data(), buff.size());
std::cout << std::string(buff.data(), buff.data()+11) << '\n';
}
It outputs hello world as expected. See it live on Coliru.
Notes:
I removed the auto_ptr and added the proper scoping.
I do not manage memory manually (new/delete), which is bad form. Instead I use std::vector and std::string.
I added the std::ios::binary flag to the fstream constructors to open in binary mode, since that is what it seems you ultimately want to do. This may not be needed with the zstr library you're using.
I made the buffer larger, as if I don't know what's in the file. Then I read from it as much space as I've allocated. When printing the result, I use the "insider knowledge" that there are 11 valid bytes. An alternative would be to initialize the vector to all zeros (the default) and just print it as a string:
auto buff = std::vector<char>( 100 );
auto ifs = std::ifstream( fileName, std::ios::binary );
ifs.read(buff.data(), buff.size() - 1); // Keep one zero for null terminator
std::cout << buff.data() << '\n';
which you can also see live on Coliru.
I also modernized in a few other ways just for fun and educational purposes:
I use constexpr on a constant known at compile-time.
I use the string literal suffix s on str to create a std::string with greater concision.
I use 'almost always auto' style for declaring objects.
Use \n instead of std::endl because you don't need the extra flush (good habit to be in).
Related
Today I did a lot of research online about how to create a directory on C++
and found a lot of way to do that, some easier than others.
I tried the _mkdir function using _mkdir("C:/Users/..."); to create a folder. Note that the argument of function will be converted into a const char*.
So far, so good, but when I want to change the path, it does not work (see the code below). I have a default string path "E:/test/new", and I want to create 10 sub-folders: new1, new2, newN, ..., new10.
To do that, I concatenate the string with a number (the counter of the for-loop), converted into char using static_cast, then I transform the string using c_str(), and assign it to a const char* variable.
The compiler has no problem compiling it, but it doesn't work. It prints 10 times "Impossible create folder n". What's wrong?
I probably made a mistake when transforming the string using c_str() to a get a const char*?.
Also, is there a way to create a folder using something else? I looked at CreateDirectory(); (API) but it uses keyword like DWORD HANDLE, etc., that are a little bit difficult to understand for a no-advanced level (I don't know what these mean).
#include <iostream>
#include <Windows.h>
#include<direct.h>
using namespace std;
int main()
{
int stat;
string path_s = "E:/test/new";
for (int i = 1; i <= 10; i++)
{
const char* path_c = (path_s + static_cast<char>(i + '0')).c_str();
stat = _mkdir(path_c);
if (!stat)
cout << "Folder created " << i << endl;
else
cout << "Impossible create folder " << i << endl;
Sleep(10);
}
return 0;
}
If your compiler supports c++17, you can use filesystem library to do what you want.
#include <filesystem>
#include <string>
#include <iostream>
namespace fs = std::filesystem;
int main(){
const std::string path = "E:/test/new";
for(int i = 1; i <= 10; ++i){
try{
if(fs::create_directory(path + std::to_string(i)))
std::cout << "Created a directory\n";
else
std::cerr << "Failed to create a directory\n";\
}catch(const std::exception& e){
std::cerr << e.what() << '\n';
}
}
return 0;
}
The problem is that (path_s + static_cast<char>(i + '0')) creates a temporary object. One whose life-time ends (and is destructed) just after c_str() has been called.
That leaves you with a pointer to a string that no longer exist, and using it in almost any way will lead to undefined behavior.
Instead save the std::string object, and call c_str() just when needed:
std::string path = path_s + std::to_string(i);
_mkdir(path.c_str());
Note that under Linux, you can use the mkdir command as follows:
#include <sys/stat.h>
...
const int dir_err = mkdir("foo", S_IRWXU | S_IRWXG | S_IROTH | S_IXOTH);
if (-1 == dir_err){
printf("Error creating directory!n");
exit(1);
}
More information on it can be gleaned from reading man 2 mkdir.
I am working in a section of code with very high performance requirements. I need to perform some formatted string operations, but I am trying to avoid memory allocations, even internal library ones.
In the past, I would have done something similar to the following (assuming C++11):
constexpr int BUFFER_SIZE = 200;
char buffer[BUFFER_SIZE];
int index = 0;
index += snprintf(&buffer[index], BUFFER_SIZE-index, "Part A: %d\n", intA);
index += snprintf(&buffer[index], BUFFER_SIZE-index, "Part B: %d\n", intB);
// etc.
I would prefer to use all C++ methods, such as ostringstream, to do this instead of the old C functions.
I realize I could use std::string::reserve and std::ostringstream to procure space ahead of time, but that will still perform at least one allocation.
Does anyone have any suggestions?
Thanks ahead of time.
Does anyone have any suggestions?
Yes, use std::ostrstream. I know it is deprecated. But I find it useful for output to static buffers. No possibility of memory leaks if an exception occurs.
No allocation of memory at all.
#include <strstream> // for std::ostrstream
#include <ostream> // for std::ends
// :
constexpr int BUFFER_SIZE = 200;
char buffer[BUFFER_SIZE];
std::ostrstream osout(buffer, sizeof(buffer));
osout << "Part A: " << intA << "Part B: " << intB << std::ends;
My thanks to all that posted suggestions (even in the comments).
I appreciate the suggestion by SJHowe, being the briefest solution to the problem, but one of the things I am looking to do with this attempt is to start coding for the C++ of the future, and not use anything deprecated.
The solution I decided to go with stems from the comment by Remy Lebeau:
#include <iostream> // For std::ostream and std::streambuf
#include <cstring> // For std::memset
template <int bufferSize>
class FixedBuffer : public std::streambuf
{
public:
FixedBuffer()
: std::streambuf()
{
std::memset(buffer, 0, sizeof(buffer));
setp(buffer, &buffer[bufferSize-1]); // Remember the -1 to preserve the terminator.
setg(buffer, buffer, &buffer[bufferSize-1]); // Technically not necessary for an std::ostream.
}
std::string get() const
{
return buffer;
}
private:
char buffer[bufferSize];
};
//...
constexpr int BUFFER_SIZE = 200;
FixedBuffer<BUFFER_SIZE> buffer;
std::ostream ostr(&buffer);
ostr << "PartA: " << intA << std::endl << "PartB: " << intB << std::endl << std::ends;
Im trying to read blocks of data from a file, but I couldn't know how to ignore the newline character when I use istream::read.
Im aware that I can use for loop to load the characters to a cstring one by one with condition to ignore new lines character, but I hope there is clever way to solve this problem.
My intention to avoid using strings or vectors.
#include <iostream>
#include <fstream>
#include <cstring>
void readIt(char* fileName) {
std::ifstream seqsFile;
seqsFile.open(fileName) ;
if (seqsFile.fail()) {
std::cout << "Failed in opening: " << fileName << std::endl;
std::exit(1);
}
seqsFile.seekg(84);
char *buffer;
buffer = new char [7];
seqsFile.read(buffer, 7);
buffer[7] = 0;
std::cout << buffer << std::endl;
}
int main(int argc, char** argv) {
readIt(argv[1]);
return 0;
}
file:
gsi|33112219|sp|O
GACATTCTGGTGGTGGACTCGGAGGCATGATAGCAGGTGCAGCTGGTGCAGCCGCAGCAGCTTATGGAGC
GCAGCAGCTTATGGAGC
current output:
GAGC
GC
desired output:
GAGCGCA
modified version:
void readIt(char* fileName) {
std::ifstream seqsFile;
seqsFile.open(fileName) ;
if (seqsFile.fail()) {
std::cout << "Failed in opening: " << fileName << std::endl;
std::exit(1);
}
seqsFile.seekg(84);
char *buffer;
buffer = new char [7];
char next ;
for ( int i = 0 ; i < 7; i++) {
seqsFile.get(next);
if (next=='\n') {
i--;
continue;
}
buffer[i] = next;
}
buffer[7]=0;
std::cout << buffer << std::endl;
}
Your program has undefined behavior since you are modifying buffer using an out of range index. You have:
buffer = new char [7]; // Allocating 7 chars.
seqsFile.read(buffer, 7); // Reading 7 chars. OK.
buffer[7] = 0; // 7 is an out of range index. Not OK.
Allocate memory for at least 8 chars.
buffer = new char [8];
Also, when you intend to read the contents of a file using istream::read, it is recommended that you open the file in binary mode.
seqsFile.open(fileName, std::ios_base::binary) ;
Well, you can not tell not to read newlines - they will appear in your buffer variable anyway and you have to handle it.
Also, you have to fix the buffer size, as R Sahu mentioned
Regarding your question, i can suggest following snippet:
while ((index = strlen(buffer)) < 7)
{
seqsFile >> &buffer[index];
}
strlen here will return size of buffer upto /0 or newline character as well
You didn't tell what to do with whitespaces, so they will be ignored as well
I am writing a program in C++ using VS2010 to read a text file and extract certain information from it. I completed the code using filestream and it worked well. However now I am asked to map the file to memory and use it rather than the file operations.
I am absolutely a newbie in case of memory mapping. A part of the code I have written is as follows.
boost::iostreams::mapped_file_source apifile;
apifile.open(LogFileName,LogFileSize);
if(!apifile.is_open())
return FILE_OPEN_ERROR;
// Get pointer to the data.
PBYTE Buffer = (PBYTE)apifile.data();
while(//read till end of the file)
{
// read a line and check if it contains a specific word
}
While using filestream I would have used eof and getline and string::find for performing the operations. But I don't have any idea on how to do it using memory mapped file.
EDIT 1:
int ProcessLogFile(string file_name)
{
LogFileName = file_name;
apifile.open(LogFileName);//boost::iostreams::mapped_file_source apifile(declared globally)
streamReader.open(apifile, std::ios::binary);//boost::iostreams::stream <boost::iostreams::mapped_file_source> streamReader(declared globally)
streamoff Curr_Offset = 0;
string read_line;
int session_id = 0;
int device_id = 0;
while(!streamReader.eof())
{
\\COLLECT OFFSETS OF DIFFERENT SESSIONS
}
streamReader.close();
}
This function worked and i got the offsets to the required structure.
Now after calling this function, I call yet another function as follows:
int GetSystemDetails()
{
streamReader.open(apifile, std::ios::binary);
string read_line;
getline(streamReader,read_line);
cout << "LINE : " << read_line;
streamReader.close();
}
I don't get any data in read_line. Is that memory mapping only for a single function? How can I use the same memory mapped file across different functions?
I agree with people questioning the use of a mmap if you just read through the file sequentially.
boost::mapped_file_source models a Device. There's two approaches to use such a Device:
use it raw (using data() as you try)
using a stream wrapper
1. Using the raw Device source
You can use the mapped_file_source to report the actual size (m.data()+m.size()).
Let's take a sample to count lines:
#include <boost/iostreams/device/mapped_file.hpp> // for mmap
#include <algorithm> // for std::find
#include <iostream> // for std::cout
#include <cstring>
int main()
{
boost::iostreams::mapped_file mmap("input.txt", boost::iostreams::mapped_file::readonly);
auto f = mmap.const_data();
auto l = f + mmap.size();
uintmax_t m_numLines = 0;
while (f && f!=l)
if ((f = static_cast<const char*>(memchr(f, '\n', l-f))))
m_numLines++, f++;
std::cout << "m_numLines = " << m_numLines << "\n";
}
You could possibly adapt this. I have several more complicated parsing examples based on memory mapped files:
Fast textfile reading in c++
Note how in the updates you can see that indeed, open()+read() was faster than the memory map due the sequential access nature
How to parse space-separated floats in C++ quickly?
2. Wrapping the source device in a istream
This gives you all the usual stream-based operations of c++ standard streams, so you can detect the end of the file like you would always:
#include <boost/iostreams/device/mapped_file.hpp> // for mmap
#include <boost/iostreams/stream.hpp> // for stream
#include <algorithm> // for std::find
#include <iostream> // for std::cout
#include <cstring>
int main()
{
using boost::iostreams::mapped_file_source;
using boost::iostreams::stream;
mapped_file_source mmap("test.cpp");
stream<mapped_file_source> is(mmap, std::ios::binary);
std::string line;
uintmax_t m_numLines = 0;
while (std::getline(is, line))
{
m_numLines++;
}
std::cout << "m_numLines = " << m_numLines << "\n";
}
I have a ostringstream variable which contains some data.
I want to get set a char * pointer to the data inside the ostringstream.
If I do the following:
std::ostringstream ofs;
.....
const char *stam = (ofs.str()).c_str();
There is a copy of the content of the string in ofs.
I want to get a pointer to that content without a copy.
Is there a way to do so?
This actually answers the question... took a while but I wanted to do it for the same reasons (efficiency vs portability is fine for my situation):
class mybuf : public std::stringbuf {
public:
// expose the terribly named end/begin pointers
char *eback() {
return std::streambuf::eback();
}
char *pptr() {
return std::streambuf::pptr();
}
};
class myos : public std::ostringstream {
mybuf d_buf;
public:
myos() {
// replace buffer
std::basic_ostream<char>::rdbuf(&d_buf);
}
char *ptr();
};
char *myos::ptr() {
// assert contiguous
assert ( tellp() == (d_buf.pptr()-d_buf.eback()) );
return d_buf.eback();
}
int main() {
myos os;
os << "hello";
std::cout << "size: " << os.tellp() << std::endl;
std::string dat(os.ptr(),os.tellp());
std::cout << "data: " << dat << std::endl;
}
This points to, yet again, the deeper, underlying problem with the standard library - a confusion between contracts and "safety". When writing a messaging service, I need a library with efficient contracts... not safety. Other times, when writing a UI, I want strong safety - and cares less about efficiency.
Although you can't get a pointer to the character buffer in the ostringstream, you can get access to its characters without copying them if you switch to using stringstream. A stringstream allows input and output (reading from and writing to the stream), whereas ostringstream allows only output (writing to the stream). Example:
std::stringstream ss;
ss << "This is a test.";
// Read stringstream from index 0. Use different values to look at any character index.
ss.seekg(0);
char ch;
while (ss.get(ch)) { // loop getting single characters
std::cout << ch;
}
ss.clear(); // Clear eof bit in case you want to read more from ss
This site has a pretty good overview of stringstreams and what you can do with them.