Internal buffer used by standard input stream (pubsetbuf) - c++

I'm trying to set the internal buffer of an input stream, but my implementation in C++17 doesn't implement pubsetbuf() for istringstream.
I've tried some other techniques, but they are slow or copy the original buffer. I'm looking for a fast method that doesn't do any copying.
It's closely related to this question about an output stream:
Setting the internal buffer used by a standard stream (pubsetbuf)
I've followed it closely, but the buffer for the input stream remains uninitialised/empty.
// Modified template from the other question about an output stream.
// This is for an input stream, but I can't get it to work.
template <typename char_type>
struct istreambuf : public std::basic_streambuf<char_type, std::char_traits<char_type> >
{
istreambuf(char_type* buffer, std::streamsize buffer_length)
{
// Set the "put" pointer to the start of the buffer and record its length.
this->setp(buffer, buffer + buffer_length);
}
};
int main()
{
ifstream infile(FILENAME, std::ifstream::binary);
if (!infile.is_open())
{
cerr << endl << "Failed to open file " << FILENAME << endl;
return 0;
}
// Works, but slow.
//istringstream local_stream;
//local_stream << infile.rdbuf();
// Works, but creates a copy.
//istringstream local_stream(&buffer[0]);
// Works, but creates a copy.
//local_stream.str(&buffer[0]);
// Read entire file into buffer.
infile.seekg(0, std::ios::end);
streampos length = infile.tellg();
infile.seekg(0, std::ios::beg);
vector<char> buffer(length);
//char* buffer = new char[length];
infile.read(&buffer[0], length);
// Doesn't work, but should point to original.
// It returns "this" (does nothing).
//local_stream.rdbuf()->pubsetbuf(&buffer[0], length);
// Works, but deprecated in C++98.
//std::istrstream local_stream(&buffer[0]);
//local_stream.rdbuf()->pubsetbuf(&buffer[0], length);
// I followed the example in the other question about an output stream,
// but I modified for an input stream. I can't get it to work. Any ideas?
istreambuf<char> istream_buffer(&buffer[0], length);
istream local_stream(&istream_buffer);
string str1, str2;
while (local_stream >> str1 && local_stream >> str2)
{
. . .
}
}

I've solved it! Spot the difference.
template <typename char_type>
struct istreambuf : public std::basic_streambuf<char_type, std::char_traits<char_type> >
{
istreambuf(char_type* buffer, std::streamsize buffer_length)
{
// Set the "put" pointer to the start of the buffer and record its length.
//this->setp(buffer, buffer + buffer_length);
// Set the "get" pointer to the start of the buffer, the next item, and record its length.
this->setg(buffer, buffer, buffer + buffer_length);
}
};
I needed to set the "get" pointer, not the "put" pointer. It works well now.

Related

Stuck on a decoding program

For my C++ class our latest lab is to write a decoding program like so:
Write a program to read a file containing an encrypted message, which
must be decoded and printed on the screen. Use the following key to
decode: input text : abcdefghijklmnopqrstuvwxyz decoded text:
iztohndbeqrkglmacsvwfuypjx
That means each 'a' in the input text should be replaced with an 'i',
each 'b' with a 'z' and so forth. Punctuation and space should be
kept as is. You will notice that all the letters in the text are
lowercase, so the second step will be to fix the captalization. First
letter of each sentence should be capitalized. Print the decoded
text to the screen. You must use an object-oriented approach on this
lab. The specification of class Message is given in header file
Message.h. You need to implement each of the member functions in the
Message.cpp, which you will turn in. The main function is also given
so you can test your class, but you don't have to turn it in. I have
added in the class definition in "Message.h" that explain how to
implement each member function. Constructor: Should open the text
file and determine its size. To do this, call getFileSize() which is
implemented in "Message.h". Check for errors when opening the input
file and don't forget to close it at the end. If there is an error,
such as the file can't be located, make sure to set length to zero.
Otherwise, use the file size to allocate space for message.
Destructor: should free the space allocated for message.
decode: decodes the message according to the given key.
fixCapitalization: capitalizes the first letter of each sentence.
dump: prints the content of message on the screen
isEmpty: returns whether message is empty of not.
You only have to turn in message.cpp. You have to make sure it works
with the header file that I'm providing, since it will be used to
compile your message.cpp.
The header file provided was:
class Message
{
private:
char *message; // holds the message
int length; // holds the the message length
static const short ALPHABET_SIZE = 26;
char code[ALPHABET_SIZE]; // holds the cypher alphabet
// iztohndbeqrkglmacsvwfuypjx
// ex: an 'a' in the original message should be converted to 'i', 'b' should be converted to 'z' and so forth
// returns the input file size in bytes
std::streamsize getFileSize(std::fstream &file) const
{
std::streamsize fsize = 0;
file.seekg (0, std::ios::end);
fsize = file.tellg();
file.seekg (0, std::ios::beg); // moves file pointer back to the beginning
return fsize;
}
public:
/*
* This constructor tries to open the file whose name is passed
* to it in filename. If file opens successfully, calls function
* getFileSize to determine how many bytes should be allocated
* for the message. Allocates space for message and reads the
* content from the file into it. Closes the file at the end.
* Member variable length should be set to the file size.
* If file cannot be found, length should be set to zero.
*/
Message(std::string filename);
// The destructor frees the space allocated to message
virtual ~Message();
// Decodes the message
void decode();
// Capitalizes first letter in each sentence
void fixCapitalization();
// Prints the content of message on the screen
void dump() const;
// Returns true if the message is empty
bool isEmpty() const;
};
What I'm really stuck on is how to use a constructor to open the file, and read the size of it.
I can't even figure out how to open the file with the constructor at all.
I tried to use ifstream in my constructor like so:
Edited 2/24/17 10:36 a.m.:
#include <iostream>
#include <fstream>
#include <string>
#include "Message.h"
using namespace std;
int main()
{
// create a message object with the content of Encrypted.txt
Message m ("Encrypted.txt");
if (m.isEmpty())
{
std::cout << "Could not read message";
return EXIT_FAILURE;
}
std::cout << "Original message: " << std::endl;
m.dump();
std::cout << std::endl << std::endl;
m.decode();
m.fixCapitalization();
std::cout << "Decoded message: " << std::endl;
m.dump();
std::cout << std::endl << std::endl;
return EXIT_SUCCESS;
}
Message::Message(std::string filename)
{
fstream infile;
infile.open(filename.c_str(), std::ifstream::in);
// read contents
// close the file
if (!infile)
{
length=0;
}else
{
// getFileSize and allocate so much space for message
std::streamsize length = getFileSize(infile);
message = new char[length + 1];
//copy contents from file to message
for (int i = 0; i < length + 1; i++)
{
infile >> new char[i];
}
}
infile.close();
}
Message::~Message()
{
}
void Message::decode()
{
}
void Message::fixCapitalization()
{
}
void Message::dump() const
{
cout << message;
}
bool Message::isEmpty() const
{
if (length == 0)
{
return true;
}else
return false;
}
This is what I've put together so far. When I run the program it tells me "Could not read message" so I assume that the file was not opened properly. Can someone check over my code for me and tell me where I went wrong?
char *message = new char;
*message = message[length];
allocates space for one char, and then reads the invalid location message[length], which is undefined.
You need to allocate length + 1 characters.
I would write it like this:
Message::Message(std::string filename)
: message(nullptr),
length(0)
{
std::ifstream infile(filename);
if (infile)
{
length = getFileSize(infile);
message = new char[length + 1];
// ...
}
}
The destructor of std::ifstream takes care of closing the stream if it was open, so you don't need to do that by hand. (Tell your teacher about this; they seem to be stuck in a C mindset.)
Side note: a more useful getFileSize would restore the previous read position and not assume that it was at the beginning:
std::streamsize getFileSize(std::fstream &file) const
{
std::streamsize old_pos = file.tellg();
file.seekg (0, std::ios::end);
std::streamsize fsize = file.tellg();
file.seekg (old_pos, std::ios::beg); // moves file pointer back to the previous position
return fsize;
}
This should help you:
Message::Message(std::string filename)
{
std::ifstream infile;
infile.open(filename.c_str(), std::ifstream::in);
// read contents & initialize other member variables.
if (infile.is_open()) {
// getFileSize and allocate so much space for message
length = getFileSize(infile);
message = new char[length + 1];
//copy contents from file to message
}
// close the file finally
infile.close()
}
EDIT for usage
int main () {
Message m("Encrypted.txt");
// other operations on m
return 0;
}

Istream to string conversion with \n characters in C++

how can I convert istream to string, when my istream also includes newline characters and I don't want to escape whitespaces?
Thank you.
If you mean how to copy the whole std::istream into a std::string then there are many ways.
Here is one:
int main()
{
// here is your istream
std::ifstream ifs("test.txt");
// copy it to your string
std::string s;
for(char c; ifs.get(c); s += c) {}
// display
std::cout << s << '\n';
}
You can just allocate a string large enough for your whole file and read it at once:
ifstream fd(filename); // open your stream (here a file stream)
if (!fd)
exit(1);
fd.seekg(0, ios_base::end); // go to end of file
size_t filesize = fd.tellg(); // dtermine size to allocate
fd.seekg(0, ios_base::beg); // go to the begin of your file
string s; // create a new string
s.resize(filesize+1); // reserve enough space to read
fd.read(&s[0], filesize); // read all the file at one
size_t bytes_read = fd.gcount(); // it could be than less bytes are read
s.resize(bytes_read); // adapt size
You can use a istreambuf_iterator like
#include <iostream>
#include <string>
#include <fstream>
int main()
{
std::ifstream ifile("test.txt"); // open
std::string str(std::istreambuf_iterator<char>(ifile), {}); // initialize
std::cout << str; // display
}

How can I assign a std::string from a char * [duplicate]

This question already has answers here:
How do I read an entire file into a std::string in C++?
(23 answers)
Closed 8 years ago.
This may be trivial, but I'm new to C++ and an getting confused here.
I have the method:
bool load_database_file( const std::string& filename, std::string& contents ) {
std::ifstream is (filename, std::ifstream::binary);
if (is) {
// get length of file:
is.seekg (0, is.end);
int length = (int)is.tellg();
is.seekg (0, is.beg);
char * buffer = new char [length];
std::cout << "Reading " << length << " characters... ";
// read data as a block:
is.read (buffer, length);
if (is)
std::cout << "all characters read successfully.";
else
std::cout << "error: only " << is.gcount() << " could be read";
is.close();
// ...buffer contains the entire file...
std::string str(buffer);
contents = str;
delete[] buffer;
}
return true ;
}
where I would like to read a file and assign it's contents to contents so that it can be read by the calling function.
My issue is that after this function is run, I see that only the first character of buffer was copied to contents.
How can I copy/convert the entire contents of buffer (a char *) to contents (a std::string).
std::string str(buffer);
Should be:
std::string str(buffer, buffer+length);
Otherwise, how can the constructor know how many bytes to allocate/copy?
By the way, your code is needlessly clumsy. Why not read directly into the string's buffer rather than using a separate buffer that you have to allocate and free just to hold the data before you allocate another buffer?

Visual Studio std::stringstream pubsetbuf does not work

pubsetbuf member of std::stringbuf is not working at all in Visual Studio 2010!
The code:
char *FileData = ... ;
unsigned long long FileDataLen = ... ;
std::stringstream *SS = new std::stringstream(std::stringstream::in | std::stringstream::out);
SS->rdbuf()->pubsetbuf( FileData, (std::streamsize)FileDataLen );
pubsetbuf does nothing in Visual Studio!!!
Workaround #1:
std::stringstream *SS = new std::stringstream( std::string(FileData, (size_type)FileDataLen ) ),std::stringstream::in | std::stringstream::out);
Workaround #2:
SS->rdbuf()->sputn(FileData, (streamsize)FileDataLen);
But both of these workarounds produce unnecessary memory copying.
I definitely need a working pubsetbuf member of std::stringbuf.
putsetbuf only makes sense for fstream (technically, for std::basic_filebuf), where the buffer and the stream are two different things.
For stringstream (technically, std::basic_stringbuf) they are one and the same std::string.
If you need a stream that works on a string that's external to it, consider std::strstream: or boost::iostreams::array_sink
basic_ios.clear()
If you need to change rdbuf, call this first or it'll fail to work.
std::ifstream file("file1.txt"); // file1.txt contains "Welcome!"
std::stringstream ss;
ss << file.rdbuf();
std::cout << ss.str() << std::endl;
Outputs "Welcome!".
Let's try again with a new file.
// Empty it
file.close();
ss.str("");
// New file
file.open("file2.txt"); // file2.txt contains "Goodbye!"
ss << file.rdbuf();
std::cout << ss.str() << std::endl;
Outputs nothing.
ss.clear();
ss << file.rdbuf();
std::cout << ss.str() << std::endl;
file.close();
Outputs "Goodbye!"
I see the same thing. I'm working on scenarios that definitely can't afford making unnecessary data copies. This seems to be somehow intentional as per comments in basic_streambuf class:
virtual basic_streambuf *__CLR_OR_THIS_CALL setbuf(_Elem *, streamsize)
{ // offer buffer to external agent (do nothing)
return (this);
}
I recently encountered the same issue of setbuf not being implemented in Visual Studio 2017.
After some searching on Stack Overflow, I found a solution for an output stream that doesn't use copying of the buffer which I modified for an input stream. Here they are for reference.
Output Stream
Source: Setting the internal buffer used by a standard stream (pubsetbuf)
#include <streambuf>
template <typename char_type>
struct ostreambuf : public std::basic_streambuf<char_type, std::char_traits<char_type> >
{
ostreambuf(char_type* buffer, std::streamsize bufferLength)
{
// set the "put" pointer the start of the buffer and record it's length.
setp(buffer, buffer + bufferLength);
}
};
Input Stream
Source: Internal buffer used by standard input stream (pubsetbuf)
#include <streambuf>
template <typename char_type>
struct istreambuf : public basic_streambuf<char_type, char_traits<char_type>>
{
istreambuf(char_type* buffer, streamsize buffer_length)
{
// Set the "get" pointer to the start of the buffer, the next item, and record its length.
this->setg(buffer, buffer, buffer + buffer_length);
}
};
int main()
{
ifstream infile(FILENAME, ifstream::binary);
// Read entire file into buffer.
infile.seekg(0, ios::end);
streampos length = infile.tellg();
infile.seekg(0, ios::beg);
vector<char> buffer(length);
//char* buffer = new char[length];
infile.read(&buffer[0], length);
infile.close();
// Create buffer and point local_stream to it.
istreambuf<char> istream_buffer(&buffer[0], length);
istream local_stream(&istream_buffer);
string str1;
while (local_stream >> str1)
{
. . .
}
}

Opening files over 5 mb and storing them in an array

I want to put each byte in a char array and rewrite the text file removing the first 100,000 characters.
int fs=0;
ifstream nm,nm1;
nm1.open("C:\\Dev-Cpp\\DCS\\Decom\\a.txt");
if(nm1.is_open())
{
nm1.seekg(0, ios::end );
fs = nm1.tellg();
}
nm1.close();
char ss[500000];
nm.open("C:\\Dev-Cpp\\DCS\\Decom\\a.txt");
nm.read(ss,fs-1);
nm.close();
ofstream om;
om.open("C:\\Dev-Cpp\\DCS\\Decom\\a.txt");
for(int i=100000;i<fs-1;i++){
om >> ss[i];
}
om.close();
Problem is i can't set the character array to a 5 million size. I tried using vector also
vector <char> ss (5000000);
int w=0;
ifstream in2("C:\\Dev-Cpp\\DCS\\Decom\\a.txt", ios::binary);
unsigned char c2;
while( in2.read((char *)&c2, 1) )
{
in2 >> ss[w];
w++;
}
Over here the size of w is almost half that of fs and a lot of characters are missing.
How to do it ?
In most implementations, char ss[5000000] tries allocating on the stack, and the size of the stack is limited as compared to the overall memory size. You can often allocate larger arrays on the heap than on the stack, like this:
char *ss = new char [5000000];
// Use ss as usual
delete[] ss; // Do not forget to delete
Note that if the file size fs is larger than 5000000, you will write past the end of the buffer. You should limit the amount of data that you read:
nm.read(ss,min(5000000,fs-1));
This part is not correct
while( in2.read((char *)&c2, 1) )
{
in2 >> ss[w];
w++;
}
bacause you first try to read one character into c2 and, if that succeeds, read another character into ss[w].
I'm not at all surprised if you lose about half the characters here!
The best way to solve your problem is to use the facilities of the standard library. That way, you also don't have to care about buffer overflows.
The following code is untested.
std::fstream file("C:\\Dev-Cpp\\DCS\\Decom\\a.txt", std::ios_base::in);
if (!file)
{
std::cerr << "could not open file C:\\Dev-Cpp\\DCS\\Decom\\a.txt for reading\n";
exit(1);
}
std::vector<char> ss; // do *not* give a size here
ss.reserve(5000000); // *expected* size
// if the file is too large, the capacity will automatically be extended
std::copy(std::istreambuf_iterator<char>(file), std::istreambuf_iterator<char>(),
std::back_inserter(ss));
file.close();
file.open("C:\\Dev-Cpp\\DCS\\Decom\\a.txt", std::ios_base::out | std::ios_base::trunc);
if (!file)
{
std::cerr << "could not open C:\\Dev-Cpp\\DCS\\Decom\\a.txt for writing\n";
exit(1);
}
if (ss.size() > 100000) // only if the file actually contained more than 100000 characters
std::copy(ss.begin()+100000, ss.end(), std::ostreambuf_iterator<char>(file));
file.close();