Can I init a iostream from a buffer without copying it? - c++

I have a buffer unsigned char *buffer filled with size bytes. I wanna init a stream from it.
I can do it as follow, which copys the buffer bytes:
string s(bytes, bytes + size);
stringstream ss(s);
I wonder if I can init a stream without that copy?

You need to implement a custom streambuf that you then pass to the istream constructor. This will allow you to access the array data as any other stream. See the following links for more details:
std::streambuf
std::istream
Boost provides the iostreams library for helping with this. In particular, it already provides the array_source class for wrapping standard arrays. The following code sample illustrates how to accomplish this:
#include <cstdlib>
#include <iostream>
#include <string>
#include <boost/iostreams/stream_buffer.hpp>
#include <boost/iostreams/device/array.hpp>
namespace io = boost::iostreams;
int main(int argc, char* argv[]) {
const char buffer[] = "buffer data\n";
io::array_source src{ buffer, strlen(buffer) };
io::stream_buffer<io::array_source> strbuf{ src };
std::istream stream{ &strbuf };
std::string line;
std::getline(stream, line);
std::cout << line << std::endl;
return EXIT_SUCCESS;
}

Related

std::basic_fstream<T> only reads zeros for all types T other than char

I am trying to read from a file in c++ with the catch that the code must work for any char type. This works fine for chars, however when I try to read utf-8 (char8_t) data I simply get zeros. The following using char works as expected.
#include <iostream>
#include <fstream>
int main()
{
typedef char c;
c* buff = new c[38];
std::basic_fstream<c> s;
s.open("test_file_1.txt", std::ios::in);
s.read(buff, 38);
std::cout << reinterpret_cast<char*>(buff) << std::endl;
delete[] buff;
return 0;
}
the file "test_file_1.txt" simple contains This file contains very important data written as plain text.
Now when I simply change typedef char c to use typedef char8_t c I get no output. When I read the data as integers the entire buffer is full of zeros which explains why there is no output in cout. I tried writing a non zero value to the buffer before reading in the data and the fstream::read does not modify the data in any way.
Any explanation of what is going on and how to fix it would be greatly appreciated. All code was compiled with g++ -std=c++20 test.cpp using gcc 11.1.0 on arch linux.
edit1: fixed ambiguity in my wording
edit2: here is the code that does not work
#include <iostream>
#include <fstream>
int main()
{
typedef char8_t c;
c* buff = new c[38];
std::basic_fstream<c> s;
s.open("test_file_1.txt", std::ios::in);
s.read(buff, 38);
std::cout << reinterpret_cast<char*>(buff) << std::endl;
delete[] buff;
return 0;
}

C++ - Is it possible to edit the data inside an ifstream?

I'm trying to use Boost-GIL to load an image. The read_image function takes ifstream as a parameter, but I will already have the image as a base64 encoded string. Is it possible for me to add the decoded image into an ifstream manually, so that I wont have to write and read from disk to get the image loaded into GIL? Another possibility could be to somehow use a string stream to add the data and cast that to an ifstream, though I haven't had luck trying that.
Boost.GIL's read_image function you mentioned seems to support istream interface. If you have an array, you can make use of boost.iostreams to represent the array as a stream.
Here is a made-up example since you do not provide a code snippet.
#include <boost/iostreams/device/array.hpp>
#include <boost/iostreams/stream.hpp>
#include <boost/gil.hpp>
#include <boost/gil/io/read_image.hpp>
#include <boost/gil/extension/io/tiff.hpp>
int main(int argc, char* argv[]) {
const char* data = "decoded-data";
boost::iostreams::stream<boost::iostreams::array_source> in{data, std::strlen(data)};
boost::gil::rgb8_image_t img;
read_image(in, img, boost::gil::tiff_tag());
return 0;
}
Alternatively, you could use std::stringstream to store your decoded image and give that to the read_image function. Something along the lines of:
#include <boost/archive/iterators/binary_from_base64.hpp>
#include <boost/archive/iterators/insert_linebreaks.hpp>
#include <boost/archive/iterators/transform_width.hpp>
#include <boost/archive/iterators/ostream_iterator.hpp>
#include <boost/gil.hpp>
#include <boost/gil/io/read_image.hpp>
#include <boost/gil/extension/io/tiff.hpp>
#include <sstream>
using base64_decode_iterator = transform_width<binary_from_base64<const char*>, 8, 6>;
int main(int argc, char* argv[]) {
const char* data = "base64-data";
std::stringstream ss;
std::copy(
base64_decode_iterator{data},
base64_decode_iterator{data + std::strlen(data)},
std::ostream_iterator<char>{ss}
);
boost::gil::rgb8_image_t img;
read_image(ss, img, boost::gil::tiff_tag());
return 0;
}

Is it possible to create an ifstream without a real file

I am using a third party library which needs a ifstream object as input and I have a stringstream object which contains everything they need. Right now, I have to write the content of stringstream to a file and then send a ifstream object to the library. I am wondering if it is possible directly convert stringstream to ifstream in memory so I don't need to write a file on disk? thanks.
I your library is really accepting std::ifstream instead of std::istream, then I found the following hack:
#include <fstream>
#include <iostream>
#include <sstream>
#include <string>
void foo(std::ifstream& fs)
{
std::string h;
fs >> h;
std::cout << h << std::endl;
}
int main()
{
std::istringstream s1("hello");
std::ifstream s2;
s2.basic_ios<char>::rdbuf(s1.rdbuf());
foo(s2);
return 0;
}
I am not sure how safe it is, however, so you might investigate the topic further.

Cast a variable of filtering_istream type to ifstream type?

I am using the filetering_istream type to save the information in a decompressed file while using 'boost/iostreams/filtering_stream.hpp'. But I want to cast it into the ifstream type. It there any way to do it? Great thanks!
The code is as follows:
#include <istream>
#include <fstream>
#include <boost/iostreams/filtering_stream.hpp>
#include <boost/iostreams/filter/gzip.hpp>
int main(){
std::ifstream file("test_data.dat.gz");
boost::iostreams::filtering_istream in;
in.push(boost::iostreams::gzip_decompressor());
in.push(file);
/* add code to convert filtering_istream 'in' into ifstream 'pfile' */
/* It seems that the following code returns a pointer NULL */
// std::ifstream* pfile = in.component<std::ifstream>(1);
return 0;
}
After trying boost::ref and boost::wrapper proposed by zett42, the ifstream really works. The only problem is that it doesn't give the phrases wanted.
In my text of .gz file, I wrote:
THIS IS A DATA FILE!
8 plus 8 is 16
But using the ifstream, I got:
is_open: 1
\213<\373Xtest_data.dat\361\360V"G\307G7OWE.\205\202\234\322b\205\314bC3.\327+>\314$
I am not sure what happened here, and can I do something to recover it?
From the reference of filtering_stream:
filtering_stream derives from std::basic_istream, std::basic_ostream
or std::basic_iostream, depending on its Mode parameter.
So no, you can't cast a filtering_stream directly to an ifstream because there is no inheritance relationship between the two.
What you can do instead, if your filter chain ends with a device that is an ifstream, you can grap that device by calling filtering_stream::component(). For streams this function returns a boost::iostreams::detail::mode_adapter (you can see the type by calling in.component_type(1)).
It's propably not a good idea to depend on an internal boost type (indicated by namespace "detail") which could change with next boost version, so one workaround is to use boost::reference_wrapper instead.
#include <iostream>
#include <istream>
#include <fstream>
#include <boost/iostreams/filtering_stream.hpp>
#include <boost/iostreams/filter/gzip.hpp>
#include <boost/core/ref.hpp>
int main(){
std::ifstream file("test_data.dat.gz");
boost::iostreams::filtering_istream in;
in.push(boost::iostreams::gzip_decompressor());
in.push(boost::ref(file));
if( auto pfile = in.component<boost::reference_wrapper<std::ifstream>>( 1 ) )
{
std::ifstream& rfile = *pfile;
std::cout << "is_open: " << rfile.is_open() << "\n";
}
}

Strange behavior with c++ io

I am using zlib to compress data for a game I am making. Here is the code I have been using
#include <SFML/Graphics.hpp>
#include <Windows.h>
#include <fstream>
#include <iostream>
#include "zlib.h"
#include "zconf.h"
using namespace std;
void compress(Bytef* toWrite, int bufferSize, char* filename)
{
uLongf comprLen = compressBound(bufferSize);
Bytef* data = new Bytef[comprLen];
compress(data, &comprLen, &toWrite[0], bufferSize);
ofstream file(filename);
file.write((char*) data, comprLen);
file.close();
cout<<comprLen;
}
int main()
{
const int X_BLOCKS = 1700;
const int Y_BLOCKS = 19;
int bufferSize = X_BLOCKS * Y_BLOCKS;
Bytef world[X_BLOCKS][Y_BLOCKS];
//fill world with integer values
compress(&world[0][0], bufferSize, "Level.lvl");
while(2);
return EXIT_SUCCESS;
}
Now I would have expected the program to simply compress the array world and save it to a file. However I noticed a weird behavior. When I prited the value for 'comprLen' it was a different length then the created file. I couldn't understand where the extra bytes in the file were coming from.
You need to open the file in binary mode:
std::ofstream file(filename, std::ios_base::binary);
without the std::ios_base::binary flag the system will replace end of line characters (\n) by end of line sequences (\n\r). Suppressing this conversion is the only purpose of the std::ios_base::binary flag.
Note that the conversion is made on the bytes written to the stream. That is, the number of actually written bytes will increase compared to the second argument to write(). Also note, that you need to make sure that you are using the "C" locale rather than some locale with a non-trivial code conversion facet (since you don't explicitly set the global std::locale in your code you should get the default which is the "C" locale).