How to convert QByteArray to std::istream or std::ifstream? - c++

I want to create istream from QByteArray at runtime, without saving a physical file in memory of QByteArray.
I found that there are many ways to do the opposite conversion, i.e. istream to QByteArray, but not this one.
How to accomplish that?

To read via std::istringstream from QByteArray seems quite easy:
testQByteArray-istream.cc:
#include <iostream>
#include <sstream>
#include <QtCore>
int main()
{
qDebug() << "Qt Version:" << QT_VERSION_STR;
// make a QByteArray
QByteArray data("Hello Qt World.");
// convert to std::string
std::istringstream in(data.toStdString());
// read from istringstream
for (;;) {
std::string buffer;
if (!std::getline(in, buffer)) break;
std::cout << "Got: '" << buffer << "'\n";
}
// done
return 0;
}
testQByteArray-istream.pro:
SOURCES = testQByteArray-istream.cc
QT = core
Compiled and tested on cygwin64:
$ qmake-qt5 testQByteArray-istream.pro
$ make
$ ./testQByteArray-istream
Qt Version: 5.9.4
Got: 'Hello Qt World.'
$
Done. Stop, wait!
without saving a physical file in memory
I'm not quite sure how to read this. Probably, it means
without copying data saved in QByteArray
I see only two solutions:
Use a QDataStream instead of std::stream. According to doc. QDataStream::QDataStream(const QByteArray &a)
Constructs a read-only data stream that operates on byte array a.
This sounds very promising that data is not copied.
DIY. Make a class derived from std::stream which may read from a QByteArray without copying.
Concerning 2. option, I found Dietmar Kühl's answer to SO: Creating an input stream from constant memory. Applying this to the above sample, it would look like this:
#include <iostream>
#include <QtCore>
// borrowed from https://stackoverflow.com/a/13059195/7478597
struct membuf: std::streambuf {
membuf(char const* base, size_t size) {
char* p(const_cast<char*>(base));
this->setg(p, p, p + size);
}
};
struct imemstream: virtual membuf, std::istream {
imemstream(char const *base, size_t size):
membuf(base, size),
std::istream(static_cast<std::streambuf*>(this)) {
}
};
int main()
{
qDebug() << "Qt Version:" << QT_VERSION_STR;
// make a QByteArray
QByteArray data("Hello Qt World.");
imemstream in(data.data(), (size_t)data.size());
// read from istringstream
for (;;) {
std::string buffer;
if (!std::getline(in, buffer)) break;
std::cout << "Got: '" << buffer << "'\n";
}
// done
return 0;
}
Compiled and tested again on cygwin64:
$ qmake-qt5 testQByteArray-istream.pro
$ make
$ ./testQByteArray-istream
Qt Version: 5.9.4
Got: 'Hello Qt World.'
$

Related

Hash output as BYTE instead of std::string?

My question/problem might be a bit newbie regarding this field, but I cannot find any solution or clear explanation to achieve what I want.
The problem
I must store and use the hash values as BYTE instead of STRING because of the size. ( it causes me troubles in other areas )
The function generates an MD5 hash for a file, used on windows OS.
Current code
std::string MD5Checksum(const path &file)
{
std::string result;
try
{
CryptoPP::Weak::MD5 md5;
CryptoPP::HashFilter f5(md5, new CryptoPP::HexEncoder(new CryptoPP::StringSink(result)));
CryptoPP::ChannelSwitch cs;
cs.AddDefaultRoute(f5);
CryptoPP::FileSource ss(file.string().c_str(), true /*pumpAll*/, new CryptoPP::Redirector(cs));
}
catch (CryptoPP::Exception const& exception)
{
//
}
return result;
}
What I tested
std::string MD5Checksum(const path &file)
{
std::string result;
try
{
CryptoPP::Weak::MD5 md5;
CryptoPP::HashFilter f5(md5, new CryptoPP::HexEncoder(new CryptoPP::StringSink(result)));
CryptoPP::ChannelSwitch cs;
cs.AddDefaultRoute(f5);
CryptoPP::FileSource ss(file.string().c_str(), true /*pumpAll*/, new CryptoPP::Redirector(cs));
}
catch (CryptoPP::Exception const& exception)
{
//
}
string decoded;
CryptoPP::StringSource ss(result, true /*pumpAll*/, new CryptoPP::StringSink(decoded));
const BYTE* data = reinterpret_cast<const BYTE*>(decoded.data());
printf(L"sizeof result: %d, sizeof data: %d"), sizeof(result), sizeof(data));
return result;
}
This seems to achieve the desired result, because the size of result string is 40 and the size of data is 8 which is a massive reduction in size for me.
However I do not see this as a good solution and I am pretty sure that there must but an easier and cleaner way of doing this.
Any examples are much appreciated.
I must store and use the hash values as BYTE instead of STRING because of the size...
You are almost there.
Both StringSource and ArraySink can handle byte arrays. You just need to use alternate constuctors. Also see StringSource and ArraySink on the Crypto++ wiki.
I would modify the code similar to the following. I'm using C++11 so I don't have std::path:
$ cat test.cxx
#include "cryptlib.h"
#include "filters.h"
#include "files.h"
#include "hex.h"
#define CRYPTOPP_ENABLE_NAMESPACE_WEAK 1
#include "md5.h"
#include <iostream>
#if defined(CRYPTOPP_NO_GLOBAL_BYTE)
using CryptoPP::byte;
#endif
bool MD5Checksum(const std::string &file, byte* digest, size_t size)
{
using namespace CryptoPP;
try
{
Weak::MD5 md5;
FileSource(file.c_str(), true /*pumpAll*/,
new HashFilter(md5, new ArraySink(digest, size)));
}
catch (Exception const& exception)
{
return false;
}
return true;
}
int main(int argc, char* argv[])
{
using namespace CryptoPP;
std::string filename = (argc >= 2 ? argv[1] : "./cryptlib.h");
byte digest[Weak::MD5::DIGESTSIZE];
if (MD5Checksum(filename, digest, sizeof(digest)))
{
std::cout << "Filename: " << filename << std::endl;
std::cout << "Digest: ";
StringSource(digest, sizeof(digest), true, new HexEncoder(new FileSink(std::cout)));
std::cout << std::endl;
}
else
{
std::cerr << "Failed to calculate digest of " << filename << std::endl;
std::exit(1);
}
return 0;
}
And then compile. I'm working from the cryptopp/ directory in my home directory:
$ g++ ./test.cxx ./libcryptopp.a -o test.exe
$
And finally:
$ ./test.exe
Filename: ./cryptlib.h
Digest: 626047BC8770BE942B26B3AD6CBD3781
In the code above, here are the sources and sinks wrapping the byte array:
StringSource(digest, sizeof(digest) ...);
ArraySink(digest, size);
If you were storing into a std::string like in How to easily apply Crypto++ hash functions?, here are the sources and sinks wrapping the std::string. They are different constructors.
std::string digest;
...
StringSource(digest, ...);
StringSink(digest);

Force print to terminal (std::cout is being piped)

Context:
I'm editing a small piece of a large program. This large program is controlling std::cout and rerouting it so that a basic piece of code like:
std::cout << "I want to see the light of the terminal!" << std::endl;
Does not display anything to the user.
Question:
How can I get something to print directly to the terminal when my standard output/error is being rerouted? (If possible)
Other notes:
I realize that I could edit the larger program, but I'm looking to use this printout for some early diagnostics before more fully integrating my code into the program. Having to mess with how the program routes the output would really extend the development cycle.
I am also currently writing to a file as a workaround but that's slightly less desirable and frankly I would like to know how to do this in the future.
I think you can do it following these steps:
Save the redirected buffer
Change buffer to console
Get your job done
Again set buffer to the saved buffer in step 1
For example
#include <sstream>
#include <iostream>
void print_to_console() {
std::cout << "Hello from print_to_console()" << std::endl;
}
void foo(){
std::cout<<"hello world"<<std::endl;
print_to_console(); // this could be printed from anything
}
int main()
{
std::stringstream ss;
//change the underlying buffer and save the old buffer
auto old_buf = std::cout.rdbuf(ss.rdbuf());
foo(); //all the std::cout goes to ss
std::cout.rdbuf(old_buf); //reset
std::cout << "<redirected-output>\n"
<< ss.str()
<< "</redirected-output>" << std::endl;
}
I haven't tested it. I took the idea and example from this accepted answer.
For convenience, you can just write a function to print in console. This function will take care of the redirection and printing.
Write to stdout and read from stdin (both are FILE descriptors).
You can wrap them in stream classes if you wish.. IE: Using streambuf and iostreamto get the same functionality as cout.
#include <iostream>
int main(int argc, const char * argv[]) {
const char* data = "DATA TO PRINT";
fwrite(data, strlen(data), sizeof(char), stdout);
return 0;
}
Small example:
#include <iostream>
class stream : public std::streambuf
{
private:
int_type overflow(int_type c = traits_type::eof());
public:
stream() {}
virtual ~stream() {}
stream(const stream& other) = delete;
stream& operator = (const stream& other) = delete;
};
stream::int_type stream::overflow(stream::int_type c)
{
if (c != traits_type::eof())
{
fwrite(&c, 1, sizeof(c), stdout);
}
return c;
}
class mcout : public std::ostream
{
public:
mcout() : std::ostream(0), sbuf() {init(&sbuf);}
virtual ~mcout() {}
private:
stream sbuf;
} mcout;
int main(int argc, const char * argv[]) {
mcout << "HELLO\n";
return 0;
}

In Qt: Can I output to `stdout`, as easy as I can output to `stderr` using qDebug()?

Up until now, I output everything using qDebug().noquote(). This is easy because it just requires a simple #import <QDebug>
Now I need everything to output to stdout, but I don't know how to do it easily. This how I was taught:
QTextStream cout(stdout, QIODevice::WriteOnly);
However, creating a new object is a tad bit more cumbersome than a simple #import <QDebug>. What is the good/least cumbersome way to handle stdout in qt?
qDebug(), qInfo(), etc. are all piped to a default message handler. But you can easily install your own that writes the debug strings to a different stream, file, or anything. All you need to do is define a message handler function and install it using qInstallMessageHandler().
Putting it all together, here's a complete example:
#include <QDebug>
void myMessageOutput(QtMsgType type, const QMessageLogContext &context, const QString &msg)
{
QTextStream cout(stdout, QIODevice::WriteOnly);
cout << msg << endl;
}
int main(int argc, char *argv[])
{
qInstallMessageHandler(myMessageOutput);
qDebug().noquote() << "Hello world!";
}
The best way is the one you mentioned. You don't have to create a new local variable:
QTextStream(stdout) << "Hello world!" << endl;
If the source text is not Latin-1 encoded then you need to convert to QString before passing it to the stream operator:
QTextStream(stdout) << QString::fromUtf8("utf8 literal") << endl;

How can I convert QByteArray to string in Qt 5.3?

I am using some functions to convert QVector's to QByteArray's, for example:
QByteArray Serialize::serialize(QVector<double> data)
{
QByteArray byteArray;
QDataStream out(&byteArray, QIODevice::WriteOnly);
out << data;
return byteArray;
}
void Serialize::deserialize(QByteArray byteArray, QVector<double> *data)
{
QDataStream in(&byteArray, QIODevice::ReadOnly);
in >> *data;
}
Now, that I have the QByteArray I need to put it in a text file, how can I convert it to QString?
I already tried the simplest way:
QString myString(data); // data - QByteArray
But myString is always empty.
I also found the toStdString() function in the documentation, but it was introduced only in Qt 5.4.
I'm using Qt 5.3.
Follows a complete example:
#include <QCoreApplication>
#include <QDebug>
#include <QVector>
#include <QByteArray>
#include <QDataStream>
QByteArray serialize(QVector<double> data)
{
QByteArray byteArray;
QDataStream out(&byteArray, QIODevice::WriteOnly);
out << data;
return byteArray;
}
void deserialize(QByteArray byteArray, QVector<double> *data)
{
QDataStream in(&byteArray, QIODevice::ReadOnly);
in >> *data;
}
int main(int argc, char *argv[])
{
QCoreApplication a(argc, argv);
QVector<double> data;
data << 1.1 << 2.2 << 3.3 << 4.4 << 5.5 << 6.6 << 7.7 << 8.8 << 9.9;
QByteArray byteArray = serialize(data);
QVector<double> dataConverted;
deserialize(byteArray, &dataConverted);
qDebug() << "Data:";
qDebug() << data;
qDebug() << "ByteArray:";
QString test(byteArray);
qDebug() << test;
qDebug() << "Data Converted:";
qDebug() << dataConverted;
return a.exec();
}
Note: The general objective of this is to generate a SQL file with all content from the SQLite database. My double vector is converted to QByteArray and stored as BLOB into the database (using the serialize function). When I need to load it from the database I use the deserialize function to convert to a double vector again. Now I need to generate the SQL file with the data in the BLOB format, then I can directly import it into another database.
The problem is that a byte array is type-agnostic data type, it simply represents a collection of individual bytes in memory. In your example code you are creating the byte array from a vector of doubles, and then converting back to another vector of doubles. No problem.
However when you pass the byte array into the QString constructor, the QString is trying to interpret the byte array as data that represents a string, for example an array of ASCII character codes.
Some string classes might let you do this, and create an instance filled with garbage, however QString appears to be doing some basic error checking and helping you out by giving you an empty string.
As for some code to print out the contents of a byte array of doubles, the deserialize method you've provided is not too bad an example.
Use QTextCodec to convert from QByteArray to QString, here's an example from official docs:
QByteArray encodedString = "...";
QTextCodec *codec = QTextCodec::codecForName("KOI8-R");
QString string = codec->toUnicode(encodedString);
And to make the example work, you need to convert the doubles to QStrings during serialization:
QByteArray serialize(QVector<double> data)
{
QByteArray byteArray;
QDataStream out(&byteArray, QIODevice::WriteOnly);
for (double d : data) {
out << QString::number(d);
}
return byteArray;
}
If you don't want to convert individual numbers to string, you can also "stringify" the QByteArray with byteArray.toHex():
qDebug() << "ByteArray:";
QString test(byteArray.toHex());

How to override cout in C++?

I have a requirement, I need to use printf and cout to display the data into console and file as well.
For printf I have done it but for cout I am struggling, how to do it?
#ifdef _MSC_VER
#define GWEN_FNULL "NUL"
#define va_copy(d,s) ((d) = (s))
#else
#define GWEN_FNULL "/dev/null"
#endif
#include <iostream>
#include <fstream>
using namespace std;
void printf (FILE * outfile, const char * format, ...)
{
va_list ap1, ap2;
int i = 5;
va_start(ap1, format);
va_copy(ap2, ap1);
vprintf(format, ap1);
vfprintf(outfile, format, ap2);
va_end(ap2);
va_end(ap1);
}
/* void COUT(const char* fmt, ...)
{
ofstream out("output-file.txt");
std::cout << "Cout to file";
out << "Cout to file";
}*/
int main (int argc, char *argv[]) {
FILE *outfile;
char *mode = "a+";
char outputFilename[] = "PRINT.log";
outfile = fopen(outputFilename, mode);
char bigfoot[] = "Hello
World!\n";
int howbad = 10;
printf(outfile, "\n--------\n");
//myout();
/* then i realized that i can't send the arguments to fn:PRINTs */
printf(outfile, "%s %i",bigfoot, howbad); /* error here! I can't send bigfoot and howbad*/
system("pause");
return 0;
}
I have done it in COUT(caps, the commented part for the code above) . But I want to use normal std::cout, so how can I override it. And it should work for both sting and variables like
int i = 5;
cout << "Hello world" << i <<endl;
Or are there anyway to capture stdout data, so that they can be easily written into file and console as well.
If you have another stream buffer, you can just replace std::cout's:
std::cout.rdbuf(some_other_rdbuf);
See http://en.cppreference.com/w/cpp/io/basic_ios/rdbuf.
You can swap the underlying buffers. Here is that done facilitated through RAII.
#include <streambuf>
class buffer_restore
{
std::ostream& os;
std::streambuf* buf;
public:
buffer_restore(std::ostream& os) : os(os), buf(os.rdbuf())
{ }
~buffer_restore()
{
os.rdbuf(buf);
}
};
int main()
{
buffer_restore b(std::cout);
std::ofstream file("file.txt");
std::cout.rdbuf(file.rdbuf());
// ...
}
Overriding the behaviour of std::cout is a really bad idea as other developers will have a hard time understanding that the use of std::cout doesn't behave as usual.
Make your intention clear with a simple class
#include <fstream>
#include <iostream>
class DualStream
{
std::ofstream file_stream;
bool valid_state;
public:
DualStream(const char* filename) // the ofstream needs a path
:
file_stream(filename), // open the file stream
valid_state(file_stream) // set the state of the DualStream according to the state of the ofstream
{
}
explicit operator bool() const
{
return valid_state;
}
template <typename T>
DualStream& operator<<(T&& t) // provide a generic operator<<
{
if ( !valid_state ) // if it previously was in a bad state, don't try anything
{
return *this;
}
if ( !(std::cout << t) ) // to console!
{
valid_state = false;
return *this;
}
if ( !(file_stream << t) ) // to file!
{
valid_state = false;
return *this;
}
return *this;
}
};
// let's test it:
int main()
{
DualStream ds("testfile");
if ( (ds << 1 << "\n" << 2 << "\n") )
{
std::cerr << "all went fine\n";
}
else
{
std::cerr << "bad bad stream\n";
}
}
This provides a clean interface and outputs the same for both the console and the file.
You may want to add a flush method or open the file in append mode.
I assume you have some code using std::cout and printf which you cannot modify, otherwise the most simple way to solve your problem would be to write to a different stream from cout and use fprintf rather than or in conjunction with printf.
By following that approach you could define both a new stream class that actually wrote both to standard output and to a given file, as well as a function that combined calls to both printf and fprintf.
However a much simpler approach is to use the tee program, originally from UNIX, which copies its input both to output and to a given file. With that you could simply call your program in this way:
your_program | tee your_log_file
Answers to this question lead to a few alternative implementations available for Windows. Personally I always install cygwin on my PC's to have UNIX/Linux utilities available.
If i guess correctly you want to log everything that goes to the output also into a file.
What you want is an observer pattern.
Replace all direct logging in your code with calls to a new relay.
The logging relay sends your messages to the observers.
One of your observers loggs the message to the screen.
The other one loggs to the file.
Avoid making your relay a singleton if possible.
This suggestion only works if you can edit all of your source files.
std::cout writes to stdout file you can do the following on Linux and Windows
#include <stdio.h>
#include <iostream>
int main()
{
freopen("test.txt", "w", stdout);
std::cout << "Hello strange stdout\n";
}
to change it back use the following taken from here
#include <stdio.h>
#include <stdlib.h>
void main(void)
{
FILE *stream ;
if((stream = freopen("file.txt", "w", stdout)) == NULL)
exit(-1);
printf("this is stdout output\n");
stream = freopen("CON", "w", stdout);
printf("And now back to the console once again\n");
}
Note: The latter is windows only
cout is normally implemented as an object instance so you can't override it in the way that you would overload / override a function or a class.
Your best bet is not to fight that - yes you could build a my_cout and #define cout my_cout but that would make your code obtuse.
For readability I'd leave cout as it is. It's a standard and everyone knows what it can and can't do.
Try using a macro - something like this (you'll need to add the includes) :
#define MY_COUT(theos,printThis) { cout << printThis ; theos << printThis; }
void test()
{
ofstream myos;
myos.open("testfile", ios::trunc|ios::out);
int i = 7;
MY_COUT(myos, "try this numbers" << i << i + 1 << endl);
myos.close()
}
There's already a Boost class for this: tee