Qt console: Log all stdin and stdout to file - c++

I have a Qt console application. All communication with the user goes over two streams:
QTextStream in(stdin);
QTextStream out(stdout);
Now I want to log the whole session to a file, but I don't want to add a log output on every location where the streams are used.
Is there an easy way to "tee" the data of both streams to the console and to a file?
The app has to run on Windows.

I suggest a very simple solution, but c++14 is recommended.
I would wrap QTextStream this way (output only as example):
#include <QTextStream>
#include <QFile>
class OutTeeTextStream
{
public:
OutTeeTextStream()
{
logfilestream = 0;
stdoutstream = new QTextStream(stdout);
file.setFileName("/tmp/outlog.log");
if(file.open(QFile::Append))
{
logfilestream = new QTextStream(&file);
}
}
~OutTeeTextStream()
{
file.close();
delete stdoutstream;
delete logfilestream;
}
OutTeeTextStream &operator<<(auto data)
{
(*stdoutstream) << data;
if(logfilestream != 0)
{
(*logfilestream) << data;
}
return (*this);
}
private:
QTextStream * stdoutstream;
QTextStream * logfilestream;
QFile file;
};
As you can see, the wrapper holds two QTextStream instances and provides a stream insertion operator, which basically broadcasts data to each of them.
Not using c++14 (e.g. c++11) will result in complaints about use of ‘auto’ in parameter declaration. In this case, one had to declare/define every insertion (and extraction) operator overload (about fifteen / seventeen in QTextStream).
Refactoring would consist of replacing every QTextStream(stdout) with OutTeeTextStream().

Related

Multiple get/post calls using a wrapper class with Qt

I am working on a Qt project with a team. I have two functions — one retrives the numerical coordinates of a place, the other downloads the map of the place — that I want to merge in one wrapper class, so that my teammates can call it easily.
#include <QCoreApplication>
#include <QFile>
#include <QHttpMultiPart>
#include <QNetworkAccessManager>
#include <QNetworkReply>
#include <iostream>
class OpenStreetMapWrapper: public QObject{
Q_OBJECT
public:
OpenStreetMapWrapper(QObject *parent=nullptr):QObject(parent){
connect(&manager, &QNetworkAccessManager::finished, this, &OpenStreetMapWrapper::handle_finished);
}
void download(const std::string &region, const std::string &department, const QFile& outfile){
QNetworkRequest request;
QUrl url = QUrl(QString::fromStdString("https://download.openstreetmap.fr/extracts/europe/france/" + region + "/" + department + ".osm.pbf"));
request.setUrl(url);
request.setAttribute(QNetworkRequest::User, outfile.fileName());
manager.get(request);
}
void searchCSV(QFile& file, QFile& outfile){
QNetworkRequest request(QUrl("https://api-adresse.data.gouv.fr/search/csv/")); // Free API provided by the French government
request.setAttribute(QNetworkRequest::User, outfile.fileName());
QHttpMultiPart *multipart = new QHttpMultiPart(QHttpMultiPart::FormDataType);
QHttpPart postpart;
postpart.setHeader(QNetworkRequest::ContentDispositionHeader,
QString("form-data; name=%1; filename=%2")
.arg("data", file.fileName()));
postpart.setBodyDevice(&file);
multipart->append(postpart);
file.setParent(multipart);
manager.post(request, multipart);
}
private:
QNetworkAccessManager manager;
void handle_finished(QNetworkReply *reply){
if(reply->error() == QNetworkReply::NoError){
QByteArray read = reply->readAll();
std::cout << read.toStdString() << std::endl; // For debugging
QString filename = reply->request().attribute(QNetworkRequest::User).toString();
QFile out(filename);
if(out.open(QIODevice::WriteOnly)){
out.write(read);
out.close();
}
}
else{
qDebug() << reply->error() << reply->errorString();
}
reply->deleteLater();
// QCoreApplication::quit(); This is done somewhere else?
}
};
#include <main.moc>
int main(int argc, char *argv[])
{
QCoreApplication a(argc, argv);
OpenStreetMapWrapper A;
QFile file("./search.csv");
file.open(QIODevice::ReadWrite);
QFile outfile("./output.csv");
outfile.open(QIODevice::ReadWrite);
// Search
A.searchCSV(file, outfile); // 1st call works
A.searchCSV(file, outfile); // 2nd call -> makes both calls fail.
// Downloader
std::string region = "corse";
std::string department = "haute_corse";
return a.exec();
}
The problem with the code above, is that when for example searchCSV is called it displays the output as needed, but if it is called twice in the code, there is no output at all. After some debugging I think the problem is that the manager and handle_finished are not connected properly, because the execution never reaches there. Is there a simple way to solve this issue? Ideally, there is just one class instant, and any method can be called any number of times.
I don't know much about Qt, but it looks like you're trying to read from file twice and my guess is that when it reaches the end of the file after the first call to A.searchCSV it is done and you can't read from it anymore - unless you reposition the QFile to the beginning of the file.
Possible solution:
A.searchCSV(file, outfile);
file.unsetError(); // possibly needed
file.seek(0); // rewind to the start of the file
A.searchCSV(file, outfile);
The two QFile (input, output) are shared between two asynchronous calls (searchCSV) that might give an undefined behavior. The input file (stream) contents will be load and push only after the connection was made (like curl does).
You should:
Make searchCSV a blocking function (wait until handle_finished() done), input file pointer should be reinitialized before an other call.
OR: Use separated input/output QFile instances

Program crashes trying to create ofstream or fopen

I don't have enough reputation points to comment to ask if they solved the problem originally stated here. I have the same problem of the program crashing in construction of an ofstream.
It is not multi-threaded access, but it can be called in quick succession. I believe it crashes on the 2nd time. I use scope to ensure the stream object is destroyed.
Why would this happen?
I tried std::fopen too. It also results in a crash.
Here is the code using the ofstream:
static bool writeConfigFile (const char * filename, const Config & cfg)
{
logsPrintLine(_SLOG_SETCODE(_SLOGC_CONFIG, 0), _SLOG_INFO, "Write config file (%s stream)", filename);
ofstream os(filename); // FIXME: Crashes here creating ofstream 2nd time
if (os.good())
{
// Uses stream insertion operator to output attributes to stream
// in human readable form (about 2kb)
outputConfig(cfg, os);
if (!os.good())
{
logsPrintLine(_SLOG_SETCODE(_SLOGC_CONFIG, 0), _SLOG_NOTICE, "Failed to write configuration file (%s)", filename);
return false;
}
logsPrintLine(_SLOG_SETCODE(_SLOGC_CONFIG, 0), _SLOG_INFO, "Configuration written to file (%s)", filename);
return true;
}
logsPrintLine(_SLOG_SETCODE(_SLOGC_CONFIG, 0), _SLOG_NOTICE, "Cannot write configuration file (%s)", filename);
return false;
}
/**
* Called when configuration settings have been read/received and validated
* #return true if successfully set, and written to file
*/
bool Config::set (SysConfigSource source, const struct SCADA_dsconfig * p)
{
Lock lock(mtxSet); // This is locking a mutex on construction of the lock. Release it on destruction.
// Setup the non-current one to switch to
Config * pCfg = pConfig.other();
unsigned i, f, n = 0;
// set attributes in pCfg based on the config received
// and some constants ...
pCfg->setWritten(writeConfigFile("test.conf", *pCfg));
if (!pCfg->isWritten())
{
// Don't set system config status here. Existing one still in use.
logsPrintLine(_SLOG_SETCODE(_SLOGC_CONFIG, 0), _SLOG_NOTICE, "Config file not written. Retain prior config.");
return false;
}
pConfig.swap(); // switch-in the new config
setSystemConfigSource(source);
toSyslog(pCfg);
notifyConfigChange();
return true;
}
Maybe post a segment of your source code in order to get an idea of where it went wrong.
Here is a very basic segment of code of how I would use fstream.. hope you will find it helpful.
#include <iostream>
#include <fstream>
#include <string>
int main() {
while (1) {
std::string testString;
std::ofstream outFile;
outFile.open("Test", std::ios_base::app); // Appends to file, does not delete existing code
std::cout << "Enter a string: ";
std::cin >> testString;
outFile << testString << std::endl;
outFile.close();
}
}
It turned out to be a device driver bus master issue. Add "ahci nobmstr" when launching devb-ahci.
Derived via http://www.qnx.com/developers/docs/qnxcar2/index.jsp?topic=%2Fcom.qnx.doc.neutrino.user_guide%2Ftopic%2Fhardware_Troubleshooting_devb-eide.html

Is it necessary to flush a QTextStream before closing a QFile?

I need to log some text messages to a file with following requirements :
Each text messages is written in a new line at the end of the file.
Be reasonably sure that each message was correctly written to the file.
So far, the function is using QTextStream and QFile:
bool FileManager::appendLine(const QString &line)
{
if(!m_file.open(QIODevice::Append | QIODevice::Text)) // m_file is a QFile
return false;
QTextStream ts(&m_file);
ts << line << endl;
bool status = (ts.status() == QTextStream::Ok);
m_file.close();
return status;
}
Point 1 is satisfied but i have doubts about Point 2.
Even Qt Doc says that it is sufficient to close() the QFile to flush all its internal buffers :
void QFileDevice::close()
Reimplemented from QIODevice::close().
Calls QFileDevice::flush() and closes the file. Errors from flush are ignored.
What about the internal buffer of the QTextStream ?
Is it necessary to call QTextStream::flush() before closing the file ?
About Point 2, i guess that reading back the line just after it has been written would be the only way to be 100% sure of that. (for example a power failure may occur while the kernel has still datas in its buffers )
Thanks.
In your case, its not, because you are appending &endl in each write!
Writting &endl to the QTextStream writes '\n' to the stream and flushes the stream. It is Equivalent to: stream << '\n' << flush;
Further, when QTextStream is flushed due to &endl, it will empty all data from its write buffer into the device and call flush() on the device.
While this particular code will work because operations with QTextStream end with an endl, it's still better to ensure that QTextStream is completely and utterly finished working with the file when you close it. Just use scopes.
bool FileManager::appendLine(const QString &line)
{
if(!m_file.open(QIODevice::Append | QIODevice::Text)) // m_file is a QFile
return false;
bool status {false};
{
QTextStream ts(&m_file);
ts << line << endl;
status = (ts.status() == QTextStream::Ok);
}
m_file.close();
return status;
}

C++ - Duplicating stdout/stderr to file while keeping console outputs

Very similar question was already asked here:
Writing to both terminal and file c++
But without a good answer. All answers suggest to use custom stream or duplicating std::cout. However I need the behavior for stdout/stderr.
Wanted behavior: For every write to stdout/stderr I want this to appear on console and also be redirected to a file.
I was thinking about redirecting the stdout to pipe and from there writing to file and console - expanding on this answer https://stackoverflow.com/a/956269/2308106
Is there any better approach to this?
EDIT1: Why stdout/stderr and not custom streams?
I'm calling (3rd party) code that I cannot modify and that is hosted within my process. So I cannot use custom streams (the called code is already writting to stderr/stdout).
EDIT2:
Based on the suggestion from JvO I tried my implementation (windows):
HANDLE hPipe = ::CreateNamedPipe(TEXT("\\\\.\\pipe\\StdErrPipe"),
PIPE_ACCESS_OUTBOUND | FILE_FLAG_FIRST_PIPE_INSTANCE,
PIPE_TYPE_BYTE,
//single instance
1,
//default buffer sizes
0,
0,
0,
NULL);
if (hPipe == INVALID_HANDLE_VALUE)
{
//Error out
}
bool fConnected = ConnectNamedPipe(hPipe, NULL) ?
TRUE : (GetLastError() == ERROR_PIPE_CONNECTED);
if (!fConnected)
{
//Error out
}
int fd = _open_osfhandle(reinterpret_cast<intptr_t>(hPipe), _O_APPEND | /*_O_WTEXT*/_O_TEXT);
if (dup2(fd, 2) == -1)
{
//Error out
}
There is still some issue though - as from the other end of the pipe I receive only a rubish (I first try to send something dirrectly - that works great; but once stderr is redirected and I write to it; the pipe receives same nonprinatble character over and over)
You can 'hijack' stdout and stderr by replacing the pointers; stdout and stderr are nothing more than FILE *. I suggest you open a pipe pair first, then used fdopen() to create a new FILE * which is assiocated with the sending end of the pipe, then point stdout to your new FILE. Use the receiving end of the pipe to extract what was written to the 'old' stdout.
Pseudo code:
int fd[2];
FILE *old_stdout, *new_stdout;
pipe(fd);
new_stdout = fdopen(fd[1], "w");
old_stdout = stdout;
stdout = new_stdout;
Now, everything you read from fd[0] can be written to a file, old_stdout, etc.
You can redirect cout. One (incomplete) example might look like this:
#include <fstream>
#include <iostream>
template<class CharT, class Traits = std::char_traits<CharT> >
struct teestream : std::basic_streambuf<CharT, Traits> {
private:
std::basic_streambuf<CharT, Traits>* m_rdbuf1;
std::basic_streambuf<CharT, Traits>* m_rdbuf2;
public:
teestream(std::basic_streambuf<CharT, Traits>* rdbuf1, std::basic_streambuf<CharT, Traits>* rdbuf2)
:m_rdbuf1(rdbuf1)
,m_rdbuf2(rdbuf2)
{}
~teestream() {
m_rdbuf1->pubsync();
m_rdbuf2->pubsync();
}
protected:
int_type overflow(int_type ch = Traits::eof()) override
{
int_type result = m_rdbuf1->sputc(ch);
if (result != Traits::eof())
{
result = m_rdbuf2->sputc(ch);
}
return result;
}
virtual int sync() override
{
int result = m_rdbuf1->pubsync();
if (result == 0)
{
result = m_rdbuf2->pubsync();
}
return result;
}
};
typedef teestream<char, std::char_traits<char>> basic_teestream;
int main()
{
std::ofstream fout("out.txt");
std::streambuf *foutbuf = fout.rdbuf(); //Get streambuf for output stream
std::streambuf *coutbuf = std::cout.rdbuf(); //Get streambuf for cout
std::streambuf *teebuf = new basic_teestream(coutbuf, foutbuf); //create new teebuf
std::cout.rdbuf(teebuf);//Redirect cout
std::cout << "hello" << std::endl;
std::cout.rdbuf(coutbuf); //Restore cout
delete teebuf; //Destroy teebuf
}
As you can see here the streambuf used by cout is replaced by one that controls the streambuf itself as well as the streambuf of a ofstream.
The code has most likely many flaws and is incomplete but you should get the idea.
Sources:
https://stackoverflow.com/a/10151286/4181011
How can I compose output streams, so output goes multiple places at once?
http://en.cppreference.com/w/cpp/io/basic_streambuf/pubsync
Implementing std::basic_streambuf subclass for manipulating input

QFile and QTextStream for a logger class

I'm trying to create a Logger class using QFile and QTextStream but I can't find an efficient way of doing it. I just want to create a log(...) function in it.
I know it works if I do the following:
void CLogger::log(QString strLog,int nType) {
QFile file(m_strFileName);
file.open( QIODevice::Append | QIODevice::Text );
QTextStream logStream(&file);
logStream << nType << "-" << strLog;
file.close();
}
But It is quite nasty. I don't want to create a QFile object at every log line I insert.
So from that, I tried several different ways like:
1) (with QFile *m_pFile as member)
CLogger::CLogger()
{
m_pFile = new QFile(m_strFileName);
}
void CLogger::log(QString strLog,int nType)
{
m_pFile->open( QIODevice::Append | QIODevice::Text );
QTextStream logStream(m_pFile);
logStream << nType << "-" << strLog;
m_pFile.close();
}
or
2) (with QFile *m_pFile and QTextStream *m_pLogStream as members)
CLogger::CLogger()
{
m_pFile = new QFile(m_strFileName);
m_pFile->open( QIODevice::Append | QIODevice::Text );
m_pLogStream = new QTextStream(m_pFile);
}
void CLogger::log(QString strLog,int nType)
{
*m_pLogStream << nType << "-" << strLog;
}
In the first case, I get:
C2248: 'QTextStream::QTextStream' : cannot access private member
declared in class 'QTextStream'
in the 2nd, *m_pLogStream is not equivalent to a QTextStream&.
What am I doing wrong?
Actually it is not such a bad solution to open (and close) the log file every time you need to log something (unless you log 1000 times every second... but then noone would be able to process that amount of data ...). This not only allows you to have a very stable log (since you do not keep the file open all the time, so you are no depending on the flushing of the oeprating system), but also would allow you to be able to implement features as log rolling, and other niceties.
If you keep the log file open, in case of an unwanted "crash" you might not get all the log lines, depending of course how your OS is handling this ungraceful exits.
Here is a piece of code we use for logging:
QMutexLocker locker(&m_lineLoggerMutex);
QFile f(getLogFileName());
doRollLogsIfNeeded(static_cast<qint64>(f.size() + lineToBelogged.length()));
// Do not open in append mode but seek() to avoid warning for unseekable
// devices, note that if open is made with WriteOnly without Append, the
// file gets truncated
if (!f.open(QIODevice::ReadWrite | QIODevice::Text))
{
QTextStream out(stdout);
out << "CANNOT OPEN LOG FILE: " << getLogFileName();
return;
}
// seek() does nothing on sequential devices, this is in essence what QFile
// does when Append flag is set in open() but without warning (on Qt 4.8.3)
// However, Qt 4.8.1 issues the warning, so check it explicitly
if (!f.isSequential())
{
f.seek(f.size());
}
QTextStream out(&f);
out << lineToBelogged;
This goes in a method, and the destructors take care of the closing of the devices.