I'm trying to write a configuration reader in C/C++ (using Low-Level I/O).
The configuration contains directions like:
App example.com {
use: python vauejs sass;
root: www/html;
ssl: Enabled;
}
How could i read the content into a std::map or struct? Google did not give me the results i'm looking for yet. I hope SO got some ideas...
What i got so far:
// File Descriptor
int fd;
// Open File
const errno_t ferr = _sopen_s(&fd, _file, _O_RDONLY, _SH_DENYWR, _S_IREAD);
// Handle returned value
switch (ferr) {
// The given path is a directory, or the file is read-only, but an open-for-writing operation was attempted.
case EACCES:
perror("[Error] Following error occurred while reading configuration");
return false;
break;
// Invalid oflag, shflag, or pmode argument, or pfh or filename was a null pointer.
case EINVAL:
perror("[Error] Following error occurred while reading configuration");
return false;
break;
// No more file descriptors available.
case EMFILE:
perror("[Error] Following error occurred while reading configuration");
return false;
break;
// File or path not found.
case ENOENT:
perror("[Error] Following error occured while reading configuration");
return false;
break;
}
// Notify Screen
if (pDevMode)
std::printf("[Configuration]: '_sopen_s' were able to open file \"%s\".\n", _file);
// Allocate memory for buffer
buffer = new (std::nothrow) char[4098 * 4];
// Did the allocation succeed?
if (buffer == nullptr) {
_close(fd);
std::perror("[Error] Following error occurred while reading configuration");
return false;
}
// Notify Screen
if (pDevMode)
std::printf("[Configuration]: Buffer Allocation succeed.\n");
// Reading content from file
const std::size_t size = _read(fd, buffer, (4098 * 4));
If you put your buffer into a std::string you can piece together a solution from various answers about splitting strings on SO.
The essential structure seems to be "stuff { key:value \n key:value \n }"
with varying amounts of whitespace. Many questions have been asked about trimming a string. Splitting a string can happen in several ways, e.g.
std::string config = "App example.com {\n"
" use: python vauejs sass;\n"
" root: www / html; \n"
" ssl: Enabled;"
"}";
std::istringstream ss(config);
std::string token;
std::getline(ss, token, '{');
std::cout << token << "... ";
std::getline(ss, token, ':');
//use handy trim function - loads of e.g.s on SO
std::cout << token << " = ";
std::getline(ss, token, '\n');
// trim function required...
std::cout << token << "...\n\n";
//as many times or in a loop..
//then check for closing }
If you have more complicated parsing consider a full-on parser.
Related
I don't have enough reputation points to comment to ask if they solved the problem originally stated here. I have the same problem of the program crashing in construction of an ofstream.
It is not multi-threaded access, but it can be called in quick succession. I believe it crashes on the 2nd time. I use scope to ensure the stream object is destroyed.
Why would this happen?
I tried std::fopen too. It also results in a crash.
Here is the code using the ofstream:
static bool writeConfigFile (const char * filename, const Config & cfg)
{
logsPrintLine(_SLOG_SETCODE(_SLOGC_CONFIG, 0), _SLOG_INFO, "Write config file (%s stream)", filename);
ofstream os(filename); // FIXME: Crashes here creating ofstream 2nd time
if (os.good())
{
// Uses stream insertion operator to output attributes to stream
// in human readable form (about 2kb)
outputConfig(cfg, os);
if (!os.good())
{
logsPrintLine(_SLOG_SETCODE(_SLOGC_CONFIG, 0), _SLOG_NOTICE, "Failed to write configuration file (%s)", filename);
return false;
}
logsPrintLine(_SLOG_SETCODE(_SLOGC_CONFIG, 0), _SLOG_INFO, "Configuration written to file (%s)", filename);
return true;
}
logsPrintLine(_SLOG_SETCODE(_SLOGC_CONFIG, 0), _SLOG_NOTICE, "Cannot write configuration file (%s)", filename);
return false;
}
/**
* Called when configuration settings have been read/received and validated
* #return true if successfully set, and written to file
*/
bool Config::set (SysConfigSource source, const struct SCADA_dsconfig * p)
{
Lock lock(mtxSet); // This is locking a mutex on construction of the lock. Release it on destruction.
// Setup the non-current one to switch to
Config * pCfg = pConfig.other();
unsigned i, f, n = 0;
// set attributes in pCfg based on the config received
// and some constants ...
pCfg->setWritten(writeConfigFile("test.conf", *pCfg));
if (!pCfg->isWritten())
{
// Don't set system config status here. Existing one still in use.
logsPrintLine(_SLOG_SETCODE(_SLOGC_CONFIG, 0), _SLOG_NOTICE, "Config file not written. Retain prior config.");
return false;
}
pConfig.swap(); // switch-in the new config
setSystemConfigSource(source);
toSyslog(pCfg);
notifyConfigChange();
return true;
}
Maybe post a segment of your source code in order to get an idea of where it went wrong.
Here is a very basic segment of code of how I would use fstream.. hope you will find it helpful.
#include <iostream>
#include <fstream>
#include <string>
int main() {
while (1) {
std::string testString;
std::ofstream outFile;
outFile.open("Test", std::ios_base::app); // Appends to file, does not delete existing code
std::cout << "Enter a string: ";
std::cin >> testString;
outFile << testString << std::endl;
outFile.close();
}
}
It turned out to be a device driver bus master issue. Add "ahci nobmstr" when launching devb-ahci.
Derived via http://www.qnx.com/developers/docs/qnxcar2/index.jsp?topic=%2Fcom.qnx.doc.neutrino.user_guide%2Ftopic%2Fhardware_Troubleshooting_devb-eide.html
My code reads lines from a fifo, constructs a functor, and queues it with a Boost thread pool. I was having a problem with occasional work items failing, but when I added logging the problem disappeared.
I've worked it down to the location of where a std::ofstream is opened, but I can't figure out why this is true. By moving that line I was able to get the problems to reoccur. Sometimes part of a line goes missing and reappears somewhere else (either preppended to or overwriting another line). This doesn't appear to be random. Given the same input the error always occurs in the same place, but if the input lines are reordered the error will occur in a different location.
As an example
# This line should read
# /dev/shm/test/bluetooth-active-symbolic.symbolic.png.csv.plot.png
# But it is missing the leading "/dev/"
shm/test/bluetooth-active-symbolic.symbolic.png.csv.plot.png
# This line should read
# POOLCMD_CLOSE
# But the missing text reappeared here
/dev/POOLCMD_CLOSE
Code
Where we fork to the background
switch (fork()) {
case -1:
std::cerr << "Error forking\n";
return 1;
case 0:
close(0);
close(1);
close(2);
dup2(stdout_file, fileno(stdout));
dup2(stdout_file, fileno(stderr));
setsid();
went_background = true;
// If this ofstream line is moved elsewhere, things break
log_file = std::ofstream("poolcmd-log.txt");
break;
default:
exit(0);
}
Where we read from the fifo
read_threads.push_back(
decltype(read_threads)::value_type(new std::thread([&]() {
char* line = 0;
size_t n = 0;
while (true) {
auto read = getline(&line, &n, fifo);
locked_print(log_file, "read: ");
locked_print(log_file, line);
if (read >= 0) {
std::string tmp(line, read - 1);
if (tmp == "POOLCMD_CLOSE") {
locked_print(log_file, "Understood POOLCMD_CLOSE\n");
break;
}
tp.add(Executer(cmd, {tmp}, quiet));
}
else {
locked_print(log_file, "read < 0\n");
break;
}
}
free(line);
fclose(fifo);
remove(fifo_name.c_str());
})));
I was wondering if you could help me understand what is going wrong here. I am trying to write a little client that engages in SMTP dialogue on port 25.
If you recall SMTP, there's a few things you need to send, and then you write the email after the DATA message, ending with a period on it's own line to send the email.
There is something problematic in the way the program handles this. It handles the dialogue fine until after the DATA message. It will recognize the period only if I type it first. After any subsequent line, any code execution seems to be lost. The if statement fails to recognize if a period has been entered. Thank you again. Attached is the relevant code..
void readstuff(int sock, char* buf) {
int r = read (sock, buf, BUFSIZE -1);
buf[r] = NULL;
cout << buf << endl;
}
void doit(int sock, string arg, char* buf) {
int r = write(sock, arg.c_str(), arg.length());
readstuff(sock, buf);
}
int main(int argc, char *argv[]) {
char buf[BUFSIZE];
// Make a socket
int sock = MakeSocket(argv[1], argv[2]);
cout << "socket is " << sock << endl;
assert(sock != -1);
// Begin dialogue
doit(sock, "HELO " + org.substr(org.find("#") + 1) + "\r\n", buf);
doit(sock, "MAIL FROM: <" + org + "> \r\n", buf);
doit(sock, "RCPT TO: <" + dest + "> \r\n", buf);
doit(sock, "DATA \r\n", buf);
readstuff(sock, buf); //should say "go ahead"
//User writes email here
while (true) {
string line = "";
getline(cin, line);
doit(sock, line + "\r\n", buf);
if (line == ".") {
readstuff(sock, buf); //should say "email cleared to send"
return 0;
}
}
}
Please read the SMTP specification, RFC 5321, particularly sections 4.1.1.4 DATA and 4.5.2 Transparency:
4.1.1.4. DATA (DATA)
The receiver normally sends a 354 response to DATA, and then treats
the lines (strings ending in <CRLF> sequences, as described in
Section 2.3.7) following the command as mail data from the sender.
This command causes the mail data to be appended to the mail data
buffer. The mail data may contain any of the 128 ASCII character
codes, although experience has indicated that use of control
characters other than SP, HT, CR, and LF may cause problems and
SHOULD be avoided when possible.
The mail data are terminated by a line containing only a period, that
is, the character sequence "<CRLF>.<CRLF>", where the first <CRLF> is
actually the terminator of the previous line (see Section 4.5.2).
This is the end of mail data indication. The first <CRLF> of this
terminating sequence is also the <CRLF> that ends the final line of
the data (message text) or, if there was no mail data, ends the DATA
command itself (the "no mail data" case does not conform to this
specification since it would require that neither the trace header
fields required by this specification nor the message header section
required by RFC 5322 [4] be transmitted). An extra <CRLF> MUST NOT
be added, as that would cause an empty line to be added to the
message. The only exception to this rule would arise if the message
body were passed to the originating SMTP-sender with a final "line"
that did not end in <CRLF>; in that case, the originating SMTP system
MUST either reject the message as invalid or add <CRLF> in order to
have the receiving SMTP server recognize the "end of data" condition.
The custom of accepting lines ending only in <LF>, as a concession to
non-conforming behavior on the part of some UNIX systems, has proven
to cause more interoperability problems than it solves, and SMTP
server systems MUST NOT do this, even in the name of improved
robustness. In particular, the sequence "<LF>.<LF>" (bare line
feeds, without carriage returns) MUST NOT be treated as equivalent to
<CRLF>.<CRLF> as the end of mail data indication.
Receipt of the end of mail data indication requires the server to
process the stored mail transaction information. This processing
consumes the information in the reverse-path buffer, the forward-path
buffer, and the mail data buffer, and on the completion of this
command these buffers are cleared. If the processing is successful,
the receiver MUST send an OK reply. If the processing fails, the
receiver MUST send a failure reply. The SMTP model does not allow
for partial failures at this point: either the message is accepted by
the server for delivery and a positive response is returned or it is
not accepted and a failure reply is returned. In sending a positive
"250 OK" completion reply to the end of data indication, the receiver
takes full responsibility for the message (see Section 6.1). Errors
that are diagnosed subsequently MUST be reported in a mail message,
as discussed in Section 4.4.
When the SMTP server accepts a message either for relaying or for
final delivery, it inserts a trace record (also referred to
interchangeably as a "time stamp line" or "Received" line) at the top
of the mail data. This trace record indicates the identity of the
host that sent the message, the identity of the host that received
the message (and is inserting this time stamp), and the date and time
the message was received. Relayed messages will have multiple time
stamp lines. Details for formation of these lines, including their
syntax, is specified in Section 4.4.
Additional discussion about the operation of the DATA command appears
in Section 3.3.
Syntax:
data = "DATA" CRLF
4.5.2. Transparency
Without some provision for data transparency, the character sequence
"<CRLF>.<CRLF>" ends the mail text and cannot be sent by the user.
In general, users are not aware of such "forbidden" sequences. To
allow all user composed text to be transmitted transparently, the
following procedures are used:
o Before sending a line of mail text, the SMTP client checks the
first character of the line. If it is a period, one additional
period is inserted at the beginning of the line.
o When a line of mail text is received by the SMTP server, it checks
the line. If the line is composed of a single period, it is
treated as the end of mail indicator. If the first character is a
period and there are other characters on the line, the first
character is deleted.
...
Your DATA command needs to account for that:
Your doit() function is expecting a response after sending a string. That is the wrong logic to use while sending the email data. You can't read a response until after the final terminating <CRLF>.<CRLF> has been sent.
you have to apply transparency to any line in the email that begins with a . character.
With that said, try something more like this:
int readLine(int sock, string &line)
{
// read a line from sock until CRLF is reached.
// I leave this as an exercise for you to implement...
line = ...;
return -1 on error, else 0;
}
int readResponse(int sock)
{
// Please read RFC 5321 section 4.2 for the PROPER format
// of an SMTP response. You should be reading from the
// socket until you receive the terminating
// "Reply-code [ SP textstring ] CRLF" line...
string line;
int r = readLine(sock, line);
if (r < 0) return r;
string code = line.substr(0, 3);
string text = line.substr(4);
if ((line.length() >= 4) && (line[3] = '-'))
{
do
{
r = readLine(sock, line);
if (r < 0) return r;
text += (" " + line.substr(4));
}
while (line.compare(0, 4, code+"-") == 0);
}
cout << code << ": " << text << endl;
return stoi(code);
}
int sendText(int sock, const string &arg)
{
const char *p = arg.c_str();
int len = arg.length();
while (len > 0)
{
int r = write(sock, p, len);
if (r <= 0) return -1;
p += r;
len -= r;
}
return 0;
}
int sendCmd(int sock, const string &arg)
{
int r = sendText(sock, arg + "\r\n");
if (r < 0) return r;
return readResponse(sock);
}
int main(int argc, char *argv[])
{
// Make a socket
int sock = MakeSocket(argv[1], argv[2]);
cout << "socket is " << sock << endl;
assert(sock != -1);
// Begin dialogue
// read the server greeting first...
if (readResponse(sock) != 220) {
// failed, do something...
}
if (sendCmd(sock, "HELO " + org.substr(org.find("#") + 1)) != 250) {
// failed, do something...
}
if (sendCmd(sock, "MAIL FROM: <" + org + ">") != 250) {
// failed, do something...
}
int r = sendCmd(sock, "RCPT TO: <" + dest + ">");
if ((r != 250) && (r != 251)) {
// failed, do something...
}
if (sendCmd(sock, "DATA") != 354) {
// failed, do something...
}
//User writes email here
while (true) {
string line;
getline(cin, line);
// A line consisting of only "." is a valid line in an email
// message, so you should not use that as a terminator in your
// input, use something else, like an EOF marker, or CTRL-C,
// or something...
if (some termination condition)
break;
// DO NOT call readResponse() here!
if (!line.empty() && (line[0] == '.')) {
if (sendText(sock, ".") < 0) {
// failed, do something...
}
}
if (sendText(sock, line) < 0) {
// failed, do something...
}
if (sendText(sock, "\r\n") < 0) {
// failed, do something...
}
}
// NOW call readResponse() here!
if (sendCmd(sock, ".") != 250) {
// failed, do something...
}
sendCmd(sock, "QUIT");
close(sock);
return 0;
}
I have code that parses a configuration file, which may send output to stdout or stderr in case of errors.
Unfortunately, when I pipe the output of my program to /dev/null, I get an exception: ios_base::clear: unspecified iostream_category error with stderror Inappropriate ioctl for device.
Here is the relevant part of my code:
try {
file.exceptions(std::ios::failbit | std::ios::badbit);
file.open(config_file);
// file.exceptions(std::ios::failbit);
}
catch (std::ios_base::failure& e) {
throw std::invalid_argument(std::string("Can't open config file ") + config_file + ": " + strerror(errno));
}
try {
errno = 0; // reset errno before I/O operation.
// ...
while (std::getline(file, line)) {
if ( [... unknown line ...] ) {
std::cerr << "Invalid line in config " << config_file << ": " << line << std::endl;
continue;
}
// debug info to STDOUT:
std::cout << config_file << ": " << line << std::endl;
}
} catch (std::ios_base::failure& err) {
std::cout << "Caught exception " << err.what() << std::endl;
if (errno != 0) {
char* theerror = strerror(errno);
file.close();
throw std::invalid_argument(std::string("Can't read config file ") + config_file + ": " + theerror);
}
}
try {
file.close();
}
catch (std::ios_base::failure& e) {
throw std::invalid_argument(std::string("Can't close config file ") + config_file + ": " + strerror(errno));
}
Here is an example of an exception:
~> ./Application test1.conf > /dev/null
test1.conf: # this is a line in the config file
Caught exception ios_base::clear: unspecified iostream_category error
When I don't pipe to /dev/null (but to stdout or a regular file), all is fine. I first suspected that the cout and cerr where causing problems, but I'm not sure.
I finally found that I could resolve this by enabling this line after opening the file, so that the badbit-type of exceptions are ignored.
file.exceptions(std::ios::failbit);
Frankly, I'm too novice in C++ to understand what is going on here.
My questions: what is causing the unspecified iostream_category exception? How can I avoid it? Is setting file.exceptions(std::ios::failbit); indeed a proper solution, or does that give other pitfalls? (A pointer to a good source detailing best practices for opening files in C++, which does included all proper exception handling, or some background explained, is highly appreciated!)
I would recommend the following approach. This is based on my own experience as well as on some of the links that have been provided above. In short I would suggest the following:
Don't turn on the exceptions when using C++ streams. They are just so hard to get right I find they make my code less readable, which sort of defeats the purpose of exceptions. (IMHO it would be better if the C++ streams used exceptions by default and in a more reasonable manner. But since it isn't built that way it is better not to force the issue and just follow the pattern that the designers seem to have had in mind.)
Rely on the fact that getline will handle the various stream bits properly. You don't need to check after each call if the bad or fail bits are set. The stream returned by getline will be implicitly cast to false when these occur.
Restructure your code to follow the RAII pattern so that you don't need to call open() or close() manually. This not only simplifies your code, but ensures that you don't forget to close it. If you are not familiar with the RAII pattern, see https://en.wikipedia.org/wiki/Resource_acquisition_is_initialization.
Don't repeatedly put things like errno or your filename into the exceptions you generate. Keep them minimal to avoid repetition and use a catch block at the bottom to handle the errors, possibly throwing a new exception that adds the details you wish to report.
With that said, I would recommend rewriting your code to look something like the following:
using namespace std;
try {
errno = 0;
// Construct the stream here (part of RAII) then you don't need to call
// open() or close() manually.
ifstream file(config_file);
if (!file.is_open()) {
throw invalid_argument("Could not open the file");
}
while (getline(file, line)) {
// do all your processing as if no errors will occur (getline will
// be explicitly cast to a false when an error occurs). If there is
// something wrong with a line (bad format, etc.) then throw an
// exception without echoing it to cerr.
}
if (file.bad()) {
throw invalid_argument("Problem while reading file");
}
}
catch (const invalid_argument& e) {
// Whatever your error handling needs to be. config_file should still
// be valid so you can add it here, you don't need to add it to each
// individual exception. Also you can echo to cerr here, you don't need
// to do it inside the loop. If you want to use errno it should still
// be set properly at this point. If you want to pass the exception to
// the next level you can either rethrow it or generate a new one that
// adds additional details (like strerror and the filename).
}
I've improved on my earlier answer by writing a couple of functions that handle the stream checks using lambdas to actually process the file. This has the following advantages:
You don't forget where to put the stream checks.
Your code concentrates on what you want to do (the file processing) and not on the system boilerplate items.
I've created two versions. With the first your lambda is given the stream and you can process it how you like. With the second your lambda is given one line at a time. In both cases if an I/O problem occurs it will throw a system_error exception. You can also throw your own exceptions in your lambda and they will be passed on properly.
namespace {
inline void throwProcessingError(const string& filename, const string& what_arg) {
throw system_error(errno, system_category(), what_arg + " " + filename);
}
}
void process_file(const string& filename, function<void (ifstream &)> fn) {
errno = 0;
ifstream strm(filename);
if (!strm.is_open()) throwProcessingError(filename, "Failed to open");
fn(strm);
if (strm.bad()) throwProcessingError(filename, "Failed while processing");
}
void process_file_line_by_line(const string& filename, function<void (const string &)> fn)
{
errno = 0;
ifstream strm(filename);
if (!strm.is_open()) throwProcessingError(filename, "Failed to open");
string line;
while (getline(strm, line)) {
fn(line);
}
if (strm.bad()) throwProcessingError(filename, "Failed while processing");
}
To use them, you would call them as follows...
process_file("myfile.txt", [](ifstream& stream) {
... my processing on stream ...
});
or
process_file_line_by_line("myfile.txt", [](const string& line) {
... process the line ...
});
I'm just trying to get the contents of a page with their headers...but it seems that my buffer of size 1024 is either too large or too small for the last packet of information coming through...I don't want to get too much or too little, if that makes sense. Here's my code. It's printing out the page just fine with all the information, but I want to ensure that it's correct.
//Build HTTP Get Request
std::stringstream ss;
ss << "GET " << url << " HTTP/1.0\r\nHost: " << strHostName << "\r\n\r\n";
std::string req = ss.str();
// Send Request
send(hSocket, req.c_str(), strlen(req.c_str()), 0);
// Read from socket into buffer.
do
{
nReadAmount = read(hSocket, pBuffer, sizeof pBuffer);
printf("%s", pBuffer);
}
while(nReadAmount != 0);
nReadAmount = read(hSocket, pBuffer, sizeof pBuffer);
printf("%s", pBuffer);
This is broken. You can only use the %s format specifier for a C-style (zero-terminated) string. How is printf supposed to know how many bytes to print? That information is in nReadAmount, but you don't use it.
Also, you call printf even if read fails.
The simplest fix:
do
{
nReadAmount = read(hSocket, pBuffer, (sizeof pBuffer) - 1);
if (nReadAmount <= 0)
break;
pBuffer[nReadAmount] = 0;
printf("%s", pBuffer);
} while(1);
The correct way to read an HTTP reply is to read until you have received a full LF-delimited line (some servers use bare LF even though the official spec says to use CRLF), which contains the response code and version, then keep reading LF-delimited lines, which are the headers, until you encounter a 0-length line, indicating the end of the headers, then you have to analyze the headers to figure out how the remaining data is encoded so you know the proper way to read it and know how it is terminated. There are several different possibilities, refer to RFC 2616 Section 4.4 for the actual rules.
In other words, your code needs to use this kind of structure instead (pseudo code):
// Send Request
send(hSocket, req.c_str(), req.length(), 0);
// Read Response
std::string line = ReadALineFromSocket(hSocket);
int rescode = ExtractResponseCode(line);
std::vector<std::string> headers;
do
{
line = ReadALineFromSocket(hSocket);
if (line.length() == 0) break;
headers.push_back(line);
}
while (true);
if (
((rescode / 100) != 1) &&
(rescode != 204) &&
(rescode != 304) &&
(request is not "HEAD")
)
{
if ((headers has "Transfer-Encoding") && (Transfer-Encoding != "identity"))
{
// read chunks until a 0-length chunk is encountered.
// refer to RFC 2616 Section 3.6 for the format of the chunks...
}
else if (headers has "Content-Length")
{
// read how many bytes the Content-Length header says...
}
else if ((headers has "Content-Type") && (Content-Type == "multipart/byteranges"))
{
// read until the terminating MIME boundary specified by Content-Type is encountered...
}
else
{
// read until the socket is disconnected...
}
}