I am encountering a problem when I am trying to write to sysfs node.
In the below code I am trying to wite to a trace_marker file. In the ftrace log, the first write is successful. But after that the write fails.
The file descriptor seemingly closes.
I do not want to open file every time before writing as writes are too frequent.
class Logger {
int mFileFd;
void logFromAnotherThread(std::string s) {
std::unique_lock<std::mutex> ul(mLogMu);
...
int count = write(mFileFd, s.c_str(), s.length());
if (count > 0)
std::cout << "Wrote n bytes: " << count << std::endl;
else
std::cout << "Errornum: " << strerror(errno) << std::endl;
...
}
Logger() {
mFileFd = open(SYSFS_NODE_WRITE, O_WRONLY);
....
}
}
First write is succesful.
I get output as-
Errornum: Bad file descriptor
My expectation is file open should be once, file descrtiptor should remain open for entire duration, and close on exit.
Edit 1:
Thank you for the suggestions on object getting destroyed. But I ensure that object is not getting destroyed.
For debugging, I had removed class/structure. Logging is now in simple C++ function calls. The file descriptor is a global variable, initialized once in main.
It does not works.
My confusion was is it something to do with the way write operations are performed on sysfs node.
Or can this be because of the number of writes are high (about 2-3 logs in 10us).
I am doing this like below, but this has an overhead of two added system calls.
#define TRACE_MARKER_FILE "/sys/kernel/debug/tracing/trace_marker"
void logdata(pid_t tid, std::string mystring) {
if(useLogger) {
std::stringstream ss;
if (funcname.length() > 0)
ss << LOGTAG << mystring;
int tempfd = open(TRACE_MARKER_FILE, O_WRONLY);
int count = write(tempfd, ss.str().c_str(), ss.str().length());
if (count == 0) {
std::cout << "Errornum: " << strerror(errno) <<std::endl;
}
close(tempfd);
}
}
I would suggest using std::ofstream which allows for proper RAII, in other words it will open the file in your constructor, then will automatically close the file in your destructor
class Logger {
std::ofstream mFile;
void logFromAnotherThread(std::string s) {
std::unique_lock<std::mutex> ul(mLogMu);
...
mFile << s;
...
}
Logger() : mFile(SYSFS_NODE_WRITE, O_WRONLY) {
....
}
}
Related
I have an application that must monitor some folders (in Windows) to detect if a file was created in that folder (real use is to detect incoming FTP files).
If a file is detected , it is read, then deleted .
Occasionally, I get a file reading error on a file that was detected.
Question is: Why?
To simulate the error, I created a simple program to reproduce it:
std::vector<std::filesystem::path> watch;
void main()
{
watch.push_back("D:\\test1"); //must exist
watch.push_back("D:\\test2");
watch_dir();
}
this example monitors 2 folders.
To simulate incoming files on the folder, another program copies files to that folder
continuously at configurable intervals (say 100 milliseconds).
To detect folder changes , WIN32 API functions FindFirstChangeNotification and WaitForMultipleObjects are used, based on this Microsoft example
https://learn.microsoft.com/en-us/windows/win32/fileio/obtaining-directory-change-notifications
detection function adapted from the example (Note: WaitForMultipleObjects blocks until a change is detected)
void watch_dir()
{
HANDLE handle[2];
memset(handle, 0, 2 * sizeof(HANDLE));
for (size_t idx = 0; idx < watch.size(); idx++)
{
std::string str = watch.at(idx).string();
LPTSTR path = (LPTSTR)str.c_str();
std::cout << "watch path " << path << std::endl;
handle[idx] = FindFirstChangeNotification(
path, // directory to watch
FALSE, // do not watch subtree
FILE_NOTIFY_CHANGE_FILE_NAME); // watch file name changes
if (handle[idx] == INVALID_HANDLE_VALUE)
{
assert(0);
ExitProcess(GetLastError());
}
}
while (TRUE)
{
std::cout << "Waiting for notification..." << std::endl;
DWORD wait_status = WaitForMultipleObjects(watch.size(), handle, FALSE, INFINITE);
std::cout << "Directory " << watch.at(wait_status) << " changed" << std::endl;
if (FindNextChangeNotification(handle[wait_status]) == FALSE)
{
assert(0);
ExitProcess(GetLastError());
}
std::filesystem::path path = watch.at(wait_status);
send_files_in_path(path);
}
}
Once a change is detected by the function above, then all files in the folder are listed
and read, by these functions
void send_files_in_path(const std::filesystem::path& ftp_path)
{
std::vector<std::filesystem::path> list = get_files(ftp_path);
for (size_t idx = 0; idx < list.size(); idx++)
{
std::string buf;
read_file(list.at(idx).string(), buf);
std::this_thread::sleep_for(std::chrono::milliseconds(10));
std::filesystem::remove(list.at(idx));
}
}
/////////////////////////////////////////////////////////////////////////////////////////////////////
//get_files
//get all ".txt" files inside a FTP folder
/////////////////////////////////////////////////////////////////////////////////////////////////////
std::vector<std::filesystem::path> get_files(const std::filesystem::path& base_archive_path)
{
std::vector<std::filesystem::path> list;
try
{
for (const auto& entry : std::filesystem::recursive_directory_iterator(base_archive_path))
{
std::filesystem::path path = entry.path();
if (!entry.is_regular_file())
{
continue;
}
std::string fname = entry.path().filename().string();
size_t len = fname.size();
size_t pos = len - 4;
//check if last 4 characters are ".txt"
if (fname.find(".txt", pos) == std::string::npos && fname.find(".TXT", pos) == std::string::npos)
{
continue;
}
SPDLOG_INFO("loading: " + entry.path().string());
list.push_back(path);
}//this path
} //try
catch (const std::exception& e)
{
SPDLOG_ERROR(e.what());
}
return list;
}
The function where the error happens is
int read_file(const std::string& fname, std::string& buf)
{
std::ifstream ifs;
std::ios_base::iostate mask = ifs.exceptions() | std::ios::failbit;
ifs.exceptions(mask);
std::this_thread::sleep_for(std::chrono::milliseconds(0));
std::cout << "opening : " << fname << std::endl;
try
{
ifs.open(fname);
if (!ifs.is_open())
{
std::cout << "open fail: " << fname << std::endl;
return -1;
}
}
catch (const std::exception& e)
{
std::cout << e.what() << std::endl;
return -1;
}
std::stringstream ss;
ss << ifs.rdbuf();
ifs.close();
buf = ss.str();
return 0;
}
the try/catch block, again, occasionally , is triggered with the error
ios_base::failbit set: iostream stream error
removing the try/catch block, and the open mask (just to try), then
ifs.is_open
fails.
A temporary solution was to detect the cases where the open() failed and repeat it.. which succeeds, because the file does exist.
Calling this with a small delay before the open call has the effect of reducing the open fails
std::this_thread::sleep_for(std::chrono::milliseconds(10));
ifs.open(fname);
But still would like to find out the reason for the occasional failure
I'm about to write an IRCBot using Boost.Asio and I have the function getMsg:
std::string getMsg()
{
buffer.clear(); //make sure buffer is empty
buffer.resize(512); //make sure it's big enough for 512char
socket.read_some(boost::asio::buffer(&buffer[0],buffer.size()));
std::size_t pos = buffer.find("PING :");
if(pos != std::string::npos)
{
sendMsg("PONG :" + buffer.substr(pos + 6));
}
return buffer;
}
In my main function when using std::cout << Text; I get an output, but when trying std::cout << "Hello", nothing seems to happen:
while(true)
{
std::string Text = Test.getMsg();
std::cout << Text; //OUTPUT
}
while(true)
{
std::string Text = Test.getMsg();
std::cout << "TEST"; //NO OUTPUT ---- WHY?
}
The error you are asking about most likely occurs because you don't flush the stdout: std::cout << "TEST" << std::flush; This has nothing to do with boost::asio.
However your asio code also has a possible error: You are looking for PING : there in a single read call which might never be received within a single read call, due to the fact of how TCP works (it's a stream, not packets). If it's UDP socket it would work.
I have a multi-threaded C++03 application that presently uses popen() to invoke itself (same binary) and ssh (different binary) again in a new process and reads the output, however, when porting to Android NDK this is posing some issues such as not not having permissions to access ssh, so I'm linking in Dropbear ssh to my application to try and avoid that issue. Further, my current popen solution requires that stdout and stderr be merged together into a single FD which is a bit messy and I'd like to stop doing that.
I would think the pipe code could be simplified by using fork() instead but wonder how to drop all of the parent's stack/memory which is not needed in the child of the fork? Here is a snippet of the old working code:
#include <iostream>
#include <stdio.h>
#include <string>
#include <errno.h>
using std::endl;
using std::cerr;
using std::cout;
using std::string;
void
doPipe()
{
// Redirect stderr to stdout with '2>&1' so that we see any error messages
// in the pipe output.
const string selfCmd = "/path/to/self/binary arg1 arg2 arg3 2>&1";
FILE *fPtr = ::popen(selfCmd.c_str(), "r");
const int bufSize = 4096;
char buf[bufSize + 1];
if (fPtr == NULL) {
cerr << "Failed attempt to popen '" << selfCmd << "'." << endl;
} else {
cout << "Result of: '" << selfCmd << "':\n";
while (true) {
if (::fgets(buf, bufSize, fPtr) == NULL) {
if (!::feof(fPtr)) {
cerr << "Failed attempt to fgets '" << selfCmd << "'." << endl;
}
break;
} else {
cout << buf;
}
}
if (pclose(fPtr) == -1) {
if (errno != 10) {
cerr << "Failed attempt to pclose '" << selfCmd << "'." << endl;
}
}
cout << "\n";
}
}
So far, this is loosely what I have done to convert to fork(), but fork needlessly duplicates the entire parent process memory space. Further, it does not quite work, because the parent never sees EOF on the outFD it is reading from the pipe(). Where else do I need to close the FDs for this to work? How can I do something like execlp() without supplying a binary path (not easily available on Android) but instead start over with the same binary and a blank image with new args?
#include <iostream>
#include <stdio.h>
#include <string>
#include <errno.h>
using std::endl;
using std::cerr;
using std::cout;
using std::string;
int
selfAction(int argc, char *argv[], int &outFD, int &errFD)
{
pid_t childPid; // Process id used for current process.
// fd[0] is the read end of the pipe and fd[1] is the write end of the pipe.
int fd[2]; // Pipe for normal communication between parent/child.
int fdErr[2]; // Pipe for error communication between parent/child.
// Create a pipe for IPC between child and parent.
const int pipeResult = pipe(fd);
if (pipeResult) {
cerr << "selfAction normal pipe failed: " << errno << ".\n";
return -1;
}
const int errorPipeResult = pipe(fdErr);
if (errorPipeResult) {
cerr << "selfAction error pipe failed: " << errno << ".\n";
return -1;
}
// Fork - error.
if ((childPid = fork()) < 0) {
cerr << "selfAction fork failed: " << errno << ".\n";
return -1;
} else if (childPid == 0) { // Fork -> child.
// Close read end of pipe.
::close(fd[0]);
::close(fdErr[0]);
// Close stdout and set fd[1] to it, this way any stdout of the child is
// piped to the parent.
::dup2(fd[1], STDOUT_FILENO);
::dup2(fdErr[1], STDERR_FILENO);
// Close write end of pipe.
::close(fd[1]);
::close(fdErr[1]);
// Exit child process.
exit(main(argc, argv));
} else { // Fork -> parent.
// Close write end of pipe.
::close(fd[1]);
::close(fdErr[1]);
// Provide fd's to our caller for stdout and stderr:
outFD = fd[0];
errFD = fdErr[0];
return 0;
}
}
void
doFork()
{
int argc = 4;
char *argv[4] = { "/path/to/self/binary", "arg1", "arg2", "arg3" };
int outFD = -1;
int errFD = -1;
int result = selfAction(argc, argv, outFD, errFD);
if (result) {
cerr << "Failed to execute selfAction." << endl;
return;
}
FILE *outFile = fdopen(outFD, "r");
FILE *errFile = fdopen(errFD, "r");
const int bufSize = 4096;
char buf[bufSize + 1];
if (outFile == NULL) {
cerr << "Failed attempt to open fork file." << endl;
return;
} else {
cout << "Result:\n";
while (true) {
if (::fgets(buf, bufSize, outFile) == NULL) {
if (!::feof(outFile)) {
cerr << "Failed attempt to fgets." << endl;
}
break;
} else {
cout << buf;
}
}
if (::close(outFD) == -1) {
if (errno != 10) {
cerr << "Failed attempt to close." << endl;
}
}
cout << "\n";
}
if (errFile == NULL) {
cerr << "Failed attempt to open fork file err." << endl;
return;
} else {
cerr << "Error result:\n";
while (true) {
if (::fgets(buf, bufSize, errFile) == NULL) {
if (!::feof(errFile)) {
cerr << "Failed attempt to fgets err." << endl;
}
break;
} else {
cerr << buf;
}
}
if (::close(errFD) == -1) {
if (errno != 10) {
cerr << "Failed attempt to close err." << endl;
}
}
cerr << "\n";
}
}
There are two kinds of child processes created in this fashion with different tasks in my application:
SSH to another machine and invoke a server that will communicate back to the parent that is acting as a client.
Compute a signature, delta, or merge file using rsync.
First of all, popen is a very thin wrapper on top of fork() followed by exec() [and some call to pipe and dup and so on to manage the ends of a pipe] .
Second, the memory is only duplicated in form of "copy-on-write" memory - meaning that unless one of the processes writes to some page, the actual physical memory is shared between the two processes.
It does mean, of course, the OS has to create a memory map with 4-8 bytes per 4KB [in typical cases] (probably plus some internal OS data to track how many copies there are of that page and stuff - but as long as the page remains the same one as the parent process, the child page uses the parent processes internal data). Compared to everything else involved in creating a new process and loading an executable file into the new process, it's a pretty small part of the time. Since you are almost immediately doing exec, not much of the parent process' memory will be touched, so very little will happen there.
My advice would be that if popen works, keep using popen. If popen doesn't quite do what you want for some reason, then use fork + exec - but make sure you know what the reason for doing so is.
Below is a simple class which attempts to write an integer to a file. The mode of writing the file is to append characters at the end of the file (In this mode, file should be created if it doesn't exist)
#include <iostream>
#include <fstream>
class TestFileStream
{
private:
std::ofstream* _myFileStream;
bool isFileOpen;
public:
TestFileStream():isFileOpen(false)
{
_myFileStream = new std::ofstream("TestFile.txt", std::ios_base::out | std::ios_base::app );
isFileOpen = _myFileStream->is_open();
if( !isFileOpen )
{
std::cout << "Unable to open log file" << std::endl;
std::cout << "Good State: " << _myFileStream->good() <<std::endl;
std::cout << "Eof State: " << _myFileStream->eof() <<std::endl;
std::cout << "Fail State: " << _myFileStream->fail() <<std::endl;
std::cout << "Bad State: " << _myFileStream->bad() <<std::endl;
}
else
{
std::cout << "Opened log file" << std::endl;
}
}
~TestFileStream()
{
_myFileStream->close();
delete _myFileStream;
_myFileStream = nullptr;
}
void WriteFile( unsigned number )
{
if ( isFileOpen )
{
(*_myFileStream) << "Number: " << number << std::endl;
}
}
};
int main()
{
// Number of iterations can be multiple.
// For testing purpose, only 1 loop iteration executes
for( unsigned iter = 1; iter != 2; ++iter )
{
TestFileStream fileWriteObj;
fileWriteObj.WriteFile( 100+iter );
}
return 0;
}
When I execute the above code, I get following log output:
Unable to open log file
Good State: 0
Eof State: 0
Fail State: 1
Bad State: 0
This seems like trivial task, but I am not able to find out whats causing the failure. Note that this question is most likely related to the following question
Just to summarize the comments, there is nothing wrong about the code you posted (apart from the rather unconventional new ostream ;) )
Note however that opening files may fail for a number of reasons (Permissions, file in use, disk unavailable, the file does not exist, the file exists...). That is why you must always test it.
If you tried to run the above code in an online emulator, then chances are file IO is disabled. Which would explain why you get that the streams fail-bit is set.
A RAII file handle looks pretty basic so I guess it has already been implemented? But I couldn't find any implementation. I found file_descriptor in boost::iostreams but I don't know if it's what I'm looking for.
std::fstreams support RAII-style usage - they can be opened and even tested at construction, and they're automatically flushed and closed in the destructor, though you could miss errors if you just assume that works so you may want to do something more explicit in code if you need the robustness.
For example:
if (std::ifstream input(filename))
... use input...
else
std::cerr << "unable to open '" << filename << "'\n";
If you really want to use file descriptors, you can tune something like the following to taste. It's a bit longer than something that just invokes close, but if you want to do robust programming you need to check for and handle errors somehow....
struct Descriptor
{
Descriptor(int fd, const char* filename = nullptr)
: fd_(fd), filename_(filename)
{
if (fd < 0)
{
std::ostringstream oss;
oss << "failed to open file";
if (filename_) oss << " '" << filename_ << '\'';
oss << ": " << strerror(errno);
throw std::runtime_error(oss.str());
}
}
~Descriptor()
{
if (fd_ != -1 && close(fd_) == -1)
{
// throwing from destructors risks termination - avoid...
std::cerr << "failed to close file";
if (filename_) std::cerr << " '" << filename_ << '\'';
std::cerr << ": " << strerror(errno) << std::endl;
}
}
operator int() const { return fd_; }
private:
int fd_;
};
Usage:
try
{
Descriptor fd(open(filename, O_RDONLY), filename);
int nbytes = read(fd, ...);
...
}
catch ...
Depends on what exactly you want.
If you really want a scoped handle, use:
std::unique_ptr<HANDLETYPE, closehandletypefunction> smartpointer;
For FILE pointers, this would look like
std::unique_ptr<FILE, int (*)(FILE *)> f(fopen("myfile.txt", "a"), fclose);
The FILE* can then be gotten with f.get(). The same thing would work with file descriptors (open and close from <fcntl.h> and <unistd.h>, respectively).
The preferred C++ way is wrapping the handle in an object with thousands of members to do everything though.
I am using boost::filesystem::ifstream (or ofstream for writing).
I was actually asking this because I wanted to be sure that my file was closed even if an exception was raised before calling file.close()
But after reading the documentation again:
In case that an object is destroyed while still associated with an open file, the destructor automatically calls the member function close.
So, this is safe :)