FileLocker_wo.h
#include <string>
namespace Utils
{
namespace FileLocker
{
bool lock_file(std::string aFileName, int& aFileDescriptor);
bool unlock_file(int& aFileDescriptor);
bool is_file_locked(std::string aFileName);
};
}
FileLocker_wo.cpp
namespace Utils
{
namespace FileLocker
{
bool lock_file(std::string aFileName, int& aFileDescriptor)
{
aFileDescriptor = open(aFileName.c_str(), O_RDWR);
if (aFileDescriptor != -1)
{
if (lockf(aFileDescriptor, F_TLOCK, 0) == 0)
{
return true;
}
std::cout << strerror(errno) << std::endl;
}
return false;
}
bool unlock_file(int& aFileDescriptor)
{
if (lockf(aFileDescriptor, F_ULOCK, 0) == 0)
{
std::cout << "unloced file" << std::endl;
close(aFileDescriptor);
return true;
}
close(aFileDescriptor);
return false;
}
bool is_file_locked(std::string aFileName)
{
int file_descriptor = open(aFileName.c_str(), O_RDWR);
if (file_descriptor != -1)
{
int ret = lockf(file_descriptor, F_TEST, 0);
if (ret == -1 && (errno == EACCES || errno == EAGAIN))
{
std::cout << "locked by another process" << std::endl;
close(file_descriptor);
return true;
}
if (ret != 0)
{
std::cout << "return value is " << ret << " " << strerror(errno) << std::endl;
}
}
close(file_descriptor);
return false;
}
}
}
p1.cpp
#include <iostream>
#include <fstream>
#include "FileLocker_wo.h"
int main()
{
int fd = -1;
if (Utils::FileLocker::lock_file("hello.txt", fd))
{
std::ofstream out("hello.txt");
out << "hello ding dong" << std::endl;
out.close();
std::cout << "locked" << std::endl;
sleep(5);
if (Utils::FileLocker::unlock_file(fd))
{
std::cout << "unlocked" << std::endl;
}
}
return 0;
}
p2.cpp
#include "FileLocker_wo.h"
#include <iostream>
#include <fstream>
int main()
{
int max_trys = 2;
int trys = 0;
bool is_locked = false;
do
{
is_locked = Utils::FileLocker::is_file_locked("hello.txt");
if (!is_locked)
{
std::cout << "not locked" << std::endl;
break;
}
std::cout << "locked" << std::endl;
sleep(1);
++trys;
}
while(trys < max_trys);
if (!is_locked)
{
std::string s;
std::ifstream in("hello.txt");
while(getline(in,s))
{
std::cout << "s is " << s << std::endl;
}
}
return 0;
}
I am trying to get a file lock in one process and checking whether there is any lock on that file in other process using lockf (p1.cpp, p2.cpp).
In p1.cpp I am locking the file hello.txt and waiting for 5 seconds. Meanwhile I start p2.cpp and checking whether any lock is there by other process, but always getting there is no lock> I am stuck with this for last 2 hours.
Can anybody tell what is wrong in this?
You've tripped over one of the nastier design errors in POSIX file locks. You probably didn't know about this because you only read the lockf manpage, not the fcntl manpage, so here's the important bit of the fcntl manpage:
If a process closes any file descriptor referring to a file, then
all of the process's locks on that file are released, regardless of
the file descriptor(s) on which the locks were obtained.
What this means is, in this bit of your code
if (Utils::FileLocker::lock_file("hello.txt", fd))
{
std::ofstream out("hello.txt");
out << "hello ding dong" << std::endl;
out.close();
you lose your lock on the file when you call out.close(), even though out is a different OS-level "open file description" than you used in lock_file!
In order to use POSIX locks safely you must ensure that you call open() on the file to be locked once and only once per process, you must never duplicate the file descriptor, and you must only close it again when you are ready to drop the lock. Because there may not be any way (even using unportable extensions) to construct an iostreams object from a file descriptor, or to extract a file descriptor from an iostreams object, the path of least resistance is to use only OS-level I/O primitives (open, close, read, write, fcntl, lseek, ftruncate) with files that you need to apply POSIX locks to.
Related
I wrote the following code:
void execute() {
std::cout << "smash pid is " << getpid() << std::endl;
}
int main()
{
int pid=fork();
if (pid==0)
{
int fd=open("my_file.txt", O_WRONLY | O_CREAT, 0666); // 3=my_file
dup2(fd,1); // replace output stream
close(fd); //close duplicate access to my_file
execute();
close (1); // close last access to my file
}
else if (pid>0)
{
std::cout << "Hello!" << std::endl;
}
return 0;
}
my question is am I doing things correctly? and can the main process still have access to print to terminal as usual?
I tried to add notes of what I am doing, please let me know if something isn't clear.
Version 2:
int main()
{
int pid=fork();
if (pid==0)
{
close (1);
int fd=open("my_file.txt", O_WRONLY | O_CREAT, 0666); // 3=my_file
execute();
close (1); // close last access to my file
}
else if (pid>0)
{
std::cout << "Hello!" << std::endl;
}
return 0;
}
my question is am I doing things correctly?
Your child will have its output directed into the opened file and the parent will write "Hello!" to the stdout that was provided to the program at startup. It looks like that's what you want, so, yes.
I'd use pid_t instead of int for the process id though, but that may be different on different platforms.
can the main process still have access to print to terminal as usual?
Yes. The dup2 in the child process does not affect the parent process in any way.
One note. Use fileno(stdout) instead of 1:
dup2(fd, fileno(stdout));
// and
close(fileno(stdout)); // not needed really
The int fileno(FILE*) function returns the internal file descriptor from the standard FILE* that is stdout. It's just a way of making it clearer to readers of the code.
You could however redirect stdout to a file using the standard C++ function std::freopen instead.
#include <unistd.h>
#include <sys/types.h>
#include <sys/wait.h>
#include <cstdio> // std::freopen
#include <iostream>
void execute() {
std::cout << "smash pid is " << getpid() << std::endl;
system("echo the redirect is inherited by the grand children too");
}
int main() {
pid_t pid = fork();
if(pid < 0) { // check for errors
std::perror("fork failed");
return 1;
}
if(pid == 0) { // child
// using std::freopen
if(std::freopen("my_file.txt", "w", stdout) == nullptr) {
std::perror("freopen failed");
return 1;
}
execute();
return 0;
}
// parent
std::cout << "Parent says hello!\n";
// wait for child
int wstatus;
if(waitpid(pid, &wstatus, 0) == pid) {
std::cout << "child exited with status " << wstatus << '\n';
} else {
std::perror("waitpid failed");
}
}
For my formation, an exercise ask us to create a program similar to the linux 'cat' command.
So to read the file, i use an ifstream, and everything work fine for regular file.
But not when i try to open /dev/ files like /dev/stdin: the 'enter' is not detected and so, getline really exit only when the fd is being closed (with a CTRL-D).
The problem seems to be around how ifstream or getline handle reading, because with the regular 'read' function from libc, this problem is not to be seen.
Here is my code:
#include <iostream>
#include <string>
#include <fstream>
#include <errno.h>
#ifndef PROGRAM_NAME
# define PROGRAM_NAME "cato9tails"
#endif
int g_exitCode = 0;
void
displayErrno(std::string &file)
{
if (errno)
{
g_exitCode = 1;
std::cerr << PROGRAM_NAME << ": " << file << ": " << strerror(errno) << std::endl;
}
}
void
handleStream(std::string file, std::istream &stream)
{
std::string read;
stream.peek(); /* try to read: will set fail bit if it is a folder. */
if (!stream.good())
displayErrno(file);
while (stream.good())
{
std::getline(stream, read);
std::cout << read;
if (stream.eof())
break;
std::cout << std::endl;
}
}
int
main(int argc, char **argv)
{
if (argc == 1)
handleStream("", std::cin);
else
{
for (int index = 1; index < argc; index++)
{
errno = 0;
std::string file = std::string(argv[index]);
std::ifstream stream(file, std::ifstream::in);
if (stream.is_open())
{
handleStream(file, stream);
stream.close();
}
else
displayErrno(file);
}
}
return (g_exitCode);
}
We can only use method from libcpp.
I have search this problem for a long time, and i only find this post where they seems to have a very similar problem to me:
https://github.com/bigartm/bigartm/pull/258#issuecomment-128131871
But found no really usable solution from them.
I tried to do a very ugly solution but... well...:
bool
isUnixStdFile(std::string file)
{
return (file == "/dev/stdin" || file == "/dev/stdout" || file == "/dev/stderr"
|| file == "/dev/fd/0" || file == "/dev/fd/1" || file == "/dev/fd/2");
}
...
if (isUnixStdFile(file))
handleStream(file, std::cin);
else
{
std::ifstream stream(file, std::ifstream::in);
...
As you can see, a lot of files are missing, this can only be called a temporary solution.
Any help would be appreciated!
The following code worked for me to deal with /dev/fd files or when using shell substitute syntax:
std::ifstream stream(file_name);
std::cout << "Opening file '" << file_name << "'" << std::endl;
if (stream.fail() || !stream.good())
{
std::cout << "Error: Failed to open file '" << file_name << "'" << std::endl;
return false;
}
while (!stream.eof() && stream.good() && stream.peek() != EOF)
{
std::getline(stream, buffer);
std::cout << buffer << std::endl;
}
stream.close();
Basically std::getline() fails when content from the special file is not ready yet.
I checked this code several times and cannot understand why does poll() return immediately?
Here file is opened for read and should wait for event. How to make it wait for input?
#include <iostream>
#include <poll.h>
#include <unistd.h>
#include <stdlib.h>
#include <fcntl.h>
#include <sys/types.h>
#include <sys/stat.h>
using namespace std;
ssize_t read_out_to_the_end(int fd){
char chunk[1024];
ssize_t ret = 0, n;
while((n = ::read(fd, chunk, sizeof chunk)) > 0){
ret += n;
cerr << "read chunk: " << n << " | ";
cerr.write(chunk, n);
cerr << endl;
}
if (n < 0) {
cerr << "err in read" << endl;
}
else if (ret == 0){
cerr << "nothing to read" << endl;
}
return ret;
}
int main() {
int bininfd = open("bin-in", O_RDONLY | O_CREAT);//, 0644/*S_IRUSR | S_IWUSR | S_IRGRP | S_IROTH*/);
if (bininfd < 0) {
perror("err in open(binin)");
return -1;
}
struct pollfd pollfds[] = {
{bininfd, POLLIN, 0},
};
auto&[pfd] = pollfds;
while (1) {
pfd.revents = 0; // cleanup, shouldn't it be redundant
int pollret = poll(pollfds, 1, -1);
if (pollret > 0) {
if (pfd.revents & POLLIN) {
cerr << "(pfd.revents & POLLIN)" << endl;
read_out_to_the_end(pfd.fd);
}
} else if (pollret == 0) {
cerr << "poll timed out" << endl;
continue;
} else {
cerr << "check for error" << endl;
continue;
}
}
}
the output is
(pfd.revents & POLLIN)
nothing to read
(pfd.revents & POLLIN)
nothing to read
(pfd.revents & POLLIN)
nothing to read
(pfd.revents & POLLIN)
nothing to read
(pfd.revents & POLLIN)
............... etc ....................
live example
UPDATE:
read_out_to_the_end() fixed. Thanks to #RemyLebeau
it works (blocks) on fifos as I expect, but not on regular files. Why?
poll() or select() never block on regular files. They always return a regular file as "ready". If you want to use poll() to do what tail -f does, you're on the wrong track.
Quoting from the SUSv4 standard:
The poll() function shall support regular files, terminal and
pseudo-terminal devices, FIFOs, pipes, sockets and [OB XSR] STREAMS-based files. The behavior of poll() on
elements of fds that refer to other types of file is unspecified.
Regular files shall always poll TRUE for reading and writing.
Since using poll() or select() on regular files is pretty much useless, newer interfaces have tried to remedy that. On BSD, you could use kqueue(2) with EVFILT_READ, and on Linux inotify(2) with IN_MODIFY. The newer epoll(7) interface on Linux will simply error out with EPERM if you try to watch a regular file.
Unfortunately, neither of those is standard.
read_out_to_the_end() has several issues:
ret is uninitialized.
The while loop is incrementing n when it should be assigning it instead. But then if the while loop hits the EOF, if( n == 0) will be true even if data was actually read before hitting EOF.
chunk may be null-terminated, but it may also receive nulls too, depending on the input data. So it should not be written to cerr (why not cout?) using operator<<, use cerr.write() instead so that you can pass it the actual number of bytes read.
Try this instead:
ssize_t read_out_to_the_end(int fd){
char chunk[1024];
ssize_t ret = 0, n;
while((n = ::read(fd, chunk, sizeof chunk)) > 0){
ret += n;
cerr << "read chunk: " << n << " | ";
cerr.write(chunk, n);
cerr << endl;
}
if (n < 0) {
cerr << "err in read" << endl;
}
else if (ret == 0){
cerr << "nothing to read" << endl;
}
return ret;
}
int main() {
int bininfd = open("bin-in", O_RDONLY | O_CREAT);//, 0644/*S_IRUSR | S_IWUSR | S_IRGRP | S_IROTH*/);
if (bininfd < 0) {
perror("err in open(binin)");
return -1;
}
pollfd pfd = {};
pfd.fd = bininfd;
pfd.events = POLLIN;
while (true) {
pfd.revents = 0; // cleanup, shouldn't it be redundant
int pollret = poll(&pfd, 1, -1);
if (pollret > 0) {
if (pfd.revents & POLLIN) {
cerr << "(pfd.revents & POLLIN)" << endl;
read_out_to_the_end(pfd.fd);
}
} else if (pollret == 0) {
cerr << "poll timed out" << endl;
continue;
} else {
cerr << "poll error " << errno << endl;
break;
}
}
}
Also, on a side note, the open() documentation says:
The mode argument specifies the file mode bits be applied when a new file is created. This argument must be supplied when O_CREAT or O_TMPFILE is specified in flags; if neither O_CREAT nor O_TMPFILE is specified, then mode is ignored. The effective mode is modified by the process's umask in the usual way: in the absence of a default ACL, the mode of the created file is (mode & ~umask). Note that this mode applies only to future accesses of the newly created file; the open() call that creates a read-only file may well return a read/write file descriptor.
This question already has answers here:
How to get the winapi id of a thread that has been created using the standard library?
(2 answers)
Closed 7 years ago.
Could it be posible to cast or convert a std::thread thread in C++ to a HANDLE in Windows?
I've been trying to manage threads in Windows with WINAPI functions for threads but I can't get it to work...
#include <thread>
#include <string>
#include <iostream>
#include <windows.h>
void Hi(std::string n){
while(true){
std::cout<<"Hi :3 "<<n<<"\n";
std::this_thread::sleep_for(std::chrono::seconds(1));
}
}
int main(void){
std::thread first(Hi, "zoditu");
first.detach();
getc(stdin);
//SuspendThread((void*)first.native_handle());
TerminateThread((void*)first.native_handle(), (unsigned long)0x00);
CloseHandle((void*)first.native_handle());
std::cout<<"No D:!!\n";
getc(stdin);
return 0;
}
But seems to do nothing because thread keeps spawning "Hi's" in the console... Could there be a way to "kill" it using WINAPI?
I don't think there is anything wrong with using the value returned by std::thread::native_handle() directly with the Win32 API functions (i.e., a conversion is not required).
The following program works for me. However, it usually (always?) crashes if the thread is terminated while it is actively executing but works just fine if the thread is suspended before terminating. As you are aware and others have pointed out it is generally not a good idea to terminate a thread.
But to answer your question the Win32 API seems to work as expected without any additional conversions. The following program works for me.
Program:
#include <windows.h>
#include <iostream>
#include <string>
#include <thread>
void foo()
{
while (true)
{
std::cout << "foo()\n";
std::this_thread::sleep_for(std::chrono::seconds(1));
}
}
int main(void)
{
std::thread first(foo);
bool isFinished = false;
while (!isFinished)
{
char ch = ::getchar();
::getchar(); // Swallow the new line character
if (ch == 'e')
{
isFinished = true;
}
else if (ch == 's')
{
DWORD result = ::SuspendThread(first.native_handle());
if (result != -1)
{
std::cout << "Successfully suspended thread\n";
}
else
{
std::cout << "Failed to suspend thread: failure resson " << ::GetLastError() << "\n";
}
}
else if (ch == 'r')
{
DWORD result = ::ResumeThread(first.native_handle());
if (result != -1)
{
std::cout << "Successfully resumed thread\n";
}
else
{
std::cout << "Failed to resume thread: failure resson " << ::GetLastError() << "\n";
}
}
else if (ch == 'k')
{
DWORD result = ::TerminateThread(first.native_handle(), 1);
if (result != 0)
{
std::cout << "Successfully terminated thread\n";
}
else
{
std::cout << "Failed to terminate thread: failure resson " << ::GetLastError() << "\n";
}
}
else
{
std::cout << "Unhandled char '" << ch << "'\n";
}
}
first.detach();
std::cout << "waiting to exit main...";
::getchar();
std::cout << "exiting...\n";
return 0;
}
Sample Output (comments added by me):
foo()
foo()
foo()
foo()
s
Successfully suspended thread // This was successful since 'foo()' is no longer printing
r
Successfully resumed thread // This was successful since 'foo()' is again printing
foo()
foo()
foo()
foo()
s
Successfully suspended thread // Worked again
k
Successfully terminated thread // Says it works...
r
Successfully resumed thread // Termination must have worked because resuming did not cause 'foo' to start printing
e
waiting to exit main...
exiting...
I have a multi-threaded C++03 application that presently uses popen() to invoke itself (same binary) and ssh (different binary) again in a new process and reads the output, however, when porting to Android NDK this is posing some issues such as not not having permissions to access ssh, so I'm linking in Dropbear ssh to my application to try and avoid that issue. Further, my current popen solution requires that stdout and stderr be merged together into a single FD which is a bit messy and I'd like to stop doing that.
I would think the pipe code could be simplified by using fork() instead but wonder how to drop all of the parent's stack/memory which is not needed in the child of the fork? Here is a snippet of the old working code:
#include <iostream>
#include <stdio.h>
#include <string>
#include <errno.h>
using std::endl;
using std::cerr;
using std::cout;
using std::string;
void
doPipe()
{
// Redirect stderr to stdout with '2>&1' so that we see any error messages
// in the pipe output.
const string selfCmd = "/path/to/self/binary arg1 arg2 arg3 2>&1";
FILE *fPtr = ::popen(selfCmd.c_str(), "r");
const int bufSize = 4096;
char buf[bufSize + 1];
if (fPtr == NULL) {
cerr << "Failed attempt to popen '" << selfCmd << "'." << endl;
} else {
cout << "Result of: '" << selfCmd << "':\n";
while (true) {
if (::fgets(buf, bufSize, fPtr) == NULL) {
if (!::feof(fPtr)) {
cerr << "Failed attempt to fgets '" << selfCmd << "'." << endl;
}
break;
} else {
cout << buf;
}
}
if (pclose(fPtr) == -1) {
if (errno != 10) {
cerr << "Failed attempt to pclose '" << selfCmd << "'." << endl;
}
}
cout << "\n";
}
}
So far, this is loosely what I have done to convert to fork(), but fork needlessly duplicates the entire parent process memory space. Further, it does not quite work, because the parent never sees EOF on the outFD it is reading from the pipe(). Where else do I need to close the FDs for this to work? How can I do something like execlp() without supplying a binary path (not easily available on Android) but instead start over with the same binary and a blank image with new args?
#include <iostream>
#include <stdio.h>
#include <string>
#include <errno.h>
using std::endl;
using std::cerr;
using std::cout;
using std::string;
int
selfAction(int argc, char *argv[], int &outFD, int &errFD)
{
pid_t childPid; // Process id used for current process.
// fd[0] is the read end of the pipe and fd[1] is the write end of the pipe.
int fd[2]; // Pipe for normal communication between parent/child.
int fdErr[2]; // Pipe for error communication between parent/child.
// Create a pipe for IPC between child and parent.
const int pipeResult = pipe(fd);
if (pipeResult) {
cerr << "selfAction normal pipe failed: " << errno << ".\n";
return -1;
}
const int errorPipeResult = pipe(fdErr);
if (errorPipeResult) {
cerr << "selfAction error pipe failed: " << errno << ".\n";
return -1;
}
// Fork - error.
if ((childPid = fork()) < 0) {
cerr << "selfAction fork failed: " << errno << ".\n";
return -1;
} else if (childPid == 0) { // Fork -> child.
// Close read end of pipe.
::close(fd[0]);
::close(fdErr[0]);
// Close stdout and set fd[1] to it, this way any stdout of the child is
// piped to the parent.
::dup2(fd[1], STDOUT_FILENO);
::dup2(fdErr[1], STDERR_FILENO);
// Close write end of pipe.
::close(fd[1]);
::close(fdErr[1]);
// Exit child process.
exit(main(argc, argv));
} else { // Fork -> parent.
// Close write end of pipe.
::close(fd[1]);
::close(fdErr[1]);
// Provide fd's to our caller for stdout and stderr:
outFD = fd[0];
errFD = fdErr[0];
return 0;
}
}
void
doFork()
{
int argc = 4;
char *argv[4] = { "/path/to/self/binary", "arg1", "arg2", "arg3" };
int outFD = -1;
int errFD = -1;
int result = selfAction(argc, argv, outFD, errFD);
if (result) {
cerr << "Failed to execute selfAction." << endl;
return;
}
FILE *outFile = fdopen(outFD, "r");
FILE *errFile = fdopen(errFD, "r");
const int bufSize = 4096;
char buf[bufSize + 1];
if (outFile == NULL) {
cerr << "Failed attempt to open fork file." << endl;
return;
} else {
cout << "Result:\n";
while (true) {
if (::fgets(buf, bufSize, outFile) == NULL) {
if (!::feof(outFile)) {
cerr << "Failed attempt to fgets." << endl;
}
break;
} else {
cout << buf;
}
}
if (::close(outFD) == -1) {
if (errno != 10) {
cerr << "Failed attempt to close." << endl;
}
}
cout << "\n";
}
if (errFile == NULL) {
cerr << "Failed attempt to open fork file err." << endl;
return;
} else {
cerr << "Error result:\n";
while (true) {
if (::fgets(buf, bufSize, errFile) == NULL) {
if (!::feof(errFile)) {
cerr << "Failed attempt to fgets err." << endl;
}
break;
} else {
cerr << buf;
}
}
if (::close(errFD) == -1) {
if (errno != 10) {
cerr << "Failed attempt to close err." << endl;
}
}
cerr << "\n";
}
}
There are two kinds of child processes created in this fashion with different tasks in my application:
SSH to another machine and invoke a server that will communicate back to the parent that is acting as a client.
Compute a signature, delta, or merge file using rsync.
First of all, popen is a very thin wrapper on top of fork() followed by exec() [and some call to pipe and dup and so on to manage the ends of a pipe] .
Second, the memory is only duplicated in form of "copy-on-write" memory - meaning that unless one of the processes writes to some page, the actual physical memory is shared between the two processes.
It does mean, of course, the OS has to create a memory map with 4-8 bytes per 4KB [in typical cases] (probably plus some internal OS data to track how many copies there are of that page and stuff - but as long as the page remains the same one as the parent process, the child page uses the parent processes internal data). Compared to everything else involved in creating a new process and loading an executable file into the new process, it's a pretty small part of the time. Since you are almost immediately doing exec, not much of the parent process' memory will be touched, so very little will happen there.
My advice would be that if popen works, keep using popen. If popen doesn't quite do what you want for some reason, then use fork + exec - but make sure you know what the reason for doing so is.