I am trying to share a structure across processes using interprocess in Boost.
I've defined the mapped file to use null mutex because I was having problems with it locking and I don't mind doing the synchronisation myself.
What I am having problems with though is finding of objects.
I have the following declaration:
typedef boost::interprocess::basic_managed_mapped_file
< char,
boost::interprocess::rbtree_best_fit<boost::interprocess::null_mutex_family,boost::interprocess::offset_ptr<void>>,
boost::interprocess::flat_map_index>
my_mapped_file;
In process A, I do:
m_managedMappedFile.reset(new my_mapped_file(bip::open_or_create, filename, filesize));
auto hdr = m_managedMappedFile->find_or_construct<Foo>(bip::unique_instance)();
auto x = m_managedMappedFile->find<Foo>(bip::unique_instance);
Which works as I would expect, i.e. it finds the object. Now, in process B:
m_managedMappedFile.reset(new my_mapped_file(bip::open_only, filename));
auto ret = m_managedMappedFile->find<Foo>(bip::unique_instance);
For some reason the find method returns null in process B. I realise I must be doing something daft, but can't figure it out.
Can anyone help?
You should not have to bypass the locking mechanism of the default bip::managed_mapped_file indexes.
See if you can run the following with success:
#include <iostream>
#include <boost/interprocess/managed_mapped_file.hpp>
namespace bip = boost::interprocess;
struct X {
int i;
};
int main()
{
{
bip::managed_mapped_file f(bip::open_or_create, "/tmp/mmf.bin", 1ul << 20);
if (!f.find<X>(bip::unique_instance).first) {
auto xp = f.find_or_construct<X>(bip::unique_instance)();
assert(xp);
xp->i = 42;
}
}
{
bip::managed_mapped_file f(bip::open_only, "/tmp/mmf.bin");
auto xp = f.find<X>(bip::unique_instance).first;
if (xp)
std::cout << "value: " << xp->i++ << "\n";
}
}
This should print 42 on the first run (or after the file has been recreated), and increasing numbers on each subsequent run.
I'm going to have a look at the implementation behind the unique_instance_t* overloads of the segment managers, but I suspect they might not work because the mutex policy was nulled. This is just a hunch though, at the moment.
I'd focus on finding out why you can't get Interprocess managed_mapped_file working in the default configuration, on your platform & installation.
Related
I'm writing a sketch of a simple thread-safe logging library and some things came on my mind. Here's the code:
#ifndef SimpleLogger_H
#define SimpleLogger_H
#include <iostream>
#include <mutex>
class SimpleLogger
{
public:
template <typename T>
static void log(T message)
{
mutex.lock();
std::cout << message;
mutex.unlock();
}
private:
static std::mutex mutex;
}LOG;
template <typename T>
SimpleLogger &operator<<(SimpleLogger &simpleLogger, T message)
{
simpleLogger.log(message);
return simpleLogger;
}
#endif //SimpleLogger_H
My idea is to use it like this:
LOG << "hello" << " world " << 8 << " I can mix integers and strings";
I understand that the line above is like the following:
auto a1 = LOG.operator<<("hello");
auto a2 = a1.operator<<(" world ");
//Another thread may want to use LOG here, and would print in the middle of my message
auto a3 = a2.operator<<(8);
auto a4 = a3.operator<<(" I can mix integers and strings");
As you can see, because << is broken into several funciton calls, there's a risk that a thread can use the LOG object in the middle of my message (I consider a message the entire cascade of << on one line)
Also, is there a way to automatically add an std::endl for the last << call? I couldn't think of a way to do this, but I saw that some logging libraries have this functionality
How do I solve these two problems?
I know it'd be preferable to use a logging library but I want to mix android, desktop and ios logging in one simple lib without need for high performance, and I'm also puzzled by how I can overcome the difficulties I encountered while writing my own
As others already mentioned, you need a local buffer to collect message before sending to the log file. In the example below, SimpleLoggerBuffer objects are designed to be used as temporary variable only. I.e. it gets destroyed at the end of the expression. The destructor flushes the buffer into the log so that you don't have to explicitly call a flush function (you may add endl there as well if you wish)
#include <iostream>
#include <sstream>
#include <mutex>
using namespace std;
class SimpleLogger
{
public:
template <typename T>
static void log(T& message)
{
mutex.lock();
std::cout << message.str();
message.flush();
mutex.unlock();
}
private:
static std::mutex mutex;
}LOG;
std::mutex SimpleLogger::mutex;
struct SimpleLoggerBuffer{
stringstream ss;
SimpleLoggerBuffer() = default;
SimpleLoggerBuffer(const SimpleLoggerBuffer&) = delete;
SimpleLoggerBuffer& operator=(const SimpleLoggerBuffer&) = delete;
SimpleLoggerBuffer& operator=(SimpleLoggerBuffer&&) = delete;
SimpleLoggerBuffer(SimpleLoggerBuffer&& buf): ss(move(buf.ss)) {
}
template <typename T>
SimpleLoggerBuffer& operator<<(T&& message)
{
ss << std::forward<T>(message);
return *this;
}
~SimpleLoggerBuffer() {
LOG.log(ss);
}
};
template <typename T>
SimpleLoggerBuffer operator<<(SimpleLogger &simpleLogger, T&& message)
{
SimpleLoggerBuffer buf;
buf.ss << std::forward<T>(message);
return buf;
}
int main() {
LOG << "hello" << " world " << 8 << " I can mix integers and strings";
}
You could create a helper class that collects all the output and prints on destruction. Outline:
#include <string>
#include <iostream>
struct Msg;
struct Log {
void print(const Msg &m);
};
struct Msg {
std::string m;
Log &l;
Msg(Log &l) : l(l) {}
~Msg() {
// Print the message on destruction
l.print(*this);
}
};
void Log::print(const Msg &m) {
// Logger specific printing... here, append newline
std::cout << m.m << std::endl;
}
Msg &&operator << (Msg &&m, const std::string &s) {
// Append operator
m.m += s;
return std::move(m);
}
// Helper to log on a specific logger. Just creates the initial message
Msg log(Log &l) { return Msg(l); }
int main()
{
Log l;
log(l) << "a" << "b" << "c";
return 0;
}
As the Msg is local, other threads will not interfere with it. Any necessary locking can be done in the Log.print method, which will receive the complete message
A simple solution is to write into files instead of standard output, and specifically, separate file for each thread. That way no locking or any other synchronization is needed. The files can later be merged if lines have parseable format.
Another solution is to write logs asynchronously from a single thread, and initially store the messages in a thread safe (possibly lock free) queue.
Also, is there a way to automatically add an std::endl for the last << call?
Unless I misunderstand, you can simply do stream << message << std::endl.
I think that you can simply use std::clog. It is thread safe and, in contrary of std::cout, designed to output instantly for logging.
From the reference page :
Unless sync_with_stdio(false) has been issued, it is safe to
concurrently access these objects from multiple threads for both
formatted and unformatted output.
I recommend you this Jason Turner's video about cout, clog and cerror.
The simplest approach is to return a temporary proxy from the first << - this can either log your stream for the duration (and unlock on destruction), or simply build a local ostringstream and flush it in a single call (again, on destruction).
Holding a lock while logging isn't great for performance (although it's better than your existing log method, which should use std::lock_guard for exception safety).
Building and discarding a temporary ostringstream is probably better, but if you care about performance you'll need to benchmark, and may well end up requiring something more elaborate (per-thread circular buffers, mmapped files or something).
I wrote a very simple code to reproduce my problem.
#include <iostream>
#include "tools.h" //contains s_sleep()
#include <thread>
using namespace std;
void change( int *i)
{
while (true)
{
*i = 4356;
}
}
int main()
{
int v=3;
cout << v <<endl;
thread t(change, &v);
s_sleep(1); //sleep one second
cout << v << endl;
t.join();
}
The output is 3, and after a second 3 again. But when I change one line to
//while ( true )
I receive 3, and a second later 4356.
How can that be?
Hope somebody can help.
Please specify what compiler you are using. I am using Microsoft Visual C++ compiler, and with my visual studio, I see for both time, the output is 3 followed by 4356.
Here is the code I ran on my computer.
#include <ctime>
#include <thread>
#include <iostream>
using namespace std;
void change(int *i) {
while (true) { // commented out this later, the result is same.
*i = 4356;
}
}
int main() {
clock_t tstart = clock();
int v = 3;
cout << v << endl;
thread t(change, &v);
while(double(clock() - tstart)/CLOCKS_PER_SEC < 3.0) { // Instead of your s_sleep
int x = 1; // Just a dummy instruction
}
cout << v << endl;
t.join();
return 0;
}
The explanation to my result is that the thread "t" does not know anything about the variable "v". It just gets a pointer of type int and it edits the value at the pointers location directly to the memory. So, when the main(first) thread
again accesses the variable "v", it simply reads the memory assigned to "v" and prints what it gets.
And also, what code is in the "tools.h"? Does it have anything to do with the variable "v".
If it doesn't, then it must be a compiler variance(Your compiler may be different than mine, maybe gcc or g++?). That means, your compiler must have cached(or something like that) the variable for faster access. And as in the current thread, the variable has not been changed, whenever it is accessed, compiler gives the old value(which the compiler sees as unchanged) of the variable. (I AM NOT SURE ABOUT THIS)
This might also be due to caching. You are first reading a variable frome one thread, then manipulating the variable from another thread and reading it again from the first thread. The compiler cannot know that it changed in the meantime.
To safely do this "v" has to be declared volatile.
A followup with reference to the upcoming feature in C++20 from n3721 "Improvements to std::future and related APIs"
#include <iostream>
#include <future>
#include <exception>
using std::cout;
using std::endl;
int main() {
auto prom_one = std::promise<std::future<int>>{};
auto fut_one = prom_one.get_future();
std::thread{[prom_one = std::move(prom_one)]() mutable {
auto prom_two = std::promise<int>{};
auto fut_two = prom_two.get_future();
prom_two.set_value(1);
prom_one.set_value(std::move(fut_two));
}}.detach();
auto inner_fut_unwrap = fut_one.unwrap();
auto inner_fut_get = fut_one.get();
auto th_one = std::thread{[&]() {
cout << inner_fut_unwrap.get() << endl;
}};
auto th_two = std::thread{[&]() {
cout << inner_fut_get.get() < endl;
}};
th_one.join();
th_two.join();
return 0;
}
In the code above, which will win the race to print 1? th_one or th_two?
To clarify what race I was talking about, there are two (potential) racy situations here, the latter being the one that is really confusing me.
The first is in the setting and unwrapping of the inner future; the unwrapped future should act as a suitable proxy for the inner future even when the actual set_value has not been called on the inner future. So unwrap() must return a proxy that exposes a thread safe interface regardless of what happens on the other side.
The other situation is what happens to a get() from a future when a proxy for it already exists elsewhere, in this example inner_fut_unwrap is the proxy for inner_fut_get. In such a situation, which should win the race? The unwrapped future or the future fetched via a call to get() on the outer future?
This code makes me worried that there is some kind of misunderstanding about what futures and promises are, and what .get() does. It's also a bit weird that we have using namespace std; followed by a lot of std::.
Let's break it down. Here's the important part:
#include <iostream>
#include <future>
int main() {
auto prom_one = std::promise<std::future<int>>{};
auto fut_one = prom_one.get_future();
auto inner_fut_unwrap = fut_one.unwrap();
auto inner_fut_get = fut_one.get();
// Boom! throws std::future_error()
So neither thread "wins" the race, since neither thread actually gets a chance to run. Note from the document you linked, for .unwrap(), on p13:
Removes the outer-most future and returns a proxy to the inner future.
So the outer-most future, fut_one, is not valid. When you call .get(), it throws std::future_error1. There is no race.
1: Not guaranteed. Technically undefined behavior.
I have the following simplified code at which while writing I thought was fine, but I have seem some random access violations.
Initially I thought as long as the arguments passed to async were on the stack, and not temporary variables, the code would be safe. I also thought that filename and extra data would destruct/considered not there at the brace where they leave scope.
It did some more research and read about the 'as if' principle that apparently compilers use for optimisation. I've often seen stack variables being optimised away in the debugger right after they have been used too.
My question here is basically, is it guaranteed that those stack variables will be around for the entire duration of the async function running. The .get() call on the future obviously synchronises the call before the two stack variables leave scope.
My current thinking is that it's not thread safe as the compiler can't see the variables being used after the call to the function, and therefore think it is safe to remove them. I can easily change the code to eliminate the problem (if there is one), but I really want to understand this.
The randomness of the AV, occurring more on some computers than others suggests it is a problem, and the scheduling order dictates whether this is a problem or not.
Any help is much appreciated.
#include <future>
#include <fstream>
#include <string>
#include <iostream>
int write_some_file(const char * const filename, int * extra_result)
{
std::ofstream fs;
try {
fs.open(filename);
} catch (std::ios_base::failure e) {
return 1;
}
fs << "Hello";
*extra_result = 1;
return 0;
}
int main(void)
{
std::string filename {"myffile.txt"};
int extraResult = 0;
auto result = std::async(std::launch::async, write_some_file, filename.c_str(), &extraResult);
// Do some other work
// ...
int returnCode = result.get();
std::cout << returnCode << std::endl;
std::cout << extraResult << std::endl;
return 0;
}
Keeping track of how many times a function is called is easy when passing the counter as an argument into the function. It's also easy when returning a one from the called function. But, I do not want to go that route. The reason behind this is because it seems like bad programming (letting the function know too much information). Is there a better way to keep track of how many times this function has been called?
I'm just looking for concepts that I could study. Providing code examples is not neccessary, but might be helpful.
Edit: I'm not actually looking for profiling tools. Let me add some code to get my point across. Because scope for funcCounter ends in main, I have no way of getting back a variable from myFunction that will increment funcCounter. I could possibly return 1 from myFunction and then increment funcCounter that way, but this doesn't seem like very good programming. Is there another way to do it?
int main()
{
int funcCounter = 0;
char *mystring = "This is a silly function.";
myFunction(mystring);
cout << "Times function is called: " << funcCounter << endl;
return 0;
}
void myFunction(char *mystring)
{
cout << mystring << endl;
}
Have a static variable in your function and keep incrementing it each time the function in called.
void my_Function(void) {
static unsigned int call_count = 0;
call_count++;
}
If you want to do it for debugging reasons, then there are tools like gcov which do this for you. (I'm pretty sure Microsoft doesn't have an alternative bundled with Microsoft Visual C++)
I would do this through the use of a profiling tool like gcov (which is for linux). These programs do the work of inserting code into your program during compilation and give you a report of how many times a function is called, where its called from, and how long the program spent executing that function.
It sounds like what you are looking for is a profiler. Depending on the platform you are using there are a slew of tools available that can help you hunt down the (ab)uses of a routine.
Please revise your question with the platform for which you need profiling tools.
If the function is part of a class, you can add a static counter to the class, plus an accessor and/or reset functions:
class X
{
private:
/* diagnostics */
static int counter = 0;
int read_counter() const { return counter; }
void reset_counter() { counter = 0; }
public:
/* real code */
fcn() {
++counter;
/* ... */
}
};
The problem with adding a static counter to a standalone function is that there's no way to get at the value.
You could add a global, of course, but instead of a raw global I'd suggest an instance of a singleton containing all your diagnostic code and data.
Use a class like this one, and simply instantiate it at the top of a function (or any other block) like is done in f() below.
Note: There is some overhead for gettimeofday() so you may want to use a different timing method, but that is a completely different topic worthy of it's own question (and has been addressed before on SO).
#include <iostream>
#include <string>
#include <map>
#include <sstream>
#include <ctime>
#include <cstdlib>
#include <sys/time.h>
class PerfStats
{
private:
std::string which_;
timeval begin_;
public:
PerfStats(std::string const &file, int line)
{
std::stringstream ss;
ss << file << ':' << line;
which_ = ss.str();
gettimeofday(&begin_, NULL);
}
~PerfStats()
{
timeval end;
gettimeofday(&end, NULL);
Times[which_] = (end.tv_sec - begin_.tv_sec) + (end.tv_usec - begin_.tv_usec)/1000000.0;
++Counts[which_];
}
static std::map<std::string, double> Times;
static std::map<std::string, unsigned int> Counts;
static void Print()
{
for(std::map<std::string, double>::iterator it = Times.begin(); it != Times.end(); ++it)
std::cout << it->first << " :\t" << it->second << "s" << std::endl;
for(std::map<std::string, unsigned int>::iterator it = Counts.begin(); it != Counts.end(); ++it)
std::cout << it->first << " :\t" << it->second << " times" << std::endl;
}
};
std::map<std::string, double> PerfStats::Times;
std::map<std::string, unsigned int> PerfStats::Counts;
void f()
{
PerfStats(__FILE__, __LINE__);
usleep(1);
}
main()
{
srand(time(NULL));
for(int i = 0; i < rand(); ++i)
f();
PerfStats::Print();
}
Sample output:
test.cpp:54 : 2e-06s
test.cpp:54 : 21639 times
Bad coding style, but maybe adding global variables and if necessary mutex locks may do the trick.