What's the proper way to have a Task that calls an arbitrary function with a known, specific return type? - templates

I have a value which is expensive to calculate and can be asked for ahead of time--something like a lazily initiated value whose initialization is actually done at the moment of definition, but in a different thread. My immediate thought was to use parallelism.-Task seems purpose-built for this exact use-case. So, let's put it in a class:
class Foo
{
import std.parallelism : Task,task;
static int calculate(int a, int b)
{
return a+b;
}
private Task!(calculate,int,int)* ourTask;
private int _val;
int val()
{
return ourTask.workForce();
}
this(int a, int b)
{
ourTask = task!calculate(a,b);
}
}
That seems all well and good... except when I want the task to be based on a non-static method, in which case I want to make the task a delegate, in which case I start having to do stuff like this:
private typeof(task(&classFunc)) working;
And then, as it turns out, typeof(task(&classFunc)), when it's asked for outside of a function body, is actually Task!(run,ReturnType!classFunc function(Parameters!classFunc))*, which you may notice is not the type actually returned by runtime function calls of that. That would be Task!(run,ReturnType!classFunc delegate(Parameters!classFunc))*, which requires me to cast to typeof(working) when I actually call task(&classFunc). This is all extremely hackish feeling.
This was my attempt at a general template solution:
/**
Provides a transparent wrapper that allows for lazy
setting of variables. When lazySet!!func(args) is called
on the value, the function will be called in a new thread;
as soon as the value's access is attempted, it'll return the
result of the task, blocking if it's not done calculating.
Accessing the value is as simple as using it like the
type it's templated for--see the unit test.
*/
shared struct LazySet(T)
{
/// You can set the value directly, as normal--this throws away the current task.
void opAssign(T n)
{
import core.atomic : atomicStore;
working = false;
atomicStore(_val,n);
}
import std.traits : ReturnType;
/**
Called the same way as std.parallelism.task;
after this is called, the next attempt to access
the value will result in the value being set from
the result of the given function before it's returned.
If the task isn't done, it'll wait on the task to be done
once accessed, using workForce.
*/
void lazySet(alias func,Args...)(Args args)
if(is(ReturnType!func == T))
{
import std.parallelism : task,taskPool;
auto t = task!func(args);
taskPool.put(t);
curTask = (() => t.workForce);
working = true;
}
/// ditto
void lazySet(F,Args...)(F fpOrDelegate, ref Args args)
if(is(ReturnType!F == T))
{
import std.parallelism : task,taskPool;
auto t = task(fpOrDelegate,args);
taskPool.put(t);
curTask = (() => t.workForce);
working = true;
}
private:
T _val;
T delegate() curTask;
bool working = false;
T val()
{
import core.atomic : atomicStore,atomicLoad;
if(working)
{
atomicStore(_val,curTask());
working = false;
}
return atomicLoad(_val);
}
// alias this is inherently public
alias val this;
}
This lets me call lazySet using any function, function pointer or delegate that returns T, and then it'll calculate the value in parallel and return it, fully calculated, next time anything tries to access the underlying value, exactly as I wanted. Unit tests I wrote to describe its functionality pass, etc., it works perfectly.
But one thing's bothering me:
curTask = (() => t.workForce);
Moving the Task around by way of creating a lambda on-the-spot that happens to have the Task in its context still seems like I'm trying to "pull one over" on the language, even if it's less "hackish-feeling" than all the casting from earlier.
Am I missing some obvious language feature that would allow me to do this more "elegantly"?

Templates that take an alias function parameter (such as the Task family) are finicky regarding their actual type, as they can receive any type of function as parameter (including in-place delegates that get inferred themselves). As the actual function that gets called is part of the type itself, you would have to pass it to your custom struct to be able to save the Task directly.
As for the legitimacy of your solution, there is nothing wrong with storing lambdas to interact with complicated (or "hidden") types later.
An alternative is to store a pointer to &t.workForce directly.
Also, in your T val() two threads could enter if(working) at the same time, but I guess due to the atomic store it wouldn't really break anything - anyway, that could be fixed by core.atomic.cas.

Related

Objects vs. Static Variables for retaining function state

I have a function which processes data that comes as a sequence. Because of this, I need to know the value of certain variables from the last function call during the current function call.
My current approach to doing this is to use static variables. My function goes something like this:
bool processData(Object message){
static int lastVar1 = -1;
int curVar1 = message.var1;
if (curVar1 > lastVar1){
// Do something
}
lastVar1 = curVar1;
}
This is just a small sample of the code; in reality I have 10+ static variables tracking different things. My gut tells me using so many static variables probably isn't a good idea, though I have nothing to back that feeling up.
My question: Is there a better way to do this?
An alternative I've been looking into is using an object whose fields are lastVar1, lastVar2, etc. However, I'm not sure if keeping an object in memory would be more efficient than using static variables.
Your question has a taste of being purely about style and opinions, though there are aspects that are not a matter of opinion: multithreading and testing.
Consider this:
bool foo(int x) {
static last_val = -1;
bool result = (x == last_val);
last_val = x;
return result;
}
You can call this function concurrently from multiple threads but it wont do the expected. Moreover you can only test the function by asserting that it does the right thing:
foo(1);
assert( foo(1) ); // silenty assumes that the last call did the right thing
To setup the preconditions for the test (first line) you already have to assume that foo(1) does the right thing, which somehow defeats the purpose of testing that call in the second line.
If the methods need the current object and the previous object, simply pass both:
bool processData(const Object& message,const Object& previous_message){
if (message.var1 > previous_message.var1){
// Do something
return true;
}
return false;
}
Of course this just shifts the issue of keeping track of the previous message to the caller, though thats straight-forward and requires not messing around with statics:
Object message, old_message;
while ( get_more( message )) {
processData(message, old_message);
old_message = message;
}

Using C++ member function to handle a callback that takes simple static function pointer

I haven't used C++ in ages. Between what I've forgotten and what has changed in C++ over time, I'm really banging my head against the wall trying to do something that would be trivially easy in JavaScript, or any other language where functions are objects, and not just simple pointers.
I think I understand the basic problem: A class member function only exists in once place in memory (there isn't a different copy of it for each class instance). The only way the function knows what "this" is is because an instance pointer is passed along as an invisible first argument to every function call. A plain-old C-style callback isn't going to know anything about passing that instance pointer.
What I need is a new function that is somehow bound to my class instance, one which knows how to pass "this" along to the member function. That's the function I need to use as a callback.
But I don't know for sure how to dynamically create such a function. I think the code below is on the right track (except for casting pointer types), but it does bother me a bit because it seems like that there'd have to be some dynamic memory allocation going on, and if so, some way to track that allocation and do clean-up later.
class SignalMonitor {
int dataPin;
unsigned short timings[RING_BUFFER_SIZE];
unsigned long lastSignalChange = 0;
int dataIndex = 0;
int syncCount = 0;
void signalHasChanged();
public:
SignalMonitor(int);
};
SignalMonitor::SignalMonitor(int dataPin) {
this->dataPin = dataPin;
function<void()> callback = bind(&SignalMonitor::signalHasChanged, this);
wiringPiISR(dataPin, INT_EDGE_BOTH, callback);
}
void SignalMonitor::signalHasChanged() {
unsigned long now = micros();
int duration = (int) min(now - this->lastSignalChange, 10000ul);
this->lastSignalChange = now;
cout << duration << '\n';
}
I feel like this is close to what I want, but I'm getting this error:
acu-rite-433Mhz-reader.cpp:58:72: error: invalid cast from type ‘std::function<void()>’ to type ‘void*’
wiringPiISR(dataPin, INT_EDGE_BOTH, reinterpret_cast<void *>(callback));
^
Here's the call signature of the function I'm trying to pass this callback to:
int wiringPiISR (int pin, int edgeType, void (*function)(void))
I've found a number of similar issues discussed when searching on this topic, but they either don't quite match what I'm trying to do, or assume much more familiarity with C++ than I currently possess. (All I remember about function pointer types is that they can get hellishly ugly very quickly!)
I tried to use lambda function as a solution, but that led to an error (besides a type mismatch error) about something being "temporary", which I'm assuming meant that the lambda function's scope was temporary.
This is a far from ideal solution (I'm beginning to think there are no ideal solutions here), but it works for me in this particular case where there aren't likely to be very many instances of my SignalMonitor class in use at the same time.
First, I turned my signalHasChanged class method into a static method that takes an instance as an argument. (I could have kept the method as a class method by going through some hairy type-casting, but it wasn't worth it.)
Then I made ten almost-identical indirect callback functions:
void smCallback0() { SignalMonitor::signalHasChanged(monitors[0]); }
void smCallback1() { SignalMonitor::signalHasChanged(monitors[1]); }
void smCallback2() { SignalMonitor::signalHasChanged(monitors[2]); }
void smCallback3() { SignalMonitor::signalHasChanged(monitors[3]); }
void smCallback4() { SignalMonitor::signalHasChanged(monitors[4]); }
void smCallback5() { SignalMonitor::signalHasChanged(monitors[5]); }
void smCallback6() { SignalMonitor::signalHasChanged(monitors[6]); }
void smCallback7() { SignalMonitor::signalHasChanged(monitors[7]); }
void smCallback8() { SignalMonitor::signalHasChanged(monitors[8]); }
void smCallback9() { SignalMonitor::signalHasChanged(monitors[9]); }
Then I stuck all of those functions into an array:
void (*_smCallbacks[MAX_MONITORS])() = {
smCallback0, smCallback1, smCallback2, smCallback3, smCallback4,
smCallback5, smCallback6, smCallback7, smCallback8, smCallback9
};
Along with the monitors array, which is an array of SignalHandler pointers, this gives me ten available callback slots. (_smCallbacks is copied into smCallbacks as a way to get around foreward reference problems.)
The init method for SignalMonitor simply searches for an available slot, plugs itself in, then sets the callback:
void SignalMonitor::init() {
for (int i = 0; i < MAX_MONITORS; ++i) {
if (monitors[i] == NULL) {
callbackIndex = i;
monitors[i] = this;
break;
}
}
if (callbackIndex < 0)
throw "Maximum number of SignalMonitor instances reached";
wiringPiISR(dataPin, INT_EDGE_BOTH, smCallbacks[callbackIndex]);
}
There's also a destructor to free up the callback slots:
SignalMonitor::~SignalMonitor() {
if (callbackIndex >= 0)
monitors[callbackIndex] = NULL;
}
It may help to consider the traditional way of handling a similar issue. Other APIs have been designed where instead of void(*function)(void), wiringPiISR would expect a function void(*function)(void *). This allows the use of
static void signalHasChanged(void *p) {
static_cast<SignalMonitor*>(p)->signalHasChanged();
}
This is not a general solution, but because Raspberry Pi has a limited number of GPIO pins, and you can't have more callback functions than you have pins, you might be able to create one callback function per pin. Then, you need a global data structure that maps the interrupt pin to which SignalMonitor instance (or instances) it should signal. The constructor would register the 'this' object to a specific pin, then set the appropriate callback function based on the pin.
The callback functions would be able to pass a pin argument to a general function, which could then look up the specific SignalMonitor instance and call a class function.
I wouldn't want to do it for 1000 pins, 1000 instances, but this hack should work for anything running on a Pi.

LevelDB --- Code in C++

The below given code is taken from LevelDB. I am giving two blocks of code for better understanding. I am unable to understand what is happening.
ThreadState is a structure and I have written here to make it easy for the reader.
struct ThreadState {
int tid; // 0..n-1 when running in n threads
Random rand; // Has different seeds for different threads
Stats stats;
SharedState* shared;
ThreadState(int index)
: tid(index),
rand(1000 + index) {
}
};
Is the marked code below an object instantiation of class Benchmark? What is happening in the marked code below?
void Run() {
PrintHeader();
Open();
const char* benchmarks = FLAGS_benchmarks;
while (benchmarks != NULL) {
{
//code ommitted
}
// Reset parameters that may be overriddden bwlow
***void (Benchmark::*method)(ThreadState*) = NULL;*** // What does this code line mean? // Benchmark is a class.
bool fresh_db = false;
int num_threads = FLAGS_threads;
if (name == Slice("fillseq")) {
fresh_db = true;
method = &Benchmark::WriteSeq;
}
If required, I can give detailed implementation of Benchmark as well.
Thanks a lot for the help!
void (Benchmark::*method)(ThreadState*) = NULL;
// What does this code line mean?
// Benchmark is a class.
The above is a pointer to a member function. Since member functions are not like regular functions (they can only be called on a valid object), you cannot take their address it the same way you would for a free function.
Therefore the above syntax is introduced. It is similar to a regular function pointer except the class specifier Benchmark::. This is essentially the type of the implicit this pointer.
In your case, method is a pointer to a member function that takes ThreadState* as a parameter, and has a void return type. The reason for using it is most probably to simplify the call. First, and based on various parameters, a member function is chosen to be called, and its "address" stored in method. After all the checks are done, there is only a single call to the chosen function via the pointer to member.
Incidentally, &Benchmark::WriteSeq is how the code obtains the "address" of the member function WriteSeq. You must use the address-of operator on the qualified function name.

Is it possible to export/wrap a complex Go struct to C?

I own a Go library, gofileseq, for which I would like to try and made a C/C++ binding.
It is pretty straightforward to be able to export functions that use simple types (ints, strings, ...). It is even easy enough to export data from custom Go types to C by defining a C struct and translating the Go type to it, to be used in the exported functions, since you are allocating C memory to do it. But with the go 1.5 cgo rules I am finding it difficult to figure out how to export functionality from a more complex struct that stores state.
Example of a struct from gofileseq that I would like to export somehow to a C++ binding:
// package fileseq
//
type FrameSet struct {
frange string
rangePtr *ranges.InclusiveRanges
}
func NewFrameSet(frange string) (*FrameSet, error) {
// bunch of processing to set up internal state
}
func (s *FrameSet) Len() int {
return s.rangePtr.Len()
}
// package ranges
//
type InclusiveRanges struct {
blocks []*InclusiveRange
}
type InclusiveRange struct {
start int
end int
step int
cachedEnd int
isEndCached bool
cachedLen int
isLenCached bool
}
As you can see, the FrameSet type that I want to expose contains a slice of pointers to an underlying type, each of which stores state.
Ideally, I would love to be able to store a void* on a C++ class, and make it just a simple proxy for calling back into exported Go functions with the void*. But the cgo rules disallow C storing a Go pointer longer than the function call. And I am failing to see how I could use an approach of defining C++ classes that could be allocated and used to operate with my Go library.
Is it possible to wrap complex types for exposure to C/C++?
Is there a pattern that would allow a C++ client to create a Go FrameSet?
Edit
One idea I can think of would be to let C++ create objects in Go that get stored on the Go side in a static map[int]*FrameSet and then return the int id to C++. Then all the C++ operations make requests into Go with the id. Does that sound like a valid solution?
Update
For now, I am proceeding with testing a solution that uses global maps and unique ids to store objects. C++ would request a new object to be created and only get back an opaque id. Then they can call all of the methods exported as functions, using that id, including requesting for it to be destroyed when done.
If there is a better approach than this, I would love to see an answer. Once I get a fully working prototype, I will add my own answer.
Update #2
I've written a blog post about the final solution that I ended up using: http://justinfx.com/2016/05/14/cpp-bindings-for-go/
The way I ended up solving this, for lack of a better solution, was to use private global maps on the Go side (ref). These maps would associate instances of the Go objects with a random uint64 id, and the id would be returned to C++ as an "opaque handle".
type frameSetMap struct {
lock *sync.RWMutex
m map[FrameSetId]*frameSetRef
rand idMaker
}
//...
func (m *frameSetMap) Add(fset fileseq.FrameSet) FrameSetId {
// fmt.Printf("frameset Add %v as %v\n", fset.String(), id)
m.lock.Lock()
id := FrameSetId(m.rand.Uint64())
m.m[id] = &frameSetRef{fset, 1}
m.lock.Unlock()
return id
}
Then I use reference counting to determine when C++ no longer needs the object, and remove it from the map:
// Go
func (m *frameSetMap) Incref(id FrameSetId) {
m.lock.RLock()
ref, ok := m.m[id]
m.lock.RUnlock()
if !ok {
return
}
atomic.AddUint32(&ref.refs, 1)
// fmt.Printf("Incref %v to %d\n", ref, refs)
}
func (m *frameSetMap) Decref(id FrameSetId) {
m.lock.RLock()
ref, ok := m.m[id]
m.lock.RUnlock()
if !ok {
return
}
refs := atomic.AddUint32(&ref.refs, ^uint32(0))
// fmt.Printf("Decref %v to %d\n", ref, refs)
if refs != 0 {
return
}
m.lock.Lock()
if atomic.LoadUint32(&ref.refs) == 0 {
// fmt.Printf("Deleting %v\n", ref)
delete(m.m, id)
}
m.lock.Unlock()
}
//C++
FileSequence::~FileSequence() {
if (m_valid) {
// std::cout << "FileSequence destroy " << m_id << std::endl;
m_valid = false;
internal::FileSequence_Decref(m_id);
m_id = 0;
m_fsetId = 0;
}
}
And all C++ interactions with the exported Go library communicate via the opaque handle:
// C++
size_t FileSequence::length() const {
return internal::FileSequence_Len(m_id);
}
Unfortunately it does mean that in a multhreaded C++ environment, all threads would go through a mutex to the map. But it is only a write lock when objects are created and destroyed, and for all method calls on an object it is a read lock.

Implementation of config file

I am making an application in C++, and it requires a config file that will be read and interpreted on launch. It will contain things such as:
Module1=true
Now, my original plan was to store it all in variables and simply have
If(module1) {
DO_STUFF();
}
However this seems wasteful as it would be checking constantly for a value that would never change. Any ideas?
Optimize the code, only if you find a bottleneck with a profiler. Branch prediction should do its thing here, module1 never changes, so if you call it in a loop, even, there shouldn't be a noticeable performance loss.
If you want to experiment, you can branch once, and make a pointer point to the right function:
using func_ptr = void (*)();
func_ptr p = [](){};
if(module1)
p = DO_STUFF;
while(...)
p();
But this is just something to profile, look at the assembly...
There are also slower, but comfortable ways you could be storing the configuration, e.g. in an array with enumerated indexes, or a map. If I were to get some value in a loop, I'd do:
auto module1 = modules[MODULE1]; // array and enumeration
//auto module1 = modules.at("module1"); // map and string
while(...)
{
if(module1)
DO_STUFF;
...
}
So I'd end up with what you already have.
performance wise a boolean check is no problem, except you start doing it millions or billions of times. Maybe you can start merging code which belongs to module1, but other than that you'd have to check for it like you currently do
This really isn't an issue. If your program requires that Module1 should be true then let it check the value and continue on. It wont affect your performance unless it is being checked too many times.
One thing you could do is make an inline function if it being checked too many times. However, you will have to make sure the function shouldnt be too big otherwise it will be a bigger bottleneck
Sorry guys, didn't spot this when I looked it up:
MDSN
So I check the boolean once on launch and then I don't need to anymore as only the correct functions are launched.
Depending on how your program is set up and how the variables change the behaviour of the code you might be able to use function pointers:
if(Module1 == true)
{
std::function<void(int)> DoStuff = Module1Stuff;
}
And then later:
while(true)
{
DoStuff(ImportantVariable);
}
See http://en.cppreference.com/w/cpp/utility/functional/function for further reference.
Not that I think it'll help all that much but it's an alternative to try out at least.
This can be solved if you know the all use cases of the values you check. For example, if you've read your config file and module1 is true - you do one thing, if it is false - another. Let's start with example:
class ConfigFileWorker {
public:
virtual void run() = 0;
};
class WithModule1Worker {
public:
void run() final override {
// do stuff as if your `Module1` is true
}
};
class WithoutModule1Worker {
public:
void run() final override {
// do stuff as if your `Module1` is false
}
};
int main() {
std::unique_ptr<ConfigFileWorker> worker;
const bool Module1 = read_config_file(file, "Module1");
if (Module1) { // you check this only once during launch and just use `worker` all the time after
worker.reset(new WithModule1Worker);
} else {
worker.reset(new WithoutModule1Worker);
}
// here and after just use the pointer with `run()` - then you will not need to check the variable all the time, you'll just perform action.
}
So you have predefined behaviour for 2 cases (true and false) and just create an object of one of them during parsing the config file on launch. This is java-like code, but of course you may use function pointers, std::function and other abstractions instead of a base class, however, base class-option has more flexibility in my opinion.