Moving Arc Mutex Ncurses window down thread still doesn't implement send - concurrency

I'm building an ncurses interface for a little thing I'm working on. For the input, I want to read it non blocking. I figured I could wrap the WINDOW in an Arc Mutex but this doesn't seem to work, as it still complains about send. Is this because the implementation of ncurses is unsafe? How can I solve this? Ideally, I'd have this work with a callback instead of the tx, so I can cut the dependency from the view up the stack, but I couldn't get that closure to Send either.
I'm using this library: https://github.com/jeaye/ncurses-rs
Simplified code:
pub struct View {
max_x: i32,
max_y: i32,
messages_window: WINDOW,
input_window: Arc<Mutex<WINDOW>>
}
pub fn init(&mut self, input_tx: mpsc::Sender<DispatchMessage>) {
let input_window = self.input_window.clone();
thread::spawn(move || {
loop {
let input_window = input_window.lock().unwrap();
draw_prompt(input_window);
let input = prompt_input(input_window);
input_tx.send(input_to_message(input)).unwrap();
}
});
}
fn prompt_input(input: WINDOW) -> String {
let mut string = String::new();
wgetstr(input, &mut string);
string
}
fn draw_prompt(input: WINDOW) {
wclear(input);
let top = 0 as chtype;
let bottom = ' ' as chtype;
let empty = ' ' as chtype;
wborder(input, empty,empty,top,bottom,empty,empty,empty,empty);
mvwprintw(input, 1, 1, ">> ");
wrefresh(input);
}
And the errors I get:
src/view.rs:40:33: 40:45 error: mismatched types:
expected `*mut ncurses::ll::WINDOW_impl`,
found `std::sync::mutex::MutexGuard<'_, *mut ncurses::ll::WINDOW_impl>`
(expected *-ptr,
found struct `std::sync::mutex::MutexGuard`) [E0308]
src/view.rs:40 draw_prompt(input_window);
^~~~~~~~~~~~
note: in expansion of closure expansion
src/view.rs:37:27: 44:14 note: expansion site
src/view.rs:40:33: 40:45 help: run `rustc --explain E0308` to see a detailed explanation
src/view.rs:41:46: 41:58 error: mismatched types:
expected `*mut ncurses::ll::WINDOW_impl`,
found `std::sync::mutex::MutexGuard<'_, *mut ncurses::ll::WINDOW_impl>`
(expected *-ptr,
found struct `std::sync::mutex::MutexGuard`) [E0308]
src/view.rs:41 let input = prompt_input(input_window);
^~~~~~~~~~~~
note: in expansion of closure expansion
src/view.rs:37:27: 44:14 note: expansion site
src/view.rs:41:46: 41:58 help: run `rustc --explain E0308` to see a detailed explanation
src/view.rs:37:13: 37:26 error: the trait `core::marker::Send` is not implemented for the type `*mut ncurses::ll::WINDOW_impl` [E0277]
src/view.rs:37 thread::spawn(move || {
^~~~~~~~~~~~~
src/view.rs:37:13: 37:26 note: `*mut ncurses::ll::WINDOW_impl` cannot be sent between threads safely
src/view.rs:37 thread::spawn(move || {
^~~~~~~~~~~~~
error: aborting due to 3 previous errors
Manually dereferencing the window removes the type errors, but I figured I'd leave it in as it might be an indication of what's wrong.

Arc<T> implements Send where T implements both Send and Sync. Mutex<T> implements Send and Sync where T implements Send. So Arc<Mutex<T>> only implements Send if T implements Send. Remember that Send means “Types able to be transferred across thread boundaries.” Arc<Mutex<T>> allows access to its contents from multiple threads, purely taking care of ownership and mutability issues, so if the underlying type cannot be transferred across thread boundaries it won’t help. You may well need to do all your ncurses operations from one thread.
Raw pointers explicitly do not implement Send because there can be no guarantees about it. You can construct types on top of it which explicitly implement Send, thus providing a guarantee that the contained raw pointer is in fact safe for transferring across thread boundaries.

Related

"error: underscore lifetimes are unstable" when implementing From<std::sync::PoisonError>

I'm working on a function that looks like this:
fn do_stuff(&mut self, a: MyStruct) -> Result<(), MyError> {
let x = try!(serde_json::to_vec(a));
let cache = Arc::clone(self.data); // Get shared reference
{
let cache = try!(cache.lock()); // Get lock
cache.push(x);
}
/* Do stuff with other resources */
Ok(())
}
Where the definition of MyError is:
#[derive(Debug)]
pub enum MyError {
Serialization(serde_json::Error),
Synch(PoisonError<MutexGuard<'_, Vec<u8>>>),
}
Before I even get to implementing From<std::sync::PoisonError> for MyError, the compiler already tells me the definition of the Synch variant of my enum is wrong:
error: underscore lifetimes are unstable (see issue #44524)
The declaration using underscore lifetimes actually came from an earlier hint from the compiler when I was trying to figure out the error I should convert from when the lock operation fails. I read the aforementioned issue and that doesn't help me.
What's the full type I should be converting from in order to catch the error from the Mutex::lock operation?
Like so:
#[derive(Debug)]
pub enum MyError<'a> {
Serialization(serde_json::Error),
Synch(PoisonError<MutexGuard<'a, Vec<u8>>>),
}
The closest explanation I can find in the book is the section on Lifetime Annotations in Struct Definitions (enums behave the same way).
The compiler suggesting unstable syntax as a solution is quite unfair.

Why this Clang ASTMatcher cause wrong polymorphic conversion?

I am Writing a tool using clang as frontend and matching some AST nodes.
I create ASTMatcher as follow:
void Rule_1_2_1::registerMatchers(MatchFinder *Finder)
{
DeclarationMatcher Matcher = decl(hasType(builtinType().bind("non-typedef"))).bind("non-typedef-decl");
Finder->addMatcher(Matcher, this);
}
void Rule_1_2_1::run(const MatchFinder::MatchResult &Result)
{
if (const BuiltinType *type = Result.Nodes.getNodeAs<BuiltinType>("non-typedef")) {
if (!type->isFloatingPoint() && !type->isInteger())
return;
if (const Decl *decl = Result.Nodes.getNodeAs<Decl>("non-typedef-decl")) {
DiagnosticsEngine &DE = Result.Context->getDiagnostics();
Context->report(this->CheckerName, this->ReportMsg, DE, decl->getLocStart(), DiagnosticIDs::Note);
}
}
}
But compiler gives me following errors:
/usr/include/clang/ASTMatchers/ASTMatchersInternal.h: In instantiation of ‘clang::ast_matchers::internal::PolymorphicMatcherWithParam1<MatcherT, P1, ReturnTypesF>::operator clang::ast_matchers::internal::Matcher<From>() const [with T = clang::Decl; MatcherT = clang::ast_matchers::internal::matcher_hasType0Matcher; P1 = clang::ast_matchers::internal::Matcher<clang::QualType>; ReturnTypesF = void(clang::ast_matchers::internal::TypeList<clang::Expr, clang::TypedefNameDecl, clang::ValueDecl>)]’:
../src/modules/gjb/Rule_1_2_1.cpp:18:81: required from here
/usr/include/clang/ASTMatchers/ASTMatchersInternal.h:1104:5: Error:static assertion failed: right polymorphic conversion
static_assert(TypeListContainsSuperOf<ReturnTypes, T>::value,
^~~~~~~~~~~~~
I know i am not familiar with Clang ASTMatcher and the documentation may be not very detailed.
Why this error happened?
line 18 is the line of Matcher defined.
I'm posting this as an answer since it is too long for a comment, but it is only a guess at your problem, not a definite solution.
The error looks like it occurs when you compile your matcher, not when you apply it. Which means you misused the API, not that it doesn't match anything in your code. The AST matcher API checks that you don't do things that make no sense, like filtering on an attribute that may not even exist.
In your case, you are looking for declarations that have some type. But asking a declaration what its type is doesn't necessarily make sense. The Decl class in Clang is the root of the entire declaration hierarchy and includes things like EmptyDecl (which represents simply a single semicolon outside a statement context) and StaticAssertDecl (static_assert), neither of which have a type.
Every node matcher has type information on what nodes it produces. Every narrowing matcher has information on what nodes it applies to. It is checked at compile time that these are compatible.
They interesting parts of the error message are not the unfortunately vague message, but the static_assert condition itself and the listing of the active parameter substitutions.
TypeListContainsSuperOf<ReturnTypes, T>::value is the condition, i.e. "the type list must contain a type that is a supertype of T".
But what is T, and what does the type list contain? The error message says: "In instantiation of with " and then lists substitutions. There we learn that:
T = clang::Decl
MatcherT = clang::ast_matchers::internal::matcher_hasType0Matcher
ReturnTypesF = void(clang::ast_matchers::internal::TypeList<clang::Expr, clang::TypedefNameDecl, clang::ValueDecl>)
ReturnTypes is not directly listed, but it's pretty obvious that it refers to the parameter type of ReturnTypesF, i.e. the TypeList in there.
This tells us the following things:
The decl() matcher produces clang::Decl nodes.
The matcher we're currently validating is the hasType() matcher.
The hasType() matcher can work on any of clang::Expr, clang::TypedefNameDecl, and clang::ValueDecl.
But Decl is a supertype of TypedefNameDecl and ValueDecl, not the other way around, and unrelated to Expr. This means that the static assertion fails. The decl() matcher does not produce nodes that hasType() can work with.
Depending on your exact goals, using valueDecl() instead might work.

Is it possible to export/wrap a complex Go struct to C?

I own a Go library, gofileseq, for which I would like to try and made a C/C++ binding.
It is pretty straightforward to be able to export functions that use simple types (ints, strings, ...). It is even easy enough to export data from custom Go types to C by defining a C struct and translating the Go type to it, to be used in the exported functions, since you are allocating C memory to do it. But with the go 1.5 cgo rules I am finding it difficult to figure out how to export functionality from a more complex struct that stores state.
Example of a struct from gofileseq that I would like to export somehow to a C++ binding:
// package fileseq
//
type FrameSet struct {
frange string
rangePtr *ranges.InclusiveRanges
}
func NewFrameSet(frange string) (*FrameSet, error) {
// bunch of processing to set up internal state
}
func (s *FrameSet) Len() int {
return s.rangePtr.Len()
}
// package ranges
//
type InclusiveRanges struct {
blocks []*InclusiveRange
}
type InclusiveRange struct {
start int
end int
step int
cachedEnd int
isEndCached bool
cachedLen int
isLenCached bool
}
As you can see, the FrameSet type that I want to expose contains a slice of pointers to an underlying type, each of which stores state.
Ideally, I would love to be able to store a void* on a C++ class, and make it just a simple proxy for calling back into exported Go functions with the void*. But the cgo rules disallow C storing a Go pointer longer than the function call. And I am failing to see how I could use an approach of defining C++ classes that could be allocated and used to operate with my Go library.
Is it possible to wrap complex types for exposure to C/C++?
Is there a pattern that would allow a C++ client to create a Go FrameSet?
Edit
One idea I can think of would be to let C++ create objects in Go that get stored on the Go side in a static map[int]*FrameSet and then return the int id to C++. Then all the C++ operations make requests into Go with the id. Does that sound like a valid solution?
Update
For now, I am proceeding with testing a solution that uses global maps and unique ids to store objects. C++ would request a new object to be created and only get back an opaque id. Then they can call all of the methods exported as functions, using that id, including requesting for it to be destroyed when done.
If there is a better approach than this, I would love to see an answer. Once I get a fully working prototype, I will add my own answer.
Update #2
I've written a blog post about the final solution that I ended up using: http://justinfx.com/2016/05/14/cpp-bindings-for-go/
The way I ended up solving this, for lack of a better solution, was to use private global maps on the Go side (ref). These maps would associate instances of the Go objects with a random uint64 id, and the id would be returned to C++ as an "opaque handle".
type frameSetMap struct {
lock *sync.RWMutex
m map[FrameSetId]*frameSetRef
rand idMaker
}
//...
func (m *frameSetMap) Add(fset fileseq.FrameSet) FrameSetId {
// fmt.Printf("frameset Add %v as %v\n", fset.String(), id)
m.lock.Lock()
id := FrameSetId(m.rand.Uint64())
m.m[id] = &frameSetRef{fset, 1}
m.lock.Unlock()
return id
}
Then I use reference counting to determine when C++ no longer needs the object, and remove it from the map:
// Go
func (m *frameSetMap) Incref(id FrameSetId) {
m.lock.RLock()
ref, ok := m.m[id]
m.lock.RUnlock()
if !ok {
return
}
atomic.AddUint32(&ref.refs, 1)
// fmt.Printf("Incref %v to %d\n", ref, refs)
}
func (m *frameSetMap) Decref(id FrameSetId) {
m.lock.RLock()
ref, ok := m.m[id]
m.lock.RUnlock()
if !ok {
return
}
refs := atomic.AddUint32(&ref.refs, ^uint32(0))
// fmt.Printf("Decref %v to %d\n", ref, refs)
if refs != 0 {
return
}
m.lock.Lock()
if atomic.LoadUint32(&ref.refs) == 0 {
// fmt.Printf("Deleting %v\n", ref)
delete(m.m, id)
}
m.lock.Unlock()
}
//C++
FileSequence::~FileSequence() {
if (m_valid) {
// std::cout << "FileSequence destroy " << m_id << std::endl;
m_valid = false;
internal::FileSequence_Decref(m_id);
m_id = 0;
m_fsetId = 0;
}
}
And all C++ interactions with the exported Go library communicate via the opaque handle:
// C++
size_t FileSequence::length() const {
return internal::FileSequence_Len(m_id);
}
Unfortunately it does mean that in a multhreaded C++ environment, all threads would go through a mutex to the map. But it is only a write lock when objects are created and destroyed, and for all method calls on an object it is a read lock.

Threading lifetime error

I'm trying to implement the game "Tower of Hanoi" using concurrency in Rust. Honestly, I tried to understand the whole lifetime thing in Rust in the last time, but I'm not done yet. That's why I get some weird lifetime errors which I don't understand. First, here is the important piece of code
fn move_plate<'a>(stack_a: &'a mut Vec<i32>, stack_b: &'a mut Vec<i32>,
stack_c: &'a mut Vec<i32>, moves: &'a mut Vec<(i32, i32)>)
{
let mut moves1: Vec<(i32, i32)> = Vec::new();
let guard1 = Thread::scoped(
move || { move_plate(stack_a, stack_c, stack_b, (1, 3, 2), &mut moves1);
});
guard1.join().ok();
}
And here is the error
error: cannot infer an appropriate lifetime due to conflicting requirements
let guard1 = Thread::scoped(move || {
move_plate(height - 1, stack_a, stack_c, stack_b, (1, 3, 2), threads, depth + 1, &mut moves1);
});
note: first, the lifetime cannot outlive the expression at 93:25...
let guard1 = Thread::scoped(move || {
note: ...so that the declared lifetime parameter bounds are satisfied
let guard1 = Thread::scoped(move || {
note: but, the lifetime must be valid for the expression at 93:45...
let guard1 = Thread::scoped(move || {
move_plate(height - 1, stack_a, stack_c, stack_b, (1, 3, 2), threads, depth + 1, &mut moves1);
});
note: ...so type `closure[]` of expression is valid during the expression
let guard1 = Thread::scoped(move || {
move_plate(height - 1, stack_a, stack_c, stack_b, (1, 3, 2), threads, depth + 1, &mut moves1);
});
error: declared lifetime bound not satisfied
let guard1 = Thread::scoped(move || {
I understand that I have to avoid the thread to outlive the function, because otherwise the reference to moves would be gone. But since I join the thread, that should be all right, shouldn't it? What am I missing at that point?
Would be really nice if somebody could help me, I'm just getting used to that cool (but complicated) kind of stuff
This is a known limitation of the Rust type system. Currently Rust allows to send data between threads only if this data satisfies Send bound, and Send implies 'static - that is, the only references which can be sent across thread boundaries are 'static ones.
There is an RFC which partially lifts this limitation, allowing non-'static references to be sent across tasks. I thought that it already has been accepted, but it is not (which is strange). The API that supports such thing has already been created (and this is likely why you are confused), but the language is yet to be adjusted.

Is it possible to use WinRT objects in STL-like containers?

I'm trying to create a simple gesture recognizer for a D3D application. The gesture recognizer works by storing each point received into a boost::circular_buffer of capacity 3 and then counting the number of similar FrameID's in the buffer, like so:
UINT Trackball::CalculateGestureSize(Windows::UI::Input::PointerPoint ^ pPoint)
{
// shift the circular buffer queue one if it's full (common case)
if (m_pointQueue.full())
{
m_pointQueue.pop_back();
}
// then store our point
m_pointQueue.push_front(*pPoint);
// now we need to see how many of the points in the
// circular buffer match the frame Id
UINT gestureLength = 0;
for (UINT i = 0; i < MAX_GESTURE_SIZE; i += 1)
{
if (m_pointQueue[i].FrameId == pPoint->FrameId)
{
gestureLength += 1;
}
}
assert(gestureLength != 0);
return gestureLength;
}
However, the compiler is unable to figure out how to instantiate this type:
// a queue of size 3 that helps determine what kind of gesture we're working with
boost::circular_buffer<Windows::UI::Input::PointerPoint> m_pointQueue;
because & and * cannot be used on WinRT objects:
boost/concept_check.hpp(195): error C3699: '&' : cannot use this indirection on type 'const Windows::UI::Input::PointerPoint' compiler replacing '&' with '^' to continue parsing
The compiler's error list then grows long very quickly due to the cascading effects of that error.
Right now, my solution is copy the necessary information for a PointerPoint into a struct and use that as the boost::circular_buffer's typename, like so:
// So WinRT objects (like Windows::UI::Input::PointerPoint) can't
// be used in STL-like containers (like boost::circular_buffer)
// because * and & operators cannot be used on them, so I'm copying
// the necessary info into this struct and using that instead
typedef struct
{
UINT FrameId;
Windows::Foundation::Point Position;
} LocalPoint;
// a queue of size 3 that helps determine what kind of gesture we're working with
boost::circular_buffer<LocalPoint> m_pointQueue;
This definitely works, but I was wondering if there's a better solution out there.
Thanks for reading and for trying to help.
If you want to put a reference type in an STL collection, you need to use the ^ form. So you'd use: boost::circular_buffer<PointerPoint^> instead of boost::circular_buffer<PointerPoint>. A Windows::Foundation::Point is a value type so it can be used in a collection directly.
I think I accidentally found a working solution by using that Windows::Foundation::Point object in my LocalPoint struct. Just wrap the WinRT object with a struct and then the operators will work just fine, but it'll add a bit of syntactic noise.
However I'm still listening for a better solution, but I'll leave this here till then.