How to share pthread synchronisation primitives between C++ and Rust? - c++

Question

I have a C++ program and a Rust program, and between them I have successfully got them talking over POSIX shared memory (C++ and rust).
What I am now trying to do is synchronise them. I already managed to create a working, but inefficient, primitive system using an atomic bool (creating the AtomicBool on the rust side like this).
However, I would really like to use a mutex/condvar to synchronise between the threads, and this is where I am stuck.
I seem to be able to initialise the C++ side of it, following this example pretty much word for word.
I have attempted to translate it directly into rust:
let raw_shm = shm.get_shm();
let mut mtx_attrs = MaybeUninit::<nix::libc::pthread_mutexattr_t>::uninit();
if unsafe { nix::libc::pthread_mutexattr_init(mtx_attrs.as_mut_ptr()) } != 0 {
panic!("failed to create mtx_attrs");
};
let mtx_attrs = unsafe { mtx_attrs.assume_init() };
let mut cond_attrs = MaybeUninit::<nix::libc::pthread_condattr_t>::uninit();
if unsafe { nix::libc::pthread_condattr_init(cond_attrs.as_mut_ptr()) } != 0 {
panic!("failed to create cond_attrs");
};
let cond_attrs = unsafe { cond_attrs.assume_init() };
if unsafe {
nix::libc::pthread_mutexattr_setpshared(
&mtx_attrs as *const _ as *mut _,
PTHREAD_PROCESS_SHARED,
)
} != 0
{
panic!("failed to set mtx as process shared");
};
if unsafe {
nix::libc::pthread_condattr_setpshared(
&cond_attrs as *const _ as *mut _,
PTHREAD_PROCESS_SHARED,
)
} != 0
{
panic!("failed to set cond as process shared");
};
// I know that these offsets are correct, having used `offsetof` on the C++ side
let mtx_start = unsafe { &raw_shm.as_slice()[3110416] };
let mtx = unsafe { &*(mtx_start as *const _ as *const pthread_mutex_t) };
let cond_start = unsafe { &raw_shm.as_slice()[3110440] };
let cond = unsafe { &*(cond_start as *const _ as *const pthread_mutex_t) };
if unsafe {
nix::libc::pthread_mutex_init(&mtx as *const _ as *mut _, &mtx_attrs as *const _ as *mut _)
} != 0
{
panic!("failed to init mtx");
};
if unsafe {
nix::libc::pthread_cond_init(
&cond as *const _ as *mut _,
&cond_attrs as *const _ as *mut _,
)
} != 0
{
panic!("failed to init cond");
};
All of that passes with return values of 0... so far so good.
I can now test it in one of two ways:
I can set the trivial C++ program going and have it stop waiting at the condvar:
if (pthread_mutex_lock(&shmp->mutex) != 0)
throw("Error locking mutex");
if (pthread_cond_wait(&shmp->condition, &shmp->mutex) != 0)
throw("Error waiting for condition variable");
and in rust:
let sig = unsafe { nix::libc::pthread_cond_signal(&cond as *const _ as *mut _) };
dbg!(sig);
Despite returning 0 (i.e. success), my C++ program is not released past the condvar; it remains waiting as if it never received a signal.
I can set of another trivial C++ program which endlessly signals the condition variable in a loop:
for (unsigned int count = 0;; count++) {
if (pthread_cond_signal(condition) != 0)
throw("Error")
// sleep for a bit
}
and then in rust, something like:
loop {
if unsafe { nix::libc::pthread_mutex_lock(&mtx as *const _ as *mut _) } > 0 {
panic!("Failed to acquire lock")
};
if unsafe {
nix::libc::pthread_cond_wait(&cond as *const _ as *mut _, &mtx as *const _ as *mut _)
} > 0
{
panic!("Failed to acquire lock")
};
}
Doing it this way around, the call to lock the mutex is successful, but I get an EINVAL on pthread_cond_wait defined here, which I cannot seem to rectify...
I feel like I'm close... any thoughts on how to get this to work? (this is mostly just a proof of concept).

Answer 1

For posterity, I have managed to get this working.
To clarify how the program is architectured, there are two binaries: one C++ and one rust. The Rust program spawns the C++ program using std::process::Command.
Error handling and imports elided for brevity.
The rust program starts and creates a new shared memory block (removing an existing block if it exists, to ensure the program always starts in a fresh state). I use the shared_memory crate to handle the details for me, and that also provides useful helpers such as access to a raw pointer to the start of the memory block.
The shared memory block is structured like the following:
#[repr(c)]
struct SharedMemoryLayout {
ready: std::sync::atomic::AtomicBool,
mutex: libc::pthread_mutex_t,
condition: libc::pthread_cond_t,
}
Shared memory blocks are initialised with zeros, so ready will always be falseto begin with.
The rust program spawns the C++ program with std::process::Command::spawn and then waits in a loop until ready is true.
let proc = Command::new("/path/to/c++/binary").spawn().unwrap();
let ptr: *mut u8 = // pointer to first byte of shared memory block;
let ready: &AtomicBool = unsafe { &*(ptr as *mut bool as *const AtomicBool) };
loop {
if ready.load(Ordering::SeqCst) {
break
} else {
thread::sleep(Duration::from_secs(1));
}
}
The C++ program opens the shared memory block and mmaps it into its local address space.
struct SharedMemoryLayout
{
std::atomic_bool ready;
pthread_mutex_t mutex;
pthread_cond_t condition;
};
int fd = shm_open("name_of_shared_memory_block", O_RDWR, S_IRUSR | S_IWUSR);
struct SharedMemoryLayout *sync = (SharedMemoryLayout *)mmap(NULL, sizeof(*sync), PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0);
The C++ program carries on and proceeds to initialise the mutex and the condition, before marking the memory block as ready.
pthread_mutexattr_t mutex_attributes;
pthread_condattr_t condition_attributes;
pthread_mutexattr_init(&mutex_attributes);
pthread_condattr_init(&condition_attributes);
pthread_mutexattr_setpshared(&mutex_attributes, PTHREAD_PROCESS_SHARED);
pthread_condattr_setpshared(&condition_attributes, PTHREAD_PROCESS_SHARED);
pthread_mutex_init(&sync->mutex, &mutex_attributes);
pthread_cond_init(&sync->condition, &condition_attributes);
pthread_mutexattr_destroy(&mutex_attributes);
pthread_condattr_destroy(&condition_attributes);
std::atomic_bool *ready = &syncp->ready;
ready->store(true);
And then enter a loop signalling on the condition:
for (unsigned int count = 0;; count++) {
// do something
sleep(1);
pthread_cond_signal(&sync->condition);
}
Now, the rust program will have been released from the loop in step 2). Materialise the mutex and condition that were initialised in step 4).
let mutex = unsafe {ptr.offset(4) as *mut pthread_mutex_t};
let condition = unsafe {ptr.offset(32) as *mut pthread_cond_t};
And now we can wait on the condition, getting notified by the C++ program.
loop {
unsafe {
pthread_mutex_lock(mutex);
pthread_cond_wait(condition, mutex);
pthread_mutex_unlock(mutex);
// Do something
}
}