Preventing a Fn from being invoked again while it's already running - concurrency

I am using inputbot to write a program that provides some global macros for my computer. For example, when I press the h key, it should execute the macro typing
Hello World
into the current application. I tried to implement it like this:
extern crate inputbot;
fn main() {
let mut callback = || {
inputbot::KeySequence("Hello World").send();
};
inputbot::KeybdKey::HKey.bind(callback);
inputbot::handle_input_events();
}
However, when I pressed the h key what I actually got was:
hHHHHHHHHHHHHHHHHHHHHHHHHHEHEHhEhEEHHhEhEhEHhEHHEHHEEHhEHlhEHEHHEHLEHLHeeleleelelelllelelleelehlhehlleeheehelheelleeleelhllllllellelolelellelleoleloloelellololol olollollelllolllol lloo ol o oo l lo lolooloooloo loo LOWOLO O L OLW WOWO L WLLOLOW L O O O O o WOWW low o oOow WWW WOW wowooWWWO oOWRWOoor W RoW oOWorororWRRWLR rLROwoRWLWOworo WorrrRWl ow o WRLR OLw o OWLDol rollWWLDWowDLlroWWo r oWDWOL dorRrwrolrdrrorlrLWDRdodRLowdllrllolrdlrddolrdlrldowldorowlrdlrorloLDLWDLoddlrddlrdldldldrrdordldrlrddrodlrrldoldlrlddldlrdlldlrdlddrlddldddlddlddd
The macro was triggering itself each time it sent the h key event. 😬
How can I prevent a Fn from being invoked again while another instance of it is still running? This is the main functionality of a small application, so there's nothing else to really worry about compatibility with.
My naive attempt to fix
this was to add a mut running variable in main, which callback would set to true while it was running, or immediately return if it was already true:
extern crate inputbot;
use std::time::Duration;
use std::thread::sleep;
fn main() {
let mut running = false;
let mut callback = || {
if running { return };
running = true;
inputbot::KeySequence("Hello World").send();
// wait to make sure keyboard events are done.
sleep(Duration::from_millis(125));
running = false;
};
inputbot::KeybdKey::HKey.bind(callback);
inputbot::handle_input_events();
}
However, this doesn't compile:
error[E0525]: expected a closure that implements the `Fn` trait, but this closure only implements `FnMut`
After some reading, my understanding is now that a Fn closure (required by inputbot's .bind() methods) can't own any mutable data, like a captured mut variable.
Maybe it's possible to wrap the variable in some kind of non-mut value? Perhaps some kind-of lock, to make the potential concurrency safe, like this pseudocde?
fn main() {
let mut running = false;
let lockedRunning = example::Lock(&running);
let mut callback = || {
{
let mut running = lockedRunning.acquire();
if running { return };
running = true;
}
inputbot::KeySequence("Hello World").send();
// wait to make sure keyboard events are done.
sleep(Duration::from_millis(125));
{
let mut running = lockedRunning.acquire();
running = false;
}
};
}

What you want here is that the function is mutually exclusive to itself.
Rust allows you to do this with the Mutex struct. It allows you to hold a lock that when acquired stops anyone else from taking it until you release it.
Specifically the functionality you want is the try_lock method which would allow you to check if the lock has already been acquired and would allow you to handle that case.
let lock = mutex.try_lock();
match lock {
Ok(_) => {
// We are the sole owners here
}
Err(TryLockError::WouldBlock) => return,
Err(TryLockError::Poisoned(_)) => {
println!("The mutex is poisoned");
return
}
}

Using an atomic value is a bit simpler than a Mutex as you don't need to worry about failure cases and it can easily be made into a static variable without using lazy-static:
use std::sync::atomic::{AtomicBool, Ordering};
fn main() {
let is_being_called = AtomicBool::new(false);
bind(move || {
if !is_being_called.compare_and_swap(false, true, Ordering::SeqCst) {
print!("I'm doing work");
is_being_called.store(false, Ordering::SeqCst);
}
});
}
I have a hunch that this is also more efficient than using a Mutex as no heap allocations need to be made, but I didn't benchmark it.
If you are in a single-threaded context and your callback is somehow (accidentally?) recursive (which closures cannot be) you can also use a Cell:
use std::cell::Cell;
fn main() {
let is_being_called = Cell::new(false);
bind(move || {
if !is_being_called.get() {
is_being_called.set(true);
print!("doing work");
is_being_called.set(false);
}
})
}
If you happen to have a FnMut closure, you don't even need the Cell and can just use a boolean:
fn main() {
let mut is_being_called = false;
bind(move || {
if !is_being_called {
is_being_called = true;
print!("doing work");
is_being_called = false;
}
})
}

Related

Unit testing a service that accepts an Fn closure as a callback

I have the following service that registers callbacks to execute at a certain epoch, identified by an i64. The service has a vector of callbacks (that are bounded by the Send + Fn() -> () traits). Each callback can be executed multiple times (hence Fn instead of FnOnce or FnMut). The Send trait is needed because the callbacks will be registered by other threads, and this service will run in the background.
So far so good, but I'd like to test that the callbacks are executed the way they should be (i.e. the i64 epoch ticking in some direction which may (or may not) cause the callback to be executed). The problem is that I cannot seem to be able to think of a way to achieve this. I'm coming from Golang in which it is quite easy to inject a mock callback and assert whether it was called since such limitations are not imposed by the compiler, however when I employ the same methods in Rust, I end up with an FnMut instead of an Fn.
use std::sync::{Arc, Mutex};
use std::collections::HashMap;
struct Service<T: Send + Fn() -> ()> {
triggers: Arc<Mutex<HashMap<i64, Vec<Box<T>>>>>,
}
impl<T: Send + Fn() -> ()> Service<T> {
pub fn build() -> Self {
Service {
triggers: Arc::new(Mutex::new(HashMap::new())),
}
}
pub fn poll(&'static self) {
let hs = Arc::clone(&self.triggers);
tokio::spawn(async move {
loop {
// do some stuff and get `val`
if let Some(v) = hs.lock().unwrap().get(&val) {
for cb in v.iter() {
cb();
}
}
}
});
()
}
pub fn register_callback(&self, val: i64, cb: Box<T>) -> () {
self.triggers
.lock()
.unwrap()
.entry(val)
.or_insert(Vec::new())
.push(cb);
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_poll() {
let c = Service::build();
let mut called = false;
let cb = || called = true;
let h: i64 = 10;
c.register_callback(h, Box::new(cb));
assert_eq!(called, false);
}
}
Any ideas on how would this sort of behavior could be tested in Rust? The only thing I can think of is perhaps some channel that would pass a local value to the test and relinquish ownership over it?
The best way would probably be to make your interface as general as possible:
// type bounds on structs are generally unnecessary so I removed it here.
struct Service<T> {
triggers: Arc<Mutex<HashMap<i64, Vec<Box<T>>>>>,
}
impl<T: Send + FnMut() -> ()> Service<T> {
pub fn build() -> Self {
Service {
triggers: Arc::new(Mutex::new(HashMap::new())),
}
}
pub fn poll(&'static self, val: i64) {
let hs = Arc::clone(&self.triggers);
tokio::spawn(async move {
loop {
// do some stuff and get `val`
if let Some(v) = hs.lock().unwrap().get_mut(&val) {
for cb in v.iter_mut() {
cb();
}
}
}
});
()
}
pub fn register_callback(&self, val: i64, cb: Box<T>) -> () {
self.triggers
.lock()
.unwrap()
.entry(val)
.or_insert(Vec::new())
.push(cb);
}
}
But if you can't generalize the interface you can just use an AtomicBool like this:
#[cfg(test)]
mod tests {
use super::*;
use std::sync::atomic::{Ordering, AtomicBool};
#[test]
fn test_poll() {
let c = Service::build();
let mut called = AtomicBool::new(false);
let cb = || called.store(true, Ordering::Relaxed);
let h: i64 = 10;
c.register_callback(h, Box::new(cb));
assert!(!called.load(Ordering::Relaxed));
}
}

Getting reference to shared data in async closure [duplicate]

I have something like this:
use std::sync::Arc;
fn main() {
let arc = Arc::new(42);
move || { arc.clone() };
move || { arc.clone() };
}
I am getting:
error[E0382]: capture of moved value: `arc`
--> src/main.rs:6:19
|
5 | move || { arc.clone() };
| ------- value moved (into closure) here
6 | move || { arc.clone() };
| ^^^ value captured here after move
|
= note: move occurs because `arc` has type `std::sync::Arc<i32>`, which does not implement the `Copy` trait
I understand why I am getting this: the clone isn't called before arc is passed to the closure. I can fix this by defining each closure in a function and clone the Arc before passing it to the closure, but is there another option?
There is no way around it. You should clone the Arc before it is used in a closure. The common pattern is to re-bind the cloned Arc to the same name in a nested scope:
use std::sync::Arc;
fn main() {
let arc = Arc::new(42);
{
let arc = arc.clone();
move || { /* do something with arc */ };
}
{
let arc = arc.clone();
move || { /* do something else with arc */ };
}
}
This is usually done together with thread::spawn():
use std::sync::{Arc, Mutex};
use std::thread;
const NUM_THREADS: usize = 4;
fn main() {
let arc = Arc::new(Mutex::new(42));
for _ in 0..NUM_THREADS {
let arc = arc.clone();
thread::spawn(move || {
let mut shared_data = arc.lock().unwrap();
*shared_data += 1;
});
}
}
is there another option?
Because this pattern of cloning things before defining a closure is somewhat common, some people have proposed adding something like clone || as an analog to move ||. I wouldn't hold out hope for this happening, but a number of comments there point out that macros can solve the case fairly well.
Several crates provide some form of this macro:
closet
capture
clone_all
It's likely that many projects define their own macro to do something similar. For example, the WASM example rust-todomvc defines:
macro_rules! enclose {
( ($( $x:ident ),*) $y:expr ) => {
{
$(let $x = $x.clone();)*
$y
}
};
}
Which can be used as:
fn main() {
let arc = Arc::new(42);
enclose! { (arc) move || arc };
enclose! { (arc) move || arc };
}

Problems about using C++ coroutines

I'm experimenting the incoming C++ coroutines with liburing.
I'm a JavaScript developer and trying to implement a function like Promise.all
My first attempt
template <typename T>
task<std::vector<T>> taskAll1(std::vector<task<T>> list) {
std::vector<T> result;
result.reserve(list.size());
for (auto&& t : list) {
result.push_back(co_await t);
}
co_return result;
}
task is modified from gor_task. I mainly make task::promise_type::initial_suspend return suspend_never to let coroutines start by default.
The usage:
// async_delay: https://github.com/CarterLi/liburing-http-demo/blob/12af992d0d87a721bbe67bb67aee8e4b0e965114/async_coro.hpp#L68
// It waits some seconds asynchronously and then prints the number of seconds it waits.
task<bool> start() {
// TODO: less verbose code?
std::vector<task<int>> vec;
vec.emplace_back(async_delay(1));
vec.emplace_back(async_delay(2));
vec.emplace_back(async_delay(3));
co_await taskAll<int>(std::move(vec));
co_return true;
}
The code works as expected. But if I rearrange the order of emplace_backs, for example let async_delay(2) come before async_delay(1), the code still runs but freezes at the end of program. I know it is because the task async_delay(1) is resolved before it's awaited, but I have no idea how to fix it.
The second attempt, what I want to to do is translating following JS code into C++
/** #param {Promise<any>[]} list */
function taskAll(list) {
return new Promise(resolve => {
const result = new Array(list.length);
let left = list.length;
list.forEach((p, i) => {
p.then(x => {
result[i] = x;
left--;
if (!left) resolve(result);
})
});
});
}
The c++ code
// completion: https://github.com/CarterLi/liburing-http-demo/blob/12af992d0d87a721bbe67bb67aee8e4b0e965114/task.hpp#L78
// It's like JS Promise and can be awaited directly without calling an async function
template <typename T>
task<std::vector<T>> taskAll(std::vector<task<T>> list) {
std::vector<T> result(list.size());
size_t left = list.size();
completion<std::vector<T>> promise;
for (size_t i=0; i<list.size(); ++i) {
[&, i]() mutable -> task<bool> {
result[i] = co_await list[i];
left--;
if (!left) promise.resolve(std::move(result));
co_return true;
}();
}
co_await promise;
co_return result;
}
The code crashes immediately. Async code is really hard to debug and I gave up.
Full code can be found in Github, please help.
For 2nd attempt, I found that I forgot to assign the task returned by lambda expression, which make the task destruct immediately after the lambda function returned.

What is the equivalent of Go's range time.Tick?

I'm new to and studying Rust currently, coming from Go. How do I implement something like long concurrent polling?
// StartGettingWeather initialize weather getter and setter
func StartGettingWeather() {
// start looping
for i := range time.Tick(time.Second * time.Duration(delay)) {
_ = i
loopCounter++
fmt.Println(time.Now().Format(time.RFC850), " counter: ", loopCounter)
mainWeatherGetter()
}
}
and I will call this func as go StartGettingWeather()
Rust threads are OS threads, they use OS scheduler and so you can emulate this with thread::sleep_ms:
use std::thread;
fn start_getting_weather() {
let mut loop_counter = 0;
loop {
loop_counter += 1;
println!("counter: {}", loop_counter);
main_weather_getter();
thread::sleep_ms(delay);
}
}
thread::spawn(move || start_getting_weather());

D parallel loop

First, how D create parallel foreach (underlying logic)?
int main(string[] args)
{
int[] arr;
arr.length = 100000000;
/* Why it is working?, it's simple foreach which working with
reference to int from arr, parallel function return ParallelForeach!R
(ParallelForeach!int[]), but I don't know what it is.
Parallel function is part od phobos library, not D builtin function, then what
kind of magic is used for this? */
foreach (ref e;parallel(arr))
{
e = 100;
}
foreach (ref e;parallel(arr))
{
e *= e;
}
return 0;
}
And second, why it is slower then simple foreach?
Finally, If I create my own taskPool (and don't use global taskPool object), program never end. Why?
parallel returns a struct (of type ParallelForeach) that implements the opApply(int delegate(...)) foreach overload.
when called the struct submits a parallel function to the private submitAndExecute which submits the same task to all threads in the pool.
this then does:
scope(failure)
{
// If an exception is thrown, all threads should bail.
atomicStore(shouldContinue, false);
}
while (atomicLoad(shouldContinue))
{
immutable myUnitIndex = atomicOp!"+="(workUnitIndex, 1);
immutable start = workUnitSize * myUnitIndex;
if(start >= len)
{
atomicStore(shouldContinue, false);
break;
}
immutable end = min(len, start + workUnitSize);
foreach(i; start..end)
{
static if(withIndex)
{
if(dg(i, range[i])) foreachErr();
}
else
{
if(dg(range[i])) foreachErr();
}
}
}
where workUnitIndex and shouldContinue are shared variables and dg is the foreach delegate
The reason it is slower is simply because of the overhead required to pass the function to the threads in the pool and atomically accessing the shared variables.
the reason your custom pool doesn't shut down is likely you don't shut down the threadpool with finish