I'm new to and studying Rust currently, coming from Go. How do I implement something like long concurrent polling?
// StartGettingWeather initialize weather getter and setter
func StartGettingWeather() {
// start looping
for i := range time.Tick(time.Second * time.Duration(delay)) {
_ = i
loopCounter++
fmt.Println(time.Now().Format(time.RFC850), " counter: ", loopCounter)
mainWeatherGetter()
}
}
and I will call this func as go StartGettingWeather()
Rust threads are OS threads, they use OS scheduler and so you can emulate this with thread::sleep_ms:
use std::thread;
fn start_getting_weather() {
let mut loop_counter = 0;
loop {
loop_counter += 1;
println!("counter: {}", loop_counter);
main_weather_getter();
thread::sleep_ms(delay);
}
}
thread::spawn(move || start_getting_weather());
Related
I have the following service that registers callbacks to execute at a certain epoch, identified by an i64. The service has a vector of callbacks (that are bounded by the Send + Fn() -> () traits). Each callback can be executed multiple times (hence Fn instead of FnOnce or FnMut). The Send trait is needed because the callbacks will be registered by other threads, and this service will run in the background.
So far so good, but I'd like to test that the callbacks are executed the way they should be (i.e. the i64 epoch ticking in some direction which may (or may not) cause the callback to be executed). The problem is that I cannot seem to be able to think of a way to achieve this. I'm coming from Golang in which it is quite easy to inject a mock callback and assert whether it was called since such limitations are not imposed by the compiler, however when I employ the same methods in Rust, I end up with an FnMut instead of an Fn.
use std::sync::{Arc, Mutex};
use std::collections::HashMap;
struct Service<T: Send + Fn() -> ()> {
triggers: Arc<Mutex<HashMap<i64, Vec<Box<T>>>>>,
}
impl<T: Send + Fn() -> ()> Service<T> {
pub fn build() -> Self {
Service {
triggers: Arc::new(Mutex::new(HashMap::new())),
}
}
pub fn poll(&'static self) {
let hs = Arc::clone(&self.triggers);
tokio::spawn(async move {
loop {
// do some stuff and get `val`
if let Some(v) = hs.lock().unwrap().get(&val) {
for cb in v.iter() {
cb();
}
}
}
});
()
}
pub fn register_callback(&self, val: i64, cb: Box<T>) -> () {
self.triggers
.lock()
.unwrap()
.entry(val)
.or_insert(Vec::new())
.push(cb);
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_poll() {
let c = Service::build();
let mut called = false;
let cb = || called = true;
let h: i64 = 10;
c.register_callback(h, Box::new(cb));
assert_eq!(called, false);
}
}
Any ideas on how would this sort of behavior could be tested in Rust? The only thing I can think of is perhaps some channel that would pass a local value to the test and relinquish ownership over it?
The best way would probably be to make your interface as general as possible:
// type bounds on structs are generally unnecessary so I removed it here.
struct Service<T> {
triggers: Arc<Mutex<HashMap<i64, Vec<Box<T>>>>>,
}
impl<T: Send + FnMut() -> ()> Service<T> {
pub fn build() -> Self {
Service {
triggers: Arc::new(Mutex::new(HashMap::new())),
}
}
pub fn poll(&'static self, val: i64) {
let hs = Arc::clone(&self.triggers);
tokio::spawn(async move {
loop {
// do some stuff and get `val`
if let Some(v) = hs.lock().unwrap().get_mut(&val) {
for cb in v.iter_mut() {
cb();
}
}
}
});
()
}
pub fn register_callback(&self, val: i64, cb: Box<T>) -> () {
self.triggers
.lock()
.unwrap()
.entry(val)
.or_insert(Vec::new())
.push(cb);
}
}
But if you can't generalize the interface you can just use an AtomicBool like this:
#[cfg(test)]
mod tests {
use super::*;
use std::sync::atomic::{Ordering, AtomicBool};
#[test]
fn test_poll() {
let c = Service::build();
let mut called = AtomicBool::new(false);
let cb = || called.store(true, Ordering::Relaxed);
let h: i64 = 10;
c.register_callback(h, Box::new(cb));
assert!(!called.load(Ordering::Relaxed));
}
}
Imagine a situation where I'd like to call a function that does some amount of processing, but is time-bound. I could write a function in golang using context.Context and select. I'd imagine something as follows:
package main
import (
"context"
"fmt"
"time"
)
func longRunning(ctx context.Context, msg string) {
stop := make(chan bool)
done := make(chan bool)
go func() {
for {
fmt.Printf("long running calculation %v...", msg)
select {
case <-stop:
fmt.Println("time to stop early!")
return
default:
}
}
done <- true
}()
select {
case <-done:
return
case <-ctx.Done():
stop <- true
return
}
}
func main() {
ctx := context.Background()
ctx, cancel := context.WithTimeout(ctx, 3*time.Second)
defer cancel()
longRunning(ctx, "wheeee")
}
Is there a pattern I can use to achieve similar results in C++? in the sample above select is able to listen on a channel in a non-blocking way. Is having a eventfd file descriptor of some kind and listening to events the way to do it?
Any suggestions or tips would be much appreciated.
Something along these lines, perhaps:
void longRunning(std::atomic<bool>& stop) {
for (;;) {
if (stop) return;
// Do a bit of work
}
}
int main() {
std::atomic<bool> stop = false;
auto future = std::async(std::launch::async, longRunning, std::ref(stop));
future.wait_for(std::chrono::seconds(num_seconds));
stop = true;
future.get(); // wait for the task to finish or exit early.
}
Demo
I am using inputbot to write a program that provides some global macros for my computer. For example, when I press the h key, it should execute the macro typing
Hello World
into the current application. I tried to implement it like this:
extern crate inputbot;
fn main() {
let mut callback = || {
inputbot::KeySequence("Hello World").send();
};
inputbot::KeybdKey::HKey.bind(callback);
inputbot::handle_input_events();
}
However, when I pressed the h key what I actually got was:
hHHHHHHHHHHHHHHHHHHHHHHHHHEHEHhEhEEHHhEhEhEHhEHHEHHEEHhEHlhEHEHHEHLEHLHeeleleelelelllelelleelehlhehlleeheehelheelleeleelhllllllellelolelellelleoleloloelellololol olollollelllolllol lloo ol o oo l lo lolooloooloo loo LOWOLO O L OLW WOWO L WLLOLOW L O O O O o WOWW low o oOow WWW WOW wowooWWWO oOWRWOoor W RoW oOWorororWRRWLR rLROwoRWLWOworo WorrrRWl ow o WRLR OLw o OWLDol rollWWLDWowDLlroWWo r oWDWOL dorRrwrolrdrrorlrLWDRdodRLowdllrllolrdlrddolrdlrldowldorowlrdlrorloLDLWDLoddlrddlrdldldldrrdordldrlrddrodlrrldoldlrlddldlrdlldlrdlddrlddldddlddlddd
The macro was triggering itself each time it sent the h key event. 😬
How can I prevent a Fn from being invoked again while another instance of it is still running? This is the main functionality of a small application, so there's nothing else to really worry about compatibility with.
My naive attempt to fix
this was to add a mut running variable in main, which callback would set to true while it was running, or immediately return if it was already true:
extern crate inputbot;
use std::time::Duration;
use std::thread::sleep;
fn main() {
let mut running = false;
let mut callback = || {
if running { return };
running = true;
inputbot::KeySequence("Hello World").send();
// wait to make sure keyboard events are done.
sleep(Duration::from_millis(125));
running = false;
};
inputbot::KeybdKey::HKey.bind(callback);
inputbot::handle_input_events();
}
However, this doesn't compile:
error[E0525]: expected a closure that implements the `Fn` trait, but this closure only implements `FnMut`
After some reading, my understanding is now that a Fn closure (required by inputbot's .bind() methods) can't own any mutable data, like a captured mut variable.
Maybe it's possible to wrap the variable in some kind of non-mut value? Perhaps some kind-of lock, to make the potential concurrency safe, like this pseudocde?
fn main() {
let mut running = false;
let lockedRunning = example::Lock(&running);
let mut callback = || {
{
let mut running = lockedRunning.acquire();
if running { return };
running = true;
}
inputbot::KeySequence("Hello World").send();
// wait to make sure keyboard events are done.
sleep(Duration::from_millis(125));
{
let mut running = lockedRunning.acquire();
running = false;
}
};
}
What you want here is that the function is mutually exclusive to itself.
Rust allows you to do this with the Mutex struct. It allows you to hold a lock that when acquired stops anyone else from taking it until you release it.
Specifically the functionality you want is the try_lock method which would allow you to check if the lock has already been acquired and would allow you to handle that case.
let lock = mutex.try_lock();
match lock {
Ok(_) => {
// We are the sole owners here
}
Err(TryLockError::WouldBlock) => return,
Err(TryLockError::Poisoned(_)) => {
println!("The mutex is poisoned");
return
}
}
Using an atomic value is a bit simpler than a Mutex as you don't need to worry about failure cases and it can easily be made into a static variable without using lazy-static:
use std::sync::atomic::{AtomicBool, Ordering};
fn main() {
let is_being_called = AtomicBool::new(false);
bind(move || {
if !is_being_called.compare_and_swap(false, true, Ordering::SeqCst) {
print!("I'm doing work");
is_being_called.store(false, Ordering::SeqCst);
}
});
}
I have a hunch that this is also more efficient than using a Mutex as no heap allocations need to be made, but I didn't benchmark it.
If you are in a single-threaded context and your callback is somehow (accidentally?) recursive (which closures cannot be) you can also use a Cell:
use std::cell::Cell;
fn main() {
let is_being_called = Cell::new(false);
bind(move || {
if !is_being_called.get() {
is_being_called.set(true);
print!("doing work");
is_being_called.set(false);
}
})
}
If you happen to have a FnMut closure, you don't even need the Cell and can just use a boolean:
fn main() {
let mut is_being_called = false;
bind(move || {
if !is_being_called {
is_being_called = true;
print!("doing work");
is_being_called = false;
}
})
}
First, how D create parallel foreach (underlying logic)?
int main(string[] args)
{
int[] arr;
arr.length = 100000000;
/* Why it is working?, it's simple foreach which working with
reference to int from arr, parallel function return ParallelForeach!R
(ParallelForeach!int[]), but I don't know what it is.
Parallel function is part od phobos library, not D builtin function, then what
kind of magic is used for this? */
foreach (ref e;parallel(arr))
{
e = 100;
}
foreach (ref e;parallel(arr))
{
e *= e;
}
return 0;
}
And second, why it is slower then simple foreach?
Finally, If I create my own taskPool (and don't use global taskPool object), program never end. Why?
parallel returns a struct (of type ParallelForeach) that implements the opApply(int delegate(...)) foreach overload.
when called the struct submits a parallel function to the private submitAndExecute which submits the same task to all threads in the pool.
this then does:
scope(failure)
{
// If an exception is thrown, all threads should bail.
atomicStore(shouldContinue, false);
}
while (atomicLoad(shouldContinue))
{
immutable myUnitIndex = atomicOp!"+="(workUnitIndex, 1);
immutable start = workUnitSize * myUnitIndex;
if(start >= len)
{
atomicStore(shouldContinue, false);
break;
}
immutable end = min(len, start + workUnitSize);
foreach(i; start..end)
{
static if(withIndex)
{
if(dg(i, range[i])) foreachErr();
}
else
{
if(dg(range[i])) foreachErr();
}
}
}
where workUnitIndex and shouldContinue are shared variables and dg is the foreach delegate
The reason it is slower is simply because of the overhead required to pass the function to the threads in the pool and atomically accessing the shared variables.
the reason your custom pool doesn't shut down is likely you don't shut down the threadpool with finish
I'm doing some stream processing in Go and got stuck trying to figure out how to do this the "Go way" without locks.
This contrived example shows the problem I'm facing.
We get one thing at a time.
There is a goroutine which buffers them into a slice called things.
When things becomes full len(things) == 100 then it is processed somehow and reset
There are n number of concurrent goroutines that need to access things before it's full
Access to the "incomplete" things from other goroutines is not predictable.
Neither doSomethingWithPartial nor doSomethingWithComplete needs to mutate things
Code:
var m sync.Mutex
var count int64
things := make([]int64, 0, 100)
// slices of data are constantly being generated and used
go func() {
for {
m.Lock()
if len(things) == 100 {
// doSomethingWithComplete does not modify things
doSomethingWithComplete(things)
things = make([]int64, 0, 100)
}
things = append(things, count)
m.Unlock()
count++
}
}()
// doSomethingWithPartial needs to access the things before they're ready
for {
m.Lock()
// doSomethingWithPartial does not modify things
doSomethingWithPartial(things)
m.Unlock()
}
I know that slices are immutable so does that mean I can remove the mutex and expect it to still work (I assume no).
How can I refactor this to use channels instead of a mutex.
Edit: Here's the solution I came up with that does not use a mutex
package main
import (
"fmt"
"sync"
"time"
)
func Incrementor() chan int {
ch := make(chan int)
go func() {
count := 0
for {
ch <- count
count++
}
}()
return ch
}
type Foo struct {
things []int
requests chan chan []int
stream chan int
C chan []int
}
func NewFoo() *Foo {
foo := &Foo{
things: make([]int, 0, 100),
requests: make(chan chan []int),
stream: Incrementor(),
C: make(chan []int),
}
go foo.Launch()
return foo
}
func (f *Foo) Launch() {
for {
select {
case ch := <-f.requests:
ch <- f.things
case thing := <-f.stream:
if len(f.things) == 100 {
f.C <- f.things
f.things = make([]int, 0, 100)
}
f.things = append(f.things, thing)
}
}
}
func (f *Foo) Things() []int {
ch := make(chan []int)
f.requests <- ch
return <-ch
}
func main() {
foo := NewFoo()
var wg sync.WaitGroup
wg.Add(10)
for i := 0; i < 10; i++ {
go func(i int) {
time.Sleep(time.Millisecond * time.Duration(i) * 100)
things := foo.Things()
fmt.Println("got things:", len(things))
wg.Done()
}(i)
}
go func() {
for _ = range foo.C {
// do something with things
}
}()
wg.Wait()
}
It should be noted that the "Go way" is probably just to use a mutex for this. It's fun to work out how to do it with a channel but a mutex is probably simpler and easier to reason about for this particular problem.