I have a function that takes two suspend functions and runs them in parallel, I'm using Arrow to do this with parZip and either
fun <E, R1, R2, R3> combineCalls(
call1: suspend () -> Either<E, R1>,
call2: suspend () -> Either<E, R2>,
combine: (R1, R2) -> R3
): suspend () -> Either<E, R3> = suspend {
either {
parZip({ call1().bind() }, { call2().bind() }) { r1, r2 ->
r1 to r2
}
}
}
So which ever of call1 or call2 fails first (returns an Either.Left) will cause the other one to be cancelled and that error will be immediately returned.
I want to test this, but I'm not sure how to ensure which call function will return first.
#Test
fun basicCombinationErrorBoth() = runTest {
val result = combineCalls({ 400.left() }, {
delay(100)
500.left()
})()
result assertEquals 400.left()
}
I'm using runTest so that it doesn't take up actual time (if we have a lot of tests with delays it adds up), but this means the delay is skipped and the test only passes sometimes as it's indeterminate which passed in suspend function returns first.
How can I use runTest and ensure that one suspend function takes more virtual time than the other?
Related
I'm wrapper a function into another one, making the function as a parameter to the other, but I'm having a hard time to modify the existing unit test.
The wrapper function is called withLock(), which do something before and after the real function. Pasting my code below:
Original standalone function V1 and its unit test
suspend fun V1 : RandomType (
// Step A
// Step B
// Step C
return RandomType
)
#Test
fun `test V1` (
// stubStepA
// stubStepB
// stubStepC
call V1()
// verify results
)
The withLock() function and how I wrap it in V2
suspend fun <T : Any> withLock(
function: suspend () -> T
): T {
acquireLock()
val res = function()
releaseLock()
return res
}
suspend fun V2 : RandomType (
return withLock() {
// Step A
// Step B
// Step C
}
)
Want to know how am I supposed to modify the unit test to be able to test withLock()
This would be a way to test it, of course acquireLock/releaseLock aren't checked directly but if check the function was executed inside withLock:
class LockTest {
suspend fun whatever() : String = "whatever"
suspend fun V2(): String {
return withLock(::whatever)
}
}
#Test
fun `test`() = runBlockingTest {
val lockTest = spyK(LockTest())
lockTest.V2()
coVerify { lockTest.whatever() }
}
I am trying to validate that a suspend function does not return anything at all in a certain test.
Consider the following situation:
val completionSignal = Channel<Unit>(capacity = 1, onBufferOverflow = BufferOverflow.DROP_LATEST)
suspend fun waitForCompletionSignal(): String {
completionSignal.receive()
return "Completion signal received"
}
I want to test this code with 2 unit tests, one that validates it returns the string when I provide the CompletionSignal with a value (thats the easy one).
And one that validates that it does not return anything when i don't give it anything. This is the hard one, since how long should I wait? And can i be sure the test fails if my code changes and suddenly the string is returned?
I got the following approach but I am missing some pieces:
#Test
fun `waitForCompletionSignal when completionSignal is provided assert result`() = runTest {
// Start waiting for result
val result = async { waitForCompletionSignal() }
// Provide completion signal
completionSignal.trySend(Unit)
// Await result and verify its correct
assertThat(result.await() == "Completion signal received")
}
#Test
fun `waitForCompletionSignal when completionSignal is not provided assert no result`() = runTest {
// Start waiting for result
val result = async { waitForCompletionSignal() }
// TODO?? some validation that succeeds if the test is just like this, but fails when i do the following:
completionSignal.trySend(Unit)
// A regular await would wait indefinately, and checking if the deferred result is completed does not work very well as well.
}
I hope the question is clear, thanks in advance.
I made an extension function on the deferred type to be able to wait for a max amount of time and after that it will return null. In my particular situation a delay time of 0 (so no delay whatsoever) is enough, but I can imagine that in some situations its useful to delay for a minimum amount of time.
#ExperimentalCoroutinesApi
suspend inline fun <reified T> Deferred<T>.awaitOrNull(
time: Long = 0,
crossinline actBlock: () -> Unit = { }
): T? = coroutineScope {
actBlock()
val timeoutJob = async<T?> {
delay(time)
null
}
return#coroutineScope select<T?> {
this#awaitOrNull.onAwait {
timeoutJob.cancel()
it
}
timeoutJob.onAwait {
this#awaitOrNull.cancel()
it
}
}
}
Using this method i can write the following tests that fail / succeed as expected:
// Succeeds
#Test
fun `waitForCompletionSignal when completionSignal is provided assert result`() = runTest {
val result = async {
waitForCompletionSignal()
}.awaitOrNull {
completionSignal.trySend(Unit)
}
assertThat(result == "Completion signal received")
}
// Succeeds
#Test
fun `waitForCompletionSignal when completionSignal is not provided assert no result`() = runTest {
val result = async {
waitForCompletionSignal()
}.awaitOrNull()
assertThat(result == null)
}
// Fails (to prove that it works)
#Test
fun `waitForCompletionSignal when completionSignal is not provided assert no result`() = runTest {
val result = async {
waitForCompletionSignal()
}.awaitOrNull {
completionSignal.trySend(Unit) // This (obviously) causes the test to fail, as I wanted.
}
assertThat(result == null)
}
Please consider the following function:
pub fn shiny_function(&mut self, cb: Arc<Mutex<dyn FnMut(usize) + Send>>) {
// Do stuff here...
}
Now, the question is, how do I write a Unit Test that checks that the callback (closure) parameter is equal to some value?
The obvious solution looks something like this:
#[test]
fn progress_cb() {
let cut = ... // cut stands for Class Under Test
cut.shiny_function(Arc::new(Mutex::new(move |percent| {
// Assert here maybe? I don't know.
})));
cut.shiny_function();
// Or maybe assert it somehow here? I don't know.
}
But the thing is the test finishes before the callback is even called. How can I tell the test harness to wait until the callback is called?
You can use the regular concurrency structs provided in the standard library to fix this issue. In this example, I use a barrier to ensure that the end of the closure is reached before the test function exits. I create the barrier with a value of 2 since wait must be called twice before the barrier is released on both threads. This behavior might not be desirable when calling shiny_function multiple times so you could also substitute another concurrency structure that only blocks in a single location.
use std::sync::{Arc, Barrier};
#[test]
fn progress_cb() {
let cut = ... // cut stands for Class Under Test
// Create a barrier for this thread and clone it to move into the closure
let barrier = Arc::new(Barrier::new(2));
let barrier_clone = barrier.clone();
cut.shiny_function(Arc::new(Mutex::new(move |percent| {
// Perform tests
assert_eq!(percent, foo);
// Once we finish we can trigger the barrier so the outer thread can continue
barrier_clone.wait();
})));
// Don't exit the function until the barrier has been resolved in the callback
barrier.wait();
}
Edit: Here is a struct you could use to if the barrier starts becoming an issue due to the closure blocking on every call and holding up later calls to shiny_function in a single test function.
use std::sync::{Arc, Mutex, Condvar};
pub struct SingleBlockingBarrier {
target: u32,
counter: Mutex<u32>,
lock: Condvar,
}
impl SingleBlockingBarrier {
pub fn new(target: u32) -> Arc<Self> {
Arc::new(SingleBlockingBarrier {
target,
counter: Mutex::new(0),
lock: Condvar::new(),
})
}
pub fn signal(&self) {
let mut guard = self.counter.lock().unwrap();
*guard += 1;
if *guard >= self.target {
self.lock.notify_all();
}
}
// Block until signal has been called the target number of times
pub fn block_until_finished(&self) {
let mut guard = self.counter.lock().unwrap();
loop {
if *guard >= self.target {
return;
}
guard = self.lock.wait(guard).unwrap();
}
}
}
#[test]
fn progress_cb() {
let cut = ... // cut stands for Class Under Test
// Create a barrier for this thread and clone it to move into the closure
let barrier = SingleBlockingBarrier::new(10);
for _ in 0..10 {
let barrier_clone = barrier.clone();
cut.shiny_function(Arc::new(Mutex::new(move |percent| {
// Perform tests
assert_eq!(percent, foo);
// Notify barrier that a worker has finished without blocking
barrier_clone.signal();
})));
}
// Block until all non-blocking barriers have been reached
barrier.block_until_finished();
}
I have a collection of tests. There are a few tests that need to access a shared resource (external library/API/hardware device). If any of these tests run in parallel, they fail.
I know I could run everything using --test-threads=1 but I find that inconvenient just for a couple of special tests.
Is there any way to keep running all tests in parallel and have an exception for a few? Ideally, I would like to say do not run X, Y, Z at the same time.
Use the serial_test crate. With this crate added, you put in your code:
#[serial]
in front of any test you want run in sequentially.
As mcarton mentions in the comments, you can use a Mutex to prevent multiple pieces of code from running at the same time:
use once_cell::sync::Lazy; // 1.4.0
use std::{sync::Mutex, thread::sleep, time::Duration};
static THE_RESOURCE: Lazy<Mutex<()>> = Lazy::new(Mutex::default);
type TestResult<T = (), E = Box<dyn std::error::Error>> = std::result::Result<T, E>;
#[test]
fn one() -> TestResult {
let _shared = THE_RESOURCE.lock()?;
eprintln!("Starting test one");
sleep(Duration::from_secs(1));
eprintln!("Finishing test one");
Ok(())
}
#[test]
fn two() -> TestResult {
let _shared = THE_RESOURCE.lock()?;
eprintln!("Starting test two");
sleep(Duration::from_secs(1));
eprintln!("Finishing test two");
Ok(())
}
If you run with cargo test -- --nocapture, you can see the difference in behavior:
No lock
running 2 tests
Starting test one
Starting test two
Finishing test two
Finishing test one
test one ... ok
test two ... ok
With lock
running 2 tests
Starting test one
Finishing test one
Starting test two
test one ... ok
Finishing test two
test two ... ok
Ideally, you'd put the external resource itself in the Mutex to make the code represent the fact that it's a singleton and remove the need to remember to lock the otherwise-unused Mutex.
This does have the massive downside that a panic in a test (a.k.a an assert! failure) will cause the Mutex to become poisoned. This will then cause subsequent tests to fail to acquire the lock. If you need to avoid that and you know the locked resource is in a good state (and () should be fine...) you can handle the poisoning:
let _shared = THE_RESOURCE.lock().unwrap_or_else(|e| e.into_inner());
If you need the ability to run a limited set of threads in parallel, you can use a semaphore. Here, I've built a poor one using Condvar with a Mutex:
use std::{
sync::{Condvar, Mutex},
thread::sleep,
time::Duration,
};
#[derive(Debug)]
struct Semaphore {
mutex: Mutex<usize>,
condvar: Condvar,
}
impl Semaphore {
fn new(count: usize) -> Self {
Semaphore {
mutex: Mutex::new(count),
condvar: Condvar::new(),
}
}
fn wait(&self) -> TestResult {
let mut count = self.mutex.lock().map_err(|_| "unable to lock")?;
while *count == 0 {
count = self.condvar.wait(count).map_err(|_| "unable to lock")?;
}
*count -= 1;
Ok(())
}
fn signal(&self) -> TestResult {
let mut count = self.mutex.lock().map_err(|_| "unable to lock")?;
*count += 1;
self.condvar.notify_one();
Ok(())
}
fn guarded(&self, f: impl FnOnce() -> TestResult) -> TestResult {
// Not panic-safe!
self.wait()?;
let x = f();
self.signal()?;
x
}
}
lazy_static! {
static ref THE_COUNT: Semaphore = Semaphore::new(4);
}
THE_COUNT.guarded(|| {
eprintln!("Starting test {}", id);
sleep(Duration::from_secs(1));
eprintln!("Finishing test {}", id);
Ok(())
})
See also:
How to limit the number of test threads in Cargo.toml?
You can always provide your own test harness. You can do that by adding a [[test]] entry to Cargo.toml:
[[test]]
name = "my_test"
# If your test file is not `tests/my_test.rs`, add this key:
#path = "path/to/my_test.rs"
harness = false
In that case, cargo test will compile my_test.rs as a normal executable file. That means you have to provide a main function and add all the "run tests" logic yourself. Yes, this is some work, but at least you can decide everything about running tests yourself.
You can also create two test files:
tests/
- sequential.rs
- parallel.rs
You then would need to run cargo test --test sequential -- --test-threads=1 and cargo test --test parallel. So it doesn't work with a single cargo test, but you don't need to write your own test harness logic.
I am building a library that interrogates its running environment to return values to the asking program. Sometimes as simple as
pub fn func_name() -> Option<String> {
match env::var("ENVIRONMENT_VARIABLE") {
Ok(s) => Some(s),
Err(e) => None
}
}
but sometimes a good bit more complicated, or even having a result composed of various environment variables. How can I test that these methods are functioning as expected?
"How do I test X" is almost always answered with "by controlling X". In this case, you need to control the environment variables:
use std::env;
fn env_is_set() -> bool {
match env::var("ENVIRONMENT_VARIABLE") {
Ok(s) => s == "yes",
_ => false
}
}
#[test]
fn when_set_yes() {
env::set_var("ENVIRONMENT_VARIABLE", "yes");
assert!(env_is_set());
}
#[test]
fn when_set_no() {
env::set_var("ENVIRONMENT_VARIABLE", "no");
assert!(!env_is_set());
}
#[test]
fn when_unset() {
env::remove_var("ENVIRONMENT_VARIABLE");
assert!(!env_is_set());
}
However, you need to be aware that environment variables are a shared resource. From the docs for set_var, emphasis mine:
Sets the environment variable k to the value v for the currently running process.
You may also need to be aware that the Rust test runner runs tests in parallel by default, so it's possible to have one test clobber another.
Additionally, you may wish to "reset" your environment variables to a known good state after the test.
Your other option (if you don't want to mess around with actually setting environment variables) is to abstract the call away. I am only just learning Rust and so I am not sure if this is "the Rust way(tm)" to do it... but this is certainly how I would do it in another language/environment:
use std::env;
pub trait QueryEnvironment {
fn get_var(&self, var: &str) -> Result<String, std::env::VarError>;
}
struct MockQuery;
struct ActualQuery;
impl QueryEnvironment for MockQuery {
fn get_var(&self, _var: &str) -> Result<String, std::env::VarError> {
Ok("Some Mocked Result".to_string()) // Returns a mocked response
}
}
impl QueryEnvironment for ActualQuery {
fn get_var(&self, var: &str) -> Result<String, std::env::VarError> {
env::var(var) // Returns an actual response
}
}
fn main() {
env::set_var("ENVIRONMENT_VARIABLE", "user"); // Just to make program execute for ActualQuery type
let mocked_query = MockQuery;
let actual_query = ActualQuery;
println!("The mocked environment value is: {}", func_name(mocked_query).unwrap());
println!("The actual environment value is: {}", func_name(actual_query).unwrap());
}
pub fn func_name<T: QueryEnvironment>(query: T) -> Option<String> {
match query.get_var("ENVIRONMENT_VARIABLE") {
Ok(s) => Some(s),
Err(_) => None
}
}
Example on the rust playground
Notice how the actual call panics. This is the implementation you would use in actual code. For your tests, you would use the mocked ones.
EDIT:
The test helpers below are now available in a dedicated crate
Disclaimer: I'm a co-author
I had the same need and implemented some small test helpers which take care of the caveats mentioned by #Shepmaster .
These test helpers enable testing like so:
#[test]
fn test_default_log_level_is_info() {
with_env_vars(
vec![
("LOGLEVEL", None),
("SOME_OTHER_VAR", Some("foo"))
],
|| {
let actual = Config::new();
assert_eq!("INFO", actual.log_level);
},
);
}
with_env_vars will take care of:
Avoiding side effects when running tests in parallel
Resetting the env variables to their original values when the test closure completes
Support for unsetting environment variables during the test closure
All of the above, also when the test-closure panics.
The helper:
use lazy_static::lazy_static;
use std::env::VarError;
use std::panic::{RefUnwindSafe, UnwindSafe};
use std::sync::Mutex;
use std::{env, panic};
lazy_static! {
static ref SERIAL_TEST: Mutex<()> = Default::default();
}
/// Sets environment variables to the given value for the duration of the closure.
/// Restores the previous values when the closure completes or panics, before unwinding the panic.
pub fn with_env_vars<F>(kvs: Vec<(&str, Option<&str>)>, closure: F)
where
F: Fn() + UnwindSafe + RefUnwindSafe,
{
let guard = SERIAL_TEST.lock().unwrap();
let mut old_kvs: Vec<(&str, Result<String, VarError>)> = Vec::new();
for (k, v) in kvs {
let old_v = env::var(k);
old_kvs.push((k, old_v));
match v {
None => env::remove_var(k),
Some(v) => env::set_var(k, v),
}
}
match panic::catch_unwind(|| {
closure();
}) {
Ok(_) => {
for (k, v) in old_kvs {
reset_env(k, v);
}
}
Err(err) => {
for (k, v) in old_kvs {
reset_env(k, v);
}
drop(guard);
panic::resume_unwind(err);
}
};
}
fn reset_env(k: &str, old: Result<String, VarError>) {
if let Ok(v) = old {
env::set_var(k, v);
} else {
env::remove_var(k);
}
}
A third option, and one I think is better, is to pass in the existing type - rather than creating a new abstraction that everyone would have to coerce to.
pub fn new<I>(vars: I)
where I: Iterator<Item = (String, String)>
{
for (x, y) in vars {
println!("{}: {}", x, y)
}
}
#[test]
fn trivial_call() {
let vars = [("fred".to_string(), "jones".to_string())];
new(vars.iter().cloned());
}
Thanks to qrlpz on #rust for helping me get this sorted for my program, just sharing the result to help others :)