AWS Kinesis Rust SDK is not compatible with Tokio Runtime - amazon-web-services

I am trying to bridging the async functionality of the SDK with sync code. I need this to create a tokio_stream (a struct with Stream trait implemented.) The problem is quite suprising,
#[cfg(test)]
mod tests {
use aws_config::meta::region::RegionProviderChain;
use aws_sdk_kinesis::{Client, Region};
use aws_config::SdkConfig;
use aws_sdk_kinesis::error::ListShardsError;
use aws_sdk_kinesis::output::ListShardsOutput;
use aws_sdk_kinesis::types::SdkError;
use futures::executor::block_on;
use tokio::runtime::Runtime;
/// It passes this test, without a problem.
#[tokio::test]
async fn testing_kinesis_stream_connection() -> () {
let region_provider = RegionProviderChain::first_try(Region::new("eu-central-1".to_string()));
let sdk_config =
aws_config::from_env()
.region(region_provider)
.load().await;
let client = Client::new(&sdk_config);
let shard_request = client.list_shards().stream_name("datafusion-stream".to_string());
let shards = shard_request.send().await.unwrap();
println!("{:?}", shards);
}
/// It passes this test, without a problem.
#[test]
fn testing_kinesis_stream_connection_without_tokio() -> () {
let runtime = tokio::runtime::Builder::new_current_thread().enable_all().build().unwrap();
let region_provider = RegionProviderChain::first_try(Region::new("eu-central-1".to_string()));
let sdk_config =
runtime.block_on(aws_config::from_env()
.region(region_provider)
.load());
let client = Client::new(&sdk_config);
let shard_request = client.list_shards().stream_name("datafusion-stream".to_string());
let shards = runtime.block_on(shard_request.send()).unwrap();
println!("{:?}", shards);
}
fn this_needs_to_be_sync_1() -> SdkConfig {
let region_provider = RegionProviderChain::first_try(Region::new("eu-central-1".to_string()));
let sdk_config =
futures::executor::block_on(aws_config::from_env()
.region(region_provider)
.load());
sdk_config
}
fn this_needs_to_be_sync_2(client: Client) -> ListShardsOutput {
let shard_request = client.list_shards().stream_name("datafusion-stream".to_string());
let shards = futures::executor::block_on(shard_request.send()).unwrap();
shards
}
/// It hangs in the second block_on.
#[tokio::test]
async fn testing_kinesis_stream_connection_sync_inside_async() -> () {
// Passes this block_on
let sdk_config = this_needs_to_be_sync_1();
let client = Client::new(&sdk_config);
// Blocks here
let shards = this_needs_to_be_sync_2(client);
println!("{:?}", shards);
}
}
I could not come up with a solution since there is no error, only hanged process.

Related

How to test Contacts Framework

hi i want to test CNContacts Store since this is my first time doing test, i don't have any idea how to conduct a unit test. This the code i want to test.
private func fetchContacts() {
var contacts: [Contact] = []
let keys: [CNKeyDescriptor] = [CNContactFormatter.descriptorForRequiredKeys(for: .fullName),
CNContactPhoneNumbersKey as CNKeyDescriptor]
let request = CNContactFetchRequest(keysToFetch: keys)
do {
try contactStore.enumerateContacts(with: request) {
(contact, stop) in
let name: String = CNContactFormatter.string(from: contact, style: .fullName) ?? contact.nickname
contacts.append(contentsOf: contact.phoneNumbers.compactMap({ phoneNumber in
let phoneNumberString: String = phoneNumber.value.stringValue
return .init(name: name, phoneNumber: phoneNumberString)
}))
}
allContacts = contacts
isContactsFetched = true
filterContacts()
}
catch {
print("unable to fetch contacts")
}
}
I'm using sourcery to generate mock from CNContactStore this is the enumerated mock i generate using sorcery
//MARK: - enumerateContacts
var enumerateContactsWithUsingBlockThrowableError: Error?
var enumerateContactsWithUsingBlockCallsCount = 0
var enumerateContactsWithUsingBlockCalled: Bool {
return enumerateContactsWithUsingBlockCallsCount > 0
}
var enumerateContactsWithUsingBlockReceivedArguments: (fetchRequest: CNContactFetchRequest, block: (CNContact, UnsafeMutablePointer<ObjCBool>) -> Void)?
var enumerateContactsWithUsingBlockReceivedInvocations: [(fetchRequest: CNContactFetchRequest, block: (CNContact, UnsafeMutablePointer<ObjCBool>) -> Void)] = []
var enumerateContactsWithUsingBlockClosure: ((CNContactFetchRequest, #escaping (CNContact, UnsafeMutablePointer<ObjCBool>) -> Void) throws -> Void)?
func enumerateContacts(with fetchRequest: CNContactFetchRequest, usingBlock block: #escaping (CNContact, UnsafeMutablePointer<ObjCBool>) -> Void) throws {
if let error = enumerateContactsWithUsingBlockThrowableError {
throw error
}
enumerateContactsWithUsingBlockCallsCount += 1
enumerateContactsWithUsingBlockReceivedArguments = (fetchRequest: fetchRequest, block: block)
enumerateContactsWithUsingBlockReceivedInvocations.append((fetchRequest: fetchRequest, block: block))
try enumerateContactsWithUsingBlockClosure?(fetchRequest, block)
}
what i did so far for unit test is this
it("should fetch contacts") {
let contact = CNContact()
let stop = UnsafeMutablePointer<ObjCBool>.allocate(capacity: 1)
stop[0] = true
// When
viewModel.onViewDidAppear()
// Then
mockContactStore.enumerateContactsWithUsingBlockClosure = { (_, args) in
args(contact, stop)
expect(mockContactStore.enumerateContactsWithUsingBlockCallsCount).to(equal(1))
}
}
Please help
if you want to test this ->
let request = CNContactFetchRequest(keysToFetch: keys)
you can do like this ->
protocol CNContactFetchRequestProtocol {
}
extension CNContactFetchRequest: CNContactFetchRequestProtocol {
}
let request: CNContactFetchRequestProtocol = CNContactFetchRequest(keysToFetch: keys)
and finally you can create mock
class MockContact: CNContactFetchRequestProtocol {
}
and then you can tests like this:
let request: CNContactFetchRequestProtocol = MockContact()

Rust Unit Test - Function call hangs at the very end and does not return

I am writing some unit tests for my Rust http server handlers. But when I am running one of the tests it get stuck at the end of the inner function. Here is relevant part of the code:
async fn generate(request: Request<Body>) -> Result<Response<Body>, hyper::Error> {
let result = process_request(request).await;
println!("This message doesn't get printed!!");
let (spec, override) = match result {
Ok((s, o)) => (s, o),
Err(process_error) => {
return Ok(Response::new(Body::from(format!("{}", process_error))));
},
};
...
Ok(Response::new(Body::from(format!("{}", response))))
}
async fn process_request(request: Request<Body>) -> Result<(Spec, Option<Config>), Error> {
let body = body::to_bytes(request.into_body()).await;
let payload: Payload = serde_json::from_slice(&body.unwrap().to_vec()).unwrap();
let spec_str = payload.spec.to_owned();
...
println!("Function runs to this point and prints this message");
Ok((spec, override))
}
#[tokio::test]
async fn test_gen() {
let payload = Payload {
spec: a_spec(),
};
let payload_json = serde_json::to_string_pretty(&payload).unwrap();
let request = Request::builder().body(Body::from(payload_json));
let result = generate(request.unwrap()).await.unwrap();
// Some asserts ...
}
I am wondering what I am doing wrong?
Looks like the inner function starts another thread, so the solution was to decorate the test with:
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
This resolved the issue for my unit tests.

How can I programmatically set target_os in a unit test?

I tried the following approaches:
fn get_system_extension(&self) -> String {
if cfg!(target_os = "windows") {
String::from(".lib")
} else {
String::new()
}
}
mod test {
#[allow(unused_imports)]
use super::*;
use std::env;
#[test]
fn get_system_extension_one() -> Result<(), CcrustyError> {
env::set_var("CARGO_BUILD_TARGET", "linux");
let result = get_system_extension();
assert_eq!(String::new(), result);
Ok(())
}
#[test]
fn get_system_extension_two() -> Result<(), CcrustyError> {
env::set_var("CARGO_CFG_UNIX", "");
let result = get_system_extension();
assert_eq!(String::new(), result);
Ok(())
}
}
According to the documentation those could be set in a file, but I have no clue how to dynamically link them to tests.
Are you looking for something something like:
#[test]
fn currect_system_extension() -> Result<(), CcrustyError> {
let system = env::var_os("CARGO_BUILD_TARGET");
let result = get_system_extension();
match system => {
"linux" => assert_eq!(String::new(), result),
"windows" => assert_eq!(".libs".to_string(), result),
_ => (),
}
Ok(())
}
Even with this you would have to compile both versions for this test to cover the code. take a look at using --all-targets
https://doc.rust-lang.org/cargo/commands/cargo-test.html

How do I suppress output from multiple threads when running cargo test?

If I execute the below testcases with cargo test, the output of one_thread_test will be suppressed as stated in the documentation.
However the output from multi_thread_test will appear on stdout. Is it possible to match the behavior of single- and multi-threaded testcases?
#[test]
fn one_thread_test() {
println!("A");
println!("B");
}
#[test]
fn multi_thread_test() {
use std::thread;
let mut threads = vec![];
for _ in 0..100 {
let t = thread::spawn(move || {
println!("from thread");
});
threads.push(t);
}
for thread in threads {
thread.join().unwrap();
}
}
Here is a quick-and-dirty workaround.
It works by sending messages to a receiver owned by a struct in the main thread. The receiver prints all of the accumulated messages when it is dropped - this is important so that panics caused by failed assertions don't prevent the printing.
use std::sync::mpsc::{channel, Sender, Receiver};
struct TestPrinter {
receiver: Receiver<String>,
sender: Sender<String>,
}
impl TestPrinter {
fn new() -> TestPrinter {
let (sender, receiver) = channel();
TestPrinter { receiver, sender }
}
fn sender(&self) -> Sender<String> {
self.sender.clone()
}
}
impl Drop for TestPrinter {
fn drop(&mut self) {
while let Some(v) = self.receiver.try_recv().ok() {
println!("later: {}", v);
}
}
}
And a convenience macro so it feels mostly like calling println!:
macro_rules! myprint {
($send: expr, $($arg:tt)*) => {
(*&$send).send(format!($($arg)*));
};
}
In order to send messages for printing, you have get a sender for each thread:
#[test]
fn multi_thread_test() {
use std::thread;
let mut threads = vec![];
let printer = TestPrinter::new();
for _ in 0..100 {
let sender = printer.sender();
let t = thread::spawn(move || {
myprint!(sender, "from thread");
});
threads.push(t);
}
for thread in threads {
thread.join().unwrap();
}
}
The actual printing happens when printer goes out of scope. It's in the main thread so it won't print during successful tests unless --nocapture is specified.

Pusher Api for Swift 3 crashes while connecting

I have integrated the Pusher framework for my application in Swift 3 using cocoa pods [ pod 'PusherSwift' ].
These are the lines of code :
let pusher = Pusher(key: "XXXXXXXXXXXXXXXXXXXX")
// subscribe to channel and bind to event
let channel = pusher.subscribe("test_channel")
let _ = channel.bind(eventName: "my_event", callback: { (data: Any?) -> Void in
if let data = data as? [String : AnyObject] {
if let message = data["message"] as? String {
print(message)
}
}
})
pusher.connect()
The app crashes at pusher.connect() at the line - self.delegate?.debugLog?(message: "[PUSHER DEBUG] Network reachable"). No crash report is shown.
open lazy var reachability: Reachability? = {
let reachability = Reachability.init()
reachability?.whenReachable = { [unowned self] reachability in
self.delegate?.debugLog?(message: "[PUSHER DEBUG] Network reachable")
if self.connectionState == .disconnected || self.connectionState == .reconnectingWhenNetworkBecomesReachable {
self.attemptReconnect()
}
}
reachability?.whenUnreachable = { [unowned self] reachability in
self.delegate?.debugLog?(message: "[PUSHER DEBUG] Network unreachable")
}
return reachability
}()
This looks like you might be getting bitten by the same issue described here.
I think it's that the PusherConnection object is taken as unowned into the reachability closure but because you're not keeping a reference to the Pusher instance outside of the viewDidLoad function then the connection object gets cleaned up whereas the reachability object does not.
So, to fix this you probably need to declare the pusher object outside of the function where you instantiate it, so that it hangs around. e.g.
class ViewController: UIViewController, PusherDelegate {
var pusher: Pusher! = nil
...
and then within viewDidLoad do pusher = Pusher(... as normal.
I don't think you need to use pusher.connect().
See for example detailed docs:
let pusher = Pusher(key: "YOUR_APP_KEY")
let myChannel = pusher.subscribe("my-channel")
myChannel.bind(eventName: "new-price", callback: { (data: Any?) -> Void in
if let data = data as? [String : AnyObject] {
if let price = data["price"] as? String, company = data["company"] as? String {
print("\(company) is now priced at \(price)")
}
}
})
Alternatively try this first and see if it connects:
let pusher = Pusher(key: "XXXXXXXXXXXXXXXXXXXX")
pusher.connect()
Then bind to your channel.