Let's report some diagnostic and register code fix for this diagnostic.
private static void OnCompilation(CompilationAnalysisContext context) {
var compilation = context.Compilation;
var diagnostic = Diagnostic.Create( Rule, (Location) null );
context.ReportDiagnostic( diagnostic );
}
public sealed override Task RegisterCodeFixesAsync(CodeFixContext context) {
var document = context.Document;
//var span = context.Span;
var diagnostic = context.Diagnostics.Single();
RegisterCodeFix( context, $"Fix compilation", diagnostic, Action );
Task<Solution> Action(CancellationToken cancellationToken) {
return Task.FromResult( document.Project.Solution );
}
}
Problem is that: there is no way to execute code fix action. Light bulb is not shown.
And even if I could execute it then document were null here.
So, is it possible to make code fix for compilation-level diagnostics?
Code fixes for project-level diagnostics are not supported.
https://github.com/dotnet/roslyn/discussions/50087#discussioncomment-230367
Related
I am using Artos runner. In our development environment we keep
<property name="stopOnFail">true</property> so I can debug my changes without having to deal with dependent test cases failing. In production environment we keep <property name="stopOnFail">false</property> so test execution does not stop upon failure and we can analyse log in the morning.
Now I have a different requirement,
I have some tests that are pre-requisite for rest of the units, so if critical test fails then I would like to skip rest of the unit, otherwise it can put our product into bad state.
Is there a way in Artos to skip rest of the unit only if specific test case or test unit fails?
Or can we perform specific steps, incase test fails to ensure we are safe to perform rest of the tests?
Depending on the requirement there are multiple ways to achieve it in Artos
First of all, ensure all of your units are having a sequence number so they execute in the same order all the time.
Let's say testUnit_1() is critical unit and it must be executed successfully in order to execute rest of the following units. In that case, set dropRemainingUnitsUponFailure = true as shown below. This will ensure that the rest of the units are dropped from the execution list if testUnit_1() fails.
#TestPlan(preparedBy = "user", preparationDate = "19/02/2019", bdd = "GIVEN..WHEN..AND..THEN..")
#TestCase(sequence = 1)
public class TestCase_1 implements TestExecutable {
#Unit(sequence = 1, dropRemainingUnitsUponFailure = true)
public void testUnit_1(TestContext context) {
context.getLogger().info("do something");
}
#Unit(sequence = 2)
public void testUnit_2(TestContext context) {
context.getLogger().info("do something");
}
#Unit(sequence = 3)
public void testUnit_3(TestContext context) {
context.getLogger().info("do something");
}
}
If the test cases that are dependent upon each other then you can do similar at test case level
Ensure TestCases are assigned with sequence number so they follow same execution order (Similar to units)
As shown below. If dropRemainingTestsUponFailure = true and dropRemainingUnitsUponFailure = true then upon testUnit_1() failure, not only rest of the units will be dropped but also remaining test cases will be dropped from execution list so you can achieve clean exit.
#TestPlan(preparedBy = "user", preparationDate = "19/02/2019", bdd = "GIVEN..WHEN..AND..THEN..")
#TestCase(sequence = 1, , dropRemainingTestsUponFailure = true)
public class TestCase_1 implements TestExecutable {
#Unit(sequence = 1, dropRemainingUnitsUponFailure = true)
public void testUnit_1(TestContext context) {
context.getLogger().info("do something");
}
#Unit(sequence = 2)
public void testUnit_2(TestContext context) {
context.getLogger().info("do something");
}
#Unit(sequence = 3)
public void testUnit_3(TestContext context) {
context.getLogger().info("do something");
}
}
In the log file you will see warning
=========================================================================
========== DROP REMAINING UNITS UPON FAILURE IS TRIGGERED ===============
================== REMAINING UNITS WILL BE DROPPED ======================
=========================================================================
and
=========================================================================
========== DROP REMAINING TESTS UPON FAILURE IS TRIGGERED ===============
================== REMAINING TESTS WILL BE DROPPED ======================
=========================================================================
so you will know what happened.
To Answer your second question
(Question: If there is any ways to perform clean up if test unit fails, so before you perform next unit test, you can recover your product from the bad state)
If I understood it correctly then it can be done using annotation #AfterFailedUnit
If you create a method as shown below in your runner class
#AfterFailedUnit
public void globalAfterFailedTestUnit(TestContext context) throws Exception {
context.getLogger().info("This method executes after failed test unit");
}
then it will be executed after each test unit failure, you should implement clean up logic in this method.
Hopefully this answers your questions
I have this function
override fun trackEvent(trackingData: TrackingData) {
trackingData.eventsList()
}
And I could have my test as below.
#Test
fun `My Test`() {
// When
myObject.trackEvent(myTrackingMock)
// Then
verify(myTrackingMock, times(1)).eventsList()
}
However, if I make it into a
override fun trackEvent(trackingData: TrackingData) {
GlobalScope.launch{
trackingData.eventsList()
}
}
How could I still get my test running? (i.e. can make the launch Synchronous?)
I created my own CoroutineScope and pass in (e.g. CoroutineScope(Dispatchers.IO) as a variable myScope)
Then have my function
override fun trackEvent(trackingData: TrackingData) {
myScope.launch{
trackingData.eventsList()
}
}
Then in my test I mock the scope by create a blockCoroutineScope as below.
class BlockCoroutineDispatcher : CoroutineDispatcher() {
override fun dispatch(context: CoroutineContext, block: Runnable) {
block.run()
}
}
private val blockCoroutineScope = CoroutineScope(BlockCoroutineDispatcher())
For my test, I'll pass the blockCoroutineScope in instead as myScope. Then the test is executed with launch as a blocking operation.
To approach the answer, try asking a related question: "How would I unit-test a function that has
Thread { trackingData.eventsList() }
in it?"
Your only hope is running a loop that repeatedly checks the expected condition, for some period time, until giving up and declaring the test failed.
When you wrote GlobalScope.launch, you waived your interest in Kotlin's structured concurrency, so you'll have to resort to unstructured and non-deterministic approaches of testing.
Probably the best recourse is to rewrite your code to use a scope under your control.
I refactored my method to
suspend fun deleteThing(serial: String): String? = coroutineScope {
This way, I can launch coroutines with launch
val jobs = mutableListOf<Job>()
var certDeleteError: String? = null
certs.forEach { certArn ->
val job = launch {
deleteCert(certArn, serial)?.let { error ->
jobs.forEach { it.cancel() }
certDeleteError = error
}
}
jobs.add(job)
}
jobs.joinAll()
For the test, I can then just use runTest and it runs all of the coroutines synchronously
#Test
fun successfullyDeletes2Certs() = runTest {
aws.deleteThing("s1")
Now you just need to mind your context where you are calling the deleteThing function. For me, it was a ktor request, so I could just call launch there also.
delete("vehicles/{vehicle-serial}/") {
launch {
aws.deleteThing(serial)
}
}
The problem
I have the following simple situation popping up all over the place. A large number of requests come to the device with a function signature like this:
Err execute( const ICommandContext &context,
const RoutineArguments &arguments,
RoutineResults &results)
There is essentially a request handling server that will call this execute the function for a variety of request types that have these signatures. We have 2 return paths in the case of an error.
The Err output type (consider it to be equivalent to an int) which is used to inform the server or system that something has gone wrong that is to do with the system, not the request. This is always sorted at the top of the function before the user request is dealt with.
RoutineResults provides a setStatus function that can be used to return failure information of the request to the client.
For this reason we have a lot of this type of code popping up:
// Failure due to request
Err error = someFunctionCall(clientInput);
if (!error.success()) {
results.setStatus(error); // Inform the client of the error
return SUCCESS; // Inform the system that we are all good
}
We have a particular request type that has around 15 parameters that come in and are sent off around the system. We would conceptually need 15 of this if error do set which seems wasteful. It is also prone to errors if we need to go through and change anything about how we return. How can we effectively delegate the setStatus and return to a short amount of code that only needs to happen once in the function?
A Macro Solution
A c system might solve this with a macro, something like:
#define M_InitTry Err error
#define M_Try(statement) if (!(error = statement).success()) { goto catch_lab; }
#define M_Catch catch_lab: if (!error.successs())
#define M_Return return error
Which would be used like this:
Err execute( const ICommandContext &context, ...) {
M_InitTry;
...
M_Try(someFunctionCall(clientInput));
M_Try(someFunctionCall(otherClientInput));
...
M_Catch {
// Other specific actions for dealing with the return.
results.setStatus(error);
error = SUCCESS;
}
M_Return;
}
This cleans the code nicely, but is not particularly nice with the goto. It will cause problems if defining variables that might be skipped by a goto.
A delegating solution
I was trying to think of a more C++ so I thought an RAII type delegate might help. Something like:
class DelegateToFunctionEnd {
typedef std::function<void(void)> EndFunction;
public:
DelegateToFunctionEnd(EndFunction endFunction) : callAtEnd(endFunction) { }
~DelegateToFunctionEnd() {
callAtEnd();
}
private:
EndFunction callAtEnd;
};
Pretty simple, it does a delegate of the action until the function return by implementing the action in the destructor. You might use it like this:
Err execute( const ICommandContext &context, ...) {
Err error;
DelegateToFunctionEnd del(std::bind(&RoutineResults::setStatus, &results, std::cref(error)));
error = someFunctionCall(clientInput));
if (error) return SUCCESS;
...
}
Live example.
This solution seems ok, but has several problems:
It is not as clear what is happening.
It is easier to make a mistake about setting error correctly.
You still need a large number of if statements to deal with the returns.
The ability to configure the terminating action is not great.
Dangerous if the user doesn't carefully consider the destruction order of items at function return.
A better solution?
This must be a problem that comes up often. Is there a general solution that provides a clean delegation of this set and returns type action?
I have some unfortunate restrictions below. Don't let these stop you from answering because it might be helpful for future people.
I am working on a c++03 limited system. We have boost, but no c++11.
Embedded system and we have silly rules about exceptions and memory allocation.
If error status codes are proving troublesome, you should consider using exceptions instead. That is, change the API of your functions
so they are guaranteed to have success as a post-condition
throw a suitable std::exception in the event of failure
It is impossible to "forget" to examine a status code if you do this. If you choose not to handle an error condition, the exception thrown by low-level code automatically percolates upwards. You only need to catch a low-level exception if
You need to do some manual roll-back or deallocation in the event of an error,
and RAII is impractical. In this case you would rethrow the expcetion.
You want to translate a low-level exception message or exception type into a high-level message, using a thrown) nested exception.
Maybe, you can write your statement as array, something like:
Err execute( const ICommandContext &context, ...)
{
const boost::function<Err()> functions[] = {
boost::bind(&someFunctionCall, std::ref(clientInput)),
boost::bind(&someFunctionCall, std::ref(otherClientInput)),
// ...
};
for (std::size_t i = 0; i != sizeof(functions) / sizeof(functions[0]); ++i) {
Err err = functions[i]();
if (!err.successs()) {
results.setStatus(err);
return SUCCESS;
}
}
return SUCCESS;
}
and if you do that several time with only different statements,
you might create
Err execute_functions(const ICommandContext &context, std::function<Err()> functions);
Maybe also provide other entry points as OnError depending of your needs.
Split the function.
The inner function returns an error code based on user input; the outer translates that to a client error, and only returns server side errors.
Inner function contains:
if(Err error = someFunctionCall(clientInput))
return error;
repeatedly. Outer has the relay to client error code, but only once.
Err just needs an operator bool. If it cannot have it, create a type that converts to/from Err and has an operator bool.
Can you add a method to error that does the check etc and return a bool.
if(!someFunctionCall(clientInput).handleSuccess(results))
{
return SUCCESS;
}
I follow the MVP pattern + UseCases to interact with a Model layer. This is a method in a Presenter I want to test:
fun loadPreviews() {
launch(UI) {
val items = previewsUseCase.getPreviews() // a suspending function
println("[method] UseCase items: $items")
println("[method] View call")
view.showPreviews(items)
}
}
My simple BDD test:
fun <T> givenSuspended(block: suspend () -> T) = BDDMockito.given(runBlocking { block() })
infix fun <T> BDDMockito.BDDMyOngoingStubbing<T>.willReturn(block: () -> T) = willReturn(block())
#Test
fun `load previews`() {
// UseCase and View are mocked in a `setUp` method
val items = listOf<PreviewItem>()
givenSuspended { previewsUseCase.getPreviews() } willReturn { items }
println("[test] before Presenter call")
runBlocking { presenter.loadPreviews() }
println("[test] after Presenter call")
println("[test] verify the View")
verify(view).showPreviews(items)
}
The test passes successfully but there's something weird in the log. I expect it to be:
"[test] before Presenter call"
"[method] UseCase items: []"
"[method] View call"
"[test] after Presenter call"
"[test] verify the View"
But it turns out to be:
[test] before Presenter call
[test] after Presenter call
[test] verify the View
[method] UseCase items: []
[method] View call
What's the reason of this behaviour and how should I fix it?
I've found out that it's because of a CoroutineDispatcher. I used to mock UI context with EmptyCoroutineContext. Switching to Unconfined has solved the problem
Update 02.04.20
The name of the question suggests that there'll be an exhaustive explanation how to unit test a suspending function. So let me explain a bit more.
The main problem with testing a suspending function is threading. Let's say we want to test this simple function that updates a property's value in a different thread:
class ItemUpdater(val item: Item) {
fun updateItemValue() {
launch(Dispatchers.Default) { item.value = 42 }
}
}
We need to somehow replace Dispatchers.Default with an another dispatcher only for testing purposes. There're two ways how we can do that. Each has its pros and cons, and which one to choose depends on your project & style of coding:
1. Inject a Dispatcher.
class ItemUpdater(
val item: Item,
val dispatcher: CoroutineDispatcher // can be a wrapper that provides multiple dispatchers but let's keep it simple
) {
fun updateItemValue() {
launch(dispatcher) { item.value = 42 }
}
}
// later in a test class
#Test
fun `item value is updated`() = runBlocking {
val item = Item()
val testDispatcher = Dispatchers.Unconfined // can be a TestCoroutineDispatcher but we still keep it simple
val updater = ItemUpdater(item, testDispatcher)
updater.updateItemValue()
assertEquals(42, item.value)
}
2. Substitute a Dispatcher.
class ItemUpdater(val item: Item) {
fun updateItemValue() {
launch(DispatchersProvider.Default) { item.value = 42 } // DispatchersProvider is our own global wrapper
}
}
// later in a test class
// -----------------------------------------------------------------------------------
// --- This block can be extracted into a JUnit Rule and replaced by a single line ---
// -----------------------------------------------------------------------------------
#Before
fun setUp() {
DispatchersProvider.Default = Dispatchers.Unconfined
}
#After
fun cleanUp() {
DispatchersProvider.Default = Dispatchers.Default
}
// -----------------------------------------------------------------------------------
#Test
fun `item value is updated`() = runBlocking {
val item = Item()
val updater = ItemUpdater(item)
updater.updateItemValue()
assertEquals(42, item.value)
}
Both of them are doing the same thing - they replace the original Dispatchers.Default in test classes. The only difference is how they do that. It's really really up to you which of them to choose so don't get biased by my own thoughts below.
IMHO: The first approach is a little too much cumbersome. Injecting dispatchers everywhere will result into polluting most of the classes' constructors with an extra DispatchersWrapper only for a testing purpose. However Google recommends this way at least for now. The second style keeps things simple and it doesn't complicate the production classes. It's like an RxJava's way of testing where you have to substitute schedulers via RxJavaPlugins. By the way, kotlinx-coroutines-test will bring the exact same functionality someday in future.
I see you found out on you own, but I'd like to explain a bit more for the people that might run into the same problem
When you do launch(UI) {}, a new coroutine is created and dispatched to the "UI" Dispatcher, that means that your coroutine now runs on a different thread.
Your runBlocking{} call create a new coroutine, but runBlocking{} will wait for this coroutine to end before continuing, your loadPreviews() function creates a coroutine, start it and then return immediately, so runBlocking() just wait for it and return.
So while runBlocking{} has returned, the coroutine that you created with launch(UI){} is still running in a different thread, that's why the order of your log is messed up
The Unconfined context is a special CoroutineContext that simply create a dispatcher that execute the coroutine right there on the current thread, so now when you execute runBlocking{}, it has to wait for the coroutine created by launch{} to end because it is running on the same thread thus blocking that thread.
I hope my explanation was clear, have a good day
Given this code:
var assert = require('assert');
function boom(){
throw new Error('BOOM');
}
assert.throws( boom(), Error );
I get this output, with node 0.4.9:
node.js:134
throw e; // process.nextTick error, or 'error' event on first tick
^
Error: BOOM
at boom ([EDITED]/assert.throws.test.js:4:9)
at Object.<anonymous> ([EDITED]/assert.throws.test.js:7:17)
at Module._compile (module.js:402:26)
at Object..js (module.js:408:10)
at Module.load (module.js:334:31)
at Function._load (module.js:293:12)
at Array.<anonymous> (module.js:421:10)
at EventEmitter._tickCallback (node.js:126:26)
This, to me, implies that an uncaught exception has occurred, as opposed to a reported, caught exception. Looking in the docs, I notice that the examples look more like this:
var assert = require('assert');
function boom(){
throw new Error('BOOM');
}
assert.throws( boom, Error );
But how do you test if it throws an exception given a certain input? For example:
var assert = require('assert');
function boom(blowup){
if(blowup)
throw new Error('BOOM');
}
assert.throws( boom, Error );
This will fail. What am I doing wrong, or what secret does everybody know but me?
The examples take a function, while your sample code calls a function and passes the result. The exception happens before the assert even gets to look at it.
Change your code to this:
var assert = require('assert');
function boom(){
throw new Error('BOOM');
}
assert.throws( boom, Error ); // note no parentheses
EDIT: To pass parameters, just make another function. After all, this is javascript!
var assert = require('assert');
function boom(blowup){
if(blowup)
throw new Error('BOOM');
}
assert.throws( function() { boom(true); }, Error );
You can use bind():
assert.throws( boom.bind(null), Error );
With arguments it is:
assert.throws( boom.bind(null, "This is a blowup"), Error );
Current node stable (v4.1) includes fat arrow function support by default (no --harmony flag required) so you can do something like:
assert.throws(()=>boom(), Error);
assert.throws(()=>boom(true), Error); // with params
Even if you have parentheses after boom() (so you're actually invoking it, instead of passing a reference to the function object), by using the fat arrow function you're wrapping it in a block, which is what assert.throws expects.
This is closely related to the issue people with with other assertion Mocha/Chai. See this answer for the description with node examples:
Mocha / Chai expect.to.throw not catching thrown errors