How to set the exit status in a Groovy Script - unit-testing

We have a Groovy Script that exits with a status of 0 when everything worked and a non-0 status for different types of failure conditions. For example if the script took a user and an email address as arguments it would exit with a status of 1 for an invalid user, and a status of 2 for an invalid email address format. We use System.exit(statusCode) for this. This works fine, but makes the the script difficult to write test cases for.
In a test we create our GroovyShell, create our Binding and call shell.run(script,args). For tests that assert failure conditions, the System.exit() causes the JVM (and the test) to exit.
Are there alternatives to using System.exit() to exit a Groovy Script? I experimented with throwing uncaught exceptions, but that clutters the output and always makes the status code 1.
In our test cases I have also experimented with using System.metaClass.static.invokeMethod to change the behavior of System.exit() to not exit the JVM, but that seems like an ugly hack.

imho System.metaClass.static.invokeMethod looks fine. It's test, and hacking is fine here.
Also you can create your own wrapper around it, like:
class ExitUtils {
static boolean enabled = true
static exit(int code) {
if (!ExitUtils.enabled) {
return //TODO set some flag?
}
System.exit(code)
}
}
and disable it for tests.

Here is the technique we eventually used.
We can't just ignore a call to System.exit() since the script would continue to run. Instead we want to throw an exception with the desired status code. We throw a (custom) ProgramExitException when System.exit() is called in our tests
class ProgramExitException extends RuntimeException {
int statusCode
public ProgramExitException(int statusCode) {
super("Exited with " + statusCode)
this.statusCode = statusCode
}
}
then we intercept System.exit() to throw this exception
/**
* Make System.exit throw ProgramExitException to fake exiting the VM
*/
System.metaClass.static.invokeMethod = { String name, args ->
if (name == 'exit')
throw new ProgramExitException(args[0])
def validMethod = System.metaClass.getStaticMetaMethod(name, args)
if (validMethod != null) {
validMethod.invoke(delegate, args)
}
else {
return System.metaClass.invokeMissingMethod(delegate, name, args)
}
}
and lastly we have GroovyShell catch any ProgramExitException and return the status code from the run method.
/**
* Catch ProgramExitException exceptions to mimic exit status codes
* without exiting the VM
*/
GroovyShell.metaClass.invokeMethod = { String name, args ->
def validMethod = GroovyShell.metaClass.getMetaMethod(name, args)
if (validMethod != null) {
try {
validMethod.invoke(delegate, args)
} catch (ProgramExitException e) {
return e.statusCode
}
}
else {
return GroovyShell.metaClass.invokeMissingMethod(delegate, name, args)
}
}
Our tests can stay looking simple, we don't need to change anything in the scripts and we get the behavior we expect from running on the command line.
assertEquals 'Unexpected status code', 0, shell.run(script,[arg1, arg2])
assertEquals 'Unexpected status code', 10, shell.run(script,[badarg1, badarg2])

Related

Mockk AssertionError: Verification failed: call 1 of 1

///Here is my class
class State {
var state: Int = 10
}
open class Car {
var state:State = State()
fun changState(data: Int = 1) {
setState(data)
}
fun setState(data: Int = 0) {
state.state = data
}
}
/// Here is my Test
#Test
fun `test 1`() {
var mockCar = mockk<Car>()
every { mockCar.changState(any()) } just runs
every { mockCar.setState(any()) } just runs
mockCar.changState(10)
verify(exactly = 1) { mockCar.changState(any()) }
verify { mockCar.setState(any()) }
}
But it fails with this error
################################
java.lang.AssertionError: Verification failed: call 1 of 1: Car(#1).setState(any())) was not called.
Calls to same mock:
Car(#1).changState(10)
############################
You need to remove verify { mockCar.setState(any()) } - there is no way that this will ever be called, because you mocked
every { mockCar.changState(any()) } just runs
This means the stubbed method will do nothing, it just runs, so to speak.
I don't recommend writing tests that only test mocks, because it will lead to a bias that the code is fine when you just use outputs of what you think is correct behavior. Instead, write a separate unit test for Car.
For your use-case a mock is not the intended thing to use, you should be using a spy instead if you mix real method calls with mocked behavior.

Unit testing suspend coroutine

a bit new to Kotlin and testing it... I am trying to test a dao object wrapper with using a suspend method which uses an awaitFirst() for an SQL return object. However, when I wrote the unit test for it, it is just stuck in a loop. And I would think it is due to the awaitFirst() is not in the same scope of the testing
Implementation:
suspend fun queryExecution(querySpec: DatabaseClient.GenericExecuteSpec): OrderDomain {
var result: Map<String, Any>?
try {
result = querySpec.fetch().first().awaitFirst()
} catch (e: Exception) {
if (e is DataAccessResourceFailureException)
throw CommunicationException(
"Cannot connect to " + DatabaseConstants.DB_NAME +
DatabaseConstants.ORDERS_TABLE + " when executing querySelect",
"querySelect",
e
)
throw InternalException("Encountered R2dbcException when executing SQL querySelect", e)
}
if (result == null)
throw ResourceNotFoundException("Resource not found in Aurora DB")
try {
return OrderDomain(result)
} catch (e: Exception) {
throw InternalException("Exception when parsing to OrderDomain entity", e)
} finally {
logger.info("querySelect;stage=end")
}
}
Unit Test:
#Test
fun `get by orderid id, null`() = runBlocking {
// Assign
Mockito.`when`(fetchSpecMock.first()).thenReturn(monoMapMock)
Mockito.`when`(monoMapMock.awaitFirst()).thenReturn(null)
// Act & Assert
val exception = assertThrows<ResourceNotFoundException> {
auroraClientWrapper.queryExecution(
databaseClient.sql("SELECT * FROM orderTable WHERE orderId=:1").bind("1", "123") orderId
)
}
assertEquals("Resource not found in Aurora DB", exception.message)
}
I noticed this issue on https://github.com/Kotlin/kotlinx.coroutines/issues/1204 but none of the work around has worked for me...
Using runBlocking within Unit Test just causes my tests to never complete. Using runBlockingTest explicitly throws an error saying "Job never completed"... Anyone has any idea? Any hack at this point?
Also I fairly understand the point of you should not be using suspend with a block because that kinda defeats the purposes of suspend since it is releasing the thread to continue later versus blocking forces the thread to wait for a result... But then how does this work?
private suspend fun queryExecution(querySpec: DatabaseClient.GenericExecuteSpec): Map {
var result: Map<String, Any>?
try {
result = withContext(Dispatchers.Default) {
querySpec.fetch().first().block()
}
return result
}
Does this mean withContext will utilize a new thread, and re-use the old thread elsewhere? Which then doesnt really optimize anything since I will still have one thread that is being blocked regardless of spawning a new context?
Found the solution.
The monoMapMock is a mock value from Mockito. Seems like the kotlinx-test coroutines can't intercept an async to return a mono. So I forced the method that I can mock, to return a real Mono value instead of a Mocked Mono. To do so, as suggested by Louis. I stop mocking it and return a real value
#Test
fun `get by orderid id, null`() = runBlocking {
// Assign
Mockito.`when`(fetchSpecMock.first()).thenReturn(Mono.empty())
Mockito.`when`(monoMapMock.awaitFirst()).thenReturn(null)
// Act & Assert
val exception = assertThrows<ResourceNotFoundException> {
auroraClientWrapper.queryExecution(
databaseClient.sql("SELECT * FROM orderTable WHERE orderId=:1").bind("1", "123") orderId
)
}
assertEquals("Resource not found in Aurora DB", exception.message)
}

how to apply in DebuggerHiddenAttribute dependent/cascade methods

I'm trying to ignore a exception that happen when run my test Methods.
I`m using a unitTest proyect.
The problem apears when
TestCleanup Method runs. It's a recursive method. This method clean all entities created in DB during the test. This is a recursive method because of dependences.
Anyway this method call to delete generic method in my ORM(Petapoco). It throws an exception if it can't delete the entity. Any problem, it run again with recursive way until delete it.
Now the problem, if i'm debugging VS stop a lot of times in Execute method because of failed deletes. But I can't modify this method to ignore it. I need a way to ignore this stops when i'm debugging tests. A way like DebuggerHiddenAttribute or similar.
Thanks!
I tried to use DebuggerHiddenAttribute, but cannot works in methods called by main method.
[TestCleanup(), DebuggerHidden]
public void CleanData()
{
ErrorDlt = new Dictionary<Guid, object>();
foreach (var entity in TestEntity.CreatedEnt)
{
try
{
CallingTest(entity);
}
catch (Exception e)
{
if (!ErrorDlt.ContainsKey(entity.Key))
ErrorDlt.Add(entity.Key, entity.Value);
}
}
if (ErrorDlt.Count > 0)
{
TestEntity.CreatedEnt = new Dictionary<Guid, object>();
ErrorDlt.ForEach(x => TestEntity.CreatedEnt.Add(x.Key, x.Value));
CleanData();
}
}
public int Execute(string sql, params object[] args)
{
try
{
OpenSharedConnection();
try
{
using (var cmd = CreateCommand(_sharedConnection, sql, args))
{
var retv = cmd.ExecuteNonQuery();
OnExecutedCommand(cmd);
return retv;
}
}
finally
{
CloseSharedConnection();
}
}
catch (Exception x)
{
OnException(x);
throw new DatabaseException(x.Message, LastSQL, LastArgs);
}
}
Error messages are not required.

Issue with a WS verifier method when migrating from Play 2.4 to Play 2.5

I have a method I need to refactor, as F.Promise has been deprecated in Play 2.5. It's pretty readable actually. It sends a request and authenticates via a custom security token and returns true if the response is 200.
public boolean verify(final String xSassToken){
WSRequest request = WS.url(mdVerifyXSassTokenURL)
.setHeader("X-SASS", xSassToken)
.setMethod("GET");
final F.Promise<WSResponse> responsePromise = request.execute();
try {
final WSResponse response = responsePromise.get(10000);
int status = response.getStatus();
if(status == 200 ) { //ok
return true;
}
} catch (Exception e) {
return false;
}
return false;
}
First thing I had to do was change this line:
final F.Promise<WSResponse> responsePromise = request.execute();
To this:
final CompletionStage<WSResponse> responsePromise = request.execute();
However, CompletionStage(T) doesn't have an equivalent get() method so I'm not sure the quickest and easiest way to get a WSResponse that I can verify the status of.
Yes, it does not. At least not directly.
What you are doing is "wrong" in the context of PlayFramework. get is a blocking call and you should avoid blocking as much as possible. That is why WS offers a non blocking API and a way to handle asynchronous results. So, first, you should probably rewrite your verify code to be async:
public CompletionStage<Boolean> verify(final String xSassToken) {
return WS.url(mdVerifyXSassTokenURL)
.setHeader("X-SASS", xSassToken)
.setMethod("GET")
.execute()
.thenApply(response -> response.getStatus() == Http.Status.OK);
}
Notice how I'm using thenApply to return a new a java.util.concurrent.CompletionStage instead of a plain boolean. That means that the code calling verify can also do the same. Per instance, an action at your controller can do something like this:
public class MyController extends Controller {
public CompletionStage<Result> action() {
return verify("whatever").thenApply(success -> {
if (success) return ok("successful request");
else return badRequest("xSassToken was not valid");
});
}
public CompletionStage<Boolean> verify(final String xSassToken) { ... }
}
This way your application will be able to handle a bigger workload without hanging.
Edit:
Since you have to maintain compatibility, this is what I would do to both evolve the design and also to keep code compatible while migrating:
/**
* #param xSassToken the token to be validated
* #return if the token is valid or not
*
* #deprecated Will be removed. Use {#link #verifyToken(String)} instead since it is non blocking.
*/
#Deprecated
public boolean verify(final String xSassToken) {
try {
return verifyToken(xSassToken).toCompletableFuture().get(10, TimeUnit.SECONDS);
} catch (Exception e) {
return false;
}
}
public CompletionStage<Boolean> verifyToken(final String xSassToken) {
return WS.url(mdVerifyXSassTokenURL)
.setHeader("X-SASS", xSassToken)
.setMethod("GET")
.execute()
.thenApply(response -> response.getStatus() == Http.Status.OK);
}
Basically, deprecate the old verify method and suggest users to migrate to new one.

Verify Spock mock with specified timeout

In Mockito there is option to verify if mock method has been called, and specify timeout for this verification (VerificationWithTimeout), for example:
verify(mock, timeout(200).atLeastOnce()).baz();
It there any equivalent to such functionality in Spock?
I was trying to use PollingConditions to satisfy a similar scenario (which wasn't helpful), but instead found satisfaction in Spock's BlockingVariables. To verify that SomeService.method() is invoked at least once in function ClassBeingTested.method() within a given timeout period:
def "verify an interaction happened at least once within 200ms"(){
given:
def result = new BlockingVariable<Boolean>(0.2) // 200ms
SomeService someServiceMock = Mock()
someServiceMock.method() >> {
result.set(true)
}
ClassBeingTested clazz = new ClassBeingTested(someService: someServiceMock)
when:
clazz.someMethod()
then:
result.get()
}
When the result is set, the blocking condition will be satisfied and result.get() would have to return true for the condition to pass. If it fails to be set within 200ms, the test will fail with a timeout exception.
There is no equivalent in Spock. (PollingConditions can only be used for conditions, not for interactions.) The closest you can get is to add a sleep() statement in the then block:
when:
...
then:
sleep(200)
(1.._) * mock.baz()
Using PollingConditions and a boolean variable, the following example evaluates a function until it satisfies an interaction.
def "test the config watcher to reload games if a new customer directory is added"() {
given:
def conditions = new PollingConditions(initialDelay: 1, timeout: 60, factor: 1.25)
def mockGameConfigLoader = Mock(GameConfigLoader)
def gameConfigWatcher= new GameConfigWatcher(mockGameConfigLoader)
boolean gamesReloaded = false
when:
conditions.eventually {
gameConfigWatcher.listenEvents()
assert gamesReloaded
}
then:
1 * mockGameConfigLoader.loadAllGameConfigs() >> {
gamesReloaded = true
}
0 * _
}
This doesn't do exactly what the question asked, but I found it a bit cleaner that using a variable. If you have other conditions to asynchronously test in addition to the interaction, then you can declare the interactions at mock creation time and then use PollingConditions to test the other conditions. PollingConditions will either fail the test or block until the conditions pass, so that by the time the interaction is tested, the method should have been called:
#MicronautTest
class KernelManagerTest extends Specification {
#Inject
KernelManager kernelManager
//create conditions
PollingConditions conditions = new PollingConditions(timeout: 10, initialDelay: 1)
class Exits {
static createKernel (String[] args) {
System.exit(args[0].toInteger())
}
}
def "handles a system exit from within kernel"() {
given:
// set custom kernel
kernelManager.kernelClass = Exits
// create custom logger
kernelManager.log = Mock(Logger) {
1 * warn("Kernel exited unexpectedly.", _ as UnexpectedExitException)
}
when:
// create new kernel (exit 1)
kernelManager.startNewKernel("1")
then:
conditions.eventually {
kernelManager.kernelInstances.size() == 0
kernelManager.kernelThreads.size() == 0
}
}
}