sbt doesn't recognize ScalaTest Table-driven property checks as tests - unit-testing

I wrote some ScalaTest Table-driven property checks and I'm trying to run them with sbt test. Looking at the report I see that ScalaTest can recognize all the JUnit tests I have (they are in the same class as the checks), it runs property checks (i.e., forAll body), but it doesn't treat forAll as a test. If it fails I see the stack trace in the report (with ScalaTest failed test exception) and sbt says there was an "error" during the test run, but it says that all of the tests passed. The total number of tests in the report includes only JUnit tests.
Is there the support for this style of tests in sbt?

forAll in PropertyChecks is not a test. It is essentially a glorified assertion. You need to place assertions inside named tests. How to do that depends on your chosen style. For example, in FunSuite, you'd write something like:
class MySpec extends FunSuite with PropertyChecks {
test("give the test a name here") {
forAll(x: Int, y: Int) {
// make assertions here
}
}
}

Instead of calling forAll, make the test class extend from org.scalatest.prop.Checkers and then in each test, call check with the Property to be tested. In this case, "Property" probably means the forAll that you've created.
So I'm going to guess that currently you have a test class that looks like:
class ExampleSuite extends AssertionsForJUnit {
val fractions = Table(
("n", "d"),
( 1, 2),
///...
)
forAll (fractions) { (n: Int, d: Int) => // ...
#Test def verifySomethingElse = ???
}
I believe what you need to do is extend from Checkers and move your forAll into a test.
class ExampleSuite extends AssertionsForJUnit with org.scalatest.prop.Checkers {
#Test def verifyFractions = {
val fractions = Table(
("n", "d"),
( 1, 2),
///...
)
check(forAll (fractions) { (n: Int, d: Int) => ???)
}
#Test def verifySomethingElse = ???
}

The standard way is to create a FunSuite test with Matchers and TableDrivenPropertyCheck
Example:
import org.scalatest._
import org.scalatest.prop.TableDrivenPropertyChecks._
class CreateSpec extends FunSuite with Matchers {
test("B-Tree-Create for different degree parameter value") {
val params = Table(("degree", "result"),
(0, Tree(Leaf(), 0)),
(2, Tree(Leaf(), 1)),
(1999, Tree(Leaf(), 1999)))
forAll(params) {(degree, result) => Algorithms.create(degree) == result}
}
}

Related

Spock interaction verification ignoring call to Mock method

The following Spock test is failing to not counting the call to the Mock method:
def setup() {
mojo = new PactCreateVersionTagMojo()
mojo.pactBrokerUrl = 'http://broker:1234'
mojo.pacticipant = 'test'
mojo.pacticipantVersion = '1234'
mojo.tag = 'testTag'
}
def 'calls pact broker client with mandatory arguments'() {
given:
mojo.brokerClient = Mock(PactBrokerClient)
when:
mojo.execute()
then:
notThrown(MojoExecutionException)
1 * mojo.brokerClient.createVersionTag(
'test', '1234', 'testTag')
}
You can find it here.
The SUT code, removing the argument validation code, is:
class PactCreateVersionTagMojo : PactBaseMojo() {
override fun execute() {
...
createVersionTag()
}
private fun createVersionTag() =
brokerClient!!.createVersionTag(pacticipant!!, pacticipantVersion.orEmpty(), tag.orEmpty())
You can find it here.
The error is as follows:
I have a very similar example on the same project that passes just fine:
def 'passes optional parameters to the pact broker client'() {
given:
mojo.latest = 'true'
mojo.to = 'prod'
mojo.brokerClient = Mock(PactBrokerClient)
when:
mojo.execute()
then:
notThrown(MojoExecutionException)
1 * mojo.brokerClient.canIDeploy('test', '1234',
new Latest.UseLatest(true), 'prod') >> new CanIDeployResult(true, '', '')
}
override fun execute() {
...
val result = brokerClient!!.canIDeploy(pacticipant!!, pacticipantVersion.orEmpty(), latest, to)
}
You can find the test above here and the SUT here.
I have investigated the call that happens during the test, and it seems as expected.
Additionally, I try to create the verification with wildcard argument constraints, but it still didn't work.
It seems to me that I have misconfigured my test, but I can't spot the difference between the test that passes and my failing test.
Your fun createVersionTag(..) looks like this:
fun createVersionTag(
pacticipant: String,
pacticipantVersion: String,
tag: String) {
}
I do not speak Kotlin, but I think you ought to open the method because otherwise it is final, which means it cannot be overridden by a subclass and thus not be mocked or stubbed by conventional means. This is also the difference to open fun canIDeploy(..).

IllegalArgumentException while running contract unit test in Kotlin (Corda)

When I try to run my Junit tests (Wrote in Kotlin) I get the following exception :
java.lang.IllegalArgumentException: Attempted to find dependent attachment for class javax/xml/bind/DatatypeConverter, but could not find a suitable candidate.
I tried to comment/decomment some lines in following code, it seems that the exception occurs when I call command()
class IRIssueTests {
class DummyCommand : TypeOnlyCommandData()
private val ledgerServices = MockServices(listOf("com.my.package.name"))
private val ALICE = TestIdentity(CordaX500Name(organisation = "Alice", locality = "TestLand", country = "US"))
#Test
fun mustIncludeIssueCommand() {
val ir = IRState(
UniqueIdentifier(),
mutableListOf(ALICE.party)
)
ledgerServices.ledger {
transaction {
output(IRContract.ID, ir)
command(listOf(ALICE.publicKey), DummyCommand())
fails()
}
transaction {
output(IRContract.ID, ir)
command(listOf(ALICE.publicKey), IRContract.Commands.Issue())
verifies()
}
}
}
}
I would like to understand why I'm getting this exception and how to resolve it to make my test passing
Kotlin tests must be run with JDK8.
Change configuration of JDK8 in your project before running tests.
This will avoid this exception

Groovy Spock File test

I'm about to start learning spock and I'm trying some basic stuff.
I want to check File's: exist() and getText() functionality
So I wrote the following test:
class MyTestSpec extends Specification {
def "test file"() {
given:
def mockFile = Mock(File,constructorArgs :["./doesNotExist.txt"])
mockFile.exists() >> true
mockFile.getText() >> "sampleText"
when:
def res = ""
if(mockFile.exists()) {
res = mockFile.getText()
}
then:
"sampleText" == res
1 * mockFile.exists()
1 * mockFile.getText()
}
}
This fails on:
Too few invocations for:
1 * mockFile.getText() (0 invocations)
Unmatched invocations (ordered by similarity):
None
When I comment the 'verifications' in 'then' block, I get:
java.lang.NullPointerException at
java.io.FileInputStream.(FileInputStream.java:138) at
groovy.util.CharsetToolkit.(CharsetToolkit.java:69) at
MyTestSpec.Test Existing Resource(MyTestSpec.groovy:83)
So my question is: How Exactly I can organize my test? Why does it assume that getText should not be called?
I'm using groovy 2.4 and spock 1.0
The solution will be:
#Grab('org.spockframework:spock-core:0.7-groovy-2.0')
#Grab('cglib:cglib:3.1')
#Grab('org.ow2.asm:asm-all:5.0.3')
import spock.lang.*
class MyTestSpec extends Specification {
def "test file"() {
given:
def mockFile = GroovyMock(File, constructorArgs :["./doesNotExist.txt"])
when:
def res = ""
if(mockFile.exists()) {
res = mockFile.getText()
}
then:
"sampleText" == res
1 * mockFile.exists() >> true
1 * mockFile.getText() >> "sampleText"
}
}
On of the problems is creating a mock. Because of a dynamic nature of groovy some functionality - e.g. getText() method for File class - is added at runtime. It requires mocks to be constructed in a different way. Have a look at spock mock implementation enum and extract:
An implementation specifically targeting Groovy callers. Supports mocking of dynamic methods, constructors, static methods, and "magic" mocking of all objects of a particular type.
The second problem is defining mock behavior and verifying the interactions. When you both mock and stub it must happen in the same interaction (here, in then block), here is he relevant part of the docs.

Code Contracts throws MethodAccess Exception when unit testing

I'm not quite sure why, but when I enable runtime contract checking, I'm getting a MethodAccessException during unit testing. I use the Machine.Specifications test framework and the ReSharper/dotCover test runner. When I test my assembly containing code contracts, I get this MethodAccessException:
Machine.Specifications.SpecificationException Should be of type System.ArgumentNullException but is of type System.MethodAccessException
at BusinessLogic.Specifications.When_testing_whether_a_set_of_cart_items_contains_a_product_and_a_null_set_is_passed.<.ctor>b__3()
in CartItemQuerySpecs.cs: line 66
The specification looks like this:
[Subject(typeof(ShoppingCartQueries), "Cart contains product")]
public class When_testing_whether_a_set_of_cart_items_contains_a_product_and_a_null_set_is_passed
: with_fake_shopping_cart_repository
{
Establish context = () => QueryResult = CartItems.AsQueryable().ForUser(UserWithNoProductsInCart);
Because of = () => Thrown = Catch.Exception(() => result = QueryResult.ContainsProduct(100));
It should_throw = () => Thrown.ShouldBeOfType<ArgumentNullException>();
static bool result;
}
The unit under test looks like this:
[Pure]
public static bool ContainsProduct(this IQueryable<CartItem> items, int id)
{
Contract.Requires<ArgumentNullException>(items != null);
return (items.Any(item => item.ProductId.Equals(id)));
}
Why would I get a MethodAccessException?

Second Use of GrailsApplication Mock in Service Test Fails

I'm unit testing a Grails service and using Mocks to mock out calls to the
GrailsApplication class. I have one test that succeeds but when I try
subsequent tests they fail. I am using demand to mock the isDomainClass
method. I have tried copying and pasting the code from the test that
succeeds to the test method that fails but the second time the same code
runs it fails saying that no more calls to isDomainClass are expected. I'm
suspecting some leakage between the methods but I can't see where it is.
Things I've tried already:
Running the tests from the command line (I'm running the tests under SpringSource Tool Suite version 2.7.0.201105292341-M2.)
Moving the failing test to a different test class (the test that runs first succeeds)
Changing the number range in the demands clause to 1..5 (second test still fails)
Here is the relevant portions of my test case:
package simulation
import grails.test.*
import org.joda.time.*
import org.codehaus.groovy.grails.commons.GrailsApplication
class ObjectSerializationServiceTests extends GrailsUnitTestCase {
def objectSerializationService
protected void setUp() {
super.setUp()
objectSerializationService = new ObjectSerializationService()
}
protected void tearDown() {
super.tearDown()
objectSerializationService = null
}
void testDomainObjectSerialization() {
def otherControl = mockFor(GrailsApplication)
otherControl.demand.isDomainClass(1..1) {true}
otherControl.demand.getDomainClass(1..1) {className ->
assert className == "simulation.TestDomainClass"
TestDomainClass.class
}
objectSerializationService.grailsApplication = otherControl.createMock()
def now = new DateTime()
def testObject = new TestDomainClass([id:57, someOtherData:"Some Other
Data", theTime:now])
def testInstances = [testObject]
mockDomain(TestDomainClass, testInstances)
def serialized = objectSerializationService.serializeObject(testObject)
def deserialized =
objectSerializationService.deserializeObject(serialized)
assert deserialized == testObject
assert serialized.objectType == SerializedObject.ObjectType.DOMAIN
otherControl.verify()
}
void testSerializableSerialization() {
def otherControl = mockFor(GrailsApplication)
otherControl.demand.isDomainClass(1..1) {true}
otherControl.demand.getDomainClass(1..1) {className ->
assert className == "simulation.TestDomainClass"
TestDomainClass.class
}
objectSerializationService.grailsApplication = otherControl.createMock()
def now = new DateTime()
def testObject = new TestDomainClass([id:57, someOtherData:"Some Other
Data", theTime:now])
def testInstances = [testObject]
mockDomain(TestDomainClass, testInstances)
def serialized = objectSerializationService.serializeObject(testObject)
def deserialized =
objectSerializationService.deserializeObject(serialized)
assert deserialized == testObject
assert serialized.objectType == SerializedObject.ObjectType.DOMAIN
otherControl.verify()
}
}
And the output:
Testcase: testDomainObjectSerialization took 0.943 sec
Testcase: testSerializableSerialization took 0.072 sec
FAILED
junit.framework.AssertionFailedError: No more calls to 'isDomainClass'
expected at this point. End of demands.
at grails.test.MockClosureProxy.doBeforeCall(MockClosureProxy.java:66)
at grails.test.AbstractClosureProxy.call(AbstractClosureProxy.java:74)
at
simulation.ObjectSerializationService.serializeObject(ObjectSerializationService.groovy:20)
at simulation.ObjectSerializationService$serializeObject.call(Unknown
Source)
at
simulation.ObjectSerializationServiceTests.testSerializableSerialization(ObjectSerializationServiceTests.groovy:68)
I got a similar error trying to use mockFor on jms Message interface in multiple test cases.
I got around it by creating a custom interface that extends from the interface that needs to be mocked. You would use the custom interface to create the mock.
e.g.
private interface GrailsApplicationTest1 extends GrailsApplication(){}
testOne(){
def control = mockFor(GrailsApplicationTest1)
//...rest of code
}
private interface GrailsApplicationTest2 extends GrailsApplication(){}
testTwo(){
def control = mockFor(GrailsApplicationTest2)
//...rest of code
}
//add more private interfaces for additional test cases..
I'm not exactly sure why but I think the mockFor behaves differently between interfaces and non-interfaces. But that's just a wild guess.