So, I have adde Flyway to my application to run integration tests with an embedded H2. I have also added a data script to db/migration/afterMigrate.sql. Everything worked fine with Spring Boot autoconfiguration up until the point that I created a second test class with #SpringBootTest. This class fails execution because Flyway tries to execute afterMigrate.sql again. I can't seem to prevent it from running, I have added the flyway-spring-test library to the project and tried using it but with no luck.
The thing I find odd is that the db migration isn't executed again for the second class, just the afterMigrate.sql script.
This was caused by a #SpyBean in the second class - the Spring context of the first one was not being reused. The workaround for this is creating an abstract class that both test classes extend from, and declaring the beans in it. This is expected behaviour from Spring but has been debated whether it should be or not - see more at https://github.com/spring-projects/spring-boot/issues/7174
Related
I created some integration tests for ASP.NET using the WebApplicationFactory class from the Microsoft.AspNetCore.Mvc.Testing package. The tests classes are running in parallel per the default XUnit behaviour. After I changed over 2 test classes to use WebApplicationFactory they ran fine and were quicker than the previous method we were using. However when I retrofitted all the test classes and tried it, the performance completely tanked and appeared to be hung in some kind of lock.
I noticed that call the factory.CreateClient() or factory.CreateDefaultClient() are very slow. Is there any option to setup a shared instance of the client and have all the tests use it? Not sure if that's possible or if it would be thread safe in a parallel situation.
We started to use jhipster for our Microservices, when migrating the old code an issue arises, that our unit tests do not start anymore cause of missing or duplicate beans.
Searching for an answer led me to theses similar questions:
Cannot run Unit Tests against REST layer
#ComponentScan in application class breaks #WebMvcTest and #SpringBootTest
As soon as I add the exclusions to the ComponentScan-Annotation of the generated application class the WebMvcTest run successfully.
Are there any reasons why these exclusions are missing in jhipster-generated application class?
Basically I created a new test file in a particular package with some bare bones test structure - no actual tests...just an empty struct type that embeds suite.Suite, and a function that takes in a *testing.T object and calls suite.Run() on said struct. This immediately caused all our other tests to start failing indeterministically.
The nature of the failures were associated with database unique key integrity violations on inserts and deletes into a single Postgres DB. This is leading me to believe that the tests were being run concurrently without calling our setup methods to prepare the environment properly between tests.
Needless to say, the moment I move this test file to another package, everything magically works!
Has anyone else run into this problem before and can possibly provide some insights?
What I've found from my use, is that "go test" runs a single package's test cases sequentially (unless t.Parallel() is called), but if you supply multiple packages (go test ./foo ./bar ./baz), each package's tests are run parallel to other packages. Definitely caused similar headaches with database testing for me.
As it turns out, this is a problem rooted in how go test works, and has nothing to do with testify. Our tests were being ran on ./... This causes the underlining go test tool to run tests in each package in parallel, as justinas pointed out. After digging around more on StackOverflow (here and here) and reading through testify's active issue on this problem, it seems that the best immediate solution is to use the -p=1 flag to limit the number of packages to be run in parallel.
However, it is still unexplained why the tests consistently passed prior to adding these new packages. A hunch is perhaps the packages/test files were sorted and ran in such a manner that concurrency wasn't an issue prior to adding the new packages/files.
I've got some unit tests (c++) running in the Visual Studio 2012 test framework.
From what I can tell, the tests are running in parallel. In this case the tests are stepping on each other - I do not want to run them in parallel!
For example, I have two tests in which I have added breakpoints and they are hit in the following order:
Test1 TEST_CLASS_INITIALIZE
Test2 TEST_CLASS_INITIALIZE
Test2 TEST_METHOD
Test1 TEST_METHOD
If the init for Test1 runs first then all of its test methods should run to completion before anything related to Test2 is launched!
After doing some internet searches I am sufficiently confused. Everything I am reading says Visual Studio 2012 does not run tests concurrently by default, and you have to jump through hoops to enable it. We certainly have not enabled it in our project.
Any ideas on what could be happening? Am I missing something fundamental here?
Am I missing something fundamental here?
Yes.
Your should never assume that another test case will work as expected. This means that it should never be a concern if the tests execute synchronously or asynchronously.
Of course there are test cases that expect some fundamental part code to work, this might be own code or a part of the framework/library you work with. When it comes to this, the programmer should know what data or object to expect as a result.
This is where Mock Objects come into play. Mock objects allow you to mimic a part of code and assure that the object provides exactly what you expect, so you don't rely on other (time consuming) services, such as HTTP requests, file stream etc.
You can read more here.
When project becomes complex, the setup takes a fair number of lines and code starts duplicating. Solution to this are Setup and TearDown methods. The naming convention differs from framework to framework, Setup might be called beforeEach or TestInitialize and TearDown can also appear as afterEach or TestCleanup. Names for NUnit, MSTest and xUnit.net can be found on xUnit.net codeplex page.
A simple example application:
it should read a config file
it should verify if config file is valid
it should update user's config
The way I would go about building and testing this:
have a method to read config and second one to verify it
have a getter/setter for user's settings
test read method if it returns desired result (object, string or however you've designed it)
create mock config which you're expecting from read method and test if method accepts it
at this point, you should create multiple mock configs, which test all possible scenarios to see if it works for all possible scenarios and fix it accordingly. This is also called code coverage.
create mock object of accepted config and use the setter to update user's config, then use to check if it was set correctly
This is a basic principle of Test-Driven Development (TDD).
If the test suite is set up as described and all tests pass, all these parts, connected together, should work perfectly. Additional test, for example End-to-End (E2E) testing isn't necessarily needed, I use them only to assure that whole application flow works and to easily catch the error (e.g. http connection error).
I'm using ShrinkWrap to start Jetty server in my integration tests.
Problem:
When I start my test jetty-server and than make mockup of my controller - mockup doesn't work!
I suggest that the reason is different classloaders: JMockit - AppClassLoader, Jetty - WebAppClassLoader.
Question:
How to make mocking works fine?
P.S.
I've googled that -javaagent:jmockit.jar option may help. But it doesn't. Is it necessary for maven project based on 1.7 jdk?
ADDITION:
I've written demo to illustrate my problem. You can find it by the reference.
About my demo:
Except of ten stokes of code, it is identical to those project.
I've only added JMockit and a single mock to illustrate the problem.
You should see JettyDeploymentIntegrationUnitTestCase.requestWebapp method: in those method we make mock which doesn't work.
You can check that Jetty & JMockit loads classes by siblings classloaders, so JMockit simply doesn't see Jetty's classes
URLClassLoader
|
|-Launcher$AppClassLoader
|-WebAppClassLoader
The JUnit test in the example project is attempting to mock the ForwardingServlet class. But, in this scenario with an embedded Jetty web server, there are actually two instances of this class, both loaded in the same JVM but through different classloaders.
The first instance of the class is loaded by the regular classloader, through which classes are loaded from the thread that starts the JUnit test runner (AppClassLoader). So, when ForwardingServlet appears in test code, it is the one defined in this classloader. This is the class given to JMockit to mock, which is exactly what happens.
But then, a copy of ForwardingServlet is loaded inside the deployed web app (from the ".class" file in the file system, so not affected by the mocking as applied by JMockit, which is in-memory only), using Jetty's WebAppClassLoader. This class is never seen by JMockit.
There are two possible solutions to this issue:
Somehow get the class object loaded by WebAppClassLoader and then mock it by calling the MockUp(Class) constructor.
Configure the Jetty server so that it does not use a custom classloader for the classes in the web app.
The second solution is the easiest, and can be done simply by adding the following call on the ContextHandler object created from the WebArchive object, before setting the handler into the Jetty Server object:
handler.setClassLoader(ClassLoader.getSystemClassLoader());
I tested this and it worked as expected, with the #Mock doGet(...) method getting executed instead of the real one in ForwardingServlet.