I'm not sure on the best name for my question, but I seem to have coded myself into a corner somewhat. Or at least I am faced with a slightly awkward design descision within my hspec test suite for my project.
My project has to make some third party API calls, and in HSpec I am trying to make it so these can be faked using a transformer implemented using StateT, that can be placed inside a test stack to enable the ability to 'fake' these API calls. I always want to stub these calls out in unit tests, I never want to hit the real API, ever.
The way I am 'stubbing' calls is by defining all of my systems effects with typeclasses and providing a different instance when in test. So for example I would have a class like:
class FireApiGetRequest m where
fireApiGetRequest :: GetRequest -> m (Either ApiError GetReponse)
instance FireApiGetRequest Hander where
fireApiGetRequest = -- real world implementation
I have two transformer stacks for my tests currently one is known as AppTestIO, and AppTestPureM.
AppTestIO is made up of a ReaderT that contains a connection pool to a real test database and some test configuration settings, and AppTestPureM is a type alias for StateT FakeDatabaseCalls Gen, much in the same way as I write out the API calls with typeclasses, database calls are also factored out into type classes and 'stubbed' out in the same way. This works well, only I would like to add a transformer layer to both stacks, that mocks the third party API. In my mind I should be able to define a transformer that can sit within both of these transformer stacks and give me the ability to fake these API calls:
type AppTestPureM = StateT FakedDatabaseCalls (ThirdPartyApiMocksT Gen)
type AppTestIO = ThirdPartyApiMocksT (AppTestT IO)
newtype AppTestT m a = AppTestT
{ unAppTestT :: ReaderT TestEnv m a
}
On the face of it this seems like a great idea because it means I can stub out these API calls regardless of whether or not the test is hitting the real database, basically.
For the instances, I have no issue defining instances for the API calls in test:
instance FireApiGetRequest ThirdPartyApiMocksT where
fireApiGetRequest = -- test implementation, access state and return value based on that
instance FireApiGetRequest m => FireApiGetRequest (StateT s m) where
fireApiGetRequest = lift . fireApiGetRequest
And I can define helper functions in my test to get the instances to return the fake data that I want:
stubApiGetRequest :: Monad m => (Either ApiError GetResponse) -> ThirdPartyApiMocksT m ()
stubApiGetRequest returnVal = undefined -- store `returnVal` in the state for use in typeclass instance
My issue arises when I actually start using these instances within my tests. For those functions in my app that hit the database (that aren't yet stubbed out with typeclass instances), they ultimately are using runSqlPool, which uses MonadUnliftIO, and MonadUnliftIO is effectively forbidding me from mixing these together.
The approach to achieve 'state' while using MonadUnliftIO is to use ReaderT + MVar instead. I don't have a huge issue with this normally, my only problem here is that my PureM stack is based around StateT, and also does not run in IO, because it is used to run 'effectful' computations using fake data in order to write the test in a pure way, with no IO. This also means I can use quickcheck to test these functions if that is something I want to do. I am aware there is quickcheck-monadic so having these functions result in IO may not be the end of the world, but I would like retain AppTestPureM a -> Gen a. Using ReaderT + MVar here in this situation would make it so that these tests then depend on IO. So these two test stacks are contradictory.
I suppose that illustrates the situation I am in now. I am not sure how exactly to proceed from here.
Related
I have a simple Publish-Subscriber that I want to write tests for.
The methods called here are all behaviours, except get_number_consumed_messages that would be a function.
class iso _SinglePubSub is UnitTest
fun name(): String => "single publish/consume"
fun apply(h: TestHelper) =>
let p = Publisher("publisher message", h.env.out)
let queue = Queue(1, h.env.out)
let c = Consumer(h.env.out)
p.publish_message(queue)
p.publish_message(queue)
c.consume_message(queue)
c.consume_message(queue)
//Run after all behaviours are done
let n = c.get_number_consumed_messages()
h.assert_eq[USize](2, n)
How would someone implement the get_number_consumed_messages function/behaviour or how would you have to modify the test function?
First of all, c.get_number_consumed_messages() must be a behaviour as well. It is the only way to let one actor communicate with another. This has the added benefit of behaviours being run in the same order as they are called, which means c.get_number_consumed_messages() would run after both calls to c.consume_message(queue).
Given that, since Consumer is also an actor, calling it with behaviours -- and not methods -- means that we cannot return data from it directly. To actually receive data from another actor, you should use the Promise pattern, for example:
use "promises"
actor Consumer
var message_count: USize = 0
be consume_message(queue: OutStream) =>
... // Do some things
message_count = message_count + 1
... // Do other things
be get_number_consumed_messages(p: Promise[USize]) =>
p(message_count)
To actually test it, you would need to follow an adapted version of the Testing Notifier Interactions pattern for long tests, for example:
use "ponytest"
use "promises"
class iso _SinglePubSub is UnitTest
fun apply(h: TestHelper) =>
h.long_test(2_000_000_000)
... // Produce and consume messages
let p = Promise[USize]
p.next[None]({(n: USize): None =>
h.assert_eq[USize](2, n)
h.complete(true) })
c.get_number_consumed_messages(p)
(Notice the extra calls to h.long_test and h.complete, as well as the promise wrapping a lambda with the end of our test.)
For more information on these concepts, I would recommend familiarizing yourself with the stdlib documentation on Promises and the "Long tests" section of Ponytest.
Using Ninject, I have the following and wish to test using FluentAssertions:
[Test]
public void InterfacesEndingWithFactoryShouldBeBoundAsFactories() {
// Given
IKernel kernel = new StandardKernel();
kernel.Bind(services => services
.From(AppDomain.CurrentDomain
.GetAssemblies()
.Where(a => !a.FullName.Contains("Tests")))
.SelectAllInterfaces()
.EndingWith("Factory")
.BindToFactory()
);
// When
var factory = kernel.Get<ICustomerManagementPresenterFactory>();
// Then
factory.Should().NotBeNull();
}
Is there any good ways to test whether the factories are actually bound properly?
I wrote an extension package for Fluent Assertions to test my Ninject bindings. Using it your test could be rewritten like this:
[Test]
public void InterfacesEndingWithFactoryShouldBeBoundAsFactories() {
// Given
IKernel kernel = new StandardKernel();
kernel.Bind(services => services
.From(AppDomain.CurrentDomain
.GetAssemblies()
.Where(a => !a.FullName.Contains("Tests")))
.SelectAllInterfaces()
.EndingWith("Factory")
.BindToFactory()
);
// When
// Then
kernel.Should().Resolve<ICustomerManagementPresenterFactory>().WithSingleInstance()
}
As suggested by #Will Marcouiller, I would also extract the code to bootstrap the kernel into it's own class so it can be invoked in your app's composition root and in your unit tests.
First off, there is no ninject infrastructure to test which types are affected and which not. Unit testing is very complicated due to the fluent syntax and it's many interfaces which are returned when calling a method. So all you can practically do is integration testing.
Now, in my opinion, the From(...) is problematic. If you'd create a component where you could replace the assemblies you pass to From , create an extra test assembly containing a few types, and then test whether lets say interface IFoo is not bound, interface IFooFactory is bound, class Foo is not bound,.. you would have a functioning integration test.
Consider however, that if there is no binding for IFooFactory and SomeClass uses IFooFactory as constructor argument, ninject will throw an exception. Now if you have a composition root simply starting up the application will tell you whether the necessary bindings exist or not.
This test is even more useful than the factory convention integration test. Consider if someone accidentally bound the factory, manually, too. This won't show up with the convention-integration test, but starting the application, ninject will throw an exception stating that there are multiple bindings for this one interface.
Sorry for the long post...
While being introduced to a brown field project, I'm having doubts regarding certain sets of unit tests and what to think. Say you had a repostory class, wrapping a stored procedure and in the developer guide book, a certain set guidelines (rules), describe how this class should be constructured. The class could look like the following:
public class PersonRepository
{
public PersonCollection FindPersonsByNameAndCity(string personName, string cityName)
{
using (new SomeProfiler("someKey"))
{
var sp = Ioc.Resolve<IPersonStoredProcedure>();
sp.addNameArguement(personName);
sp.addCityArguement(cityName);
return sp.invoke();
}
} }
Now, I would of course write some integration tests, testing that the SP can be invoked, and that the behavior is as expected. However, would I write unit tests that assert that:
Constructor for SomeProfiler with the input parameter "someKey" is called
The Constructor of PersonStoredProcedure is called
The addNameArgument method on the stored procedure is called with parameter personName
The addCityArgument method on the stored procedure is called with parameter cityName
The invoke method is called on the stored procedure -
If so, I would potentially be testing the whole structure of a method, besides the behavior. My initial thought is that it is overkill. However, in regards to the coding practices enforced by the team, these test ensure a uniform and 'correct' structure and that the next layer is called correctly (from DAL to DB, BLL to DAL etc).
In my case these type of tests, are performed for each layer of the application.
Follow up question - the use of the SomeProfiler class smells a little like a convention to me - Instead creating explicit tests for this, could one create convention styled test by using static code analysis or unittest + reflection?
Thanks in advance.
I think that your initial thought was right - this is an overkill. Although you can use reflection to make sure that the class has the methods you expect I'm not sure you want to test it that way.
Perhaps instead of unit testing you should use some tool such as FxCop/StyleCop or nDepend to make sure all of the classes in a specific assembly/dll has these properties.
Having said that I'm a believer of "only code what you need" why test that a method exist, either you use it somewhere in your code and in that can you can test the specific case or you don't - and so it's irrelevant.
Unit tests should focus on behavior, not implementation. So writing a test to verify that certain arguments are set or passed in doesn't add much value to your testing strategy.
As the example provided appears to be communicating with your database, it can't truly be considered a "unit test" as it must communicate with physical dependencies that have additional setup and preconditions, such as availability of the environment, database schema, existing data, stored-procedures, etc. Any test you write is actually verifying these preconditions as well.
In it's present condition, your best bet for these types of tests is to test the behavior provided by the class -- invoke a method on your repository and then validate that the results are what you expected. However, you'll suddenly realize that there's a hidden cost here -- the database maintains state between test runs, and you'll need additional setup or tear-down logic to ensure that the database is in a well-known state.
While I realize the intent of the question was about the testing a "black box", it seems obvious that there's some hidden magic here in your API. My preference to solve the well-known state problem is to use an in-memory database that is scoped to the current test, which isolates me from environment considerations and enables me to parallelize my integration tests. I'd wager that under the current design, there is no "seam" to programmatically introduce a database configuration so you're "hemmed in". In my experience, magic hurts.
However, a slight change to the existing design solves this problem and the "magic" goes away:
public class PersonRepository : IPersonRepository
{
private ConnectionManager _mgr;
public PersonRepository(ConnectionManager mgr)
{
_mgr = mgr;
}
public PersonCollection FindPersonsByNameAndCity(string personName, string cityName)
{
using (var p = _mgr.CreateProfiler("somekey"))
{
var sp = new PersonStoredProcedure(p);
sp.addArguement("name", personName);
sp.addArguement("city", cityName);
return sp.invoke();
}
}
}
I have this simple method which calls the TFS (Team foundation server) API to get WorkItemCollection object. I have just converted in to an entity class and also added it in cache. As you can see this is very simple.
How should i unit test this method. Only the important bit it does is calls TFS API. Is it worth testing such methods? If yes then how should we test it?
One way I can think is I can mock call to Query.QueryWorkItemStore(query) and return an object of type “WorkItemCollection” and see finally this method converts “WorkItemCollection” to List. And check if it was added to cache or not.
Also as I am using dependency injection pattern her so I am injecting dependency for
cache
Query
Should I only pass dependency of mocked type (Using MOQ) or I should pass proper class type.
public virtual List<Sprint> Sprint(string query)
{
List<Sprint> sprints =
Cache.Get<List<Sprint>>(query);
if (sprints == null)
{
WorkItemCollection items =
Query.QueryWorkItemStore(query);
sprints = new List<Sprint>();
foreach (WorkItem i in items)
{
Sprint sprint = new Sprint
{
ID = i.Id,
IterationPath = i.IterationPath,
AreaPath = i.AreaPath,
Title = i.Title,
State = i.State,
Goal = i.Description,
};
sprints.Add(sprint);
}
Cache.Add(sprints, query,
this.CacheExpiryInterval);
}
return sprints;
}
Should I only pass dependency of mocked type (Using MOQ) or I should pass proper class type.
In your unit tests, you should pass a mock. There are several reasons:
A mock is transparent: it allows you to check that the code under test did the right thing with the mock.
A mock gives you full control, allowing you to test scenarios that are difficult or impossible to create with the real server (e.g. throw IOException)
A mock is predictable. A real server is not - it may not even be available when you run your tests.
Things you do on a mock don't influence the outside world. You don't want to change data or crash the server by running your tests.
A test with mocks is faster. No connection to the server or real database queries have to be made.
That being said, automated integration tests which include a real server are also very useful. You just have to keep in mind that they will have lower code coverage, will be more fragile, and will be more expensive to create/run/maintain. Keep your unit tests and your integration tests separate.
edit: some collaborator objects like your Cache object may also be very unit-test friendly. If they have the same advantages as that of a mock that I list above, then you don't need to create a mock. For example, you typically don't need to mock a collection.
I've created a unit test that tests interactions on my ViewModel class in a Silverlight application. To be able to do this test, I'm mocking the service interface, injected to the ViewModel. I'm using Moq framework to do the mocking.
to be able to verify bounded object in the ViewModel is converted properly, I've used a callback:
[Test]
public void SaveProposal_Will_Map_Proposal_To_WebService_Parameter()
{
var vm = CreateNewCampaignViewModel();
var proposal = CreateNewProposal(1, "New Proposal");
Services.Setup(x => x.SaveProposalAsync(It.IsAny<saveProposalParam>())).Callback((saveProposalParam p) =>
{
Assert.That(p.plainProposal, Is.Not.Null);
Assert.That(p.plainProposal.POrderItem.orderItemId, Is.EqualTo(1));
Assert.That(p.plainProposal.POrderItem.orderName, Is.EqualTo("New Proposal"));
});
proposal.State = ObjectStates.Added;
vm.CurrentProposal = proposal;
vm.Save();
}
It is working fine, but if you've noticed, using this mechanism the Assert and Act part of the unit test have switched their parts (Assert comes before Acting). Is there a better way to do this, while preserving correct AAA order?
I'm not sure that you've changed the semantics of the AAA order. Consider the execution of the test. Your mocked interface will not be called until the Action invokes it. Therefore, during execution, your program still follows the Arrange, Act, and Assert flow.
The alternative would be to use Data Injection and create an interface between your CampaignViewModel and the web service that it uses. You can then create a class in your UnitTests that saves your parameter information and Assert on that class member/property rather than use Moq to create a proxy on the fly.
Moq should not be used to simulate storage or assignment. Rather, use Moq to provide dummy mechanisms and values to allow your Unit Tests to execute. If Asserting storage is a requirement, then take the time to create a class that will hold on to your values.