Has anyone had any lucking creating their custom test class and using the IProvideDynamicTestMethods interface? I have a case where I need to dynamically generate test methods and this seems to be what I need. I goal is to have test methods generated based on some files I am testing.
Jeff Wilcox mentions this as a new feature in SL3 ( http://www.jeff.wilcox.name/2008/09/rc0-new-test-features/ , see the dynamic test methods section), but I was unable to find any examples of this.
I don't know how to register my custom test class (it inherits from ITestClass). I looked at the SL4 unit testing source to see how the test class are discovered and I found the following in UnitTestFrameworkAssembly.cs source link
/// <summary>
/// Reflect and retrieve the test class metadata wrappers for
/// the test assembly.
/// </summary>
/// <returns>Returns a collection of test class metadata
/// interface objects.</returns>
public ICollection<ITestClass> GetTestClasses()
{
ICollection<Type> classes = ReflectionUtility.GetTypesWithAttribute(_assembly, ProviderAttributes.TestClass);
List<ITestClass> tests = new List<ITestClass>(classes.Count);
foreach (Type type in classes)
{
tests.Add(new TestClass(this, type));
}
return tests;
}
It looks like it will always use the built-in TestClass.
Am I missing something? I don't how to get the test framework to use my custom TestClass
Any pointers appreciated.
Thanks
It looks like the only way to set this up right now is to write your own UnitTestProvider for the test runner. The good thing is the code is available, so it's not too hard to do this.
You can either grab the code from the default Vstt from codeplex here and make any changes you you like to it. I'm still experimenting with this right now, so lets see how this goes. I got this idea by looking at this github project.
The key was to plug in your own provider, which was done in Raven.Tests.Silverlight.UnitTestProvider.Example/App.xaml.cs.
Hope this helps someone
Related
I inherited a solution with 4 projects: Front-End Project, Business Project, Data Project and a Test Project.
The test project is quite... let's say empty... and now after I changed a few things on some searches methods of the business class I would like to generate some tests to validate the changes i've made.
So my question is: Is there a automatic way to generate a "empty frame test class" to test my actual code? Something like "right click the class you want to test and click generate test class and choose the project where it will be created" maybe?!?
Details:
I'm using VS 2012 Ultimate
There's no tests for the class I'm working on
There is built in functionality that allows you to create unit test classes. I am not sure if that also works in combination with NUnit though.
Anyway, I never used it. What I do is:
add a test class to the test project
decorate the class with the [TestFixture] attribute
write the first method of what I want to test
decorate the method with the [Test] attribute
write the test
and start the TDD cycle.
A typical test class skeleton will look like this
using NUnit.Framework;
namespace Tests.Framework
{
[TestFixture]
public class SomeClassTests
{
[Test]
public void AMeaningfulTestMethodName()
{
// the test
}
}
}
I also have Resharper at my aid so that I can run the test from visual studio straight away.
Since it's so little effort for me to add a new test fixture to the project, I don't see the need of adding it via templates. The most annoying part of templates is that they overgenerate. Templates will generate [SetUp] and [TearDown] fixtures which I don't always need. I like to keep my classes as clean as possible. But it's a matter of taste.
Here are some links that you might find helpful if you want to:
save your own predefined test class template
want to use the built in functionality of visual studio
follow a msdn walkthrough regarding the topic
We have a folder full of JSON text files that need to be set to a single URI. Currently it's all done with a single xUnit "[Fact]" as below
[Fact]
public void TestAllCases()
{
PileOfTests pot = new PileOfTests();
pot.RunAll();
}
pot.RunAll() then parses the folder, loads the JSON files (say 50 files). Each is then hammered against the URI to see is each returns HTTP 200 ("ok"). If any fail, we're currently printing it as a fail by using
System.Console.WriteLine("\n >> FAILED ! << " + testname + "\n");
This does ensure that failures catch our eye but xUnit thinks all tests failed (understandably). Most importantly, we can't specify to xunit "here, run only this specific test". It's all or nothing the way it's currently built.
How can I programmatically add test cases? I'd like to add them when I read the number and names of the *.json files.
The simple answer is:
No, not directly. But there exists an, albeit a bit hacky, workaround, which is presented below.
Current situation (as of xUnit 1.9.1)
By specifiying the [RunWith(typeof(CustomRunner))] on a class, one can instruct xUnit to use the CustomRunner class - which must implement Xunit.Sdk.ITestClassCommand - to enumerate the tests available on the test class decorated with this attribute.
But unfortunately, while the invocation of test methods has been decoupled from System.Reflection + the actual methods,
the way of passing the tests to run to the test runner haven't.
Somewhere down in the xUnit framework code for invoking a specific test method, there is a call to typeof(YourTestClass).GetMethod(testName).
This means that if the class implementing the test discovery returns a test name that doesn't refer to a real method on the test class, the test is shown in the xUnit GUI - but any attempts to run / invoke it end up with a TargetInvocationException.
Workaround
If one thinks about it, the workaround itself is relatively straightforward.
A working implementation of it can be found here.
The presented solution first reads in the names of the files which should appear as different tests in the xUnit GUI.
It then uses System.Reflection.Emit to dynamically generate an assembly with a test class containing a dedicated test method for each of the input files.
The only thing that each of the generated methods does is to invoke the RunTest(string fileName) method on the class that specified the [EnumerateFilesFixture(...)] attribute. See linked gist for further explanation.
Hope this helps; feel free to use the example implementation if you like.
Short version of my questions:
Can anyone point me toward some good, detailed sources from which I
can learn how to implement testing in my MVC 3 application, using
NUnit, Ninject 2, and Moq?
Can anyone here help clarify for me how Controller-Repository
decoupling, mocking, and dependency injection work together?
Longer version of my questions:
What I'm trying to do ...
I am currently beginning to create an MVC 3 application, which will use Entity Framework 4, with a database first approach. I want to do this right, so I am trying to design the classes, layers, etc., to be highly testable. But, I have little to no experience with unit testing or integration testing, other than an academic understanding of them.
After lots of research, I've settle on using
NUnit as my testing framework
Ninject 2 as my dependency injection framework
Moq as my mocking framework.
I know the topic of which framework is best, etc., could enter into this, but at this point I really don't know enough about any of it to form a solid opinion. So, I just decided to go with these free solutions which seem to be well liked and well maintained.
What I've learned so far ...
I've spent some time working through some of this stuff, reading resources such as:
Implementing the Repository and Unit of Work Patterns in an
ASP.NET MVC Application
Building Testable ASP.NET MVC Applications
NerdDinner Step 12: Unit Testing
Using Repository and Unit of Work patterns with Entity Framework
4.0
From these resources, I've managed to workout the need for a Repository pattern, complete with repository interfaces, in order to decouple my controllers and my data access logic. I have written some of that into my application already, but I admit I am not clear as to the mechanics of the whole thing, and whether I am doing this decoupling in support of mocking, or dependency injection, or both. As such, I certainly wouldn't mind hearing from you guys about this too. Any clarity I can gain on this stuff will help me at this point.
Where things got muddy for me ...
I thought I was grasping this stuff pretty well until I started trying to wrap my head around Ninject, as described in Building Testable ASP.NET MVC Applications, cited above. Specifically, I got completely lost around the point in which the author begins describing the implementation of a Service layer, about half way into the document.
Anyway, I am now looking for more resources to study, in order to try to get various perspectives around this stuff until it begins to make sense to me.
Summarizing all of this, boiling it down to specific questions, I am wondering the following:
Can anyone point me toward some good, detailed sources from which I
can learn how to implement testing in my MVC 3 application, using
NUnit, Ninject 2, and Moq?
Can anyone here help clarify for me how Controller-Repository
decoupling, mocking, and dependency injection work together?
EDIT:
I just discovered the Ninject official wiki on Github, so I'm going to start working through that to see if it starts clarifying things for me. But, I'm still very interested in the SO community thoughts on all of this :)
If you are using the Ninject.MVC3 nuget package, then some of the article you linked that was causing confusion will not be required. That package has everything you need to start injecting your controllers which is probably the biggest pain point.
Upon installing that package, it will create a NinjectMVC3.cs file in the App_Start folder, inside that class is a RegisterServices method. This is where you should create the bindings between your interfaces and your implementations
private static void RegisterServices(IKernel kernel)
{
kernel.Bind<IRepository>().To<MyRepositoryImpl>();
kernel.Bind<IWebData>().To<MyWebDAtaImpl>();
}
Now in your controller you can use constructor injection.
public class HomeController : Controller {
private readonly IRepository _Repo;
private readonly IWebData _WebData;
public HomeController(IRepository repo, IWebData webData) {
_Repo = repo;
_WebData = webData;
}
}
If you are after very high test coverage, then basically anytime one logical piece of code (say controller) needs to talk to another (say database) you should create an interface and implementation, add the definition binding to RegisterService and add a new constructor argument.
This applies not only to Controller, but any class, so in the example above if your repository implementation needed an instance of WebData for something, you would add the readonly field and the constructor to your repository implementation.
Then when it comes to testing, what you want to do is provide mocked version of all required interfaces, so that the only thing you are testing is the code in the method you are writing the test for. So in my example, say that IRepository has a
bool TryCreateUser(string username);
Which is called by a controller method
public ActionResult CreateUser(string username) {
if (_Repo.TryCreateUser(username))
return RedirectToAction("CreatedUser");
else
return RedirectToAction("Error");
}
What you are really trying to test here is that if statement and the return types, you do not want to have to create a real repository that will return true or false based on special values you give it. This is where you want to mock.
public void TestCreateUserSucceeds() {
var repo = new Mock<IRepository>();
repo.Setup(d=> d.TryCreateUser(It.IsAny<string>())).Returns(true);
var controller = new HomeController(repo);
var result = controller.CreateUser("test");
Assert.IsNotNull(result);
Assert.IsOfType<RedirectToActionResult>(result)
Assert.AreEqual("CreatedUser", ((RedirectToActionResult)result).RouteData["Action"]);
}
^ That won't compile for you as I know xUnit better, and do not remember the property names on RedirectToActionResult from the top of my head.
So to sum up, if you want one piece of code to talk to another, whack an interface in between. This then allows you to mock the second piece of code so that when you test the first you can control the output and be sure you are testing only the code in question.
I think it was this point that really made the penny drop for me with all this, you do this not necessarily becase the code demands it, but because the testing demands it.
One last piece of advice specific to MVC, any time you need to access the basic web objects, HttpContext, HttpRequest etc, wrap all these behind an interface as well (like the IWebData in my example) because while you can mock these using the *Base classes, it becomes painful very quickly as they have a lot of internal dependencies you also need to mock.
Also with Moq, set the MockBehaviour to Strict when creating mocks and it will tell you if anything is being called that you have not provided a mock for.
Here is the application that I'm creating. It is open source and available on github, and utilizes all of the required stuff - MVC3, NUnit, Moq, Ninject - https://github.com/alexanderbeletsky/trackyt.net/tree/master/src
Contoller-Repository decoupling is simple. All data operations are moved toward the Repository. Repository is an implementation of some IRepository type. The controller never creates repositories inside itself (with the new operator) but rather receives them either by constructor argument or property.
.
public class HomeController {
public HomeController (IUserRepository users) {
}
}
This technique is called "Inversion of Control." To support inversion of control you have to provide some "Dependency Injection" framework. Ninject is a good one. Inside Ninject you associate some particular interface with an implementation class:
Bind<IUserRepository>().To<UserRepository>();
You also substitute the default controller factory with your custom one. Inside the custom one you delegate the call to the Ninject kernel:
public class TrackyControllerFactory : DefaultControllerFactory
{
private IKernel _kernel = new StandardKernel(new TrackyServices());
protected override IController GetControllerInstance(
System.Web.Routing.RequestContext requestContext,
Type controllerType)
{
if (controllerType == null)
{
return null;
}
return _kernel.Get(controllerType) as IController;
}
}
When the MVC infrastructure is about to create a new controller, the call is delegated to the custom controller factory GetControllerInstance method, which delegates it to Ninject. Ninject sees that to create that controller the constructor has one argument of type IUserRepository. By using the declared binding, it sees that "I need to create a UserRepository to satisfy the IUserRepository need." It creates the instance and passes it to the constructor.
The constructor is never aware of what exact instance would be passed inside. It all depends on the binding you provide for that.
Code examples:
https://github.com/alexanderbeletsky/trackyt.net/blob/master/src/Web/Infrastructure/TrackyServices.cs https://github.com/alexanderbeletsky/trackyt.net/blob/master/src/Web/Infrastructure/TrackyControllerFactory.cs https://github.com/alexanderbeletsky/trackyt.net/blob/master/src/Web/Controllers/LoginController.cs
Check it out : DDD Melbourne video - New development workflow
The whole ASP.NET MVC 3 development process was very well presented.
The third party tools I like most are:
Using NuGet to install Ninject to enable DI throughout the MVC3
framework
Using NuGet to install nSubstite to create mocks to enable unit
testing
This is a tough one because not too many people use Pex & Moles or so I think (even though Pex is a really great product - much better than any other unit testing tool)
I have a Data project that has a very simple model with just one entity (DBItem). I've also written a DBRepository within this project, that manipulates this EF model. Repository has a method called GetItems() that returns a list of business layer items (BLItem) and looks similar to this (simplified example):
public IList<BLItem> GetItems()
{
using (var ctx = new EFContext("name=MyWebConfigConnectionName"))
{
DateTime limit = DateTime.Today.AddDays(-10);
IList<DBItem> result = ctx.Items.Where(i => i.Changed > limit).ToList();
return result.ConvertAll(i => i.ToBusinessObject());
}
}
So now I'd like to create some unit tests for this particular method. I'm using Pex & Moles. I created my moles and stubs for my EF object context.
I would like to write parametrised unit test (I know I've first written my production code, but I had to, since I'm testing Pex & Moles) that tests that this method returns valid list of items.
This is my test class:
[PexClass]
public class RepoTest
{
[PexMethod]
public void GetItemsTest(ObjectSet<DBItem> items)
{
MEFContext.ConstructorString = (#this, name) => {
var mole = new SEFContext();
};
DBRepository repo = new DBRepository();
IList<BLItem> result = repo.GetItems();
IList<DBItem> manual = items.Where(i => i.Changed > DateTime.Today.AddDays(-10));
if (result.Count != manual.Count)
{
throw new Exception();
}
}
}
Then I run Pex Explorations for this particular parametrised unit test, but I get an error path bounds exceeded. Pex starts this test by providing null to this test method (so items = null). This is the code, that Pex is running:
[Test]
[PexGeneratedBy(typeof(RepoTest))]
[Ignore("the test state was: path bounds exceeded")]
public void DBRepository_GetTasks22301()
{
this.GetItemsTest((ObjectSet<DBItem>)null);
}
This was additional comment provided by Pex:
The test case ran too long for these inputs, and Pex stopped the analysis. Please notice: The method Oblivious.Data.Test.Repositories.TaskRepositoryTest.b__0 was called 50 times; please check that the code is not stuck in an infinite loop or recursion. Otherwise, click on 'Set MaxStack=200', and run Pex again.
Update attribute [PexMethod(MaxStack = 200)]
Question
Am I doing this the correct way or not? Should I use EFContext stub instead? Do I have to add additional attributes to test method so Moles host will be running (I'm not sure it does now). I'm running just Pex & Moles. No VS test or nUnit or anything else.
I guess I should probably set some limit to Pex how many items should it provide for this particular test method.
Moles is not designed to test the parts of your application that have external dependencies (e.g. file access, network access, database access, etc). Instead, Moles allows you to mock these parts of your app so that way you can do true unit testing on the parts that don't have external dependencies.
So I think you should just mock your EF objects and queries, e.g., by creating in-memory lists and having query methods return fake data from those lists based on whatever criteria is relevant.
I am just getting to grips with pex also ... my issues surrounded me wanting to use it with moq ;)
anyway ...
I have some methods similar to your that have the same problem. When i increased the max they went away. Presumably pex was satisfied that it had sufficiently explored the branches. I have methods where i have had to increase the timeout on the code contract validation also.
One thing that you should probably be doign though is passing in all the dependant objects as parameters ... ie dont instantiate the repo in the method but pass it in.
A general problem you have is that you are instantiating big objects in your method. I do the same in my DAL classes, but then i am not tryign to unit test them in isolation. I build up datasets and use this to test my data access code against.
I use pex on my business logic and objects.
If i were to try and test my DAL code id have to use IOC to pass the datacontext into the methods - which would then make testing possible as you can mock the data context.
You should use Entity Framework Repository Pattern: http://www.codeproject.com/KB/database/ImplRepositoryPatternEF.aspx
I'm working on a web application that is built over Sitecore CMS. I was wondering if we could unit test for example a method that takes some data from Sitecore makes some processing with it and spits out a result. I would like to test all the logic within the method via a unit test.
I pretty confused after searching the internet wide and deep. Some say that this kind of testing is actually integration testing and not unit testing and I should test only the code that has no Sitecore calls, others say that this is not possible because the Sitecore context would be missing.
I would like to ask for your help experienced fellow programmers:
Can I unit test a method that contains Sitecore calls ? If YES, how ? If NO, why ? Is there any workaround ?
The project is at its beginning, so there will be no problem in choosing between unit testing frameworks such as MSTest or Nunit, if it is the case that the solution is related to the unit testing framework of choice.
It's pretty hard to find out anything about Sitecore without providing email and living through the sales pitch, so I'll just provide a generic approach on how to do something like this.
First and foremost, you assume that the Sitecore API is guaranteed to work - i.e. it's a framework - and you don't unit test it. You should be unit testing your interactions with it.
Then, download MOQ and read the quick start on how to use it. This is my preferred mocking framework. Feel free to use other frameworks if you wish.
Hopefully, Sitecore API provides a way for you to create data objects without dealing with persistence - i.e. to simply create a new instance of whatever it is you are interested in. Here is my imaginary API:
public class Post {
public string Body {get;set;}
public DateTime LastModified {get;set;}
public string Title {get;set;}
}
public interface ISiteCorePosts {
public IEnumerable<Post> GetPostsByUser(int userId);
}
In this case unit testing should be fairly easy. With a bit of Dependency Injection, you can inject the SiteCore interfaces into your component and then unit test it.
public class MyPostProcessor {
private readonly ISiteCorePosts m_postRepository;
public MyPostProcessor(ISiteCorePosts postRepository) {
m_postRepository = postRepository;
}
public void ProcessPosts(int userId) {
var posts = m_postRepository.GetPostsByUser(userId);
//do something with posts
}
}
public class MyPostProcessorTest {
[TestMethod]
ProcessPostsShouldCallGetPostsByUser() {
var siteCorePostsMock = new Mock<ISiteCorePosts>();
//Sets up the mock to return a list of posts when called with userId = 5
siteCorePostsMock.Setup(m=>m.GetPostsByUser(5)).Returns(new List<Post>{/*fake posts*/});
MyPostProcessor target = new MyPostProcessor(siteCorePostsMock.Object);
target.ProcessPosts(5);
//Verifies that all setups are called
siteCorePostsMock.VerifyAll();
}
}
If ISiteCorePosts is not, in fact, an interface and is a concrete class whose methods are not virtual and thus cannot be mocked, you will need to use Facade pattern to wrap the SiteCore interaction to make it more testing friendly.
public class SiteCorePostsFacade {
SiteCorePosts m_Posts = new SiteCorePosts();
//important - make this method virtual so it can be mocked without needing an interface
public virtual IEnumerable<Post> GetPostsByUser(int userId) {
return m_Posts.GetPostsByUser(userId);
}
}
You then proceed to use SiteCorePostsFacade as though it was an interface in the previous example. Good thing about MOQ is that it allows you to mock concrete classes with virtual methods, not just interfaces.
With this approach, you should be able to inject all sorts of data into your application to test all interactions with SiteCore API.
we have used a custom WebControl placed on a WebForm for our integration tests some years now, which wraps the NUnit Test Suite runner functionality much like the NUnit GUI. It show a pretty grid of executed tests with links to fixtures and categories to execute specific tests. Its created much like described here http://adeneys.wordpress.com/2010/04/13/new-technique-for-unit-testing-renderings-in-sitecore/ (the custom test runner part). Our implementation can also return raw NUnit xml for further processing by for example a build server.
I've tried MSTest a while back and it also works when specified that it should launch a WebDev / IIS site to test. It works but is extremely slow compared to above solution.
Happy testing!
Short answer:
You need to mock calls to SiteCore CMS.
Long answer:
I am not aware about SiteCore CMS. But, from your question looks like it is something that is external to your application. Components external to your system should always be used via interface. This has two benefits:
If you want to use another CMS system, you can easily do as your application is just talking to an interface.
It helps you with behavior testing by mocking the interface.
The code you write is your responsibility and hence you should only unit test that piece of code. Your unit tests should ensure that your code calls appropriate SiteCode CMS methods in various scenarios (behavior tests). You can do this using mocking. I use moq for mocking.
As tugga said, it depends upon how tightly the code you want to test is coupled to SiteCore. If it's something like:
SomeSiteCoreService siteCoreDependency = new SomeSiteCoreService()
Then this would be very difficult to test. If SiteCore provides you an interface, then you have more flexibility to unit test it. You could pass the implementation into your method either (contstructor, class property, or method parameter) and then you can send in a fake implementation of that service.
If they do not provide you with an interface, then you have to do a little more work. You would write an adapter interface of your own and the default implementation would delegate to the 3rd party dependency.
public interface ICMSAdapter{
void DoSomethingWithCMS()
}
public class SiteCoreCMSAdapter: ICMSAdapter{
SiteCoreService _cms = new SiteCoreService();
public void DoSomethingWithCMS(){
_cms.DoSomething();
}
That keeps your 3rd party dependencies at arms length and provides seams to all sorts of cool things, like unit tests and you do interception style architecture and do your own thing before and after the call.
}
I was able to get unit tests to interact with sitecore api in VS 2015. The same test throws a StackOverflow exception when run in VS 2012.
For example, this method call runs fine in VS2015 but not VS2015:
Context.SetActiveSite("mysite");
quick note: this assumes you have a site named mysite setup in your config file