I have objects that gets initialized by serializing an XML file.
I'm thinking of including the test data in the test project as an embedded resource instead of "hard-coding" the data in the test method itself.
Embedded Hard-coded approach:
[Test]
public void IsMale_CheckIfGenderIsMale_ReturnsTrue()
{
Human human = new Human();
human.Gender = Gender.Male;
Gender expected = Gender.Male;
Assert.IsTrue((human.Gender == expected));
}
XML approach:
[Test]
public void IsMale_CheckIfGenderIsMale_ReturnsTrue()
{
Human human = Human.Initialize("Human_Male.xml");
Gender expected = Gender.Male;
Assert.IsTrue((human.Gender == expected));
}
Which is a better approach?
We often use embedded xml files in our project for testing. Because we want to test the creation of objects using xml.
It is also good practice to separate the creation of objects from the class itself, for example:
Human human = HumanFactory.Create("Human_Male.xml");
The human class would then have a constructor which takes parameters like gender, and can be called from the HumanFactory class. It's a separation of concerns that will keep the logic of your class separate from the mechanism of building it.
And when you need a human object for a test, you can choose either to create it directly or use the factory and create it from xml.
I would also simplify the HumanFactory class further, by giving it an xml string (or Stream) instead of a filename. This can make it easier to test - because you can just include the xml in your test code and not in a file:
Human human = HumanFactory.Create("<human gender="m"></human>");
Assert.AreEqual(human.Gender, Gender.Male);
Otherwise your test depends on either:
the test setup code deploying the "Human_Male.xml" file to the file system, and removing it afterwards. Although Visual Studio test has a nice mechanism to do this (test settings deployment will copy files into the test run folder as part of the test run setup).
Including a path to the embedded resource in your test code. If you should ever restructure your project layout, you will need to change the test code to match. And make sure you use a relative path.
The only problem with external files is that they may change.
My preferred option is to have a series of initialisation functions within the test file that can be called from the test method as and when required. This also makes for easier and quicker reading for others that may pick up your code, or even yourself in six months time.
Another option is to base some of your object classes around an interface/abstract type so that you can mock them (even in your runtime code) by creating a test class instead.
For example (code agnostic):
base abstract class/interface: IHuman IHuman.Gender
property/attribute IHuman.Height property/attribute IHuman.Weight
property/attribute
class HumanMale inherits IHuman HumanMale.Gender overrides IHuman
Gender to return Male.
class HumanFemale inherits IHuman HumanFemale.Gender overrides IHuman
Gender to return Female.
All code then uses IHuman and you can even use a factory to create IHuman types to create a legion of cybermen :)
Related
We have a folder full of JSON text files that need to be set to a single URI. Currently it's all done with a single xUnit "[Fact]" as below
[Fact]
public void TestAllCases()
{
PileOfTests pot = new PileOfTests();
pot.RunAll();
}
pot.RunAll() then parses the folder, loads the JSON files (say 50 files). Each is then hammered against the URI to see is each returns HTTP 200 ("ok"). If any fail, we're currently printing it as a fail by using
System.Console.WriteLine("\n >> FAILED ! << " + testname + "\n");
This does ensure that failures catch our eye but xUnit thinks all tests failed (understandably). Most importantly, we can't specify to xunit "here, run only this specific test". It's all or nothing the way it's currently built.
How can I programmatically add test cases? I'd like to add them when I read the number and names of the *.json files.
The simple answer is:
No, not directly. But there exists an, albeit a bit hacky, workaround, which is presented below.
Current situation (as of xUnit 1.9.1)
By specifiying the [RunWith(typeof(CustomRunner))] on a class, one can instruct xUnit to use the CustomRunner class - which must implement Xunit.Sdk.ITestClassCommand - to enumerate the tests available on the test class decorated with this attribute.
But unfortunately, while the invocation of test methods has been decoupled from System.Reflection + the actual methods,
the way of passing the tests to run to the test runner haven't.
Somewhere down in the xUnit framework code for invoking a specific test method, there is a call to typeof(YourTestClass).GetMethod(testName).
This means that if the class implementing the test discovery returns a test name that doesn't refer to a real method on the test class, the test is shown in the xUnit GUI - but any attempts to run / invoke it end up with a TargetInvocationException.
Workaround
If one thinks about it, the workaround itself is relatively straightforward.
A working implementation of it can be found here.
The presented solution first reads in the names of the files which should appear as different tests in the xUnit GUI.
It then uses System.Reflection.Emit to dynamically generate an assembly with a test class containing a dedicated test method for each of the input files.
The only thing that each of the generated methods does is to invoke the RunTest(string fileName) method on the class that specified the [EnumerateFilesFixture(...)] attribute. See linked gist for further explanation.
Hope this helps; feel free to use the example implementation if you like.
Sorry for the long post...
While being introduced to a brown field project, I'm having doubts regarding certain sets of unit tests and what to think. Say you had a repostory class, wrapping a stored procedure and in the developer guide book, a certain set guidelines (rules), describe how this class should be constructured. The class could look like the following:
public class PersonRepository
{
public PersonCollection FindPersonsByNameAndCity(string personName, string cityName)
{
using (new SomeProfiler("someKey"))
{
var sp = Ioc.Resolve<IPersonStoredProcedure>();
sp.addNameArguement(personName);
sp.addCityArguement(cityName);
return sp.invoke();
}
} }
Now, I would of course write some integration tests, testing that the SP can be invoked, and that the behavior is as expected. However, would I write unit tests that assert that:
Constructor for SomeProfiler with the input parameter "someKey" is called
The Constructor of PersonStoredProcedure is called
The addNameArgument method on the stored procedure is called with parameter personName
The addCityArgument method on the stored procedure is called with parameter cityName
The invoke method is called on the stored procedure -
If so, I would potentially be testing the whole structure of a method, besides the behavior. My initial thought is that it is overkill. However, in regards to the coding practices enforced by the team, these test ensure a uniform and 'correct' structure and that the next layer is called correctly (from DAL to DB, BLL to DAL etc).
In my case these type of tests, are performed for each layer of the application.
Follow up question - the use of the SomeProfiler class smells a little like a convention to me - Instead creating explicit tests for this, could one create convention styled test by using static code analysis or unittest + reflection?
Thanks in advance.
I think that your initial thought was right - this is an overkill. Although you can use reflection to make sure that the class has the methods you expect I'm not sure you want to test it that way.
Perhaps instead of unit testing you should use some tool such as FxCop/StyleCop or nDepend to make sure all of the classes in a specific assembly/dll has these properties.
Having said that I'm a believer of "only code what you need" why test that a method exist, either you use it somewhere in your code and in that can you can test the specific case or you don't - and so it's irrelevant.
Unit tests should focus on behavior, not implementation. So writing a test to verify that certain arguments are set or passed in doesn't add much value to your testing strategy.
As the example provided appears to be communicating with your database, it can't truly be considered a "unit test" as it must communicate with physical dependencies that have additional setup and preconditions, such as availability of the environment, database schema, existing data, stored-procedures, etc. Any test you write is actually verifying these preconditions as well.
In it's present condition, your best bet for these types of tests is to test the behavior provided by the class -- invoke a method on your repository and then validate that the results are what you expected. However, you'll suddenly realize that there's a hidden cost here -- the database maintains state between test runs, and you'll need additional setup or tear-down logic to ensure that the database is in a well-known state.
While I realize the intent of the question was about the testing a "black box", it seems obvious that there's some hidden magic here in your API. My preference to solve the well-known state problem is to use an in-memory database that is scoped to the current test, which isolates me from environment considerations and enables me to parallelize my integration tests. I'd wager that under the current design, there is no "seam" to programmatically introduce a database configuration so you're "hemmed in". In my experience, magic hurts.
However, a slight change to the existing design solves this problem and the "magic" goes away:
public class PersonRepository : IPersonRepository
{
private ConnectionManager _mgr;
public PersonRepository(ConnectionManager mgr)
{
_mgr = mgr;
}
public PersonCollection FindPersonsByNameAndCity(string personName, string cityName)
{
using (var p = _mgr.CreateProfiler("somekey"))
{
var sp = new PersonStoredProcedure(p);
sp.addArguement("name", personName);
sp.addArguement("city", cityName);
return sp.invoke();
}
}
}
I'm trying to create a unit test for a code similar to this:
foreach (string domainName in Directory.GetDirectories(server.Path))
{
HandleDomainDirectory(session, server, domainName);
}
The problem is that I'm using the System.IO.Directory class in my code.
How can I create a testing method that won't be dependent on any folder I have on my hard disk.
In other words, How can I fake the response of "Directory.GetDirectories(server.Path)"?
(Please note, I do control the "server" object in my class, therefore i can give any path i want)
Thanks.
Rather than calling Directory.GetDirectories(server.Path) directly, you could create an interface like IDirectoryResolver with a single method that takes a path string and returns the list of directories. The class containing your code above would then need a property or field of type IDirectoryResolver, which can be injected through the constructor or a setter.
For your production code, you would then create a new class that implements the IDirectoryResolver interface. This class could use the Directory.GetDirectories method in its implementation of the interface method.
For unit testing, you could create a MockDirectoryResolver class which implements IDirectoryResolver (or use a mocking library to create a mock instance for the interface). The mock implementation can do whatever you need it to do.
You would inject a wrapper class.
public class DirectoryFetcher
{
public virtual List<string> GetDirectoriesIn(string directory)
{
return Directory.GetDirectories(directory);
}
}
And then inject that:
foreach(string directory in _directoryFetcher.GetDirectoriesIn(server.Path))
{
// Whatever
}
You can then Mock that guy at the injection point (this example uses Moq, and constructor injection):
Mock<DirectoryFetcher> mockFetcher = new Mock<DirectoryFetcher>();
mockFetcher.Setup(x => x.GetDirectoriesIn("SomeDirectory")).Returns(new List<string>
{
"SampleDirectory1",
"SampleDirectory2"
});
MyObjectToTest testObj = new MyObjectToTest(mockFetcher.Object);
// Do Test
When communicating with the outside world, such as file system, databases, web services etc. , you should always consider using wrapper classes like the others before me suggested. Testability is one major argument, but an even bigger one is: The out side world changes, and you have no control over it. Folders move, user rights changes, new disk drives appears and old ones are removed. You only want to care about stuff like that in one place. Hence, the wrapper -- let's call it DirectoryResolver like Andy White suggested ealier.
So, wrap your file system calls, extract an interface, and inject that interface where you need to communicate with the file system.
The best solution I've found was to use Moles. The code is very specific, and must do very specific thing. Wrapping it with wrapper class will be redundant. The only reason I needed wrapper class is in order to write tests. Moles allows me to write the tests without any wrapper class :)
I wrote a simple class to manage business objects.
class Manager
{
string[] GetNames();
BObject GetObject(string name);
void Saveobject(BObject obj);
}
It serializes /deserializes the objects as files on a local disk. I wrote Unit tests for the class and run them. That was fine so far. The problem happens when my test were run on build server because of file access permission I was not allowed to write files on the server. It's obvious I cannot test that way.
I think how to unit test this. One approach I can see is to extract an interface and creat a mock object for testing. But I want to test the class itself. How can I do it?
The class presumably calls file system operations File.OpenRead(), File.OpenWrite() etc. (I assume that this is C# due to the camel casing.) Then, you could create an interface for those operations, e.g.:
public interface IFileSystem {
StreamReader OpenRead(string fileName);
StreamWriter OpenWrite(string fileName);
}
and make the constructor of Manager take an instance of IFileSystem. Then, write a (non-mock) class that implements IFileSystem by calling the actual File.OpenRead() and File.OpenWrite() methods and use this one in the production code. In the tests, you use a mock framework, as mentioned by #Digger (my personal preference is Moq, but I haven't tried Rhino Mocks, so I have nothing negative to say about it) to mock out IFileSystem and use the mock to verify that the methods were called with the correct serialized data.
EDIT: Per request, an example in NUnit with Moq (I don't have an IDE here, so it's untested; feel free to correct it):
[Test]
public void BObjectShouldBeSerializedToFile() {
var fileSystemMock = new Mock<IFileSystem>();
var stream = new MemoryStream();
fileSystemMock.Setup(f => f.OpenWrite("theFileNameYouExpect.txt")).Returns(new StreamWriter(stream)).Verifiable();
var manager = new Manager(fileSystemMock.Object);
manager.SaveObject(new BObject(...));
stream.Seek(0, SeekOrigin.Begin);
Assert.That(...); // Perform asserts on the stream contents here
fileSystemMock.Verify(); // Not really necessary, but verify that `OpenWrite` was called
}
It depends on how much logic is contained in your class, in my opinion.
If there's some complicated logic inside your manager, it makes sense to abstract your file operations as per Aasmund's suggestion so that the logic can be tested independently of the file system. I do this when something is finicky enough to warrant the extra dependencies.
On the other hand, if there's very little logic other than calling into your serialization/deserialization code, then it's often acceptable to skip the unit tests and run integration tests that test the full cycle (create a BObject in memory, persist it via calling SaveObject, read it back out using GetObject, ensure that it is equal/equivalent to the one you persisted in the first place).
If your build environment can't run integration tests, then I'd look into setting it up so that it's possible.
I was wondering whether the object to test should be a field and thus set up during a SetUp method (ie. JUnit, nUnit, MS Test, …).
Consider the following examples (this is C♯ with MsTest, but the idea should be similar for any other language and testing framework):
public class SomeStuff
{
public string Value { get; private set; }
public SomeStuff(string value)
{
this.Value = value;
}
}
[TestClass]
public class SomeStuffTestWithSetUp
{
private string value;
private SomeStuff someStuff;
[TestInitialize]
public void MyTestInitialize()
{
this.value = Guid.NewGuid().ToString();
this.someStuff = new SomeStuff(this.value);
}
[TestCleanup]
public void MyTestCleanup()
{
this.someStuff = null;
this.value = string.Empty;
}
[TestMethod]
public void TestGetValue()
{
Assert.AreEqual(this.value, this.someStuff.Value);
}
}
[TestClass]
public class SomeStuffTestWithoutSetup
{
[TestMethod]
public void TestGetValue()
{
string value = Guid.NewGuid().ToString();
SomeStuff someStuff = new SomeStuff(value);
Assert.AreEqual(value, someStuff.Value);
}
}
Of course, with just one test method, the first example is much too long, but with more test methods, this could be safe quite some redundant code.
What are the pros and cons of each approach? Are there any “Best Practices”?
It's a slippery slope once you start initializing fields & generally setting up the context of your test within the test method itself. This leads to large test methods and really really unmanageable fixtures that don't explain themselves very well.
Instead, you should look at the BDD style naming & test organization. Make one fixture per context, rather than one fixture per system-under-test. Then your [setup] truly does setup the context, and your tests can be simple one-liner asserts.
It's much easier to read when you see a test output that does this:
OrderFulfillmentServiceTests.cs
with_an_order_from_a_new_customer
it should check their credit from the credit service
it should give no discount
with valid credit check
it should decrement inventory
it should ship the goods
with a customer in texas or california
it should add appropriate sales tax
with an order from a gold customer
it should NOT check credit
it should get expedited shipping added for free
Our tests are now really good documentation for our system. Each "with_an..." is a test fixture, and the items below it are tests. Within those, you setup the context (the state of the world as the class name describes) and then the test does the simple assert that verifies what the method name says it does.
The second approach is much more readable, and much easier to visually trace.
However, the first approach means less repetition.
What I've found is that I tend to use the SetUp to create objects (especially for things with a number of dependencies), and then set the values used in the test itself. From experience, this provides about the right amount of code-reuse versus readability/traceability.
From talking with Kent Beck about the design of jUnit I know that Test Classes were a way to share setup between Tests, so using the common initialization was the intent. However, along with that, that means splitting tests that require different setup into separate test classes that have revealing names.
Personally, I use Setup and Teardown methods for two distinct reasons, although I assume that others will have different reasons.
Use Setup and Teardown methods when there is common initiation logic that is used by all tests and a single instance of the object(s) created in the Setup are designed to be reused.
Use Setup and Teardown methods when the time it takes for creating and destroying any object(s) created takes enough time to slow down the unit testing process when repeated in each TestMethod.
To give you an idea of how often I run accross these scenarios, in a project that I am working on now, only two of my test classes (out of about eighty) have an explicit need for Setup and Teardown methods, both times it was to satisfy my second reason due to the 10 second max I have enabled for each test execution.
I also prefer the readability of having the object(s) created and destroyed within the TestMethod, although it is not a breaking or selling point for me.
The approach I take is somewhere in the middle - I use TearDown and SetUp to create a test "sandbox" directory (and delete it when done), as well as to initialize some test member variables with some default values that will be used to test the classes. I then set up some "helper methods" - One is generally called InstantiateClass() I use that to call with the default parameters (if any) which I can override as necessary in each explicit test.
[Test]
public void TestSomething()
{
_myVar = "value";
InstantiateClass();
RunTheClass();
Assert.IsTrue(this, that);
}
In practice, I find set up methods make it hard to reason about a test that is failing and have to scroll to somewhere near the top of the file (which can be very large) to figure out what collaborator has broken (not easy with mocking) and there is no clickable reference to navigate in your IDE. In short, you lose spatial locality.
Static helper methods reveal the collaborators more explicitly, and you avoid fields which unnecessarily widen the scope of variables.