I'm writing a series of unit tests for a class that requires a property list stored in the bundle. I keep a mock version of this property list in my unit test target and inject the unit test bundle into the class such that it can be read. One of my unit test deals with handling the case where the file cannot be found, so I simply delete it from the bundle like so:
NSString *plistPath = [self.bundle pathForResource:kInAppPurchasesPlistName ofType:#"plist"];
[[NSFileManager defaultManager] removeItemAtPath:plistPath error:nil];
However this leads to problems as the file is now deleted from the bundle, even between tests. If I set my tests up correctly this may not be an issue, but with randomized execution order it is.
I do "recreate" the bundle between unit tests:
- (void)setUp {
[...]
self.bundle = [NSBundle bundleForClass:[self class]];
[...]
}
- (void)tearDown {
[...]
self.bundle = nil;
[...]
}
Although I'm suspecting that the underlying bundle does not change or reset state by doing this.
What's the best way of accomplishing this? In summary, I need a way that will allow me to "inject" a fake file into my class which reads it from the bundle. I want to delete this file in some tests, but for the state to be completely reset after each individual test.
Tests should be isolated so that you can run them individually, or in any order. If a test actually deletes something, you'll have to put it back.
The danger of making changes to the file system is that it's persistent mutable state, shared across tests. Instead of talking directly to the NSFileManager, I recommend injecting it instead. Depending on your code, this could be done through an initialization parameter (preferred) or a property (second best). You can even have your production code use a lazy property so that it uses [NSFileManager defaultManager] by default, unless a different instance has been injected.
To inject a fake version of NSFileManager that your tests can control, I recommend using OCMockito or OCMock.
Related
I'm using log4net, trying to get logging in my unit tests. If I manually call
log4net.Config.XmlConfigurator.Configure();
Since that works, that seems to eliminate all of the "bad config, config location" issues.
it works, but there are a large number of test classes, so that is not good.
I added
[assembly: log4net.Config.XmlConfigurator(Watch=true)]
to the assemblyinfo of my test project, but when I run (either via native MSTest, or Resharper test runner) I get no logging.
Help?
Source
[AssemblyInitialize()]
public static void MyTestInitialize(TestContext testContext)
{
// Take care the log4net.config file is added to the deployment files of the testconfig
FileInfo fileInfo;
string fullPath = Path.Combine(System.Environment.CurrentDirectory, "log4net.config");
fileInfo = new FileInfo(fullPath);
As it says in the documentation for assembly attributes
Therefore if you use configuration attributes you must invoke log4net
to allow it to read the attributes. A simple call to
LogManager.GetLogger will cause the attributes on the calling assembly
to be read and processed. Therefore it is imperative to make a logging
call as early as possible during the application start-up, and
certainly before any external assemblies have been loaded and invoked.
Because the unit test runners load the test assembly in order to find and the tests, it isn't possible to initialise log4net using an assembly attribute in unit test projects, and you will have to use the XmlConfigurator.
Edit: as linked in a comment by OP this can be done in one place for the whole test project by using the AssemblyInitializeAttribute
In my program I have a situation that I can simplify to the following:
An IRepository of which I create a MemoryRepository and a SqlRepository implementation
A Mapper that gets the IRepository constructor injected.
A Mapper.Map() that contains business logic I want to test
I created a test where the Mapper receives the MemoryRepository.
By explicitly setting a property on the memory repository that will be used in the business logic I can now test this logic.
If I use injection however, I wouldn't have access to this repository anymore.
A bit of code tells you more then a 1000 normal words, here is the pastebin link.
How would you go about this?
If I understand your question correctly basically you are concerned that your repository was created AND injected when the test class was instantiated and so in your test method you cannot modify the state of your repository since it is already inside your mapper and, of course, your mapper should not expose the internals of the repository.
If that is the case then I don't think you have to worry, just modify the state of the myMemoryCategoryRepository and execute the mapper's method. Your mapper should behave accordingly because what you injected is a reference to the repository so the object inside the mapper is the same one as the one you would be modifying.
Dim myMemoryCategoryRepository As MemoryCategoryRepository = MemoryKernel.Instance.Get(Of MemoryCategoryRepository)()
Dim myCategoryMapper As CategoryMapper = New CategoryMapper(myMemoryCategoryRepository)
<TestMethod()> _
Public Sub GetCategoryStartDate_CategoryStartDateAndContractStartDate_ContractStartDateIsOldestDate()
myMemoryCategoryRepository.AnyFlag = True
myCategoryMapper.Execute()
Assert.AreEqual(expectedValue, myCategoryMapper.Value)
End Sub
Not entirely sure what you're asking here, are you testing the mapper or the repository? If you're testing the mapper, then fake the repository. You've already got the seams in place, either use a framework or create a fake repository manually in your tests that makes whatever happy noises you want for the sake of testing Mapper and create the mapper by passing in your fake into the constructor.
So by your own simplification,
Create a fake Repository inheriting from IRepository
Inject the fake into your Mapper that you are going to test
Test Mapper.Map()
If you need to verify some information on the Repository, use a Mock rather than a Stub.
Difference Between Mocks and Stubs
I have written test case for deletion of entity. In test case I simply pick first record by select query and pass its id to deletion method. Entity I want to delete can have some child entities restricting it from deletion. So I suppose I should create a entity first in my deletion test case and destroy same then so that I don't face issues of child dependency.
Is it good practice to write code for creation of entity before deletion. Its kind of testing creation method before deletion method.Please suggest
Edit:
I am working on Rail platform, so I have features like loading database with fixtures (not using currently, facing some error with same, see this https://stackoverflow.com/questions/5288142/rails-fixture-expects-table-name-to-be-prefixed-with-module-name-how-to-disable ). And yes I am using configuration to restore database state after test case run.
In unit-testing, you usually perform some sort of set-up before you run your tests.
Many testing frameworks support this sort of operation. Normally you don't do it through external queries though; for instance, you could directly create an object with certain properties, instead of performing an externally-exposed create query.
Because you directly create the object in the first place, you are not testing your creation query code (unless the way you internally create objects is flawed, but if you are concerned about that, you can test it too), and your deletion code is the only thing being tested.
In test case i simply pick first
record by select query
This is wrong. You should not execute queries during unit testing.
Test that I see can be:
Delete an existent;
Delete a non
existent Entity;
Delete a child;
Delete a non existent child;
If your unit testing framework allows test dependencies, i.e. run test X only if test Y passes and pass Y's return value as a parameter to X, you can get away with it. Here's how that would look in PHP:
function setUp() {
$this->dao = new UserDao(...);
}
function testCreate() {
$user = $this->dao->create('Bob');
assertThat($user, notNullValue());
// more assertions about the new user
return $user->getId();
}
/**
* #depends testCreate
*/
function testDelete($id) {
assertThat($this->dao->delete($id), is(true));
}
PHPUnit will skip testDelete() if testCreate() fails. This is a good work-around if you cannot setup a standard test data set before each test run.
Is it good practice to write code for creation of entity before deletion. Its kind of testing creation method before deletion method.Please suggest
Yes, it's good practice to create the entity whose deletion you are testing, so that the test does not depend on external state, and is repeatable independent of other tests.
This doesn't test creation, but uses creation in order to set up for testing deletion.
If you have multiple tests relying on the same data, the creation can be pulled out to a method that you call in each of your tests that needs that data. Most test frameworks also have a mechanism for specifying setup methods that are run before each test, and you could put the creation there if the data is needed for all tests in a test class.
I wrote a simple class to manage business objects.
class Manager
{
string[] GetNames();
BObject GetObject(string name);
void Saveobject(BObject obj);
}
It serializes /deserializes the objects as files on a local disk. I wrote Unit tests for the class and run them. That was fine so far. The problem happens when my test were run on build server because of file access permission I was not allowed to write files on the server. It's obvious I cannot test that way.
I think how to unit test this. One approach I can see is to extract an interface and creat a mock object for testing. But I want to test the class itself. How can I do it?
The class presumably calls file system operations File.OpenRead(), File.OpenWrite() etc. (I assume that this is C# due to the camel casing.) Then, you could create an interface for those operations, e.g.:
public interface IFileSystem {
StreamReader OpenRead(string fileName);
StreamWriter OpenWrite(string fileName);
}
and make the constructor of Manager take an instance of IFileSystem. Then, write a (non-mock) class that implements IFileSystem by calling the actual File.OpenRead() and File.OpenWrite() methods and use this one in the production code. In the tests, you use a mock framework, as mentioned by #Digger (my personal preference is Moq, but I haven't tried Rhino Mocks, so I have nothing negative to say about it) to mock out IFileSystem and use the mock to verify that the methods were called with the correct serialized data.
EDIT: Per request, an example in NUnit with Moq (I don't have an IDE here, so it's untested; feel free to correct it):
[Test]
public void BObjectShouldBeSerializedToFile() {
var fileSystemMock = new Mock<IFileSystem>();
var stream = new MemoryStream();
fileSystemMock.Setup(f => f.OpenWrite("theFileNameYouExpect.txt")).Returns(new StreamWriter(stream)).Verifiable();
var manager = new Manager(fileSystemMock.Object);
manager.SaveObject(new BObject(...));
stream.Seek(0, SeekOrigin.Begin);
Assert.That(...); // Perform asserts on the stream contents here
fileSystemMock.Verify(); // Not really necessary, but verify that `OpenWrite` was called
}
It depends on how much logic is contained in your class, in my opinion.
If there's some complicated logic inside your manager, it makes sense to abstract your file operations as per Aasmund's suggestion so that the logic can be tested independently of the file system. I do this when something is finicky enough to warrant the extra dependencies.
On the other hand, if there's very little logic other than calling into your serialization/deserialization code, then it's often acceptable to skip the unit tests and run integration tests that test the full cycle (create a BObject in memory, persist it via calling SaveObject, read it back out using GetObject, ensure that it is equal/equivalent to the one you persisted in the first place).
If your build environment can't run integration tests, then I'd look into setting it up so that it's possible.
I am writing a component that, given a ZIP file, needs to:
Unzip the file.
Find a specific dll among the unzipped files.
Load that dll through reflection and invoke a method on it.
I'd like to unit test this component.
I'm tempted to write code that deals directly with the file system:
void DoIt()
{
Zip.Unzip(theZipFile, "C:\\foo\\Unzipped");
System.IO.File myDll = File.Open("C:\\foo\\Unzipped\\SuperSecret.bar");
myDll.InvokeSomeSpecialMethod();
}
But folks often say, "Don't write unit tests that rely on the file system, database, network, etc."
If I were to write this in a unit-test friendly way, I suppose it would look like this:
void DoIt(IZipper zipper, IFileSystem fileSystem, IDllRunner runner)
{
string path = zipper.Unzip(theZipFile);
IFakeFile file = fileSystem.Open(path);
runner.Run(file);
}
Yay! Now it's testable; I can feed in test doubles (mocks) to the DoIt method. But at what cost? I've now had to define 3 new interfaces just to make this testable. And what, exactly, am I testing? I'm testing that my DoIt function properly interacts with its dependencies. It doesn't test that the zip file was unzipped properly, etc.
It doesn't feel like I'm testing functionality anymore. It feels like I'm just testing class interactions.
My question is this: what's the proper way to unit test something that is dependent on the file system?
edit I'm using .NET, but the concept could apply Java or native code too.
Yay! Now it's testable; I can feed in test doubles (mocks) to the DoIt method. But at what cost? I've now had to define 3 new interfaces just to make this testable. And what, exactly, am I testing? I'm testing that my DoIt function properly interacts with its dependencies. It doesn't test that the zip file was unzipped properly, etc.
You have hit the nail right on its head. What you want to test is the logic of your method, not necessarily whether a true file can be addressed. You don´t need to test (in this unit test) whether a file is correctly unzipped, your method takes that for granted. The interfaces are valuable by itself because they provide abstractions that you can program against, rather than implicitly or explicitly relying on one concrete implementation.
Your question exposes one of the hardest parts of testing for developers just getting into it:
"What the hell do I test?"
Your example isn't very interesting because it just glues some API calls together so if you were to write a unit test for it you would end up just asserting that methods were called. Tests like this tightly couple your implementation details to the test. This is bad because now you have to change the test every time you change the implementation details of your method because changing the implementation details breaks your test(s)!
Having bad tests is actually worse than having no tests at all.
In your example:
void DoIt(IZipper zipper, IFileSystem fileSystem, IDllRunner runner)
{
string path = zipper.Unzip(theZipFile);
IFakeFile file = fileSystem.Open(path);
runner.Run(file);
}
While you can pass in mocks, there's no logic in the method to test. If you were to attempt a unit test for this it might look something like this:
// Assuming that zipper, fileSystem, and runner are mocks
void testDoIt()
{
// mock behavior of the mock objects
when(zipper.Unzip(any(File.class)).thenReturn("some path");
when(fileSystem.Open("some path")).thenReturn(mock(IFakeFile.class));
// run the test
someObject.DoIt(zipper, fileSystem, runner);
// verify things were called
verify(zipper).Unzip(any(File.class));
verify(fileSystem).Open("some path"));
verify(runner).Run(file);
}
Congratulations, you basically copy-pasted the implementation details of your DoIt() method into a test. Happy maintaining.
When you write tests you want to test the WHAT and not the HOW.
See Black Box Testing for more.
The WHAT is the name of your method (or at least it should be). The HOW are all the little implementation details that live inside your method. Good tests allow you to swap out the HOW without breaking the WHAT.
Think about it this way, ask yourself:
"If I change the implementation details of this method (without altering the public contract) will it break my test(s)?"
If the answer is yes, you are testing the HOW and not the WHAT.
To answer your specific question about testing code with file system dependencies, let's say you had something a bit more interesting going on with a file and you wanted to save the Base64 encoded contents of a byte[] to a file. You can use streams for this to test that your code does the right thing without having to check how it does it. One example might be something like this (in Java):
interface StreamFactory {
OutputStream outStream();
InputStream inStream();
}
class Base64FileWriter {
public void write(byte[] contents, StreamFactory streamFactory) {
OutputStream outputStream = streamFactory.outStream();
outputStream.write(Base64.encodeBase64(contents));
}
}
#Test
public void save_shouldBase64EncodeContents() {
OutputStream outputStream = new ByteArrayOutputStream();
StreamFactory streamFactory = mock(StreamFactory.class);
when(streamFactory.outStream()).thenReturn(outputStream);
// Run the method under test
Base64FileWriter fileWriter = new Base64FileWriter();
fileWriter.write("Man".getBytes(), streamFactory);
// Assert we saved the base64 encoded contents
assertThat(outputStream.toString()).isEqualTo("TWFu");
}
The test uses a ByteArrayOutputStream but in the application (using dependency injection) the real StreamFactory (perhaps called FileStreamFactory) would return FileOutputStream from outputStream() and would write to a File.
What was interesting about the write method here is that it was writing the contents out Base64 encoded, so that's what we tested for. For your DoIt() method, this would be more appropriately tested with an integration test.
There's really nothing wrong with this, it's just a question of whether you call it a unit test or an integration test. You just have to make sure that if you do interact with the file system, there are no unintended side effects. Specifically, make sure that you clean up after youself -- delete any temporary files you created -- and that you don't accidentally overwrite an existing file that happened to have the same filename as a temporary file you were using. Always use relative paths and not absolute paths.
It would also be a good idea to chdir() into a temporary directory before running your test, and chdir() back afterwards.
I am reticent to pollute my code with types and concepts that exist only to facilitate unit testing. Sure, if it makes the design cleaner and better then great, but I think that is often not the case.
My take on this is that your unit tests would do as much as they can which may not be 100% coverage. In fact, it may only be 10%. The point is, your unit tests should be fast and have no external dependencies. They might test cases like "this method throws an ArgumentNullException when you pass in null for this parameter".
I would then add integration tests (also automated and probably using the same unit testing framework) that can have external dependencies and test end-to-end scenarios such as these.
When measuring code coverage, I measure both unit and integration tests.
There's nothing wrong with hitting the file system, just consider it an integration test rather than a unit test. I'd swap the hard coded path with a relative path and create a TestData subfolder to contain the zips for the unit tests.
If your integration tests take too long to run then separate them out so they aren't running as often as your quick unit tests.
I agree, sometimes I think interaction based testing can cause too much coupling and often ends up not providing enough value. You really want to test unzipping the file here not just verify you are calling the right methods.
One way would be to write the unzip method to take InputStreams. Then the unit test could construct such an InputStream from a byte array using ByteArrayInputStream. The contents of that byte array could be a constant in the unit test code.
This seems to be more of an integration test as you are depending on a specific detail (the file system) that could change, in theory.
I would abstract the code that deals with the OS into it's own module (class, assembly, jar, whatever). In your case you want to load a specific DLL if found, so make an IDllLoader interface and DllLoader class. Have your app acquire the DLL from the DllLoader using the interface and test that .. you're not responsible for the unzip code afterall right?
Assuming that "file system interactions" are well tested in the framework itself, create your method to work with streams, and test this. Opening a FileStream and passing it to the method can be left out of your tests, as FileStream.Open is well tested by the framework creators.
You should not test class interaction and function calling. instead you should consider integration testing. Test the required result and not the file loading operation.
As others have said, the first is fine as an integration test. The second tests only what the function is supposed to actually do, which is all a unit test should do.
As shown, the second example looks a little pointless, but it does give you the opportunity to test how the function responds to errors in any of the steps. You don't have any error checking in the example, but in the real system you may have, and the dependency injection would let you test all the responses to any errors. Then the cost will have been worth it.
For unit test I would suggest that you include the test file in your project(EAR file or equivalent) then use a relative path in the unit tests i.e. "../testdata/testfile".
As long as your project is correctly exported/imported than your unit test should work.