I'm writing a BDD unit test for a public method. The method changes a private property (private var) so I'd like to write an expect() and ensure it's being set correctly. Since it's private, I can't work out how access it from the unit test target.
For Objective-C, I'd just add an extension header. Are there any similar tricks in Swift? As a note, the property has a didSet() with some code as well.
(Note that Swift 2 adds the #testable attribute which can make internal methods and properties available for testing. See #JeremyP's comments below for some more information.)
No. In Swift, private is private. The compiler can use this fact to optimize, so depending on how you use that property, it is legal for the compiler to have removed it, inlined it, or done any other thing that would give the correct behavior based on the code actually in that file. (Whether the optimizer is actually that smart today or not, it's allowed to be.)
Now of course if you declare your class to be #objc, then you can break those optimizations, and you can go poking around with ObjC to read it. And there are bizarre workarounds that can let you use Swift to call arbitrary #objc exposed methods (like a zero-timeout NSTimer). But don't do that.
This is a classic testing problem, and the classic testing answer is don't test this way. Don't test internal state. If it is literally impossible to tell from the outside that something has happened, then there is nothing to test. Redesign the object so that it is testable across its public interface. And usually that means composition and mocks.
Probably the most common version of this problem is caching. It's very hard to test that something is actually cached, since the only difference may be that it is retrieved faster. But it's still testable. Move the caching functionality into another object, and let your object-under-test accept a custom caching object. Then you can pass a mock that records whether the right cache calls were made (or networking calls, or database calls, or whatever the internal state holds).
Basically the answer is: redesign so that it's easier to test.
OK, but you really, really, really need it... how to do it? OK, it is possible without breaking the world.
Create a function inside the file to be tested that exposes the thing you want. Not a method. Just a free function. Then you can put that helper function in an #if TEST, and set TEST in your testing configuration. Ideally I'd make the function actually test the thing you care about rather than exposing the variable (and in that case, maybe you can let the function be internal or even public). But either way.
Related
I have a repository with a method called ConvertToBusinessEntity which converts the data from the data source to a business object. This method is supposed to be used by other methods in the repository namely the Get, GetAll, etc.
This method is unit tested. I check if the data is being retrieved correctly from the data source and the values are being put in the entity correct properties.
But now I need to test and create the GetEntity method which is supposed to call ConvertToBusinessEntity. The logic behind ConvertToBusiness is tested. I just need to know how to verify that a method in the class being tested is called (not a dependency).
Does anyone know how to do this or any alternative method?
I thought of an alternative method but I am not sure if it's the best. I could extend the class under test and have a counter increasing each time the method is called.
Another one would be to mock the class itself.
What do you think? Any other suggestions?
Having ConvertToBusinessEntity in repository is not very good idea. Responsibility of repository is working with data store (CRUD). Mapping of data types is responsibility of some mapper class. Otherwise your repository has too many responsibilities. Take a look on what you are trying to test:
I check if the data is being retrieved correctly from the data source
and the values are being put in the entity correct properties
You see this and? Your test can fail on two completely different reasons. Also you should change repository on two completely different reasons. Best approach here is persisting business entities directly. Modern ORMs allows doing that without polluting business entity with attributes or forcing it inheriting some data-access specific class.
If you really want to have data mapping logic in repository, then make it private (actually only repository should require conversion of business entity to some data-access object) and don't care how this logic is implemented. Let it be part of internal class implementation. You should care only about repository being able to accept or return filled business entities - that's the responsibility of repository. It doesn't matter how mapping is implemented in repository. You should test what repository does, instead of how. So just check that expected business objects are returned by repository.
I just need to know how to verify that a method in the class being tested is called (not a dependency).
But do you really need to do that? If your GetEntity method operates correctly, do you really care how it operates? Do you really care if it performs its function by delegating to ConvertToBusiness, or by some other means?
I recommend instead that you
Think of each method as having a specification.
That specification describes what the outputs and publicly visible manipulations it must make. That do not describe how a method performs its function; that is an implementation detail that could change.
Your unit tests check only that your methods conform to their specification.
You might nevertheless use your knowledge about the implementation to choose good test cases.
But, you might declare, if I do that I am not unit testing my method code; my test of GetEntity depends on both the GetEntity method and the ConvertToBusiness method: two units, so an integration test rather than a unit test. But do you mock the methods of the runtime environment? Of course not. The line between unit and integration testing is not so clear.
More philosophically, you can not create good mock objects in many cases. The reason is that, for most methods, the manner in which an object delegates to associated objects is undefined. Whether it does delegate, and how, is left by the specification as an implementation detail. The only requirement is that, on delegating, the method satisfies the preconditions of its delegate. In such a situation, only a fully functional (non-mock) delegate will do. If the real object checks its preconditions, failure to satisfy a precondition on delegating will cause a test failure. And debugging that test failure will be easy.
I am very sorry if this question seems stupid, i am a newbie to TCL and TCLtest, I am trying to perform unit test on a few TCLOO programs, and i am having difficulties testing the private methods ( the methods called using keyword 'my' ). Guidance needed
Leaving aside the question of whether you should test private methods, you can get at the methods by one of these schemes:
Use [info object namespace $inst]::my $methodname to call it, which takes advantage of the fact that you can use introspection to find out the real name of my (and that's guaranteed to work; it's needed for when you're doing callbacks with commands like vwait, trace, and Tk's bind).
Use oo::objdefine $inst export $methodname to make the method public for the particular instance. At that point, you can just do $inst $methodname as normal.
Consequence: You should not use a TclOO object's private methods for things that have to be protected heavily (by contrast with, say, a private field in a Java object). The correct level for handling such shrouding of information is either to put it in a master interpreter (with the untrusted code evaluating in a safe slave) or to keep the protected information at the underlying implementation (i.e., C) level. The best option of those two depends on the details of your program; it's usually pretty obvious which is the right choice (you don't write C just for this if you're otherwise just writing Tcl code).
This might look like OT, but bear with me.
Are you sure you have to test private methods? That sounds like testing the implementantion, and thats something you shouldnt do. You should be testing the behavior of your class, and that is tested through its public methods.
If you have a complicated chunk of code in one of the private methods, and you feel it needs to be tested spearately, consider refactoring the code into two separate classes. Make the method that needs testing public in one of the two classes.
That way you avoid having a "god class" that does everything and you get to test what you wanted to test. You might want to read more about Single Responsibility Principle.
If you need specific book titles on refactoring, id recommend "Clean Code" by Robert C. Martin. I love that book!
If a function just calls another function or performs actions. How do I test it? Currently, I enforce all the functions should return a value so that I could assert the function return values. However, I think this approach mass up the API because in the production code. I don't need those functions to return value. Any good solutions?
I think mock object might be a possible solution. I want to know when should I use assert and when should I use mock objects? Is there any general guide line?
Thank you
Let's use BufferedStream.Flush() as an example method that doesn't return anything; how would we test this method if we had written it ourselves?
There is always some observable effect, otherwise the method would not exist. So the answer can be to test for the effect:
[Test]
public void FlushWritesToUnderlyingStream()
{
var memory = new byte[10];
var memoryStream = new MemoryStream(memory);
var buffered = new BufferedStream(memoryStream);
buffered.Write(0xFF);
Assert.AreEqual(0x00, memory[0]); // not yet flushed, memory unchanged
buffered.Flush();
Assert.AreEqual(0xFF, memory[0]); // now it has changed
}
The trick is to structure your code so that these effects aren't too hard to observe in a test:
explicitly pass collaborator objects,
just like how the memoryStream is passed
to the BufferedStream in the constructor.
This is called dependency
injection.
program against an interface, just
like how BufferedStream is programmed
against the Stream interface. This enables
you to pass simpler, test-friendly implementations (like MemoryStream in this case) or use a mocking framework (like MoQ or RhinoMocks), which is all great for unit testing.
Sorry for not answering straight but ... are you sure you have the exact balance in your testing?
I wonder if you are not testing too much ?
Do you really need to test a function that merely delegates to another?
Returns only for the tests
I agree with you when you write you don't want to add return values that are useful only for the tests, not for production. This clutters your API, making it less clear, which is a huge cost in the end.
Also, your return value could seem correct to the test, but nothing says that the implementation is returning the return value that corresponds to the implementation, so the test is probably not proving anything anyway...
Costs
Note that testing has an initial cost, the cost of writing the test.
If the implementation is very easy, the risk of failure is ridiculously low, but the time spend testing still accumulates (over hundred or thousands cases, it ends up being pretty serious).
But more than that, each time you refactor your production code, you will probably have to refactor your tests also. So the maintenance cost of your tests will be high.
Testing the implementation
Testing what a method does (what other methods it calls, etc) is critized, just like testing a private method... There are several points made:
this is fragile and costly : any code refactoring will break the tests, so this increases the maintenance cost
Testing a private method does not bring much safety to your production code, because your production code is not making that call. It's like verifying something you won't actually need.
When a code delegates effectively to another, the implementation is so simple that the risk of mistakes is very low, and the code almost never changes, so what works once (when you write it) will never break...
Yes, mock is generally the way to go, if you want to test that a certain function is called and that certain parameters are passed in.
Here's how to do it in Typemock (C#):
Isolate.Verify.WasCalledWithAnyArguments(()=> myInstance.WeatherService("","", null,0));
Isolate.Verify.WasCalledWithExactArguments(()=> myInstance. StockQuote("","", null,0));
In general, you should use Assert as much as possible, until when you can't have it ( For example, when you have to test whether you call an external Web service API properly, in this case you can't/ don't want to communicate with the web service directly). In this case you use mock to verify that a certain web service method is correctly called with correct parameters.
"I want to know when should I use assert and when should I use mock objects? Is there any general guide line?"
There's an absolute, fixed and important rule.
Your tests must contain assert. The presence of assert is what you use to see if the test passed or failed. A test is a method that calls the "component under test" (a function, an object, whatever) in a specific fixture, and makes specific assertions about the component's behavior.
A test asserts something about the component being tested. Every test must have an assert, or it isn't a test. If it doesn't have assert, it's not clear what you're doing.
A mock is a replacement for a component to simplify the test configuration. It is a "mock" or "imitation" or "false" component that replaces a real component. You use mocks to replace something and simplify your testing.
Let's say you're going to test function a. And function a calls function b.
The tests for function a must have an assert (or it's not a test).
The tests for a may need a mock for function b. To isolate the two functions, you test a with a mock for function b.
The tests for function b must have an assert (or it's not a test).
The tests for b may not need anything mocked. Or, perhaps b makes an OS API call. This may need to be mocked. Or perhaps b writes to a file. This may need to be mocked.
So, I'm starting to write some logic for a simple program (toy game on the side). You have a specific ship (called a setup) that is a ship + modules. You start with an empty setup based off a ship and then add modules to that setup. Ships also have a numbered array of module positions.
var setup = new Setup(ship); // ship is a stub (IShip) defined someplace else
var module = new Mock<IModule>().Object;
setup.AddModule(module, 1); // 1 = which position
So, this is the code in my test method. I now need to assert on this code. Well, I need a getter method right?
Assert.AreEqual(module, setup.GetModule(1));
This might sound really dumb and I'm worrying about nothing, but for some stupid reason I'm concerned with adding a method just to assert that a test passed.
Is this fine and is in fact part of the design process that TDD is pushing out? For instance I know I need an AddModule method because I want to test it, and the fact that this requires a GetModule method to test is simply an evolution of my design via TDD.
Or is this kind of a smell because I don't even know if I'll really need GetModule in my code and it will only be used in a test?
For example, adding a module is going to ultimately affect different stats of a setup (armor, shield, firepower, etc). The thing is those are going to be complex, and I wanted to start with a simple test. But in the end, those are the public attributes I care about -- a setup is defined by its stats, not by a list of modules.
Interesting question. I'm glad to hear you're writing the tests first.
If you let the design manifest itself through the tests, you're more likely to build only the parts you'll need. But is this the best design? Maybe not, but don't let that discourage you -- your add method works!
It may be too early to tell if you'll need the GetModule method later. For now, build up the functionality you need and go green, then slowly refactor it (going from red to green again) to get the design you want.
Part of evolving the design is to start with baby steps like a simple method and then grow into the complex stats (eventually dropping this method and changing the test) when enough supports it. When doing TDD, don't expect that the first test you write is targeting the ideal interface. It is OK to have some messiness that will get dropped as you evolve the design.
That being said, if you see no public purpose to the method, try to limit its visibility as much as is reasonable to the test code. Although even that should eventually go away as you get to build out the rest of the system and have something real to test as a side effect of the set method.
I would be wary of introducing a public method in my class that is only used for testing.
There are various ways how you could test this:
Reflection: The GetModule method is a private method in your class (this could also work if your 'stats' are private) and you can access it in your test method via reflection. This will work well, the only trouble is you will not get any compiler errors if you change the name of the private method or add / delete some variables (but, of course, your test will fail and you will know early)
Inheritance: The GetModule method could be protected (only inheritance visible) and your test class could inherit from the main class. This way your test class gets access to this method, but this is not really exposed to the outside world.
Assert the side-effect: This is where you really think about what it means to add a module to the system. If it is going to affect some 'stats' as you put it, you could write tests which assert that the stats are appropriately modified.
So I have a factory class and I'm trying to work out what the unit tests should do. From this question I could verify that the interface returned is of a particular concrete type that I would expect.
What should I check for if the factory is returning concrete types (because there is no need - at the moment - for interfaces to be used)? Currently I'm doing something like the following:
[Test]
public void CreateSomeClassWithDependencies()
{
// m_factory is instantiated in the SetUp method
var someClass = m_factory.CreateSomeClassWithDependencies();
Assert.IsNotNull(someClass);
}
The problem with this is that the Assert.IsNotNull seems somewhat redundant.
Also, my factory method might be setting up the dependencies of that particular class like so:
public SomeClass CreateSomeClassWithDependencies()
{
return new SomeClass(CreateADependency(), CreateAnotherDependency(),
CreateAThirdDependency());
}
And I want to make sure that my factory method sets up all these dependencies correctly. Is there no other way to do this then to make those dependencies public/internal properties which I then check for in the unit test? (I'm not a big fan of modifying the test subjects to suit the testing)
Edit: In response to Robert Harvey's question, I'm using NUnit as my unit testing framework (but I wouldn't have thought that it would make too much of a difference)
Often, there's nothing wrong with creating public properties that can be used for state-based testing. Yes: It's code you created to enable a test scenario, but does it hurt your API? Is it conceivable that other clients would find the same property useful later on?
There's a fine line between test-specific code and Test-Driven Design. We shouldn't introduce code that has no other potential than to satisfy a testing requirement, but it's quite alright to introduce new code that follow generally accepted design principles. We let the testing drive our design - that's why we call it TDD :)
Adding one or more properties to a class to give the user a better possibility of inspecting that class is, in my opinion, often a reasonable thing to do, so I don't think you should dismiss introducing such properties.
Apart from that, I second nader's answer :)
If the factory is returning concrete types, and you're guaranteeing that your factory always returns a concrete type, and not null, then no, there isn't too much value in the test. It does allows you to make sure, over time that this expectation isn't violated, and things like exceptions aren't thrown.
This style of test simply makes sure that, as you make changes in the future, your factory behaviour won't change without you knowing.
If your language supports it, for your dependencies, you can use reflection. This isn't always the easiest to maintain, and couples your tests very tightly to your implementation. You have to decide if that's acceptable. This approach tends to be very brittle.
But you really seem to be trying to separate which classes are constructed, from how the constructors are called. You might just be better off with using a DI framework to get that kind of flexibility.
By new-ing up all your types as you need them, you don't give yourself many seams (a seam is a place where you can alter behaviour in your program without editing in that place) to work with.
With the example as you give it though, you could derive a class from the factory. Then override / mock CreateADependency(), CreateAnotherDependency() and CreateAThirdDependency(). Now when you call CreateSomeClassWithDependencies(), you are able to sense whether or not the correct dependencies were created.
Note: the definition of "seam" comes from Michael Feather's book, "Working Effectively with Legacy Code". It contains examples of many techniques to add testability to untested code. You may find it very useful.
What we do is create the dependancies with factories, and we use a dependancy injection framework to substitute mock factories for the real ones when the test is run. Then we set up the appropriate expectations on those mock factories.
You can always check stuff with reflection. There is no need to expose something just for unit tests. I find it quite rare that I need to reach in with reflection and it may be a sign of bad design.
Looking at your sample code, yes the Assert not null seems redundant, depending on the way you designed your factory, some will return null objects from the factory as opposed to exceptioning out.
As I understand it you want to test that the dependencies are built correctly and passed to the new instance?
If I was not able to use a framework like google guice, I would probably do it something like this (here using JMock and Hamcrest):
#Test
public void CreateSomeClassWithDependencies()
{
dependencyFactory = context.mock(DependencyFactory.class);
classAFactory = context.mock(ClassAFactory.class);
myDependency0 = context.mock(MyDependency0.class);
myDependency1 = context.mock(MyDependency1.class);
myDependency2 = context.mock(MyDependency2.class);
myClassA = context.mock(ClassA.class);
context.checking(new Expectations(){{
oneOf(dependencyFactory).createDependency0(); will(returnValue(myDependency0));
oneOf(dependencyFactory).createDependency1(); will(returnValue(myDependency1));
oneOf(dependencyFactory).createDependency2(); will(returnValue(myDependency2));
oneOf(classAFactory).createClassA(myDependency0, myDependency1, myDependency2);
will(returnValue(myClassA));
}});
builder = new ClassABuilder(dependencyFactory, classAFactory);
assertThat(builder.make(), equalTo(myClassA));
}
(if you cannot mock ClassA you can assign a non-mock version to myClassA using new)