What is the practical purpose of re-running unit tests throughout development? - unit-testing

Question

I've read over and over that it's important to create unit tests for all (or at least most) of your methods, and run them repeatedly throughout development. This made perfect sense to me at first, but now as I'm beginning to implement these tests myself I'm feeling less sure. From what I can see once you've made a test pass, it will always pass, since all the data it is using is mocked up. I feel like there's something I'm not getting.
Let's say you write a method like this:
/* Verifies email address (just for illustration, not robust code) */
bool VerifyEmail(String email){
return Regex.Match(email, "^\w+#\w+\.com$");
}
Maybe you would write a unit test like this:
/* Again, not robust, just for illustration */
void TestVerifyEmail(){
Dictionary<String, bool> testCases = new Dictionary<String, bool>(
{"fake#fake.com", true},
{"fake#!!!.com", false},
{"#fake.com", false},
{"fake#fake.cme", false}
);
forEach(String test in testCases.Keys()){
Test.Assert(VerifyEmail(test) == testCases[test]);
}
}
Unless you go and change the test cases, the results of the test function will never ever change, no matter what else happens to the rest of the code, because VerifyEmail() is isolated.
This is an especially simple case, but in most unit test examples I see, even ones that are meant not to operate in a vacuum, they always use totally mocked-up data, and so the test results will never change unless the test itself is changed.
What is the point of running unit tests over and over if, it seems to me, the results will never change? Since all the tests place the chunk of code they're testing into an isolated environment with mocked-up data, the unit tests will pass every time.
I totally get writing unit tests when initially creating the code to ensure it works the way you want, such as in TDD, but once you've done that, what's the point to ever running it again later?

Answer 1

Ideally, unit tests aren't written to verify that the code you wrote works; unit tests are written to ensure that requirements are met. If the unit test suite includes test coverage for every possible requirement (positive and negative) then once you write enough code to pass every test, your project is complete. The benefit to this is that if additional requirements are added later, someone can refactor the project to add in the additional lines and as long as every unit test passes, then the original requirements are still accomplished.

Answer 2

You have a valid point. Theoretically you could use a dependencies to only rerun tests that could have broken with new changes. In practice, though we don't. Why? Mostly because we don't trust that we could write the dependencies correctly. Remember that we are talking about functional dependencies, not just include-headers. I do not know a tool to auto generate those.

Answer 3

It may help to keep in mind that there are two common definitions of "unit test". Both share the constraints that tests should be fast, deterministic, isolated from each other.
There is a school that adds an additional constraint that the system under test should be isolated from all other collaborators in the system. But that definition isn't universal -- it's normal in the "Chicago Style" for the system under test to be a composite made from many different parts.
Martin Fowler:
As xunit testing became more popular in the 2000's the notion of solitary tests came back, at least for some people. We saw the rise of Mock Objects and frameworks to support mocking. Two schools of xunit testing developed, which I call the classic and mockist styles. One of the differences between the two styles is that mockists insist upon solitary unit tests, while classicists prefer sociable tests.
When private implementation details of the system under test can be refactored into separate parts, it becomes less cost effective to track which tests are dependent on which fragments of production code.
Furthermore, you are regularly going to be running some unit tests; at a minimum, after each refactoring you should be checking that you didn't introduce a regression, so you should be running all of the tests that depend on the code you just changed.
BUT, we're talking about tests that are fast and isolated from one another. Given that you are already taking a moment to run some tests, the marginal costs of running "more" tests are pretty small.
Of course, small isn't zero; I believe that in most cases developers use a weighting strategy to determine how often a tests needs to be run; run the tests that are likely to detect a problem more of than those that aren't likely to detect a problem, conditioned on what part of the code base you are actively working on.