Unit tests are often deployed with software releases to validate the install - i.e. do the install, run the tests and if they pass then the install is good.
I'm about to embark on a project that will involve delivering prototype software library releases to customers. The unit tests will be delivered as part of each release and in addition to using the tests to validate the install, I plan on using the unit tests that test the API as a "contract" for how the release should be used. If the user uses the release in a similar manner to how it is used by the unit tests then great. If they use it some other way then all bets are off.
Has anybody tried this before? Any thoughts on whether this is a good/bad idea?
Edit: To highlight a good point raised by ChrisA and Dan in replies below, the "unit tests that test the API" are better called the integration tests and their intent is to exercise the API and the software to demonstrate the functionality of the software from a customer perspective.
Sounds like a good idea to me. I (we all?) routinely use unit tests internally to do just that. In using my unit tests to validate that I haven't broken anything I'm also implicitly verifying that my API contract hasn't changed. It seems like a natural usage of unit tests to deploy them in the fashion you're talking about.
Agile methodologies say: Tests are specifications, so this is a very good idea.
I fully expect to be flamed for this, but I don't understand how a set of unit tests proves anything at all about the kind of things a customer cares about, namely whether the application meets his business requirements.
Here's an example: I've just finished converting a chunk of code to fix a big mistake we made. It was a classic case of over-engineering, and the changes have touched about a dozen windows forms and about as many classes.
It's taken me a couple of days, it's now a lot simpler, we gained some features for free, and we lost a ton of code that did stuff that we now know we never really needed.
Every single one of those forms worked perfectly before. The public methods did exactly what they needed to do, and the underlying data accesses were just fine.
So any unit test would have passed.
Except, sadly, they did the wrong thing - which we didn't realise, except in retrospect. It's as if we'd built a prototype and only after trying to use it, realised that it wasn't right.
So now we have a leaner, meaner, fitter application.
But the things that were wrong, were wrong at a level where unit tests could never have revealed them, so I'm just not understanding how shipping a set of unit tests with an install does anything except give a false sense of security.
Maybe I'm not understanding something, but it seems to me that unless the thing that is shipped functions at the same level as the tests supplied, they prove nothing.
It's actually a pretty good idea, and extremely pleasant as an API user.
This technique can actually also be used the other way round : when you're using a "legacy" API, you can use unit tests to document the way you think the API behaves and to validate that it actually behaves as planned.
If you are releasing a code library, this sounds great.
If you are releasing an ordinary software product with which your users will interact only via a GUI, your unit tests may not be working at the same level of abstraction or may not the most useful tool to assess the behaviour of your product. A really good user manual (yes, this is possible) might be better for that.
If you're interested in providing a set of specifications with your code, perhaps you should investigate some of the behavior-driven development tools (nbehave, jbehave, rspec, etc.). These frameworks provide support for describing your tests in given/when/then syntax and outputting formatted results that are in a natural language. See nbehave for an example of a BDD tool for .NET. You can find an excellent description of BDD here
Another option may be for you to write tests using an acceptance testing framework such as fit or fitnesse (or the java-only concordion) and deliver these acceptance tests with the code. Both fit/fitnesse and concordion allow specification of the tests in plain HTML or even Word documents.
The benefit of either approach (BDD or acceptance testing frameworks) is that the results the user sees are more human-readable and understandable.
Tests will check requirements.
Requirements define functionality
=> Tests will check functionality.
The problem is, that only functionality can be checked that can be covered by unit tests. Integration or whole system tests won't work.
Otherwise, it's the main approach of TDD to check functionality via unit tests.
Meszaros calls this "Tests as documentation"
Related
As I am writing tests, some of them have a lot of logic in them. Most of this logic could easily be unit tested, which would provide a higher level of trust in the tests.
I can see a way to do this, which would be to create a class TestHelpers, to put in /classes, and write tests for TestHelpers along with the regular tests.
I could not find any opinion on such a practice on the web, probably because the keywords to the problem are tricky ("tests for tests").
I am wondering whether this sounds like good practice, whether people have already done this, whether there is any advice on that, whether this points to bad design, or something of the sort.
I am running into this while doing characterization tests. I know there are some frameworks for it, but I am writing it on my own, because it's not that complicated, and it gives me more clarity. Also, I can imagine that one can easily run into the same issue with unit tests.
To give an example, at some point I am testing a function that connects to Twitter's API service and retrieves some data. In order to test that the data is correct, I need to test whether it's a json encoded string, whether the structure matches twitter's data structure, whether each value has the correct type, etc. The function that does all these checks with the retrieved data would typically be interesting to test on its own.
Any idea or opinion on this practice ?
One of the aphorisms about TDD is that "the tests test the code, and the code tests the tests." That is, because of the red-green-refactor cycle, you see the code fail the test and then after making it work, you see it pass the test - and that alone is enough to give you pretty good confidence that the test (and all the test utility code it calls) works correctly. For characterization tests, you don't have this red-green-refactor cycle, so it may be of value for you to write tests for your test utility methods.
I think it's too much to test tests themselves. alfasin right, who would test tests for tests then? And it is no coincidence that you can't find much info on this topic. That's because it's just not a common practice. Usually, well-written tests should cover arrangement logic in tests themselves. But I understand your aspiration - how to be sure that test is "well-written"? The most dangerous thing here is to have passing test, that should normally fail(but passes due to bug in it). Having such a test is even worse than not having test at all. But to be honest, I did not have much such cases in practice. My advice is just to focus on writing good tests that cover all execution paths of your logic and I think you'll be fine :)
While it sounds perverse, I have on occasion written automated tests for some of my testing infrastructure, if it gets fairly complex. The tests that test this testing infrastructure then tend to be simple, so the "what about testing the tests for the test?" question becomes moot, in my experience.
Note that this is mainly occurring in a library designed explicitly to aid testing for other people (people writing Qt apps, in this case), though I have done it for some stand-alone apps before: for example, when writing tests for Kate's Vim mode's auto-completion integration, the fake auto-completer test-helper code used for mimicking the auto-completion for a variety of configurations got complex enough that I actually started developing it test-first.
And it's probably worth mentioning that e.g. Google Mock has hundreds of tests written for it :)
The tests at my new job are nothing like the tests I have encountered before.
When they're writing their unit tests (presumably before the code), they create a class starting with "When". The name describes the scenario under which the tests will run (the fixture). They'll created subclasses for each branch through the code. All of the tests within the class start with "should" and they test different aspects of the code after running. So, they will have a method for verifying that each mock (DOC) is called correctly and for checking the return value, if applicable. I am a little confused by this method because it means the exact same execution code is being run for each test and this seems wasteful. I was wondering if there is a technique similar to this that they may have adapted. A link explaining the style and how it is supposed to be implemented would be great. I sounds similar to some approaches of BDD I've seen.
I also noticed that they've moved the repeated calls to "execute" the SUT into the setup methods. This causes issues when they are expecting exceptions, because they can't use built-in tools for performing the check (Python unittest's assertRaises). This also means storing the return value as a backing field of the test class. They also have to store many of the mocks as backing fields. Across class hierarchies it becomes difficult to tell the configuration of each mock.
They also test code a little differently. It really comes down to what they consider an integration test. They mock out anything that steals the context away from the function being tested. This can mean private methods within the same class. I have always limited mocking to resources that can affect the results of the test, such as databases, the file system or dates. I can see some value in this approach. However, the way it is being used now, I can see it leading to fragile tests (tests that break with every code change). I get concerned because without an integration test, in this case, you could be using a 3rd party API incorrectly but your unit tests would still pass. I'd like to learn more about this approach as well.
So, any resources about where to learn more about some of these approaches would be nice. I'd hate to pass up a great learning opportunity just because I don't understand they way they are doing things. I would also like to stop focusing on the negatives of these approaches and see where the benefits come in.
If I understood you explanation in the first paragraph correctly, that's quite similar to what I often do. (Depending on whether the testing framework makes it easy or not. Also many mocking frameworks don't support it, but spy frameworks like Mockito do better.)
For example see the stack example here which has a common setup (adding things to the stack) and then a bunch of independent tests which each check one thing. Here's still another example, this time one where none of the tests (#Test) modify the common fixture (#Before), but each of them focuses on checking just one independent thing that should happen. If the tests are very well focused, then it should be possible to change the production code to make any single test fail while all other tests pass (I wrote about that recently in Unit Test Focus Isolation).
The main idea is to have each test check a single feature/behavior, so that when tests fail it's easier to find out why it failed. See this TDD tutorial for more examples and to learn that style.
I'm not worried about the same code paths executed multiple times, when it takes a millisecond to run one test (if it takes more than a couple of seconds to run all unit tests, the tests are probably too big). From your explanation I'm more worried that the tests might be too tightly coupled to the implementation, instead of the feature, if it's systematic that there is one test for each mock. The name of the test would be a good indicator of how well structured or how fragile the tests are - does it describe a feature or how that feature is implemented.
About mocking, a good book to read is Growing Object-Oriented Software Guided by Tests. One should not mock 3rd party APIs (APIs which you don't own and can't modify), for the reason you already mentioned, but one should create an abstraction over it which better fits the needs of the system using it and works the way you want it. That abstraction needs to be integration tested with the 3rd party API, but in all tests using the abstraction you can mock it.
First, the pattern that you are using is based on Cucumber - here's a link. The style is from the BDD (Behavior-driven development) approach. It has two advantages over traditional TDD:
Language - one of the tenants of BDD is that the language you use influences the thoughts you have by forcing you to speak in the language of the end user, you will end up writing different tests than when you write tests from the focus of a programmer
Tests lock code - BDD locks the code at the appropriate level. One problem common in testing is that you write a large number of tests, which makes your codebase more brittle as when you change the code you must also change a large number of tests too. BDD forces you to lock the behavior of your code, rather than the implementation of your code. This way, when a test breaks, it is more likely to be meaningful.
It is worth noting that you do not have to use the Cucumber style of testing to achieve these affects and using it does add an extra layer of overhead. But very few programmers have been successful in keeping the BDD mindset while using traditional xUnit tools (TDD).
It also sounds like you have some scenarios where you would like to say 'When I do , then verify '. Because the current BDD xUnit frameworks only allow you to verify primitives (strings, ints, doubles, booleans....), this usually results in a large number of individual tests (one for each Assert). It is possible to do more complicated verifications using a Golden Master paradigm test tool, such as ApprovalTests. Here's a video example of this.
Finally, here's a link to Dan North's blog - he started it all.
I've been writing software for years, but have never mastered the art of testing. My typical testing includes thorough run-throughs on my machines, and then testing in various operating systems via VMware. Mainly a brute-force play-with-it-until-it-breaks-or-doesn't approach. Where possible I work on actual hardware, but this isn't always practical.
My question is twofold:
How do medium-sized professional development houses do their testing?
What common techniques or procedures (outside of unit testing) can apply to a developer team of one. I'm looking for practicality.
Thank you for your time and input.
Step 1: Unit Testing
Divide your software into components (which can be anything from single functions up to whole programs) and unit-test those components thoroughly, especially as relates to the API and behavior that the rest of the application can see. (Don't forget to check for failure modes too, but beware of binding too carefully to the exact nature of a failure; it's often good enough to just test for the presence of the right class of exception rather than its exact message.) Make sure those tests pass; you're honing them in on a specification of what the component should be doing. (Automated test running helps here, as does a CI system.) This is important because of…
Step 2: Integration Testing
Test that the compositions of components that make up the application work (this is integration testing). Ideally you'll only be finding bugs in the specifications of things at this point (hah!) and wherever you identify that a component is wrong despite passing its unit tests, that tells you that there is a bug. Whenever things fail to work together despite being told to do so, you've probably got a bug in your specs from the previous step so you typically resolve these things by adding more detail to your unit tests and fixing the components until they work.
Note that to make good integration, you want to keep this stage so that the integration itself is sufficiently simple that it is in the “Obviously No Bugs” class of programs instead of the larger “No Obvious Bugs” class. An integration framework like Spring or a scripting language can help a lot here (though with the latter you have to guard against creating components on the sly; if you create a component then admit it and make sure it has a proper usage contract and unit tests to ensure that it meets its contract).
Where you can, you can make components by composing others together; these higher-level components need to be unit tested as characterized in Step 1 above. This might sound like extra work – it probably is – but it does have the advantage of meaning that you can use automated tests for larger parts of the program. (Alas, it's harder to do all integration tests with an automated test tool; such things tend to work better doing unit tests where you can mock out all the irrelevant parts.) But this doesn't save you from…
Step 3: Acceptance Testing
This is where the overall application is tested to see if it actually does what is desired. This might be automatable, but usually isn't. This is level where you bring in users to let them see whether things are what they expected, though you might want to use internal testers a bit first. How easy this all is depends on the nature of the application.
Note also that user interfaces tend to spend more time in this step than the others, precisely because what makes for a good UI is difficult to impossible to pin down in algorithms (it relates much more to human psychology after all).
A final note: What I've written here sounds like testing is meant to be a laborious process that takes ages at the end of a project. It isn't! You can often get parts of an application done before others, do an integration of those parts (with mocks for the other bits) and test quite a bit of how acceptable this sub-application really is. Of course, when doing this take care to stop users from believing that everything is done; one way is to have dialog boxes that pop up and say things like “magic to happen here”. Silly but effective. :-)
For a small team unit tests or automatic integration tests are crucial. Because there are no hands and time for manual testing - the more you automate the better. This includes Continuous Integration.
Set up a separate 'beta' environment that is as close to your production environment as possible. Do most of your tests there - this way you will pick up all the things you've forgotten in your 'release plan'.
As a proffesional tester my suggestion is that you should have a healthy mix of automated and manual testing. The Examples below are in .net but it should be easy to find a tool for whatever technique you are using.
AUTOMATED TESTING
Unit Testing
Use NUnit to test your classes, functions and interaction between them.
http://www.nunit.org/index.php
Automated Functional Testing
If it's possible you should automate a lot of the functional testing. Some frame works have functional testing built into them. Otherwise you have to use a tool for it. If you are developing web sites/applications you might want to look at Selenium.
http://www.peterkrantz.com/2005/selenium-for-aspnet/
Continuous Integration
Use CI to make sure all your automated tests run every time someone in your team makes a commit to the project.
http://martinfowler.com/articles/continuousIntegration.html
MANUAL TESTING
As much as I love automated testing it is, IMHO, not a substitute for manual testing. The main reason being that an automated can only do what it is told and only verify what it has been informed to view as pass/fail. A human can use it's intelligence to find faults and raise questions that appear while testing something else.
Exploratory Testing
ET is a very low cost and effective way to find defects in a project. It take advantage of the intelligence of a human being and a teaches the testers/developers more about the project than any other testing technique i know of. Doing an ET session aimed at every feature deployed in the test environment is not only an effective way to find problems fast, but also a good way to learn and fun!
http://www.satisfice.com/articles/et-article.pdf
My testing tool examples are Java based, but I will try to suggest tools which are ported to multiple languages or are language agnostic.
Use unit testing tools like junit (ported to a variety of languages). This will allow you to refactor your code safely. Most code bugs should result in the addition or correction of at least one test.
Use revision control, and setup an automated build environment that check out the code and builds the code. It should then run the automated test suite. If the application uses a database the build environment should have its own database. Use different code branched for production (released) and development code.
Use integration testing tools like HTTPunit or Synergy to test web applications. Tools of this type are basically language agnostic, but your may want to choose a tool which can be extended in the language(s) you are using. For non-web applications, there may not be an equivelent tool for your platform. You may also want to use a performance tool like JMeter.
These tools have some setup costs, but a quick payback. Overall development time may be the same or less than if you didn't use the tools.
Acceptance tests generally don't lend themselves to automated testing. Where they do, included them in the integrated testing. Get acceptance feedback as early and often as possible.
How do the pros do it? That all depends on who the 'Pro' is... There are dozens of different approaches to testing, and plenty of experts to tell you that their way is the one true way. Agile gurus will tell you a very different story from the waterfall gurus. The ISTBQ guys will tell you a very different from the Context-Driven guys.
Unfortunately there is no one true way, and you have to figure it out for yourself. Your approach to testing depends on too many factors. That's probably not very helpful, but you just need to be aware that any answer you get here will be only one option of many, and it may be completely inappropriate for your situation.
Personally, after several years in software testing, I have decided to align myself with the Context Driven school of software testing. See: http://www.context-driven-testing.com
Secondly, from your description of you current approach, that sounds a lot like exploratory testing to me. You may find this material interesting: satisfice.com/sbtm/
One thing you can do (combined with all the previous suggestions) is identify the risky and critical areas of your app and try to focus your testing efforts on these areas.
Do you think Unit Tests are a good way to show your fellow programmers how to use an API?
I was listening to the Stackoverflow Podcast this week and I now realize that unit testing is not appropriate in all situations (I.E. it can cost you time if you go for 100% code-coverage). I agree with this as I have suffered from the "OCD code coverage disorder in the past), and have now mended my ways.
However to further appropriate my knowledge of the subject, I'd like to know if unit testing is a good way to bring in new programmers that are unfamiliar with the project's APIs. (It sure seems easier than just writing documentation...although I like it when there's documentation later...)
I think Unit testing is a fantastic way to document APIs. It doesn't necessarily replace good commenting or API docs but it is a very practical way to help people get into the nitty gritty of your code. Moreover good unit testing promotes good design, so chances are your code will be easier to understand as a result.
Unit testing, IMO, isn't a substitute for documentation in any way. You write tests to find and exercise the corner cases of your code, to make sure that all boundary conditions are met. These are usually not the most appropriate examples to give to someone who is trying to learn how the method works.
You also typically don't give as much explanation of why what's happening is happening in a unit test.
Finally, the contents of unit tests typically aren't as accessable to documentation generation tools, and are usually separated from the code they're testing (oddities like Python's doctests notwithstanding).
Good documentation usually includes good examples. It's hard for me to imagine a better set of examples than exactly the ones that show what's expected of a correct implementation!
In addition, maintenance is a crucial issue. It's good practice to deal with a defect by adding a test that exposes the defect, and then by making that test succeed without failing prior tests. If unit tests are regarded as part of the documentation, then this new test will be part of the documentation's change log, thus helping subsequent developers learn from the experience.
No. Tests can be used as a secondary reference, they at least have the benefit of being examples that actually compile, but they are not a substitute for good API documentation.
Beware of zealots claiming almost mystical powers for unit tests saying things like "the tests are the design", "the tests are the specification", "the tests are the documentation". No. The tests are the tests and they don't give you an excuse to cut corners elsewhere.
Documentation should not be substituted with unit test code, period.
Documentation is geared towards the programmer that is trying to use, and understand, your API.
Unit tests are geared towards covering corner cases, that although they should be documented, should not be the focus for a new user of your API.
Documentation should have a gradual build-up of complexity, unit tests should be as simple as possible yet as complex as necessary in order to test the functionality.
For instance, you might have many unit tests that look very alike, but have minute differences, just to cover various oddball corner cases and assert the correct behavior.
Rather than having the user of your API decipher these unit tests, figure out the differences, and figure out why this should produce different (or possibly the same) behavior is not a good teaching aid.
Unit tests are for maintainers of your api, the programmers that will fix bugs in it, or add new features to it, or refactor it, or ....
Documentation is for programmers that will use your api.
These are not the same target audiences.
One such subtle difference might be that while a unit test asserts that when a function gets passed a negative value, it will do something specific, the documentation would instead go into details about why that particular solution was picked. If all the documentation does is just rewrite the code into english, then there is not much point in it, so documentation usually is a lot more verbose and explanatory than the code.
Although not a replacement, Unit Tests can be an excellent way of demonstrating the use of a class or API, particularly for library functions where the amount of code to perform tests is minimal.
For example out Math library is well unit-tested. If someone has a question (e.g. how to find information about ray/volume intersections) then the first-place I send them are the unit tests.
(Why do we test something that could be considered so stable? Well for one thing we support five platforms and gradually add platform-specific SIMD implementations based on benchmarking, for another the unit tests provide an excellent framework for adding new platforms or functionality).
FWIW I thought this weeks Podcasts discussion of unit testing was bang-on. Unit Testing (and Test Driven Development) is a great method for developing the components of your application, but for testing the application itself you quickly reach a point where the code needed for testing becomes disproportionate and brittle.
G'day,
I am working with a group of offshore developers who have been using the term unit testing quite loosely.
Their QA document talks about writing unit tests and then performing unit testing of the system.
This doesn't line up with my interpretation of what unit testing is at all.
I am used to unit testing being a test or suite of tests that are being used to exercise a single class, usually as a black box. The class under test may require other classes to be included by the implementation but generally it is a single class that is being exercised by the unit test(s).
Then you have system functional testing, intergration testing, acceptance testing, etc.
I want to know is this a bit pedantic on my part? Or is this what you think of when referring to unit tests and unit testing?
Edit: Rob Wells. I need to clarify that approaching such testing from a black box perspective is only one aspect. When using mock objects to verify internal behaviours, you are really testing from a white box perspective because you know what you want to happen inside the box.
Unit tests are generally used by developers to test isolated sections of code. They cover border cases, error cases, and normal cases. They are intended to demonstrate the correctness of a limited segment of code. If all of your unit tests pass, then you have demonstrated that your isolated segments of code do what they are supposed to do.
When you do integration testing, you are looking at end-to-end cases, to see if all the segments that have passed unit testing work together. Functional testing checks to see if the code meets the requirements as specified. Acceptance testing is done by end users to see if they approve of the final product.
I try to implement unit tests to test only a single method. and I make an effort to crete "mock" classes for dependant classes and methods used by the method I am testing...
... so that the exercise of the code in that method does not in fact call code in other methods the unit test is not supposed to be "Testing" (There are other unit tests for those methods) This way, a failure of the unit test reliably indicates a failure of the method the unit test is testing...
Mock classes are designed to "simulate" the interface and behavior of dependant classes so that the method I am testing can call them and they will behave ina standard, well-defined way according to system requirements. In order to make this approach work, calls to such dependant classes and to their methods must be made on a well defined interface, so that the "tester" process can "inject" the Mock version of teh dependant class into the class being tested instead of the actual production version... . This is kinda like a common design pattern referred to as "Dependency Injection", or "Inversion of Control" (IOC)
There are several third party tools on the market to help you implement this kind of pattern. One I have heard of is called "Rhino-Mock" or something like that...
Edit: Rob Wells. #Charles. Thanks for this. I'd forgotten using mock objects to completely replace using other classes except for the one under test.
A couple of other things I've remembered after you mentioning mock objects is that:
they can be used to simulate errors being returned by the included classes.
they can be used to raise specific exceptions to check exception handling in the class under test.
they can be used to simulate items where setup costs are high, e.g. a large SQL DB back end.
they can be used to verify the contents of an incoming request.
For more information, have a look at Martin Fowler's paper called "Mocks Aren't Stubs" and The Pragmatic Programmers's article "Mock Objects"
There is no reason why unit tests can't span multiple classes, or even submodules, as long as the test is treating only one consistent business operation. Think about "calculateWage", a method of a BO that uses different strategies to calculate the salary of a person. That's one unit test in my opinion.
I have heard of techniques in which many of the unit tests are done first, and development is done around them. Someone has just commented saying that this is "Test Driven Development" - TDD (Thanks Elie).
But if this is an offshore operation that's possibly going to charge you more money because they're spending time doing these unit tests - then I'd be careful. Get a second opinion from someone, experienced with unit tests who will verify that they're actually doing as they say.
From my understanding, unit testing will add a bit more time to any development project, but of course may offer some quality control. Nonetheless, this is the type of quality control I would want with an in house project. This may just be something the offshore company throws out there to give you a warm fuzzy.
There is a difference between the process you use to test and the technology that is used to support it. The various frameworks used for unit testing are generally very flexible and can be used for testing small units of code, large units and even testing entire processes. This flexibility can lead to confusion.
My recommendation is that whatever specific methodology or process you adopt that you segregate the various Unit Test into distinct assemblies or modules. The exact arrangement depends on your code and your company's organization.
The accumulative effect of using the Unit Test framework is that much of the testing of the code is automated. Adopted correctly developers can evaluate changes to the code code better with out going through a full Q&A process. As for the Q & A Process itself it makes their time more productive as the quality of the code coming out of development should be higher.
Understand it is not THE answer to all quality issues it just a useful tool like the other you use.
Wikipedia would seem to suggest that unit testing is about testing the smallest amount of code which would be a method on a class in the case of OO programming.
Some may have a more general term of what they mean by unit tests, where some may think of some integration tests as being unit tests where the unit is a mixture of components.
there is a traditional view of http://en.wikipedia.org/wiki/Software_testing as part of http://en.wikipedia.org/wiki/Software_engineering. but i like the idea of an agile unit test: a test is an agile unit test if it is fast enough so that the programmers always run it.
A unit test is the smallest and only piece of confidence you can get yourself on your way to being done. That is what matters, iteratively building a shield against regression and spec deviation, not how you actually integrate it to your Object-Oriented architecture.
This is almost a repeat of the "What is a 'Unit'?" question.
"Unit" can be defined flexibly. If their document doesn't have a definition of "unit", you'll need to clarify that.
It might be that they think of unit as a big assembly of code. Which is not the most desirable definition.
While I agree that you have several layers of testing (unit, module, package, application), I also think that much of this can be done with unit testing tools. Leading to "what is a unit?" questions coming up all the time.
Unit depends on context. For an individual developer, unit must be Class. Sometimes, it will also mean module or package.
For a team, however, their unit may be a package or a complete application.
What does it matter what we think? The issue here is your unhappiness with the terms they use in the document. Why don't you discuss it with them?
Ten years ago, before the current usage of "unit testing" as tests written in code, the same designation was applied to manual tests. I worked for a software development firm with a very formalized software development process. We had to write "unit tests" before writing any code. In that era, the unit tests were written in a text document (such as in Word). They described the exact steps that the user was to follow in using the app. For example, they described the exact input to type on the page to set up a new customer. Or, the user was to search for a particular product, and see that the displayed information matched the test document. So, the test was a script that the tester followed, where they also recorded the results. When the new incarnation of unit testing came along, it was confusing for a while trying to figure out if they meant the old, human tests or the new, coded tests.
I lead a group of offshore team too. Supposely we have a set of unit tests...but it doesn't mean much. :) So we rely much more on the functional and testers for quality. The inherit problem with unit testing is that you have perfect knowledge of the functionals, and you trust the developers. In the real world, that's hard to assume..