What tools are generally used for unit testing and especially continuous integration for embedded systems?
I am especially thinking that you usually have to cross-compile and deploy, and also that you can't easily visualize the target platform. Also it can be difficult to run test-code and frameworks.
What could I use too alleviate these difficulties?
(I think it should be some kind of dual targeting, where the build server runs its tests on a easier target)
For unit testing, take a look at Unity.
http://sourceforge.net/apps/trac/unity/wiki
It is a really lightweight test harness (2 x .h and 1 x .c file) supported by Ruby scripts. We have been using in an embedded ARM7 target system for unit testing (redirecting test reporting over a serial port).
It is also supported by CMock for (surprise, surprise) Mocking. Even though not extensive, the great thing about these is they are so easy to use.
Regarding CI, then Hudson (now Jenkins) is very good if you're Linux based.
Also look at CppUTest and check out James Grenning's book "TDD for Embedded C" at http://renaissancesoftware.net/
At work I use the embUnit framework:
http://embunit.sourceforge.net/embunit/index.html
The nice thing about this framework is, that it's lean. It does not require any external libraries (not even libc). You can hook your own output function with ease so if you work on a system where the only connection to the outside world is jtag or an UART, then embUnit will still work.
I have used RCUNIT and CANTATA++ for unit testing embedded code on the PC. Any Nunit should easily integrate into any continuous test platform. We found it an lot easier to just simulate the hardware on the PC and only test on the target during final integration.
Hardware interface abstraction is crucial for unit testing embedded code on the PC. This works well with continuous integration since it is run on a pc with just the hardware access simulated. With a little effort we could test 95% of the code on a PC for continues integration.
You could also look at these questions:
Unit Testing C Code
Testing Frameworks for C
Unit Testing Embedded Software
I've seen C(pp)unit used on a system that let you launch to the target via JTAG.
Helps to have console comms etc sorted.
But it can work.
I'm working on an open-source tool for that, that is either a minimal test or can provide detailed information when run through the debugger.
Related
I mean something similar to VectorCAST or maybe CANTATA++
What I want: I've got C code for target system I'm going to test and I've got target hardware with a little channel to developers PC (UART, tcp/ip and so on). I want software, that gives me capabilities to write tests for embedded system on a host PC on C++ and automates transfer calls of embedded code on a target system with uart etc.
I think It might be written for two weekends on PyGCCXML and protobuf (and protobuf generator for embedded C), but maybe such system is already exists.
Consider Embedded Unit, a unit test framework for embedded C systems.
Maybe something like:
Parasoft c/c++ test
Embunit
I am having a product which runs on Vxworks on end product hardware. But development is done in Visual studio using cross compiler and downloaded to hardware for testing. I am planning to write unit test cases for product. My question is because my development is done on windows and how can I run unit test cases as it is not resemebling real scenario?
Any inputs are welcome
I suspect that you have code that interacts a lot with VxWorks through system calls. Putting a layer of abstraction in there will be hard.
Are you using c or c++?
If you are using c++ and you can identify parts of the system that:
are subject to frequent change; and
are mostly handling internal data; or
just relating to a predefined/formalized subset of the surrounding system (e.g. protocol handling or individual PLC control logic modules).
Then you should first inject c++ interface(s) between the module and the rest of the system. This module should only relate to the interface(s)/adapter(s). Then you have an isolated piece that can be strapped into a visual studio test harness.
Then you should try to identify areas in your system that are error prone to bugs, subject to (frequent) change or audit. You will probably never achieve even 50% coverage on the target system, but you can achieve a system where 90% of the daily coding happens within the covered 40% of the code base.
Not possible without additional effort (new project; compile twice; for your host and let it run on your host) In this case search for an development environment that support your target for unit tests, for example http://www.parasoft.com/jsp/products/embedded_cpptest.jsp
Go on reading at parasoft C++ unit test question .
You do it just like any other system:
Write the unit tests
compile and load into target system
run unit tests
verify results
Where it runs is immaterial. The biggest difficulty is on an embedded system with limited output capability. But even if there is only a single LED, it should still be possible to signal success and failure. Only it is a little more abstract than showing "passed".
I am a graduate student and I am trying to propose a project for a advanced testing course.
Since I am a embedded guy, I do want to test something challenging related to embedded systems.
uC/OS-II is an very nice open source light-weighted OS for embedded systems. So I want to propose the testing for it for my course project.
But I don't know the feasibility of testing uC/OS. Is it doable?
I am using Blackfin and SHARC (are from Analog Devices) now and they are compatible with uC/OS (said on the uC/OS website).
In terms of testing tools, I think CUnit might work. Also we have a unit testing tool call EmbeddedUnit, which runs on VDSP (development environment for Analog Devices processors).
I have no experience with uC/OS, but my understanding is we should compile it and then include the .obj files and the header files into the project then we can use and test the functions in uC/OS.
Am I right?
Is it doable? Yes it is. We had a project that needed to be portable to many different environments uCos-II, Linux and VxWorks. In order to do that we wrote as simple abstraction layer that gave us a common API on all platforms to the OS features we chose to have enabled. We then wrote a Unit Test to test the abstraction layer, and had a unit test case for each OS feature we wanted to test (Msg Queues, Semaphores, Event Flags etc). We used that to verify our abstraction layer was functional and working on all 3 host environments.
uCos-II is delivered as very clean c-code that can easily be used in any number of tools such as code coverage etc.
Good luck.
we have one main application, which executes up to 5 different exes. These exes run independently and communicate with each other via UDP. Changing this architecture is not planned at the moment.
We want to migrate this whole thing from VS6 to VS2008.
I'm thinking about adding unit tests to make sure that after migration everything still works. Right now, there is not a single unit test.
So now I have a couple of questions:
Which unit test framework works fine with VS6 and VS2008? CppUnit seems to work with both compilers, at least I got it running.
How do I implement the unit tests in the above mentioned architecture? I see a problem that I have executables and no libraries, so it seems to be a bit difficult for me at that point
Is creating the unit tests for both platforms worth the effort or do you have a simpler suggestion?
Suggestions, best practices, new ideas welcome :)
Thanks
If they communicate via UDP and you want to make sure the integration between pieces works now and later, you could write automated tests against the UDP interfaces in MSTest or NUnit in VS2008.
Write the tests in VS2008 and send input/verify output of the interfaces to the the .exe applications. Then, when you switch to .NET, just point your tests at the new endpoint for the .NET .exe and run them to verify all input and output are the same.
I think you would get more bang for your time to write the unit tests exclusively for the new app, but write tests against the external UDP interfaces to verify the migration was successful.
If you are willing to consider each of the executables as as 'module', then there is a way to introduce tests without being too invasive.
This would create modules that will be a bit large for many people to be comfortable with (typical sizes when people talk about modules is a class or a source file), but your communication mechanism provides a nice way to hook in the tests.
With the architecture that you have and the fact that you don't have an existing test harness for the code, I would write a test harness that can communicate with each of the executables in isolation, using the UDP interfaces.
When executing the tests, only one executable is running (besides the test code). The test code simulates the other executables by listening on their respective UDP ports and providing responses as the testcase requires. The testcase also stimulates the executable under test by sending UDP requests to it. If there is also a GUI/TUI involved, it might be needed that parts of the tests are executed manually.
My new project is targeting an embedded ARM processor. I have a build system that uses a cross-compiler running on an Ubuntu linux box. I like to use unit testing as much as possible, but I'm a little bit confused about how to proceed with this setup.
I can't see how to run unit tests on the ARM device itself (somebody correct me if I'm wrong). I think that my best option is to compile the code on the build machine using its own native compiler for the unit tests. Is this approach fundamentally flawed? Is unit testing on a different platform a waste of time?
I'm planning to use CppUnit on the build machine using the native compiler for the unit tests. Then I'll cross compile the code for the ARM processor and do integration and system testing on the target device itself. How would you structure the source code and the test code to keep this from turning into a tangled mess?
With embedded device it depends on what interfaces (hardware) you have.
For example the motion control cards I deal with uses a command line interface. The IDE they ship uses it as it primary method of interacting with the cards. It works the same way regardless if I am using PCI, IDE, Serial, or Ethernet.
The DLL they ship for programming give access to the command line interface. So I can send a string, and read back the response. So what I do for my unit tests is have a physical card hooked (or in) my development machine. I send it commands after uploading the software, read the response and if they are correct it passes the test.
I also have extra hardware, a black box if you will, that simulates a machine that motion control card is normally hooked up too. It helps with the automated sets but there is a manual phase as I have to set switches to simulate different setups on the machine.
I have achieved a greater degree of automation by taking a digital I/O card and using it outputs to feed into the inputs of the motion control card and the same in reverse.
I found that for most hardware you have to have some type of simulator hardware.
The exception being the rare package that comes with a software simulator.
I know this isn't probably ideal as not every developer can have one of these on their desk. My hardware simulator so I can give it to whoever it working on the motion control software at the time. If it can't be portable then having a dedicated testing or hardware development computer would be in order.
Finally it boils down on the specifics of your hardware and what support the manufacturer gives in terms of software and simulators. To help you more you will need to post more specifics.
In ten-plus years in the embedded industry, I've seen it done quite a few ways. At my current company:
one of our products has enough horsepower (and space) to run tests on the target board. It's somewhat slow, and we can't stick all the python on the box we'd like, but it works well.
one of our products doesn't have the space, so we compile all the libs we can in x86 (anything that isn't hardware-dependent) and run unit tests on desktops. It's not perfect, but far better than nothing.
one of our components is a super-lightweight power-miser on exotic hardware, so virtually no unit tests are possible. Core algorithms (DES, etc.) are tested on x86 as above, but much of the code simply has to be tested as a whole, in situ. This entails lot of code reviews.