As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 10 years ago.
What best practices have you used in unit testing embedded software that are peculiar to embedded systems?
Embedded software may have come a long way in the last 10 years but we generally did the following:
for algorithms that didn't depend on the target hardware, we simply had unit tests that were built and tested on a non-embedded platform.
for stuff that did require the hardware, unit tests were conditionally compiled into the code to use whatever hardware was available. In our case, it was a serial port on the target pushing the results to another, more capable, machine where the tests were checked for correctness.
Depending on the hardware, you could sometimes dummy up a "virtual" device on a non-embedded platform. This usually consisted of having another thread of execution (or signal function) changing memory used by the program. Useful for memory mapped I/O but not IRQs and such.
typically, you could only unit test a small subset of the complete code at a time (due to memory constraints).
for testing of time-sensitive things, we didn't. Plain and simple. The hardware we used (8051 and 68302) was not always functional if it ran too slow. That sort of debugging had to be done initially with a CRO (oscilloscope) and (when we had some more money) an ICE (in-circuit emulator).
Hopefully the situation has improved since I last did it. I wouldn't wish that pain on my worst enemy.
There can be a lot to be gained by unit testing in a PC environment (compiling your code with a PC C compiler and running your code in a PC unit testing framework), with several provisos:
This doesn't apply to testing your low-level code, including start-up code, RAM tests, hardware drivers. You'll have to use more direct unit testing of those.
Your embedded system's compiler has to be trustworthy, so you're not hunting for bugs created by the compiler.
Your code has to be layered architecture, with hardware abstraction. You may need to write hardware driver simulators for your PC unit testing framework.
You should always use the stdint.h types such as uint16_t rather than plain unsigned int etc.
We've followed these rules, and found that after unit testing the application-layer code in a PC unit test framework, we can have a good amount of confidence that it works well.
Advantages of unit testing on the PC platform:
You don't face the problem of running out of ROM space on your embedded platform due to adding a unit testing framework.
The compile-link-run cycle is typically faster and simpler on the PC platform (and avoids the 'write/download' step which can potentially be several minutes).
You have more options for visualising progress (some embedded applications have limited I/O peripherals), storing input/output data for analysis, running more time-consuming tests.
You can use readily available PC-based unit test frameworks that aren't available/suitable for an embedded platform.
Embedded systems is a wide topic but in general, let's think of it as a specific-purpose product that combines both hardware and software. My embedded background is from mobile phones which is just a small subset of all embedded systems. I'll try to keep the following points a bit on the abstract side:
Abstract out hardware dependencies whenever possible. This way you can run your unit tests on mocked "hardware" and also test various rare/exceptional cases that would be harder to test on target. To prevent abstraction costs, you can use e.g. conditional compilation.
Have as little as possible depend on the hardware.
Unit tests running on an emulator or cross-compiler environment still does not guarantee the code works on target hardware. You must test on target as well. Test on target as early as possible.
You might want to check out Test Driven Development for Embedded C by James W. Grenning. The book is scheduled to be published in August 2010, but the beta book is available now on The Pragmatic Bookshelf.
Voice of inexperience here, but this is something I've been thinking about as well lately. It seems to me that the best approach would be either
A) Write as much of your hardware-independent application code as you can in a PC environment, before you write it on the target, and write your unit tests at the same time (doing it this on the PC first should help force you to separate the hardware-independent stuff). This way you can use your choice of unit testers, then test the hardware-dependent stuff the old fashioned way - with RS-232 and/or oscilloscopes and I/O pins signalling time-dependent data, depending on how fast it has to run.
B) Write it all on the target hardware, but have a make target to conditionally compile a unit test build that will run unit tests and output the results (or data that can be analyzed for results) via RS-232 or some other means. If you don't have a lot of memory, this can be tricky.
Edit 7/3/2009
I just had another thought about how to unit test hardware dependent stuff. If your hardware events are happening too fast to record with RS-232, but you don't want to manually sift through tons of oscilloscope data checking to see if your I/O pin flags rise and fall as expected, you can use a PC card with integrated DIO (such as National Instruments' line of Data Acquisition cards) to automatically evaluate the timing of those signals. You would then just need to write the software on your PC to control the data acquisition card to synchronize with the currently running unit test.
We manage to get quite a bit of hardware dependent code tested using a simulator, we use Keil's simulator and IDE (not affiliated just use their tools). We write the simulator scripts to drive the 'hardware' in a way we expect it to react and we are able to pretty reliably test our working code. Granted it can take some effort to model the hardware for some tests, but for most things this works very well and allows us to get a lot done without any hardware available. We have been able to get near complete system working in the simulator before having access to hardware and have had very few issues to deal with once putting the code on the real thing. This can also significantly speed up production of code since everything can be done on the PC with the more in-depth debugger available while simulating the chip vs trying to do everything on the hardware.
Have gotten this to work reliably for complex control systems, memory interfaces, custom SPI driven ICs and even a mono-display.
There's lots of good answers here, some things that haven't been mentioned is to have diagnostic code running in order to:
Log HAL events (interrupts, bus messages, etc)
Have code to keep track of your resources, (all active semaphores, thread activity)
Have a capture ram mechanism to copy the heap and memory content to persistent storage (hard disk or equivalent) to detect and debug deadlocks, livelocks, memory leaks, buffer overflows, etc.
When I was facing this last year I really wanted to test on the embedded platform itself. I was developing a library and I was using the RTOS calls and other features of the embedded platform. There wasn't anything specific available so I adapted the UnitTest++ code to my purposes. I program on the NetBurner family and since it has an embedded web server, it was pretty straight forward to write a web based GUI test runner that give the classic RED/GREEN feedback. It turned out pretty well, and now unit testing is much easier and I feel much more confident knowing the code works on the actual hardware. I even use the unit testing framework to do integration tests. At first I mocks/stub the hardware and inject that interface to test. But eventually I write some man-in-the-loop tests that exercise the actual hardware. It turns out to be a much simpler way to learn about the hardware and have an easy way to recover from embedded traps. Since the tests all run from AJAX callbacks to the web server a trap only happens as the result of manually invoking a test and the system always restarts cleanly a few seconds after the trap.
The NetBurner is fast enough that the write/compile/download/run test cycle is about 30 seconds.
Lots of embedded processors are available on eval boards, so although you may not have your real i/o devices, often you can execute a good deal of your algorithms and logic on one of these kinds of things, often w/hardware debugging available via jtag. And 'unit' tests usually are more about your logic than your i/o anyway. Problem is usually getting your test artifacts back out of one of these environments.
Split the code between device-dependent & device-independent. The independent code can be unit-tested without too much pain. The dependent code will simply need to be hand-tested until you have a smooth communications interface.
If you're writing the communications interface, I'm sorry.
Related
I have this very generic question about unit test. Should we write environment dependent unit tests? The unit tests may depend on file system, a specific connection or presence of a file.
I am not sure if there is a answer to it. But I found it takes time to decouple the unit tests from the environment. Does that mean it is ok to create environment dependent unit tests? Or it always points out that there is a design problem when we need to write environment dependent unit tests?
TL;DR
Whether environment dependent tests are indeed a design flaw or not is strongly dependent on your application, programming language and specific problem. Keep in mind, that tests are never "complete" or "perfect" and try not to achieve perfection at the expense of more and more time. Using modern frameworks, making your tests re-usable and mocking your environment can take a lot of the pain out of testing.
The longer answer
IMHO, there's no perfect answer to this question. Unit Tests should be as generic and re-usable as possible without trying to cover environmental edge cases at any price.
Example: Most environments should have an AMD64 processor nowadays (not regarding phones), so in most cases it is not needed to create unit tests for X32 or ARM environments, ...
Your question is technology agnostic, so it's hard to tell, if the need for environment dependent unit tests is a design flaw. If you're using Java (see below), I'd rather think it is, using C++, it might not be...
Some rules of thumb I can give from my experience (and therefore a little opinion-based):
Follow the pareto principle
Don't try to achieve 100% of what is possible in theory, but concretate on the about 80% that are also likely to be encountered. Let's say, you expect your application to run on Windows Clients. So possibly your tests should cover the environmental characteristics of Windows 10 and 11. Older Windows versions, Linux, Mac OS, ... should not be needed.
Or, if your application mainly has as Spanish audience, you should not care so much about how it can deal with chinese characters or canadian data protection rules.
Exclude environmental specialties as much as sane and possible when coding your application
If you can manage to make your application code mostly independent from your environment, the same will "automatically" apply to your tests. (Also, see the point about not re-inventing the wheel below)
As you didn't ask for a specific programming language, I'll refer to Java here, but similar concepts exist in most modern languages.
E. g., Java abstracts very much from your environment, so you can use File.separator, if you don't know if your application will be running on Windows (\ as separator) or Linux (/).
Not using OS-specific APIs (like Java NI) avoids environment-related problems as well. If not dealing closely with hardware access, there shouldn't be any need to use them.
Adapt your tests
Like the "real" application, usually your tests are never "done". A new problem with your software arises? Fix it and add a specific test case for this kind of bug (trying to be quite generic). If it was a problem related to the 20% of cases or environments you did not consider before, be happy - now you've got a little hold of them as well.
Package your test data with your application
Referring to Java again, you could place your test files in the resources folder and load them dynamically while testing - not need to bother the real file system of the testing machine.
Also, if you're using a database, you could spin up an in-memory instance (like H2) during testing. This will not exactly mirror your production database system, but makes testing very easy without the danger of breaking something. Anyway, database abstraction is also very sophisticated nowadays, so despite of some edge cases, you should not see any difference in behaviour if you're using a good database abstraction layer (which itself in turn make you less environment-dependent).
Make your tests re-usable
Using a tool like Maven or Gradle, you can make your tests run on a bigger variety of environments, if they are packaged as just pointed out. The need for a specific test environment will decrease.
Mocking is your friend
As with the "fake" database mentioned before, you can mock a lot of things. There are frameworks like Mockito allowing white box and black box testing with mocked objects. "Helpers" like WireMock allow you to even mock the answer you get from an external service. And there are a lot more possibilities out there!
Don't re-invent the wheel
If you use one of todays advanced application frameworks (like Spring, Symfony, Boost, ...) they'll do a lot of the work for you if you make proper use of them. You'll have to read the docs and burrow into them, but then you will realize how much easier they can make your life. This does not apply to frameworks only, but also to libraries, services, components, ... And the good thing is, all of that will also come into place when it's about testing! Possibly most of your environmental dependencies might vanish if you use a good software foundation.
As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 10 years ago.
I am looking to start from scratch to learn to program embedded systems. After some time looking around I found myself a bit confused.
I can program both C and C++ but I just don't know where to start with embedded programming, should I buy some kind of device to practice on, use a microprocessor emulator (if so, which one?) - stuff like that, any advice or resource on where to start is very welcome.
In my opinion, skip Arduino. I've always seen it as kind of a dumbed down system for non-programmers. Go for something that lets you apply your C knowledge to getting as low level as possible, at first. You want to understand how interrupt vectors work, how your limited RAM limits your stack, how to debug.
Check out Freescale's 8-bit and 16-bit Microcontrollers, especially the HCS08 or HCS12. There are some $100-200 and some sub-$100 development systems with built in debug interface (Background Debug Mode). These are also higher performance and memory than Arduino. The CodeWarrior software is free (code size limitations, but in most cases the limitation number is greater than the amount of flash on these devices), and fully functional. I don't know if you can set code debug breakpoints with Arduino, but you can with these. There is another benefit to CodeWarrior -- while you do at first want to delve into datasheets to understand how memory-mapped registers for the various modules operate (eg, how a flag should be cleared, how to set a mode, whatever), CodeWarrior comes with Processor Expert which will generate functions for you for specific HCSxx family derivatives and their specific modules. Since most of these products reuse the logic between derivatives, with some minor differences, it makes sense to reuse code. Processor Expert has come a long way since its beginning 10 or so years ago. In the long run it is a huge savings to development time, as these functions take care of writing the very low level actions (eg, configure a PWM timer output pin for a xx/yy duty cycle with one C function call).
Then you can use some of the OS's that will run on these, or you can move up to ARM or ePPC. I know of at least one HCS12 demo board that comes with Linux and a webserver, which you could always wipe out for your first development, and then put it back when you're ready. Freescale is also very good with providing tutorials, application notes, and documentation, except that their site is sometimes hard to navigate. I suppose that's just a symptom of a large ecosystem. Good luck!
There are several embedded platforms out there that are popular with hobbyists and very easy to use. The Arduino is probably the most popular. Boards are cheap ($20-30) and easy to use, and there's plenty of good software. The main web site for the Arduino platform is http://arduino.cc. You'll find plenty of links there to other resources that'll help you get started.
You may want to check out SparkFun, which is primarily a hobbyist-level supplier of electronics parts (including Arduino and other useful boards) and tools. They've got a lot of great content on their web site to help you get started with both the hardware and software.
why dont you start with something like that:
http://www.bytecraft.com/downloads/firststeps.pdf
this should cover a lot of the ground you are looking for.
or a lot can be found here: http://www.eetimes.com/electrical-engineers/education-training
have fun with your first steps =)
One could divide embedded systems into two classes:
Those that run a full OS. If there is a documented way to get the OS and boot it up, then these aren't really handled any different than desktops, except you will need to build or download a cross-compiler. For things like phones, of course, elaborate development environments with debuggers are provided by the environment's sponsor.
Those that run a program on what's called "bare metal". These will have little more than your program and the language runtime loaded. (Some of these are really simple and run either a simple interpreter or assembly language. These have the advantage of not needed complex programming just to set up the integrated peripherals.)
You may not need an emulator or other hardware debugging setup, in fact, you may not even need hardware to get started. If possible, try to do initial development in a virtual machine.
As always, you get what you pay for. In a commercial project lots of environment is available but these are presumably not used much for robot projects and the like.
There are lots of platforms that come with reasonable amounts of support. The latest rage (due to its remarkable US$35 price point) is the Raspberry Pi, however, it is not immediately available at this time.
You have tons of options! I guess you'll start with an emulator (it doesn't matter what type of embedded OS you'll use, probably you won't use a x86 CPU). If you'll work with a X86 CPU you don't need any emulator but you may need a Virtual Machine.
You tagged your question with "arm" so I guess you want to use an ARM processor (then you'll need an emulator). Start to look here, you'll find tutorials and resources to begin with embedded programming.
I would suggest trying out some of TI's EZ430 series of development tools. Some of the tools can be purchased for as little as $20 and have basic functionality, as well as ways to expand them out. You can easily buy two eZ430-F2013 and have them talk to each other or to another computer.
TI comes with Code Composer Studio for their MSP430s which is free and full featured (it's based on Eclipse) so the programming environment is very user friendly. TI also provides a large number of code samples which cover most of the system's functionality.
With the MSP430, you will be programming on "bare metal" code as its said, so you're not just going to be programming C, you're going to be toggling outputs and bit-banging. But it looks like you're interested in learning it down at this level, so this is a great place to start.
On the other hand, if you want an embedded OS (such as Linux) check out the gumstix website.
Because your question provides little information as to application type, performance requirements, and budget etc., no answer will be particularity well focussed.
However given that I would suggest that you pick an ARM architecture device, simply for the reason that this covers devices from a few dollars with performance in the 10's of MIPS, and small on-chip memories to application processors on boards costing a few hundred dollars capable of running Linux, WinCE or Android for example.
Like it or not ARM is ubiquitous in the embedded systems world; everything else is niche in terms of design-in and market share. A Cortex-M3 based device on a simple development/evaluation board is a good place to start. You will be frustrated however if you do not factor in the cost of tools and debug hardware.
Closed. This question is off-topic. It is not currently accepting answers.
Closed 9 years ago.
Locked. This question and its answers are locked because the question is off-topic but has historical significance. It is not currently accepting new answers or interactions.
How do emulators work? When I see NES/SNES or C64 emulators, it astounds me.
Do you have to emulate the processor of those machines by interpreting its particular assembly instructions? What else goes into it? How are they typically designed?
Can you give any advice for someone interested in writing an emulator (particularly a game system)?
Emulation is a multi-faceted area. Here are the basic ideas and functional components. I'm going to break it into pieces and then fill in the details via edits. Many of the things I'm going to describe will require knowledge of the inner workings of processors -- assembly knowledge is necessary. If I'm a bit too vague on certain things, please ask questions so I can continue to improve this answer.
Basic idea:
Emulation works by handling the behavior of the processor and the individual components. You build each individual piece of the system and then connect the pieces much like wires do in hardware.
Processor emulation:
There are three ways of handling processor emulation:
Interpretation
Dynamic recompilation
Static recompilation
With all of these paths, you have the same overall goal: execute a piece of code to modify processor state and interact with 'hardware'. Processor state is a conglomeration of the processor registers, interrupt handlers, etc for a given processor target. For the 6502, you'd have a number of 8-bit integers representing registers: A, X, Y, P, and S; you'd also have a 16-bit PC register.
With interpretation, you start at the IP (instruction pointer -- also called PC, program counter) and read the instruction from memory. Your code parses this instruction and uses this information to alter processor state as specified by your processor. The core problem with interpretation is that it's very slow; each time you handle a given instruction, you have to decode it and perform the requisite operation.
With dynamic recompilation, you iterate over the code much like interpretation, but instead of just executing opcodes, you build up a list of operations. Once you reach a branch instruction, you compile this list of operations to machine code for your host platform, then you cache this compiled code and execute it. Then when you hit a given instruction group again, you only have to execute the code from the cache. (BTW, most people don't actually make a list of instructions but compile them to machine code on the fly -- this makes it more difficult to optimize, but that's out of the scope of this answer, unless enough people are interested)
With static recompilation, you do the same as in dynamic recompilation, but you follow branches. You end up building a chunk of code that represents all of the code in the program, which can then be executed with no further interference. This would be a great mechanism if it weren't for the following problems:
Code that isn't in the program to begin with (e.g. compressed, encrypted, generated/modified at runtime, etc) won't be recompiled, so it won't run
It's been proven that finding all the code in a given binary is equivalent to the Halting problem
These combine to make static recompilation completely infeasible in 99% of cases. For more information, Michael Steil has done some great research into static recompilation -- the best I've seen.
The other side to processor emulation is the way in which you interact with hardware. This really has two sides:
Processor timing
Interrupt handling
Processor timing:
Certain platforms -- especially older consoles like the NES, SNES, etc -- require your emulator to have strict timing to be completely compatible. With the NES, you have the PPU (pixel processing unit) which requires that the CPU put pixels into its memory at precise moments. If you use interpretation, you can easily count cycles and emulate proper timing; with dynamic/static recompilation, things are a /lot/ more complex.
Interrupt handling:
Interrupts are the primary mechanism that the CPU communicates with hardware. Generally, your hardware components will tell the CPU what interrupts it cares about. This is pretty straightforward -- when your code throws a given interrupt, you look at the interrupt handler table and call the proper callback.
Hardware emulation:
There are two sides to emulating a given hardware device:
Emulating the functionality of the device
Emulating the actual device interfaces
Take the case of a hard-drive. The functionality is emulated by creating the backing storage, read/write/format routines, etc. This part is generally very straightforward.
The actual interface of the device is a bit more complex. This is generally some combination of memory mapped registers (e.g. parts of memory that the device watches for changes to do signaling) and interrupts. For a hard-drive, you may have a memory mapped area where you place read commands, writes, etc, then read this data back.
I'd go into more detail, but there are a million ways you can go with it. If you have any specific questions here, feel free to ask and I'll add the info.
Resources:
I think I've given a pretty good intro here, but there are a ton of additional areas. I'm more than happy to help with any questions; I've been very vague in most of this simply due to the immense complexity.
Obligatory Wikipedia links:
Emulator
Dynamic recompilation
General emulation resources:
Zophar -- This is where I got my start with emulation, first downloading emulators and eventually plundering their immense archives of documentation. This is the absolute best resource you can possibly have.
NGEmu -- Not many direct resources, but their forums are unbeatable.
RomHacking.net -- The documents section contains resources regarding machine architecture for popular consoles
Emulator projects to reference:
IronBabel -- This is an emulation platform for .NET, written in Nemerle and recompiles code to C# on the fly. Disclaimer: This is my project, so pardon the shameless plug.
BSnes -- An awesome SNES emulator with the goal of cycle-perfect accuracy.
MAME -- The arcade emulator. Great reference.
6502asm.com -- This is a JavaScript 6502 emulator with a cool little forum.
dynarec'd 6502asm -- This is a little hack I did over a day or two. I took the existing emulator from 6502asm.com and changed it to dynamically recompile the code to JavaScript for massive speed increases.
Processor recompilation references:
The research into static recompilation done by Michael Steil (referenced above) culminated in this paper and you can find source and such here.
Addendum:
It's been well over a year since this answer was submitted and with all the attention it's been getting, I figured it's time to update some things.
Perhaps the most exciting thing in emulation right now is libcpu, started by the aforementioned Michael Steil. It's a library intended to support a large number of CPU cores, which use LLVM for recompilation (static and dynamic!). It's got huge potential, and I think it'll do great things for emulation.
emu-docs has also been brought to my attention, which houses a great repository of system documentation, which is very useful for emulation purposes. I haven't spent much time there, but it looks like they have a lot of great resources.
I'm glad this post has been helpful, and I'm hoping I can get off my arse and finish up my book on the subject by the end of the year/early next year.
A guy named Victor Moya del Barrio wrote his thesis on this topic. A lot of good information on 152 pages. You can download the PDF here.
If you don't want to register with scribd, you can google for the PDF title, "Study of the techniques for emulation programming". There are a couple of different sources for the PDF.
Emulation may seem daunting but is actually quite easier than simulating.
Any processor typically has a well-written specification that describes states, interactions, etc.
If you did not care about performance at all, then you could easily emulate most older processors using very elegant object oriented programs. For example, an X86 processor would need something to maintain the state of registers (easy), something to maintain the state of memory (easy), and something that would take each incoming command and apply it to the current state of the machine. If you really wanted accuracy, you would also emulate memory translations, caching, etc., but that is doable.
In fact, many microchip and CPU manufacturers test programs against an emulator of the chip and then against the chip itself, which helps them find out if there are issues in the specifications of the chip, or in the actual implementation of the chip in hardware. For example, it is possible to write a chip specification that would result in deadlocks, and when a deadline occurs in the hardware it's important to see if it could be reproduced in the specification since that indicates a greater problem than something in the chip implementation.
Of course, emulators for video games usually care about performance so they don't use naive implementations, and they also include code that interfaces with the host system's OS, for example to use drawing and sound.
Considering the very slow performance of old video games (NES/SNES, etc.), emulation is quite easy on modern systems. In fact, it's even more amazing that you could just download a set of every SNES game ever or any Atari 2600 game ever, considering that when these systems were popular having free access to every cartridge would have been a dream come true.
I know that this question is a bit old, but I would like to add something to the discussion. Most of the answers here center around emulators interpreting the machine instructions of the systems they emulate.
However, there is a very well-known exception to this called "UltraHLE" (WIKIpedia article). UltraHLE, one of the most famous emulators ever created, emulated commercial Nintendo 64 games (with decent performance on home computers) at a time when it was widely considered impossible to do so. As a matter of fact, Nintendo was still producing new titles for the Nintendo 64 when UltraHLE was created!
For the first time, I saw articles about emulators in print magazines where before, I had only seen them discussed on the web.
The concept of UltraHLE was to make possible the impossible by emulating C library calls instead of machine level calls.
Something worth taking a look at is Imran Nazar's attempt at writing a Gameboy emulator in JavaScript.
Having created my own emulator of the BBC Microcomputer of the 80s (type VBeeb into Google), there are a number of things to know.
You're not emulating the real thing as such, that would be a replica. Instead, you're emulating State. A good example is a calculator, the real thing has buttons, screen, case etc. But to emulate a calculator you only need to emulate whether buttons are up or down, which segments of LCD are on, etc. Basically, a set of numbers representing all the possible combinations of things that can change in a calculator.
You only need the interface of the emulator to appear and behave like the real thing. The more convincing this is the closer the emulation is. What goes on behind the scenes can be anything you like. But, for ease of writing an emulator, there is a mental mapping that happens between the real system, i.e. chips, displays, keyboards, circuit boards, and the abstract computer code.
To emulate a computer system, it's easiest to break it up into smaller chunks and emulate those chunks individually. Then string the whole lot together for the finished product. Much like a set of black boxes with inputs and outputs, which lends itself beautifully to object oriented programming. You can further subdivide these chunks to make life easier.
Practically speaking, you're generally looking to write for speed and fidelity of emulation. This is because software on the target system will (may) run more slowly than the original hardware on the source system. That may constrain the choice of programming language, compilers, target system etc.
Further to that you have to circumscribe what you're prepared to emulate, for example its not necessary to emulate the voltage state of transistors in a microprocessor, but its probably necessary to emulate the state of the register set of the microprocessor.
Generally speaking the smaller the level of detail of emulation, the more fidelity you'll get to the original system.
Finally, information for older systems may be incomplete or non-existent. So getting hold of original equipment is essential, or at least prising apart another good emulator that someone else has written!
Yes, you have to interpret the whole binary machine code mess "by hand". Not only that, most of the time you also have to simulate some exotic hardware that doesn't have an equivalent on the target machine.
The simple approach is to interpret the instructions one-by-one. That works well, but it's slow. A faster approach is recompilation - translating the source machine code to target machine code. This is more complicated, as most instructions will not map one-to-one. Instead you will have to make elaborate work-arounds that involve additional code. But in the end it's much faster. Most modern emulators do this.
When you develop an emulator you are interpreting the processor assembly that the system is working on (Z80, 8080, PS CPU, etc.).
You also need to emulate all peripherals that the system has (video output, controller).
You should start writing emulators for the simpe systems like the good old Game Boy (that use a Z80 processor, am I not not mistaking) OR for C64.
Emulator are very hard to create since there are many hacks (as in unusual
effects), timing issues, etc that you need to simulate.
For an example of this, see http://queue.acm.org/detail.cfm?id=1755886.
That will also show you why you ‘need’ a multi-GHz CPU for emulating a 1MHz one.
Also check out Darek Mihocka's Emulators.com for great advice on instruction-level optimization for JITs, and many other goodies on building efficient emulators.
I've never done anything so fancy as to emulate a game console but I did take a course once where the assignment was to write an emulator for the machine described in Andrew Tanenbaums Structured Computer Organization. That was fun an gave me a lot of aha moments. You might want to pick that book up before diving in to writing a real emulator.
Advice on emulating a real system or your own thing?
I can say that emulators work by emulating the ENTIRE hardware. Maybe not down to the circuit (as moving bits around like the HW would do. Moving the byte is the end result so copying the byte is fine). Emulator are very hard to create since there are many hacks (as in unusual effects), timing issues, etc that you need to simulate. If one (input) piece is wrong the entire system can do down or at best have a bug/glitch.
The Shared Source Device Emulator contains buildable source code to a PocketPC/Smartphone emulator (Requires Visual Studio, runs on Windows). I worked on V1 and V2 of the binary release.
It tackles many emulation issues:
- efficient address translation from guest virtual to guest physical to host virtual
- JIT compilation of guest code
- simulation of peripheral devices such as network adapters, touchscreen and audio
- UI integration, for host keyboard and mouse
- save/restore of state, for simulation of resume from low-power mode
To add the answer provided by #Cody Brocious
In the context of virtualization where you are emulating a new system(CPU , I/O etc ) to a virtual machine we can see the following categories of emulators.
Interpretation: bochs is an example of interpreter , it is a x86 PC emulator,it takes each instruction from guest system translates it in another set of instruction( of the host ISA) to produce the intended effect.Yes it is very slow , it doesn't cache anything so every instruction goes through the same cycle.
Dynamic emalator: Qemu is a dynamic emulator. It does on the fly translation of guest instruction also caches results.The best part is that executes as many instructions as possible directly on the host system so that emulation is faster. Also as mentioned by Cody, it divides the code into blocks ( 1 single flow of execution).
Static emulator: As far I know there are no static emulator that can be helpful in virtualization.
How I would start emulation.
1.Get books based around low level programming, you'll need it for the "pretend" operating system of the Nintendo...game boy...
2.Get books on emulation specifically, and maybe os development. (you won't be making an os, but the closest to it.
3.look at some open source emulators, especially ones of the system you want to make an emulator for.
4.copy snippets of the more complex code into your IDE/compliler. This will save you writing out long code. This is what I do for os development, use a district of linux
I wrote an article about emulating the Chip-8 system in JavaScript.
It's a great place to start as the system isn't very complicated, but you still learn how opcodes, the stack, registers, etc work.
I will be writing a longer guide soon for the NES.
I'm working on a small fun projects that builds a robot.
We as the programmers are working parallel to the people building the robot. So it is very often the case that we are trying to run changed software and the builders have changed the hardware. If the software tests are not running it is always a hard thing to figure out if the software or the hardware fails or even worse if the integration fails.
There are some hard parts with an automatic testing for this issues.
We have figured out some ways of breaking things down so we have rc control to let the robot go through some movements without software assuring that he still works.
Then we start some software tests that make the robot going some defined figures to show that the software behaves in the same way as before. But this always comes down to a very time consuming task because you can't automate it and someone has to start the test, watch the test and try to figure out if the robot did what it should do.
Another problem is that constant testing with our real hardware is wearing out parts of our hardware, joint, motors, gear wheels and so on.
But not testing has proven to cause so much trouble and consume so much time that I would like to know what kind of techniques are used in other projects which are dealing with hardware software interaction and if there are tools out there that can be used.
The interface between the robot and the software must be defined first ; not necessarily exhaustively, this could be done incrementally. Start small, for instance with basic moves (forward, backward), then, once it has been fully tested, both in isolation and integrated, add some behaviour (e.g. turn left, turn right), retest. That way, the whole team can use what it learned all along the project to extend the interface, possibly minimizing interface reworks.
The Progress before Hardware article describes such a process in greater details, focusing on the Test-Driven-Development (TDD) aspect.
See also answers to the How to do TDD with hardware question.
I think it's a very interesting situation.
I believe there's no problem with your testing process. If you mock your robot and test against this mock, it's all good.
If the hardware robot acts different as your mocked robot, there's another big problem: The communication.
The interface between the software and the hardware is the "protocol" specification. In my opinion it must not be changed without discussion. The hardware guys may not change it and you software guys not, too! You only may change it together. In your situation, everybody changes it on his own.
In your situation, your teams seem to work against each other. So try to focus your efforts in your interface and especially your communication, not in your integration test that won't work anyway.
A suggestion of mine would be to use a robot's software mock as the one and only specification. So you can rely on your mock and there's a central point that defines the connection between hard- and software.
When the software guys want to change it, ok. They have to discuss it with you and you will change the software mock. If the hardware was changed and the mock not, you have an apology, because you develop against your specification.
Good luck!
I had an idea I was mulling over with some colleagues. None of us knew whether or not it exists currently.
The Basic Premise is to have a system that has 100% uptime but can become more efficient dynamically.
Here is the scenario: * So we hash out a system quickly to a
specified set of interfaces, it has
zero optimizations, yet we are
confident that it is 100% stable
though (dubious, but for the sake of
this scenario please play
along) * We then profile
the original classes, and start to
program replacements for the
bottlenecks.
* The original and the replacement are initiated simultaneously and
synchronized.
* An original is allowed to run to completion: if a replacement hasn´t
completed it is vetoed by the system
as a replacement for the
original.
* A replacement must always return the same value as the original, for a
specified number of times, and for a
specific range of values, before it is
adopted as a replacement for the
original.
* If exception occurs after a replacement is adopted, the system
automatically tries the same operation
with a class which was superseded by
it.
Have you seen a similar concept in practise? Critique Please ...
Below are comments written after the initial question in regards to
posts:
* The system demonstrates a Darwinian approach to system evolution.
* The original and replacement would run in parallel not in series.
* Race-conditions are an inherent issue to multi-threaded apps and I
acknowledge them.
I believe this idea to be an interesting theoretical debate, but not very practical for the following reasons:
To make sure the new version of the code works well, you need to have superb automatic tests, which is a goal that is very hard to achieve and one that many companies fail to develop. You can only go on with implementing the system after such automatic tests are in place.
The whole point of this system is performance tuning, that is - a specific version of the code is replaced by a version that supersedes it in performance. For most applications today, performance is of minor importance. Meaning, the overall performance of most applications is adequate - just think about it, you probably rarely find yourself complaining that "this application is excruciatingly slow", instead you usually find yourself complaining on the lack of specific feature, stability issues, UI issues etc. Even when you do complain about slowness, it's usually an overall slowness of your system and not just a specific applications (there are exceptions, of course).
For applications or modules where performance is a big issue, the way to improve them is usually to identify the bottlenecks, write a new version and test is independently of the system first, using some kind of benchmarking. Benchmarking the new version of the entire application might also be necessary of course, but in general I think this process would only take place a very small number of times (following the 20%-80% rule). Doing this process "manually" in these cases is probably easier and more cost-effective than the described system.
What happens when you add features, fix non-performance related bugs etc.? You don't get any benefit from the system.
Running the two versions in conjunction to compare their performance has far more problems than you might think - not only you might have race conditions, but if the input is not an appropriate benchmark, you might get the wrong result (e.g. if you get loads of small data packets and that is in 90% of the time the input is large data packets). Furthermore, it might just be impossible (for example, if the actual code changes the data, you can't run them in conjunction).
The only "environment" where this sounds useful and actually "a must" is a "genetic" system that generates new versions of the code by itself, but that's a whole different story and not really widely applicable...
A system that runs performance benchmarks while operating is going to be slower than one that doesn't. If the goal is to optimise speed, why wouldn't you benchmark independently and import the fastest routines once they are proven to be faster?
And your idea of starting routines simultaneously could introduce race conditions.
Also, if a goal is to ensure 100% uptime you would not want to introduce untested routines since they might generate uncatchable exceptions.
Perhaps your ideas have merit as a harness for benchmarking rather than an operational system?
Have I seen a similar concept in practice? No. But I'll propose an approach anyway.
It seems like most of your objectives would be meet by some sort of super source control system, which could be implemented with CruiseControl.
CruiseControl can run unit tests to ensure correctness of the new version.
You'd have to write a CruiseControl builder pluggin that would execute the new version of your system against a series of existing benchmarks to ensure that the new version is an improvement.
If the CruiseControl build loop passes, then the new version would be accepted. Such a process would take considerable effort to implement, but I think it feasible. The unit tests and benchmark builder would have to be pretty slick.
I think an Inversion of Control Container like OSGi or Spring could do most of what you are talking about. (dynamic loading by name)
You could build on top of their stuff. Then implement your code to
divide work units into discrete modules / classes (strategy pattern)
identify each module by unique name and associate a capability with it
when a module is requested it is requested by capability and at random one of the modules with that capability is used.
keep performance stats (get system tick before and after execution and store the result)
if an exception occurs mark that module as do not use and log the exception.
If the modules do their work by message passing you can store the message until the operation completes successfully and redo with another module if an exception occurs.
For design ideas for high availability systems, check out Erlang.
I don't think code will learn to be better, by itself. However, some runtime parameters can easily adjust onto optimal values, but that would be just regular programming, right?
About the on-the-fly change, I've shared the wondering and would be building it on top of Lua, or similar dynamic language. One could have parts that are loaded, and if they are replaced, reloaded into use. No rocket science in that, either. If the "old code" is still running, it's perfectly all right, since unlike with DLL's, the file is needed only when reading it in, not while executing code that came from there.
Usefulness? Naa...