I have set up the AVR compiler for using with an Atmel microcontroller using this guide.
I don't have access to strings, vectors etc. How can this be added?
The quick answer is that they are not available and you need to write your own wrapper classes to get this sort of functionality.
If you want to use c++ for the embedded platform you won't have access to all of the standard library. Importantly though, you don't want all of the standard library as it's too heavyweight for some embedded projects. Some language features (like exception handling) might not be possible on the platform you are choosing or might be too expensive given the resources available to you. The lack of some language features makes it impossible to implement certain standard containers, for example the containers that can throw exceptions might not be able to be implemented in a standards-conforming way on some platforms. Additionally there's some c++ constructs that might be available but would be a bad idea to use on the embedded platform. Dynamic allocation of memory via new and delete will very likely run you into a significant number of problems as you don't have a lot of memory and issues such as memory fragmentation are very difficult to deal with. (you would probably want to look into placement new along with some other memory allocation scheme to avoid some of these issues if you needed dynamic memory for some reason)
If you want to have the benefits of containers like std::array and std::string you will need to write your own memory management classes. One of the main benefits of using the std containers is the way in which they greatly simplify your memory management (compared with using raw C-style-arrays). If you are doing a large embedded c++ project you can write your own wrappers for the memory management using RAII and other basic c++ language constructs. For the most part you need to avoid dynamic memory allocation and exception handling when making these classes.
One of the things I find has a good ROI is making some structs/classes that wrap an array along with the length of the array. By keeping the sizes connected you can keep your code a lot clearer. Frequently I find myself writing something like this:
template<typename T, uint8_t MAX_SIZE>
class array_helper{
public:
typedef T value_type;
array_wrapper():
m_data()
{}
T& operator[](unsigned int idx){
return m_data[idx];
}
T* data(){
return this->m_data;
}
const uint8_t s_max_size = MAX_SIZE;
private:
T m_data[MAX_SIZE];
};
You would want to expand on this to do what you need, but hopefully this gives you an idea.
do not do this.
using dynamic memory allocation on avr is not recommendable, since it has not a MMU and only very limited RAM and dynamic memory allocation requires some overhead for bookkeeping.
also there is the danger of memory fragmentation.
on such tiny processors you should only use static and autmatic fixed size memory buffers.
that ensures deterministic run time behavior.
Related
Is it possible to "memory protect" pointers so that it's actually impossible to change them in the code, so that attempting to change them in the code results in a bus error? I am not referring to const but some type of deeper immutability assurance on the OS level. The question applies to any OS.
(Carmack mentions something like that here: https://youtu.be/Uooh0Y9fC_M?t=1h41m)
Since your question is quite general, here are some general ideas.
From within your application, the language gives you a number of constructs to protect your data. For example, data that is not meant to be exposed should be flagged as private. This does not prevent clients from reverse engineering your class though, for example if you have
class C {
public: int a;
private: int b;
};
then usually you will be able to access b through int* pB = &(c.a) + 1. This is not standard by any definition, but on most systems this will probably work. In general, because C++ allows very low-level memory management you can basically access any part of your applications memory from anywhere within the application, though making sensible (ab)use of this requires some reverse engineering.
When you expose public data, you can return const references and pointers, but of course it is very easy to still change this memory:
const* T myImmutable = new T();
const_cast<T*>(myImmutable)->change(); // Oops
Again, this is not standard C++ as compilers will use the const qualifier to perform optimizations, and things may break when you circumvent that, but the language does not stop you from doing this.
If you want to protect your memory from external changes, things become a bit trickier. In general, the OS makes sure that the memory assigned to different processes is separated and you cannot just go and write in the memory space of other processes. However, there are some functions in the Windows API (Read/WriteProcessMemory) that one may use. Of course, this requires heavy reverse engineering to determine exactly where in memory the pointer to be changed is located, but it is possible. Some ways of protecting against this are VirtualProtect, as mentioned in Dani's answer. There are increasingly complex things that you can do, like keeping checksums of important data or
[writing a] driver that monitors the SSDT and then catches when WriteProcessMemory or ReadProcessMemory is executed and you can squash those calls if they're pointed at the game
but as the first answer on the page I found that on correctly points out:
Safety doesn't exist. The only thing you can do is make it as hard as possible to crack
which is a lesson that you should never forget!
Are there any methods to give the compiler hints that some objects may have a more static behaviour, and allocate things on the stack instead of heap ?
For example a string object might have a kind of a constant size inside some functions.
I'm asking this because I'm trying to improve performance for an application by using OpenMP. I've already improved the serial part from going from 50 to 20 seconds, and it goes to 12 seconds with parallelism (mentioning that most of the code can be run in parallel). I'm trying to continue improvement. I think one limitation is related to continuous allocation and release of dynamic memory inside the same process.
The serial optimizations, so far, were related to merging to a more ANSI C approach, with a more hardcoded allocation of variables (they are allocated dynamically, but considering a worst case scenario, so everything is allocated once).
Now I'm pretty much stuck, because I've reached a part of the code which has a lot of C++ approach.
The standard std::basic_string template (of which std::string is a specialization) accepts an allocator as its third argument, and you might supply your own stack-based allocator instead of std::allocator, but that would be brittle and tricky (you could use alloca(3) and ensure that all the allocations are inlined; if they are not alloca won't work as you want it.). I don't recommend this approach.
A more viable approach could be to have your own arena or region based allocator. See std::allocator_traits
You could perhaps simply use the C snprintf(3) on a large enough local buffer (e.g. char buf[128];)
I think you are looking for a small buffer optimization.
Detailed description can be found Here.
Basic idea is to add an union to the class, that will hold buffer:
class string
{
union Buffer
{
char* _begin;
char[16] _local;
};
Buffer _buffer;
size_t _size;
size_t _capacity;
// ...
};
so you are looking for deficiencies by using static analysis to find performance regressions?
It's a good idea, cppcheck has some of those, but those are very rudimentary.
I'm not aware of any tool that does that so far.
There are however tools that do different things:
jemalloc
jemalloc has a allocation profiler. (See: http://www.canonware.com/jemalloc/)
Perhaps this is of some help to you. I haven't tried it myself sofar, but I would expect it to post object lifetimes and the objects that produce the highest pressure on the allocator (to find the most hurting parts first).
cacheGrind
Valgrind has also a cache and branch prediction simulator. http://valgrind.org/docs/manual/cg-manual.html
clang-check
if you find yourself having too much freetime you can try to run your own checking tools using clang-check.
google perftools
The google perf tools also have a heap profiler. https://code.google.com/p/gperftools/
I know reference counter technique but never heard of mark-sweep technique until today, when reading the book named "Concepts of programming language".
According to the book:
The original mark-sweep process of garbage collection operates as follow: The runtime system allocates storage cells as requested and disconnects pointers from cells as necessary, without regard of storage reclamation ( allowing garbage to accumulate), until it has allocated all available cells. At this point, a mark-sweep process is begun to gather all the garbage left floating-around in the heap. To facilitate the process, every heap cells has an extra indicator bit or field that is used by the collection algorithm.
From my limited understanding, smart-pointers in C++ libraries use reference counting technique. I wonder is there any library in C++ using this kind of implementation for smart-pointers? And since the book is purely theoretical, I could not visualize how the implementation is done. An example to demonstrate this idea would be greatly valuable. Please correct me if I'm wrong.
Thanks,
There is one difficulty to using garbage collection in C++, it's to identify what is pointer and what is not.
If you can tweak a compiler to provide this information for each and every object type, then you're done, but if you cannot, then you need to use conservative approach: that is scanning the memory searching for any pattern that may look like a pointer. There is also the difficulty of "bit stuffing" here, where people stuff bits into pointers (the higher bits are mostly unused in 64 bits) or XOR two different pointers to "save space".
Now, in C++0x the Standard Committee introduced a standard ABI to help implementing Garbage Collection. In n3225 you can find it at 20.9.11 Pointer safety [util.dynamic.safety]. This supposes that people will implement those functions for their types, of course:
void declare_reachable(void* p); // throw std::bad_alloc
template <typename T> T* undeclare_reachable(T* p) noexcept;
void declare_no_pointers(char* p, size_t n) noexcept;
void undeclare_no_pointers(char* p, size_t n) noexcept;
pointer_safety get_pointer_safety() noexcept;
When implemented, it will authorize you to plug any garbage collection scheme (defining those functions) into your application. It will of course require some work of course to actually provide those operations wherever they are needed. One solution could be to simply override new and delete but it does not account for pointer arithmetic...
Finally, there are many strategies for Garbage Collection: Reference Counting (with Cycle Detection algorithms) and Mark And Sweep are the main different systems, but they come in various flavors (Generational or not, Copying/Compacting or not, ...).
Although they may have upgraded it by now, Mozilla Firefox used to use a hybrid approach in which reference-counted smart pointers were used when possible, with a mark-and-sweep garbage collector running in parallel to clean up reference cycles. It's possible other projects have adopted this approach, though I'm not fully sure.
The main reason that I could see C++ programmers avoiding this type of garbage collection is that it means that object destructors would run asynchronously. This means that if any objects were created that held on to important resources, such as network connections or physical hardware, the cleanup wouldn't be guaranteed to occur in a timely fashion. Moreover, the destructors would have to be very careful to use appropriate synchronization if they were to access shared resources, while in a single-threaded, straight reference-counting solution this wouldn't be necessary.
The other complexity of this approach is that C++ allows for raw arithmetic operations on pointers, which greatly complicates the implementation of any garbage collector. It's possible to conservatively solve this problem (look at the Boehm GC, for example), though it's a significant barrier to building a system of this sort.
Would it be possible in C++ to create a custom allocator that works simply like this:
{
// Limit memory to 1024 KB
ScopedMemoryPool memoryPool(1024 * 1024);
// From here on all heap allocations ('new', 'malloc', ...) take memory from the pool.
// If the pool is depleted these calls result in an exception being thrown.
// Examples:
std::vector<int> integers(10);
int a * = new int [10];
}
I couldn't find something like this in the boost libraries, or anywhere else.
Is there a fundamental problem that makes this impossible?
You would need to create a custom allocator that you pass in as a template param to vector. This custom allocator would essentially wrap the access to your pool and do whatever size validations that it wants.
Yes you can make such a construct, it's used in many games, but you'll basically need to implement your own containers and call memory allocation methods of that pool that you've created.
You could also experiment with writing a custom allocator for the STL containers, although it seems that that sort of work is generally advised against. (I've done it before and it was tedious, but I don't remember any specific problems.)
Mind- writing your own memory allocator is not for the faint of heart. You could take a look at Doug Lea's malloc, which provides "memory spaces", which you could use in your scoping construct somehow.
I will answer a different question. Look at 'efficient c++' book. One of the things they discuss is implementing this kind of thing. That was for a web server
For this particular thing you can either mess at the c++ layer by overriding new and supplying custom allocators to the STL.
Or you can mess at the malloc level, start with a custom malloc and work from there (like dmalloc)
Is there a fundamental problem that makes this impossible?
Arguing about program behavior would become fundamentally impossible. All sorts of weird issues will come up. Certain sections of the code may or may not execute though this will seeminly have no effect on the next sections which may work un-hindered. Certain sections may always fail. Dealing with the standard-library or any other third party library will become extremely difficult. There may be fragmentations at run-time at times and at times not.
If intent is that all allocations within that scope occur with that allocator object, then it's essentially a thread-local variable.
So, there will be multithreading issues if you use a static or global variable to implement it. Otherwise, not a bad workaround for the statelessness of allocators.
(Of course, you'll need to pass a second template argument eg vector< int, UseScopedPool >.)
I am working on a group senior project for my university and I have run into a major hurdle in trying to get my code to work.
The compiler that we have for our 8 bit Atmel microcontroller does not support the new or delete operators, and it does not support the C++ STL. I could program it in C, but I have to implement an A* algorithm which I have never done before. While I have tried C initially I soon realized that I never did pure C before. Trying to model objects with structs and functions is slowing me down since I am so used to the much cleaner C++ syntax.
Regardless, the exact wording for my compilers shortcomings can be found here: http://www.nongnu.org/avr-libc/user-manual/FAQ.html#faq_cplusplus
To overcome them and still use C++ I have considered the following possibilities.
1) Don't allocate anything, just use templates to generate fixed arrays on the stack.
2) Allocate and find some hack to call the constructor for objects once I have allocated the space for them. Placement new isn't an option since new isn't an operator.
3) Just use C and suck it up, its a microcontroller why am I getting fancy?
4) Find a better compiler which will probably cost $$$.
The second option is the hardest but it would have the biggest pay off in terms of how I can write this code. However, I imagine that debugging it could be a huge pain if I get it wrong. I'm thinking of creating objects on the stack, copying their bits into the allocated space, and then zeroing the bits in the object so it doesn't call its destructor. To do that I would access the bits directly with an unsigned char pointer and the sizeof operator to get the byte count.
That sounds terrible and I don't know if it could work reliably, but I am considering it. I know vtables can be a problem but I don't intend on having any vtables since it is just an 8 bit microcontroller.
Don't fight your tools. If the only compiler you have for your embedded system is a C compiler, learn C - it's not difficult. Trying to produce some bastardised version of the two languages just to solve a fairly simple programming problem will only
end in tears.
To look at it another way, if your embedded platform didn't even support a C compiler, but only an assembler, would your first impulse be to sit down and write a C++ compiler in assembler? I hope not, I hope you would instead sit down and learn to use
the assembler to complete your assignment - writing a C++ compiler (or even a C compiler) would be totally inappropriate use of your time, and would almost certainly result in failure.
Just for the record, zeroing the bits in an object won't affect whether the destructor gets called (unless the compiler has a special quirk that enables this behaviour). Just write some logging statements in your destructor to test this out.
Structuring your program not to allocate anything is probably the way the system was designed. I've not worked with embedded systems before, however I have read some experienced embedded shops that discourage use of dynamic memory because the runtime environment has scarce amounts of it.
However, if you must, you can still use placement new. If you don't have the <new> header, here are the relevant lines directly from it on my version of GCC:
// Default placement versions of operator new.
inline void* operator new(std::size_t, void* __p) throw() { return __p; }
inline void* operator new[](std::size_t, void* __p) throw() { return __p; }
// Default placement versions of operator delete.
inline void operator delete (void*, void*) throw() { }
inline void operator delete[](void*, void*) throw() { }
Stick that somewhere in a header file included by every source file that uses placement new/delete.
Sample file that tests this:
#include <cstdio>
#include <new>
int
main(int argc, char** argv)
{
typedef char const* cstr;
char foobar[16];
cstr* str = new (&foobar) cstr(argc > 1 ? argv[1] : "Hello, world!");
std::puts(*str);
str->~cstr();
}
On my version of GCC, this does not use libstdc++ at all (if -fno-exceptions is used).
Now, if you want to combine that with malloc (if your platform provides this), then you can do this:
#include <cstdio>
#include <cstdlib>
inline void* operator new (std::size_t n) {return std::malloc(n);}
inline void* operator new[](std::size_t n) {return std::malloc(n);}
inline void operator delete (void* p) {std::free(p);}
inline void operator delete[](void* p) {std::free(p);}
int
main(int argc, char** argv)
{
typedef char const* cstr;
cstr* str = new cstr(argc > 1 ? argv[1] : "Hello, world!");
std::puts(*str);
delete str;
}
This allows you to use the standard new/delete that you're familiar with, without requiring use of libstdc++.
Good luck!
I think you are approaching the problem from a viewpoint that is less than optimum.
You are focusing on the compiler (or lack thereof) instead of focusing on the HARDWARE.
The most probable answer to your main questions is "because the hardware doesn't support all that C++ stuff". Embedded hardware (microcontrolers) are noted for the customization of the hardware design - memory maps, interrupt handlers, I/O, etc.
In my opinion, you should FIRST spend some time with the hardware book for the microcontroller, learning the ins and outs of the device - i.e. how it was designed and for what primary purpose. Some were designed for fast memory manipulation, some for fast I/O handling, some for A/D type work, some for signal processing. The type of microcontroller dictates the assembler instructions they wrote for it, and that dictates what any higher-level compiler can do efficiently.
If this is important, spend some time to look at the assembler as well - it will tell you what the designers considered important. It will also tell you a lot about how much you can get from a high-level compiler.
Generally, microcontrollers don't support C++ because the design really doesn't care about objects, or fancy memory handling (from the C++ perspective). It can be done, but you are often trying to pound a round peg in a square hole to get constructors and destructors (and 'new' and 'delete') to work in the micro environment.
IF you have a C compiler for this unit, consider it a blessing. A good C compiler is often "more than enough" to create excellent embedded software.
Cheers,
-Richard
Just because it doesn't have these tools doesn't mean you can't benefit from C++. If the project is large enough, access to Object Oriented design alone could be motivation enough.
If it doesn't support 'new' then it's probably because it doesn't make sense to make an automatic distinction between a heap and the stack. This might be because of your memory configuration. It might also be because memory resources are so constrained only very careful allocation makes sense. If you absolutely have to implement your own 'new' operator, you might look into adapting Doug Lea's malloc. I believe he began his allocator in a similar circumstance (reimplementing C++'s new).
I love the STL but it's still possible to do useful stuff without it. Depending on the scope of the project you might be better off just using an array.
I had a similar compiler that implemented a bizarre version of the Embedded-C++ standard. We had operator new which would call constructors for us and destructors were called in most cases. The compiler/runtime vendor went and implemented try and catch using setjmp and longjmp as a convenience to the engineer. The problem was that they never mentioned that a throw would not cause destructors of local objects to be invoked!
Anyway, our group inherited the code base after someone wrote an application acting like it was Standard C++: using RAII techniques and all of the other goodness. We ended up rewriting it in what a number of us call object-oriented C instead. You might want to consider just biting the bullet and writing in straight C. Instead of constructors, have an explicitly called initialization method. Destructors become an explicitly called termination method. There isn't much of C++ that you can't mimic in C pretty quickly. Yes, MI is a pain in the ... but single inheritance is pretty easy. Take a look at this PDF for some ideas. It almost describes the approach that we took. I really wish I had written our method down somewhere...
You may find some helpful code on my A* tutorial website. Although the code I wrote to support this uses STL in should be easy to strip the STL support out. In addition there is a pool allocator included with it (fsa.h) that I wrote to speed up STL on game consoles. It is C++ code, but I ported it originally from C and I don't think it would be hard to do it the other way. The code is tested by over 10,000 people so it's a good base to start from.
Replacing the STL structures I'm using is no problem since it is limited to Vectors. I use one of the vectors as a priority queue using the heap functions (make_heap and push_heap). You can replace that with my old C code which has a priority queue implemented in C that should just drop into your code. (Which only does one alloc, so you can replace that with a pointer to a reserved area of your memory.
As you can see in this code fragment from the header, the main difference in C code is that there's no this pointer, no object, so your code typically takes an object pointer as the first argument.
void PQueueInitialise( PQUEUE *pq, int32 MaxElements, uint32 MaxRating, bool32 bIsAscending );
void PQueueFree( PQUEUE *pq );
int8 PQueuePush( PQUEUE *pq, void *item, uint32 (*PGetRating) ( void * ) );
int32 PQueueIsFull( PQUEUE *pq );
int32 PQueueIsEmpty( PQUEUE *pq );
void *PQueuePop( PQUEUE *pq, uint32 (*PGetRating) ( void * ) );
Why not write it first on your desktop computer, taking into consideration the limitations of the compiler, debug it, make sure it works perfectly and only then move to the embedded environment?
when doing embedded work, I once couldn't even link the C runtime for memory constraints, but the hardware had a DMA (dynamic memory allocator) instruction so I wrote my own malloc with that hardware, your hardware likely has a similar feature, so you could write a malloc and then a new based on the malloc.
Anyways in the end I used 99% stack allocations, and a few limits sets os static objects that I would recycle, by builiding in place. This might me a good solution.