I am using SFML, which handles most of it's memory through std::vector<Uint32> and other types similar to that. All declarations use the default Allocator. The SFML library is compiled into .lib files that are statically linked against in Visual Studio. (I possess all the source code to it however if needed.)
My question is, what would be the least painful method of forcing SFML to use my custom small-objects/thread-safe allocator for STL containers, instead of the default? The less I need to alter the library itself, the better of course!
Edit:
If it helps at all, forget the implementation of the library; I can recompile that on the whim. For the sake of the question, imagine it's my code I am using. My goal is to change the default Allocator that all STL containers use, essentially.
Edit2:
If that is not at all possible, would overriding new itself be a proper method? I have read around that the default allocator is nothing more than sugar coating on the new/delete operations.
Well, the most 'bruteforce' or foolproof method would seem to create a preload library that implements malloc/free (and friends?)
This is usually the way in which heap debuggers/bounds checkers operate.
However, I know nothing about the SFML allocator requirements, so it might not work if the space is very limited. In that case, I suggest 'marshaling' (fancy word for copying) the data over to the custom-allocated regions when needed
Tangentially related:
Eletronics Art has a 'port' of STL for game development. It is heavilty geared to custom allocators (in fact, it comes without a default one!). You can have a look at
EASTL http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2007/n2271.html
at github
To change default new operator, just implement this functions (at global scope):
void* operator new (std::size_t size) throw (std::bad_alloc);
You may find more information there: http://www.cplusplus.com/reference/new/operator%20new/
I am not completely sure it would work with dynamic libraries.
Related
For a piece of multiplatform c++ code I am writing, I need a shared pointer. Currently the project does not use boost, and pulling it in would be extremely difficult or impossible from an administrative view. However, I can use select C++11 features, including shared pointers.
There is a problem with the standard shared pointers, they guarantee thread safety. That means that on some platforms/compilers, like GCC ( http://tinyurl.com/GCCSharedPtrLockPolicy ) atomics and mutexes will be needlessly used, but at least I can check and work around issues incurred by this. Then for other platforms ( http://tinyurl.com/msvscSharedPtr ) there does not even appear to be a way to check, what thread safety mechanisms are used. The original boost pointer provides only the most basic of thread safety guarantees ( http://tinyurl.com/SharedPtrThreadSafety ).
My core issue here is that on some platforms Atomics can cause costly synchronizations between CPU caches and unneeded Mutexes can cause calls to the OS that that may delay for not entirely related reasons. This code will be multi-threaded, but we have other synchronization methods for moving data between threads. A thread-safe shared pointer is simply not needed or wanted.
Normally, I would prefer to benchmark and make my decision, but because of the platforms this will run on, and be ported too, I cannot practically do so. I would need to test on some of the less popular platforms, where less optimized compilers tend to exist, but I do not have that ability currently.
I will try to make a push to get Boost pointers, but that is unlikely, what are my other options for when that fails? In the mean time I will research trying to get just the Shared_ptr out of boost, but I do not think that will be easy.
I could roll my own. This seems like a terrible idea, why would I have to re-invent something this basic.
If there is a library with that is simple and has liberal enough licensing, then I could simply copy their shared_ptr code and simplify rolling my own.
Edit: Pulling in anything from boost other than header only libraries has been struck out. I will be researching Loki as one of the answerers suggested. If that fails and no answers materialize here, I will roll my own :( .
I'd take a look at the one in Loki. Loki is considerably smaller than boost, and the smart pointer implementation in Loki is highly configurable.
boost shared_ptr supports single threading usage
you can #define the macro BOOST_SP_DISABLE_THREADS on a project-wide basis to switch to ordinary non-atomic reference count updates
citation from boost shared_ptr
gcc has a class __shared_ptr which takes a lock-policy. shared_ptr derives off this.
One of the policies is _S_single, which is for single threaded code (ie: non locked/non atomic reference count)
In c++11 you can use template aliases which will allow you to use the non-standard __shared_ptr class which shared_ptr derives off
template<typename T>
using st_ptr = __shared_ptr<T, __gnu_cxx::_S_single>;
If you don't have a conformant compiler yet you can roll your own by just inheriting off __shared_ptr and exposing the interface (in fact this is how gcc currently does it because of the fact that template aliases were not available prior to 4.7)
Look in bits/shared_ptr.h to see how shared_ptr derives off __shared_ptr - it will be trivial to roll your own.
It is quite possible to write/adapt your own smart pointer class for use in legacy projects that the newer libraries do not support. I have written my own for just such a reason and it works well with MSVC++ 4.2 and onwards.
See ootips.org/yonat/4dev/smart-pointers.html which is what I based mine on. Definitely a possibility if you want a small solution. Just the header and .cpp file required.
The key point to watch out for is the lack of the explicit keyword in older compilers. Another is that you may want to allow implicit conversion to the raw pointer to allow your APIs to remain less affected (we did this) and in that case you should also take care to prevent conversion to other types of pointers as well.
This question already has answers here:
Closed 11 years ago.
Possible Duplicate:
To STL or !STL, that is the question
Are there cases where one should avoid the use of the C++ STL in his / her project?
When you choose to use a framework like Qt you might consider using lists, vectors, etc from Qt rather that the STL. Not using STL in this case saves you from having to convert from STL to the Qt equivalents when you need to use them in your GUI.
This is debatable and not everyone wants to use everything from Qt
from http://doc.qt.nokia.com/latest/containers.html
These container classes are designed to be lighter, safer, and easier to use than the STL containers. If you are unfamiliar with the STL, or prefer to do things the "Qt way", you can use these classes instead of the STL classes.
If you cannot use RTTI and/or exceptions, you might experience that parts of STL won't work. This is the case e.g. for native Android apps. So if it doesn't give you what you need, it's a reason not to use it!
Not really. There's no excuse to ban the use of an entire library- unless that lib only serves one function, which is not the case with the Standard library. The provided facilities should be evaluated on a per-function basis- for example, you may well argue that you need a container that performs a more specific purpose than vector, but that is no excuse to ban the use of deque, iostream or for_each too.
More importantly, code generated via template will not be more bloated than the equivalent code written by hand. You won't save code bloat by refusing to use std::vector and then writing your equivalent vector for float and double. Especially in 2011, the size of an executable is pretty meaningless compared to the sizes of other things like media in the vast, vast majority of situations.
If you care a lot about executable size, then you might want to avoid using STL in your program.
For example, uTorrent doesn't use STL and that is one reason why it's so small.
Since STL does rely on templates a lot (it is Standard TEMPLATE Library, after all), many times you use templates, the compiler has to generate extra code for every type you use when dealing with STL.
This is compile time polymorphism and will increase your executable size the more you use it.
If you exclude STL from your project (and use templates sparingly or not at all), your code size will get smaller. Note that it won't necessarily be faster.
Also note that I'm not talking about a program's memory usage during execution, since that will depend on how many objects your allocating during your app's lifetime.
I'm talking about your binary's executable.
If you want an example, note that a simple Hello world program, when compiled, might be bigger than a cleverly code demo which can include an entire 3D engine (run time generated) in a very small executable.
Some info regarding uTorrent's size:
Official FAQ (from 2008), this question doesn't appear in recent FAQ.
How was uTorrent programmed to be so efficient?
Second post regarding this.
Third post regarding this.
Note that, even though uTorrent is >300kb and is compressed with UPX, it is still really small when you take into account what it's capable of doing.
I would say that there may be occasions where you do not use a particular feature of STL in your project for a given circumstance because you can custom write it better for your needs. STL collections are generic by nature.
You might want in your code:
Lock-free containers that are thread-safe. (STL ones are not).
A string class that is immutable by nature and copies the actual data "by reference" (with some mechanism).
An efficient string-building class that is not ostringstream (not part of STL anyway but you may mean all the standard library)
algorithms that use Map and Reduce (not to be confused with std::map. Map and Reduce is a way to iterate over a collection using multiple threads or processes, possibly even distributed on different machines).
Hey, look, so much of boost was written because what the Standard Library provided at the time did not really address the needs of the programmer and thus provided alternatives.
I am not sure if this is what you meant or if you particular meant STL should be "banned" at all times (eg device driver programming where templates are considered bloaty even though that is not always the case).
If you are working to particular standards that forbid it.
For example, the MISRA C/C++ guidelines are aimed at automotive and embedded systems and forbid using dynamic memory allocation, so you might choose to avoid STL containers altogether.
Note: The MISRA guideline is just an example of a standard that might influence your choice to use STL. That particular guideline doesn't rule out using all of the STL. But (I believe) it rules out using STL containers as they rely on runtime allocation of memory.
It can increase executable size. if you're running on an embedded platform you may wish to exclude the STL.
When you use something like the Qt library that implements identical functionality you may not need the STL. May depend on other needs, like performance.
The only reason is if you are working on embedded systems with low memory, or if your project coding guidelines explicitly forbid STL.
I can't other reasonable reason to manually roll your own incompatible, bug ridden implementation of some of the features on STL.
TR18015 deals with some of the limitation of the STL. It looks at it from a different angle - what compilers could do better - but still is an interesting (if in-depth) read.
I'd be careful in general with microprocessors and small embedded systems. First, compiler optimizations are not up to what you know from desktops, and you run into hardware limits much sooner.
Having said that, it depends a lot on the libraries you use. I/O streams are notoriously slow (and require a careful implementation to not be), whereas std::vector is merely a thin wrapper.
I have a library written in C++/CLI and I want to open it up. I want it to be as cross-platform as possible and be able to write bindings to it for other languages to use (Java, Python, etc, etc). To do this, the library needs to be in plain C++ for maximum flexibility. I figure that the logical structures are already there, I just need to replace the .NET libraries it uses with the standard C++ ones. Is this a misguided notion? What should I watch out for when making this transition?
It might be more trouble than it's worth. Here is what you might come across:
There is no garbage collection in C++. This is the big one. This may require a significant redesign of your library just to convert. If you are using at least C++ tr1, or the boost library, you can sort of get there by using shared_ptr, but there are important fundamental differences. For example, you must be wary of circular dependencies. Also, they make debugging difficult without specific support for them in the debugger.
Functions in .Net classes which have no equivalent in C++ stl or the standard library. Probably the biggest hurtle will be any string manipulation code you have written since there are lot of differences there.
Class libraries/assemblies are not built-in to C++ - every platform has its own method of creating dynamic or shared libraries, and there isn't much support for C++ shared libraries - only C libraries in many cases. Be prepared to make everything a static library.
You must manage all your resources yourself.
Never done a port of C++/Cli to C++, but this comes to my mind:
Make sure that you dont have memory leaks. Use smart-pointers instead of gcnew, if possible (if not, make sure your code is exception safe nontheless).
Make sure your libraries interface only consists of builtin types (builtin does not include types of the STL! however this is not coercively necessary if you go open source)
My company are currently not won over by the Boost libraries and while I've used them and have been getting them pushed through for some work, some projects due to their nature will not be allowed to use Boost. Basically libraries, like Boost, cannot be brought in for work so I am limited to the libraries available by default (Currently using Visual Studio 2005).
So... My question is, if I am unable to use Boost::shared_ptr and its little brothers, what is the alternative when using STL containers with pointers?
One option I see is writing a container class like a shared_ptr that looks after a given pointer, but I'd like to know if there are other alternatives first.
If they're not going to accept boost, I presume other "not developed here" libraries are out of the question.
It seems to me you're left with two options:
Roll your own shared_ptr.
Use raw pointers, and manage the memory yourself.
Neither is ideal, and each comes with it's own pain. Your saving grace might be that you have all of the source to boost available to you. You can use it as a model for writing your own shared_ptr.
In Visual Studio 2008 there is available std::tr1::shared_ptr. I'm not sure it is available in VS2005, you should check.
That definetly depends on what you want to do. It's not as if shared_ptr are absolutely necessary for a project that uses pointers.
If you really need them, import those classes/templates/functions you really need to your own project if possible without importing the whole boost lib.
Without knowing the background, it's hard to say why boost's libraries aren't permitted. If the reason is to avoid complex dependencies, you can easily work around the issue: Almost all boost libraries work with only a simply #include header: in short, they don't need linking and thus avoid dll-hell or any variant thereof.
So, if external libraries aren't appreciated because of the complexities involved in linking (whether statically or dynamically) them, you can simply copy the boost headers you need into the project by hand and use them directly.
For clarity and to make future upgrades and maintenance easier I'd avoid renaming the boost libraries (so future coders know where the code came from). If "they" don't want such simple code inclusions, well, you could make the argument that quite a few boost headers are headed for inclusion in the spec, and that they'll save everyone a bunch of headaches and time. Legally, the boost license is specifically designed to be as easy and safe to integrate as possible: all files have an explicit license which permits all relevant things, and almost all libs have exactly the same license.
I'm curious though: why exactly aren't boost headers permitted?
We are using c++ to develop an application that runs in Windows CE 4 on an embedded system.
One of our constraint is that all the memory used by the application shall be allocated during startup only. We wrote a lot of containers and algorithms that are using only preallocated memory instead of allocating new one.
Do you think it is possible for us to use the boost libraries instead of our own containers in these conditions?
Any comments and/or advice are welcomed!
Thanks a lot,
Nic
We use boost for embedded systems. With boost you can pick and choose what you use. We use smart_ptr and boost::bind in all of our projects. We write software for cheap cell phones.
And if Windows CE can run on your hardware I would expect that parts of boost would be applicable.
There are parts of boost that have no allocation and you might find them useful.
I would pick and choose based on your requirements.
Like anything that you use, you need to know the costs.
You could write your own allocator for the container, which allocates from a fixed size static buffer. Depending on the usage patterns of the container the allocator could be as simple as incrementing a pointer (e.g. when you only insert stuff into the container once at app startup, and don't continuously add/remove elements.)
Replacing your containers with Boost containers is NOT a good idea. The work to make appropriate custom allocators wouldn't be that bad, but you'd be violating the spirit of your 'allocate at startup' rule. The idea behind this rule (in my experience) is generally to make sure that you don't have to deal with out of memory type situations at run-time. The idea is to make sure that you have all the memory you could possibly need RIGHT AT THE START, so that there's no possibility of any part of the system coming up short of memory later on.
If you used the Boost containers with a custom allocator, you'd suddenly have to deal with the possibility that the pool the container is allocating from could go empty, thus eliminating the purpose of the 'allocate at startup' rule.
In the situation of a limited memory device, I would avoid any kind of container more complex than a statically allocated array.
Boost is a set of libraries. Some of them are focussed on template metaprogramming. Those don't even use any memory at runtime. But your question seems to be about replacing your containers. I'd doubt that is possible except using custom allocators. But even then, it's most likely you would be using plain STL containers and not boost. Boost only provides the TR1 containers, for those compilers that do not yet include TR1.
Do not use Boost.
It is a big library and your basic memory allocation requirements are very different from those of the libraries designers.
Even if you can get a current version of Boost to work according to your requirements with custom allocators it may break with a new version of Boost.
Feel free to look at the Boost source code though for some useful ideas but use your own implementation for what you need.
I'm looking into this right now — I would like to use circular buffers, lock-free containers, and asynchronous I/O, and instead of allocating dynamic memory, I'd prefer to use memory pools.
The biggest problem I've seen so far is that shared_ptr is used in a lot of places, with no easy way to replace it with intrusive_ptr. Since shared_ptr allocates dynamic memory to keep track of the reference count, I can't use it in an embedded system.
Fixing this looks doable, but a lot of work — I have to expand the template specification of any class that contains a shared_ptr so that the specific type of shared-pointer can be changed to intrusive_ptr if desired. So now I have to consider how much work that'll be, versus how much work it'll be to write my own version of the Boost features I need. Not a pleasant place to be.
I hope someone points out why I'm wrong about this.