consider the following code for a dynamic loaded library built with g++-4.7 on linux, -fPIC and linked with -rdynamic option:
struct Wrapper
{
libraryUnregisterCbMap_t instance;
Wrapper() : instance() { HDebugLog("Wrapper CTOR!");}
~Wrapper() { HDebugLog("Wrapper DESTRUCTOR!"); }
};
inline libraryUnregisterCbMap_t& getLibraryUnregisterMap()
{
static Wrapper unregisterLibraryMap;
HDebugLog("getLibraryUnregisterMap: we have " <<unregisterLibraryMap.instance.size() << " elements. the address of the map is " << &unregisterLibraryMap.instance);
return unregisterLibraryMap.instance;
}
void registerLibrary(callbackContainer_t* p)
{
auto& map = getLibraryUnregisterMap();
}
void unregisterLibrary()
{
auto& map = getLibraryUnregisterMap();
}
void __attribute__ ((constructor)) library_init()
{
static callbackContainer_t cbContainer;
HDebugLog("Library constructor: address of static cbContainer is: " << &cbContainer );
registerLibrary( &cbContainer);
}
void __attribute__ ((destructor)) library_fini()
{ unregisterLibrary(); }
the interesting/annoying part for me is that library_fini() is not being called after i call lt_dlclose, so it seems to be rather useless for finalisation, as when i load this module during a run, the destructor of Wrapper instance happens before the call to library_fini. Needless to say, this default behavior does not make any sense.
How do i change this meaningless behaviour? i need to finalise my static data in my library finalization routine. Why lt_dlclose is not invoking library_fini()?
Let me first admit that I'm out of my depth here. That said, googling turned up a thread that, at least to my limited knowledge, appears to address a similar problem to yours:
http://lists.apple.com/archives/xcode-users/2005/Aug/msg00133.html
Do you happen to be doing whatever you're doing on OSX? There's something in the thread (maybe the second follow-up) about OSX behaving differently, i.e. not calling destructors but just setting memory to be free.
Apologies if the link isn't useful. Just thought I'd have a go since no one else has answered at this point.
Edit:
Again, out of my depth - but I found two more links that might be relevant:
http://phoxis.org/2011/04/27/c-language-constructors-and-destructors-with-gcc/
in the comments, people mention having problems with destructors when they use exit, and having to use the atexit function to overcome these problems
http://clang-developers.42468.n3.nabble.com/Priority-settings-for-static-variables-and-attribute-destructor-td4030466.html
global resource destructed before attribute ((destructor)) function is called. Suggested solution is to use priorities with the destructor.
Related
I have some code where a static method is called, and the static std::unordered_map within the same file is not initialized. I understand the static initialization between two compile units is "undefined" and there are many SO questions on the topic; however, when I use an std::vector the issue does not occur. Also, the code can execute, but I am confused as to why these specific compile orders do not work. SO, my questions are:
There is another SO question (which I've been unable to find!) about static initialization and dynamic initialization of static variables. Is this error due to std::undored_map actually being a dynamic initialization?
is there a way to get this code to initialize the std::unordered_map as I expected? I'm actually trying to create a static library .lib or .a. When I link the static library, it generally needs to come last, and so the error occurs.
are there any workarounds for this? One option I've thought of is to create both an std::vector and an std::unordered_map. Use the std::vector while the std::unordered_map is uninitialized (via bool _map_is_initialized). Change the initialization of the std::unordered_map to be explicitly dynamic by calling a function which iterates over the values in the std::vector to produce the std::unordered_map.
Linux
g++ -std=c++1y -g -c thing.cpp
g++ -std=c++1y -g -c main.cpp
g++ -g main.o thing.o -o main
./main
This results in a Floating point exception (core dumped) error. Through gdb, I was able to figure out that hashtable_policy.h trys __num % __den; where __den==0. Also using gdb, it appears as though Thing::Things is uninitialized.
(gdb) break thing.cpp:12
(gdb) run
(gdb) print Thing::Things
No symbol "Things" in specified context.
(gdb) print thing
$1 = (Thing *) 0x618c20
Windows
cl /EHsc /Zi /c main.cpp
cl /EHsc /Zi /c thing.cpp
link /debug main.obj thing.obj
main
In my actual code, this resulted in a very clear segmentation fault; however, this example just opens a popup that says the application failed. ... I have not done better diagnostics.
Code
thing.cpp
#include<iostream>
#include "thing.hpp"
std::vector<Thing*> Before; // EDIT: added
std::unordered_map<std::string, Thing*> Thing::Things;
std::vector<Thing*> After; // EDIT: added
Thing::Thing(std::string name) : name(name) {
}
bool Thing::Register(Thing *thing) {
std::cout << "no worries, vectors initialized..." << std::endl;
Thing::Before.push_back(thing); // EDIT: added
Thing::After.push_back(thing); // EDIT: added
std::cout << "added to vectors, about to fail..." << std::endl;
Thing::Things[thing->name] = thing;
return true;
}
thing.hpp
#pragma once
#include <string>
#include <unordered_map>
class Thing {
public:
static std::vector<Thing*> Before; // EDIT: added
static std::unordered_map<std::string, Thing*> Things;
static std::vector<Thing*> After; // EDIT: added
static bool Register(Thing* thing);
std::string name;
Thing(std::string name);
};
#define ADD_THING(thing_name) \
static bool thing_name## _is_defined = Thing::Register(new Thing( #thing_name ));
main.cpp
#include "thing.hpp"
#include <iostream>
ADD_THING(obligatory);
ADD_THING(foo);
ADD_THING(bar);
int main(int argc, char* argv[]) {
std::cout << "before loop" << std::endl;
for (auto thing : Thing::Things) {
std::cout << "thing.name: " << thing.first << std::endl;
}
return 0;
}
EDIT
If the order within a given compile unit is guaranteed, why do static std::vector<Thing*> Thing::Before and static std::vector<Thing*> Thing::After get initialized, but static std::unordered_map<std::string, Thing*> Thing::Things does not?
As noted in the comments, static initialization order is not defined. Who knows the difference between vector and map. Maybe your compiler initializes classes with even number of characters in their name first.
If you're running c++11 or greater, static initialization of function local items is guaranteed to be thread safe. They will be initialized the first time control passes through the declaration statement.
// Header
class Thing {
public:
static std::unordered_map<std::string, Thing*>& Things();
static bool Register(Thing* thing);
// CPP
std::unordered_map<std::string, Thing*>& Thing::Things()
{
static std::unordered_map<std::string, Thing*> things;
return things;
}
This will initialize the first time you ask for the Things, and avoids all the potential randomness of static initialization.
Static initialization is tricky. As this answer states, the standard provides some guarantees as to the order of initialization within a single "translation unit" (normally a .cpp source file), but none whatsoever concerning what order initializations in different translation units will follow.
When you added the Before and After vectors to the code, you observed that unlike the calls to ordered_map::operator[], the calls to vector::push_back() did not crash the process and concluded that the objects were being initialized out of order within a single translation unit, contrary to the standard's guarantees. There is a hidden assumption there, namely that since push_back() did not cause a crash, the vector must therefore have been initialized. This turns out not to be the case: that method call on an uninitialized object is almost certainly corrupting memory somewhere, but won't necessarily cause a crash. A better way of checking whether or not the constructor is being called would be to run the code in a debugger, and set breakpoints on the lines which contain the objects' definitions, for instance std::vector<Thing*> Before in thing.cpp. This will show that initialization will occur as predicted in the standard.
The best option for avoiding the "fiasco", as described here, is "construct on first use". In the case of your example code, this would involve changing any direct use of Thing::Things, such as this line:
Thing::Things[thing->name] = thing;
To a method, say Thing::GetThings(), which initializes the object and returns a reference to it. lcs' answer provides an example of this, but beware: although it solves the static initialization problem, using a scoped static object may introduce an even more pernicious problem: crashes on program exit due to static deinitialization order. For that reason, allocating the object with the new keyword is preferred:
std::unordered_map<std::string, Thing*>& Thing::GetThings()
{
static std::unordered_map<std::string, Thing*>* pThings =
new std::unordered_map<std::string, Thing*>();
return *pThings;
}
That instance will of course never be delete'd, which feels an awful lot like a memory leak. But even if it weren't a pointer, de-initialization would only occur at program shutdown. So, unless the object's destructor performs some important function like flushing a file's contents to disk, the only difference that matters is the fact that using a pointer avoids the possibility of a crash on exit.
Consider the following code:
file_1.hpp:
typedef void (*func_ptr)(void);
func_ptr file1_get_function(void);
file1.cpp:
// file_1.cpp
#include "file_1.hpp"
static void some_func(void)
{
do_stuff();
}
func_ptr file1_get_function(void)
{
return some_func;
}
file2.cpp
#include "file1.hpp"
void file2_func(void)
{
func_ptr function_pointer_to_file1 = file1_get_function();
function_pointer_to_file1();
}
While I believe the above example is technically possible - to call a function with internal linkage only via a function pointer, is it bad practice to do so? Could there be some funky compiler optimizations that take place (auto inline, for instance) that would make this situation problematic?
There's no problem, this is fine. In fact , IMHO, it is a good practice which lets your function be called without polluting the space of externally visible symbols.
It would also be appropriate to use this technique in the context of a function lookup table, e.g. a calculator which passes in a string representing an operator name, and expects back a function pointer to the function for doing that operation.
The compiler/linker isn't allowed to make optimizations which break correct code and this is correct code.
Historical note: back in C89, externally visible symbols had to be unique on the first 6 characters; this was relaxed in C99 and also commonly by compiler extension.
In order for this to work, you have to expose some portion of it as external and that's the clue most compilers will need.
Is there a chance that there's a broken compiler out there that will make mincemeat of this strange practice because they didn't foresee someone doing it? I can't answer that.
I can only think of false reasons to want to do this though: Finger print hiding, which fails because you have to expose it in the function pointer decl, unless you are planning to cast your way around things, in which case the question is "how badly is this going to hurt".
The other reason would be facading callbacks - you have some super-sensitive static local function in module m and you now want to expose the functionality in another module for callback purposes, but you want to audit that so you want a facade:
static void voodoo_function() {
}
fnptr get_voodoo_function(const char* file, int line) {
// you tagged the question as C++, so C++ io it is.
std::cout << "requested voodoo function from " << file << ":" << line << "\n";
return voodoo_function;
}
...
// question tagged as c++, so I'm using c++ syntax
auto* fn = get_voodoo_function(__FILE__, __LINE__);
but that's not really helping much, you really want a wrapper around execution of the function.
At the end of the day, there is a much simpler way to expose a function pointer. Provide an accessor function.
static void voodoo_function() {}
void do_voodoo_function() {
// provide external access to voodoo
voodoo_function();
}
Because here you provide the compiler with an optimization opportunity - when you link, if you specify whole program optimization, it can detect that this is a facade that it can eliminate, because you let it worry about function pointers.
But is there a really compelling reason not just to remove the static from infront of voodoo_function other than not exposing the internal name for it? And if so, why is the internal name so precious that you would go to these lengths to hide that?
static void ban_account_if_user_is_ugly() {
...;
}
fnptr do_that_thing() {
ban_account_if_user_is_ugly();
}
vs
void do_that_thing() { // ban account if user is ugly
...
}
--- EDIT ---
Conversion. Your function pointer is int(*)(int) but your static function is unsigned int(*)(unsigned int) and you don't want to have to cast it.
Again: Just providing a facade function would solve the problem, and it will transform into a function pointer later. Converting it to a function pointer by hand can only be a stumbling block for the compiler's whole program optimization.
But if you're casting, lets consider this:
// v1
fnptr get_fn_ptr() {
// brute force cast because otherwise it's 'hassle'
return (fnptr)(static_fn);
}
int facade_fn(int i) {
auto ui = static_cast<unsigned int>(i);
auto result = static_fn(ui);
return static_cast<int>(result);
}
Ok unsigned to signed, not a big deal. And then someone comes along and changes what fnptr needs to be to void(int, float);. One of the above becomes a weird runtime crash and one becomes a compile error.
Somewhat related to my previous question here
Is there a way to get the calling Object from within a function or method in d?
example:
class Foo
{
public void bar()
{
auto ci = whoCalledMe();
// ci should be something that points me to baz.qux, _if_ baz.qux made the call
}
}
class Baz
{
void qux()
{
auto foo = new Foo();
foo.bar();
}
}
Questions:
Does something like whoCalledMe exist? and if so, what is it called?
if something does exist, can it be used at compile time (in a template) and if so, how?
Alternatively;
is it possible to get access to the call stack at runtime? like with php's debug_backtrace?
To expand on what CyberShadow said, since you can get the fully qualified name of the function by using __FUNCTION__, you can also get the function as a symbol using a mixin:
import std.stdio;
import std.typetuple;
void callee(string file=__FILE__, int line=__LINE__, string func=__FUNCTION__)()
{
alias callerFunc = TypeTuple!(mixin(func))[0];
static assert(&caller == &callerFunc);
callerFunc(); // will eventually overflow the stack
}
void caller()
{
callee();
}
void main()
{
caller();
}
The stack will overflow here since these two functions end up calling each other recursively indefinitely.
It's not directly possible to get information about your "caller". You might have some luck getting the address from the call stack, but this is a low-level operation and depends on things such as whether your program was compiled with stack frames. After you have the address, you could in theory convert it to a function name and line number, provided debugging symbols are available for your program's binary, but (again) this is highly platform-specific and depends on the toolchain used to compile your program.
As an alternative, you might find this helpful:
void callee(string file=__FILE__, int line=__LINE__, string func=__FUNCTION__)()
{
writefln("I was called by %s, which is in %s at line %d!", func, file, line);
}
void caller()
{
// Thanks to IFTI, we can call the function as usual.
callee();
}
But note that you can't use this trick for non-final class methods, because every call to the function will generate a new template instance (and the compiler needs to know the address of all virtual methods of a class beforehand).
Finding the caller is something debuggers do and generally requires having built the program with symbolic debug information switches turned on. Reading the debug info to figure this out is highly system dependent and is pretty advanced.
The exception unwinding mechanism also finds the caller, but those tables are not generated for functions that don't need them, and the tables do not include the name of the function.
The situation I have is that I am trying to initialize a file scoped variable, std::string, in a shared object constructor. It will probably be more clear in code:
#include <string>
#include <dlfcn.h>
#include <cstring>
static std::string pathToDaemon; // daemon should always be in the same dir as my *.so
__attribute__((constructor))
static void SetPath()
{
int lastSlash(0):
Dl_info dl_info;
memset(&dl_info, 0, sizeof(dl_info));
if((dladdr((void*)SetPath, &dl_info)) == 0)
throw up;
pathToDaemon = dl_info.dli_fname; // **whoops, segfault here**
lastSlash = pathToDaemon.find_last_of('/');
if(std::string::npos == lastSlash)
{
// no slash, but in this dir
pathToDaemon = "progd";
}
else
{
pathToDaemon.erase(pathToDaemon.begin() + (lastSlash+1), pathToDaemon.end());
pathToDaemon.append("progd");
}
std::cout << "DEBUG: path to daemon is: " << pathToDaemon << std::endl;
}
I have a very simple program that does this same thing: a test driver program for concept if you will. The code in that looks just like this: a "shared object ctor" which uses dladdr() to store off the path of the *.so file when the file is loaded.
Modifications I've tried:
namespace {
std::string pathToDaemon;
__attribute__((constructor))
void SetPath() {
// function def
}
}
or
static std::string pathToDaemon;
__attribute__((constructor))
void SetPath() { // this function not static
// function def
}
and
std::string pathToDaemon; // variable not static
__attribute__((constructor))
void SetPath() { // this function not static
// function def
}
The example you see above sits in a file that is compiled into both a static object library and a DLL. The compilation process:
options for static.a: --std=C++0x -c -Os.
options for shared.so: -Wl,--whole-archive /path/to/static.a -Wl,--no-whole-archive -lz -lrt -ldl -Wl,-Bstatic -lboost_python -lboost_thread -lboost_regex -lboost_system -Wl,-Bdynamic -fPIC -shared -o mymodule.so [a plethora of more objects which wrap into python the static stuff]
The hoops I have to jump through in the bigger project make a much more complicated build process than my little test driver program requires. This makes me think that the problem lies there. Can anyone please shed some light on what I'm missing?
Thanks,
Andy
I think it's worth giving the answer that I've found. The problem was due to the complex nature of the shared library loading. I discovered after some digging that I could reproduce the problem in my test bed program when compiling the code with optimizations enabled. That confirmed the hypothesis that the variable truly didn't exist when it was being accessed by the constructor function.
GCC includes some extra tools for C++ which allow for developers to force certain things to happen at particular times during code initialization. More precisely, it allows for certain things to take place in particular order rather than particular times.
For example:
int someVar(55) __attribute__((init_priority(101)));
// This function is a lower priority than the initialization above
// so, this will happen *after*
__attribute__((constructor(102)))
void SomeFunc() {
// do important stuff
if(someVar == 55) {
// do something here that important too
someVar = 44;
}
}
I was able to use these tools to success in the test bed program even with optimizations enabled. The happiness which ensued was short lived when applied to my much larger library. Ultimately, the problem was due to the nature of such a large amount of code and the problematic way in which the variables are brought into existence. It just wasn't reliable to use these mechanisms.
Since I wanted to avoid repeated calls for evaluating the path, i.e.
std::string GetPath() {
Dl_info dl_info;
dladdr((void*)GetPath, &dl_info);
// do wonderful stuff to find the path
return dl_info.dli_fname;
}
The solution turned out to be much simpler than I was trying to make it:
namespace {
std::string PathToProgram() {
Dl_info dl_info;
dladdr((void*)PathToProgram, &dl_info);
std::string pathVar(dl_info.dli_fname);
// do amazing things to find the last slash and remove the shared object
// from that path and append the name of the external daemon
return pathVar;
}
std::string DaemonPath() {
// I'd forgotten that static variables, like this, are initialized
// only once due to compiler magic.
static const std::string pathToDaemon(PathToProgram());
return pathToDaemon;
}
}
As you can see, exactly what I wanted with less confusion. Everything happens only once, except calls to DaemonPath(), and everything remains within the translation unit.
I hope this helps someone who runs into this in the future.
Andy
Maybe you could try running valgrind on your program
In you self posted solution above, you have changed your »interface« (for the code that reads your pathToDaemon / DaemonPath()) from »Accessing a file scoped variable« to »calling a function in anonymous namespace« - so far ok.
But the implementation of DaemonPath() is not done in a thread-safe way. I though that thread-safeness matters, because your are wrote »-lboost_thread« in your question. So you may think about to change the implementation thread-safe. There are many discussions and solutions about singleton pattern and thread-safeness available, e.g.:
Article from Scott Meyers
Stack Overflow
The fact is, that your DaemonPath() will invoked (maybe far) after loading of the library is done. Note, that only the 1st call to the singleton pattern is critical in a multithreaded environment.
As an alternative, you may add a simple »early« call to your DaemonPath() function like this:
namespace {
std::string PathToProgram() {
... your code from above ...
}
std::string DaemonPath() {
... your code from above ...
}
__attribute__((constructor)) void MyPathInit() {
DaemonPath();
}
}
or in a more portable way like this:
namespace {
std::string PathToProgram() {
... your code from above ...
}
std::string DaemonPath() {
... your code from above ...
}
class MyPathInit {
public:
MyPathInit() {
DaemonPath();
}
} myPathInit;
}
Of course, this approach don't makes your singleton pattern thread-safe. But sometimes, there are situations, we can be sure that there are no concurrent thread accesses (e.g. at initialization time when the shared lib is loading). If this conditions matches for you, this approach could be a way to bypass thread-safeness problem without the use of thread locking (mutex...).
I have a core file I am examining. And I am just stumped at what can be the possible causes for this. Here is the behavoir:
extern sampleclas* someobj;
void func()
{
someobj->MemFuncCall("This is a sample str");
}
My crash is inside MemFuncCall. But when I examine core file, someobj has an address, say abc(this address is properly initialized and not corrupted) , which is different from this pointer in the function stacktrace: sampleclass::MemFuncCall(this=xyz, "This is a sample str")
I was assuming that this pointer will always be the same as address for someobj i.e. abc should always be equal to xyz.
What are the possible cases where these 2 addresses can be different???
Fyi, This app is single threaded.
Ship optimizations can make things appear very strange in a debugger. Recompile in debug mode (optimizations off), and repo.
Another possible explaination is if the calling convention (or definition in general) is wrong for MemFuncCall (there is a disagreement between the header you compiled with and when MemFuncCall was compiled). You have to try hard to get this wrong, though.
It is possible. Maybe some kind of buffer overrun? Maybe the calling convention (or definition in general) is wrong for MemFuncCall (there is a mismatch between the header you compiled with and when MemFuncCall was compiled).
Hard to say.
But since this is single threaded I would try following technique. Usually memory layout in apps is the same between reruns of application. So start your application under debugger, stop it immediately and put two memory breakpoints on addresses 0xabc and 0xxyz. You have good chance of hitting breakpoints once someone is modifying this memory. Maybe than stack traces will help?
In the case of multiple inheritance the this pointer can be different from the pointer to the "real" object:
struct A {
int a;
void fa() { std::cout << "A::this=" << this << std::endl; }
};
struct B {
int b;
void fb() { std::cout << "B::this=" << this << std::endl; }
};
struct C : A, B {
};
int main() {
C obj;
obj.fa();
obj.fb();
}
Here inside obj.fa(), this will point to the A part of obj while inside fb() it will point to the B part. So the this pointers can be different in different methods, and also different from &obj.
As a reason for the crash a possibility would be that someobj was deleted before and the pointer is no longer valid.
since the pointer someobj is defined externally it could the that there is some inconsistencies between your compilation units. try to clean everything and rebuild the project
One thing I can think of is a corruption of vtable.