I have declared some global variables to be _Cilk_shared. They are used in the functions that I want to offload They are used in some functions I do not want to be offloaded as well.
So initially I only declared those functions that I need to offload as _Cilk_shared and call those function using _Cilk_offload.
It compiles fine. And when I run it on host only it gives correct result.
Then I run it with MIC. it gives me runtime error about can not load library blablabla undefined symbol followed by function names that I did not declare as _cilk_shared. Those functions are not needed to be _cilk_shared as well.
So I have to change those functions to _cilk_shared. Run it again. This time MIC gives correct result.
And I checked whether those functions (which I did not want to offload and did not declare as _cilk_shared initially ) are offloaded or not, by Using
#ifdef __MIC__
printf(" Running on MIC\n");
#else
printf("No MIC\n");
#endif
The result is that they are not offloaded....
So I am wondering why it wants me to declare those functions as _Cilk_shared?
When a function is declared for offloading -
if the function is defined in that file, the compiler generates
object code that will run on the processor and object code that will
run on the coprocessor.
if the function is used in that file, it
generates object code that will call the processor version and code
that will call the coprocessor version (unless the call is inside an
area that is protected by '"ifdef MIC"
So why might the loader want a function to have been declared for offload even if you never call it in an offloaded region?
Well if the function is part of a class, then the entire class needs to be declared for offload because the class 'knows' what functions are part of it. When you load the object code for a class, you load in the whole class. In this case if you are going to share objects of that class, it should be declared as _Cilk_shared.
You can get into similar problems if the functions are in a dynamically linked library. The dynamically linked library 'knows' what functions are part of it and it 'knows' what functions are required by it. So, when you run your code, the loader wants to find all the files the library 'knows' it needs. In this case, the simplest thing is to just make offloaded versions of all the libraries. Other things you can try - try either statically linking the library containing those files or try creating libraries that don't 'know' they need things they really don't - break your library into pieces or try explicitly create separate host and coprocessor version of the library as you would for code you are going to run natively on the coprocessor.
If this doesn't explain what you are seeing, let me know.
Related
Let's say I have this lib
//// testlib.h
#pragma once
#include <iostream>
void __declspec(dllexport) test();
int __declspec(dllexport) a();
If I omit the definition for a() and test() from my testlib.cpp, the library still compiles, because the interface [1] is still valid. (If I use them from a client app then they won't link, obviously)
Is there a way I can ensure that when the obj is created (which I gather is the compiler's job) it actually looks for the definitions of the functions that I explicitly exported, and fails if doesn't ?
This is not related to any real world issue. Just curious.
[1] MSVC docs
No, it's not possible.
Partially because a dllexport declaration might legally not even be implemented in the same DLL, let alone library, but be a mere forward declaration for something provided by yet another DLL.
Especially it's impossible to decide on the object level. It's just another forward declaration like any other.
You can dump the exported symbols once the DLL has been linked, but there is no common tool for checking completeness.
Ultimately, you can't do without a client test application which attempts to load all exported interfaces. You can't check that on compile time yet. Even just successfully linking the test application isn't enough, you have to actually run it.
It gets even worse if there are delay-loaded DLLs (and yes, there usually are), because now you can't even check for completeness unless you actually call at least one symbol from each involved DLL.
Tools like Dependency Walker etc. exist for this very reason.
You asked
"Is there a way I can ensure that when the obj is created (which I gather is the compiler's job) it actually looks for the definitions of the functions that I explicitly exported, and fails if doesn't ?"
When you load a DLL then the actual function binding is happenning at runtime(late binding of functions) so it is not possible for the compiler to know if the function definition are available in the DLL or not. Hope this answer to your query.
How do I (programmatically) ensure that a dll contains definitions for all exported functions?
In general, in standard C++11 (read the document n3337 and see this C++ reference), you cannot
Because C++11 does not know about DLL or dynamic linking. It might sometimes make sense to dynamically load code which does not define a function which you pragmatically know will never be called (e.g. an incomplete DLL for drawing shapes, but you happen to know that circles would never be drawn in your particular usage, so the loaded DLL might not define any class Circle related code. In standard C++11, every called function should be defined somewhere (in some other translation unit).
Look also into Qt vision of plugins.
Read Levine's Linkers and loaders book. Notice that on Linux, plugins loaded with dlopen(3) have a different semantics than Windows DLLs. The evil is in the details
In practice, you might consider using some recent variant of the GCC compiler and develop your GCC plugin to check that. This could require several weeks of work.
Alternatively, adapt the Clang static analyzer for your needs. Again, budget several weeks of work.
See also this draft report and think about C++ cross compilers (e.g. compiling on Windows a plugin for a RaspBerry Pi)
Consider also runtime code generation frameworks like asmjit or libgccjit. You might think of generating at runtime the missing stubs or functions (and fill appropriately function pointers with them, or even vtables). C++ exceptions could also be an additional issue.
If your DLL contains calls to the functions, the linker will fail if a definition isn't provided for those functions. It doesn't matter if the calls are never executed, only that they exist.
//// testlib.h
#pragma once
#include <iostream>
#ifndef DLLEXPORT
#define DLLEXPORT(TYPE) TYPE __declspec(dllexport)
#endif
DLLEXPORT(void) test();
DLLEXPORT(int) a();
//// testlib_verify.c
#define DLLEXPORT(TYPE)
void DummyFunc()
{
#include testlib.h
}
This macro-based solution only works for functions with no parameters, but it should be easy to extend.
I have a project in pure C - st usb library and I need to migrate it to c++ and change same structures into classes. I removed all c++ "protections" like:
#ifdef __cplusplus
extern "C" {
#endif
#ifdef __cplusplus
}
#endif
I changed all files extensions from .c to .cpp (except HAL library).
I realized that c++ .hex is 7kB smaller then c .hex. When I looked into .map file I saw that many functions are missing. I thought that staticfunctions caused that, but removing static key word didn't help. Does anyone have idea what could cause that some functions weren't compiled. When extensions are .c everything is fine.
I can think of two main reasons:
Inlining. The compiler can decide there is no need to emit the function as a standalone function if all usages can be inlined.
Unused code. The compiler can see that a function isn't used anywhere in your code and decide to eliminate it from the final result.
If the result is to be used as sort of library, that your environment calls specific functions without having an explicit call in your own code, I think the best method is to compile and link your code as a library (probably dynamic lib) and exporting those functions as library interface (visibility in gcc, dllexport in MSVC) will force the compiler/linker to include them in the produced binary even if they don't see why they are needed right now.
(Of course, this is a wild guess about your target environment.)
Another option is to turn off the specific compiler/linker optimizations for removing dead code and forcing emitting standalone function instance for inlined functions, but I think it's very indirect approach, have a wider effect and can complicate maintenance later.
C++ functions are given different signatures than C functions. Since you lost functionality and the code is much smaller, it's likely that a function that requires C linkage is being compiled as C++ and the signature mismatch is preventing proper linking.
A likely place for this to happen is in the interrupt vector table. If the handler function is compiled with C++ linkage, the handler address won't make it into a table that's compiled with C.
Double check the interrupt vectors and make sure that they reference the correct functions. If they are correct, check any other code compiled with C that might reference an external symbol compiled with C++.
I was wondering if there is any way to make something like plugin in Java, so main program loads sub-programs and execute their functions. I thought about set of small programs that return some value, but I want to make the plugins able to modify some of the main program objects (maybe pass the pointer somehow?).
Is there any way to do this?
Thanx for any replies.
It is possible to dynamically load code at run-time in the form of dynamically linked libraries (DLL:s in Windows).
The approach (when dealing with Windows at least) I use is about this:
Create a dll with some exported functions (declared with __declspec(dllexport))
Load the created file at run-time with the LoadLibrary function in the Windows-API
Get the function-pointer to a member function by its name with the function GetProcAddress-function using the name of the function.
The last part may be a little confusing sometimes as C++ uses name-mangling to keep track of return types and such things. This means that the name stored in the DLL is not the name of the function, but a name also containing descriptions of the function's parameter types and such things.
To prevent the name-mangling, you can declare the exported functions with extern "C", such as
extern "C" __declspec(dllexport) int myFunction(...);
However, this will not allow for classes in the function declaration.
An important thing to consider when passing pointers to data structures between the dynamically linked library and the "main program" is to make sure that the declaration of the type is the same in the two files (easily accomplished by sharing the header declaring the type), otherwise there will be severe errors when executing your program.
Again, this is probably Windows-specific, but it might give you a hint of the correct procedure on your system.
I've got an application that's using a static library I made. One .cpp file in the library has a static variable declaration, whose ctor calls a function on a singleton that does something- e.g. adds a string.
Now when I use that library from the application, my singleton doesn't seem to contain any traces of the string that was supposed to be added.
I'm definitely missing something but I don't know what..
If you have an object in a static library that is not EXPLICITLY used in the application. Then the linker will not pull that object from the lib into the application.
There is a big difference between static and dynamic libraries.
Dynamic Library:
At compile time nothing is pulled from the dynamic library. Extra code is added to explicitly load and resolve the symbols at run-time. At run time the whole library is loaded and thus object initializers are called (though when is implementation detail).
Static libraries are handled very differently:
When you link against a static library it pulls all the items that are not defined in application that are defined in the library into the application. This is repeated until there are no more dependencies that the library can resolve. The side effect of this is that objects/functions not explicitly used are not pulled form the library (thus global variables that are not directly accessed will not be pulled).
My memory of this is a bit hazy, but you might be getting hit with an initialization order problem. There are no guarantees in which order static variable initializers in different files get called, so if your singleton isn't initialized yet when your static variable in the library is being initialized, that might produce the effect you're seeing.
The way I've gotten around these problems is to have some sort of an explicit init function that does this stuff and that I call at the start of main or something. You might be able to fiddle with the order in which you give the object file and library arguments to the compiler (or linker, actually) because that's also worked for me, but that solution is a bit fragile because it depends not only on using the specific linker but probably also the specific version.
Refactor the classes doing static initialization so they do not depend on any other such classes. That is, make each class's initialization independent and self-sufficient.
I'm using static initialisation to ease the process of registering some classes with a factory in C++. Unfortunately, I think the compiler is optimising out the 'unused' objects which are meant to do the useful work in their constructors. Is there any way to tell the compiler not to optimise out a global variable?
class SomeClass {
public:
SomeClass() {
/* do something useful */
}
};
SomeClass instance;
My breakpoint in SomeClass's constructor doesn't get hit. In my actual code, SomeClass is in a header file and instance is in a source file, more or less alone.
EDIT: As guessed by KJAWolf, this code is actually compiled into a static lib, not the executable. Its purpose is to register some types also provided by the static lib with a static list of types and their creators, for a factory to then read from on construction. Since these types are provided with the lib, adding this code to the executable is undesirable.
Also I discovered that by moving the code to another source file that contains other existing code, it works fine. It seems that having a file purely consisting of these global objects is what's causing the problem. It's as if that translation unit was entirely ignored.
The compiler is not allowed to optimiza away global objects.
Even if they are never used.
Somthing else is happening in your code.
Now if you built a static library with your global object and that global object is not referenced from the executable it will not be pulled into the executable by the linker.
The compiler should never optimize away such globals - if it does that, it is simply broken.
You can force that one object (your list of types) pulls some other objects with it by partially linking them before building the complete static lib.
With GNU linker:
ld -Ur -o TypeBundle.o type1.o type2.o type3.o static_list.o
ld -static -o MyStaticLib.a type_bundle.o other_object.o another_object.o ...
Thus, whenever the static list is referenced by code using the library, the complete "TypeBundle.o" object will get linked into the resulting binary, including type1.o, type2.o, and type3.o.
While at it, do check out the meaning of "-Ur" in the manual.
To build off of Arthur Ulfeldt, volatile tells the compiler that this variable can change outside of the knowledge of the compiler. I've used it for put a statement to allow the debugger to set a breakpoint. It's also useful for hardware registers that can change based on the environment or that need a special sequence. i.e. Serial Port receive register and certain watchdog registers.
you could use
#pragma optimize off
int globalVar
#pragma optimize on
but I dunno if that only works in Visual Studio ( http://msdn.microsoft.com/en-us/library/chh3fb0k(VS.80).aspx ).
You could also tell the compiler to not optimize at all, especially if you're debugging...
Are you using gcc with gdb? There was a problem in the past where gdb could not accurately set breakpoints in constructors.
Also, are you using an optimization level which allows the compiler to inline methods in the class definition.
You need to use -whole-archive when linking. See the answer here:
ld linker question: the --whole-archive option
I have same setup & problem on VS2008.
I found that if you declare you class with dllexport it will not optimize.
class __declspec( dllexport ) Cxxx
{
.
}
However this generates a lot of warnings in my case because I must declare all classes used in this class also as dllexport.
All optimizations are off (in debug mode), still this is optimized. Also volatile/pragma optimize off. On global variable created of this class (in same cpp file) etc does not work.
Just found that dllexport does require at least to include header files of these classes in some other cpp file from exe to work! So only option is to add a file with calls to some static members for each class, and add this file to all projects used these classes.
It would not be a compiler, but the library linker (or the tool that pools object files into the .lib), who can decide that the whole file is not used, so discard it.
A workaround would be to add an empty function to that file and call it from a file that contains stuff that is called directly.
How about using the keyword volatile? It will prevent the compiler from too much optimization.