One of the namespaces in my program is spread between two files. One provides the "engine", the other uses the "engine" to perform various commands. All of the initializations are performed on the "engine" side, including caching parameters fetched from setup library.
So, there's engine.cpp with:
#include <stdio.h>
#include "ns.h"
namespace MyNS
{
unsigned char variable = 0;
void init()
{
variable = 5;
printf("Init: var = %d\n",variable);
}
void handler()
{
// printf("Handler: var = %d\n",variable);
}
}
The variable happens never to be used again in engine.cpp but it's extensively used in commands.cpp.
#include <stdio.h>
#include "ns.h"
namespace MyNS
{
extern unsigned char variable;
void command()
{
printf("Command: var = %d\n",variable);
}
}
After compiling and linking, I'm getting:
Init: var = 5
Command: var = 1
Now, if I uncomment the printf() in handler() I'm getting:
Engine: var = 5
Command: var = 5
Handler: var = 5
What would be the "correct" way to force GCC not to optimize it away in such a way that accessing it through extern from the other file would fetch the right value? Preferably without reducing the -O level for the rest of the application?
(for completeness case, main.h and ns.h: )
#include "ns.h"
int main(int argc, char** argv)
{
MyNS::init();
MyNS::command();
MyNS::handler();
return 0;
}
namespace MyNS
{
void init();
void command();
void handler();
}
This minimized testcase doesn't exhibit this particular behavior; it seems one needs this situation to occur in much more complex environment to happen...
eh... the solution was quite trivial.
I exchanged places of the declaration and definition of the variable.
engine.cpp:
extern unsigned char variable;
command.cpp:
unsigned char variable = 0;
That way the compiler has no doubts about need for this variable's existence while compiling commands and in engine it has to reach to the existing instance, it can't just create a temporary one on the spot.
EDIT: Now I've discovered another peculiarity. The value changes depending on where it's written to. The section of code in question is:
1: varso = SharedObject::Instance()->varso;
2: memset(det_map,0,sizeof(det_map));
3: memset(gr_map,0xFF,sizeof(gr_map));
4: memset(gr_ped,false,sizeof(gr_ped));
5: memset(&stan,0,sizeof(stan));
6: stan.SOTUstage = 1;
7: PR_SOTU = varso->NrPSOTU;
The variable occurs near a place where several arrays are initialized with memset. The variable in question is PR_SOTU (the uppercase is inherited from when it was still a macro, and since it acts along with several other macros acting in a very similar context, it's likely to stay that way).
If move the assignment from its line 7 and place it after lines 1, 2 or 3, it receives the correct value 5. Placed after line 4 it gets the value 18. Anything below, and the value is 1. I moved definition of the variable to a different place (it was the last on the list of all namespace-globals, now it's first) to exclude possibility something writes at that specific memory location, but the behavior remains.
Related
I have integration tests in my Catch2 project that depend on some expensive global state being set up. I'd like to only initialize that global state when the test runner will actually be testing the systems that depend on it.
What I have appears to work, but it's a little horrifying... it depends on rather a lot of implementation details in the Catch config.
Here's my main:
#define CATCH_CONFIG_RUNNER
#include "catch.hpp"
...
int main(int argc, const char* argv[])
{
// Construct a fake TestCaseInfo to match against the [integration] tag
const char * expensive_tag = "integration";
Catch::SourceLineInfo fake_source_line("?", 0)
Catch::TestCaseInfo fake_test_case("?", "?", "?", {expensive_tag}, fake_source_line);
Catch::Session session;
session.applyCommandLine(argc, argv);
auto test_spec = session.config().testSpec();
const bool want_integration_tests = test_spec.matches(fake_test_spec);
if(want_integration_tests)
{
do_expensive_setup();
}
return session.run();
}
And then my test file is just:
#include "catch.hpp"
...
TEST_CASE("expensive-to-initialize system", "[.integration]")
{
REQUIRE(expensive_setup_is_done());
SECTION("has property 1") { ... }
SECTION("has property 2") { ... }
...
}
Note that because there are multiple sections (and, in my actual project, multiple test cases) that depend on the global setup, I can't just move the initialization into the top of the TEST_CASE.
Is there a better way?
Just do the initialization on demand, using something like std::call_once:
TEST_CASE("expensive-to-initialize system", "[.integration]")
{
static std::once_flag init_flag;
std::call_once(init_flag, do_expensive_setup);
// ...
}
This will ensure that do_expensive_setup is called one time, but only if needed. If there are multiple places that need this setup, just wrap this in a function.
Note that if do_expensive_setup throws, it will may be called a second time. But once the function successfully exits, that's it.
I’m trying to JIT compile some functions in an existing C/C++ program at runtime, but I’m running into some trouble with global variable initialization. Specifically, the approach I’ve taken is to use Clang to precompile the program into IR bitcode modules in addition to the executable. At runtime, the program loads the modules, transforms them (program specialization), compiles and executes them. As it turns out, I have some global variables that get initialized and modified during execution of the “host” program. Currently, these globals are also getting initialized in the JIT compiled code, whereas I’d like them to be mapped to the host global variables instead. Can someone help me with this?
A small repro is excerpted below. Full source code is here. The file somefunc.cpp gets precompiled during build, and is loaded in the main() function in testCompile.cpp. The global variable xyz is initialized to point to 25 in somefunc.cpp, but I’d like it to point to 10 as in main() instead. In other words, the assertion in main() should succeed.
I tried a few different ways to solve this problem. The ChangeGlobal() function attempts (unsuccessfully) to achieve this updateGlobalMapping(). The second, more hacky approach uses a new global variable initialized appropriately. I can get this latter approach to work for some types of globals, but is there a more elegant approach than this?
//————— somefunc.h ————————
extern int *xyz;
//—————— somefunc.cpp ——————
int abc = 25;
int *xyz = &abc;
int somefunc() {
return *xyz;
}
//—————— testCompile.cpp ——————
class JitCompiler {
public:
JitCompiler(const std::string module_file);
void LoadModule(const std::string& file);
template <typename FnType>
FnType CompileFunc(FnType fn, const std::string& fn_name);
void ChangeGlobal();
private:
std::unique_ptr<LLVMContext> context_;
Module *module_;
std::unique_ptr<ExecutionEngine> engine_;
};
void JitCompiler::ChangeGlobal() {
// ----------------- #1: UpdateGlobalMapping -----------------
//auto g = engine_->FindGlobalVariableNamed("xyz");
//engine_->updateGlobalMapping(g, &xyz);
//assert(engine_->getGlobalValueAddress("xyz") == (uint64_t) &xyz);
// ----------------- #2: Replace with new global ————————
// ------- Ugly hack that works for globals of type T** ----------
auto g = engine_->FindGlobalVariableNamed("xyz");
Constant *addr_i = ConstantInt::get(*context_, APInt(64, (uint64_t) xyz));
auto addr = ConstantExpr::getIntToPtr(
addr_i, g->getType()->getPointerElementType());
GlobalVariable *n = new GlobalVariable(
*module_,
g->getType()->getPointerElementType(),
g->isConstant(),
g->getLinkage(),
addr,
g->getName() + "_new");
g->replaceAllUsesWith(n);
n->takeName(g);
g->eraseFromParent();
}
int main() {
xyz = new int (10);
JitCompiler jit("somefunc.bc");
jit.ChangeGlobal();
auto fn = jit.CompileFunc(&somefunc, "somefunc");
assert(somefunc() == fn());
}
A better approach is the combination of the two you presented, that is, to create a new global with external linkage mapped to &xyz and substitute it for the original:
auto g = engine_->FindGlobalVariableNamed("xyz");
GlobalVariable *n = new GlobalVariable(
g->getType()->getPointerElementType(),
g->isConstant(),
ExternalLinkage
nullptr,
g->getName() + "_new");
engine_->updateGlobalMapping(n, &xyz);
g->replaceAllUsesWith(n);
n->takeName(g);
g->eraseFromParent();
I have MCU with flash memory breaked in sections(as usual).
Linker places .struct_init, .struct_init_const, .struct_not_init sections to addresses belongs to flash memory section20. It is hardcoded in linker script.
Consider following test code:
test.h
typedef struct
{
int val1;
int val2;
} mystruct_t;
test.cpp
#include "test.h"
// each variable is placed in dedicated section
// sections are placed in flash section20
// linker exports symbols with address of eaach section
__attribute__((section(".struct_init")))
mystruct_t struct_init = {
.val1 = 1,.val2 = 2};
__attribute__((section(".struct_init_const")))
extern const mystruct_t struct_init_const = {
.val1 = 1, .val2 = 2};
__attribute__((section(".struct_not_init")))
mystruct_t struct_not_init;
main.cpp
#include <stdint.h>
// This symbols exported by linker
// contains addresses of corresponding sections
extern uintptr_t LNK_STRUCT_INIT_ADDR;
extern uintptr_t LNK_STRUCT_INIT_CONST_ADDR;
extern uintptr_t LNK_STRUCT_NOT_INIT_ADDR;
// Pointers for indirect access to data
mystruct_t* struct_init_ptr = (mystruct_t*)LNK_STRUCT_INIT_ADDR;
const mystruct_t* struct_init_const_ptr = (const mystruct_t*)LNK_STRUCT_INIT_CONST_ADDR;
mystruct_t* struct_not_init_ptr = (mystruct_t*)LNK_STRUCT_NOT_INIT_ADDR;
// Extern variables declarations for DIRECT access data
extern mystruct_t struct_init;
extern const mystruct_t struct_init_const;
extern mystruct_t struct_not_init;
// This is some variables representing config values
// They can be more complex objects(classes) with internal state and logic..
int param1_direct;
int param1_init_const_direct;
int param1_not_init_direct;
int param1_indirect;
int param2_init_const_indirect;
int param1_not_init_indirect;
int main(void)
{
// local variables init with direct access
int param1_direct_local = struct_init.val1;
int param1_init_const_direct_local = struct_init_const.val1;
int param1_not_init_direct_local = struct_not_init.val1;
// local variables init with indirect access
int param1_indirect_local = struct_init_ptr->val1;
int param2_init_const_indirect_local = struct_init_const_ptr->val1;
int param1_not_init_indirect_local = struct_not_init_ptr->val1;
//global variables init direct
param1_direct = struct_init.val1;
param1_init_const_direct = struct_init_const.val1;
param1_not_init_direct = struct_not_init.val1;
//global variables init indirect
param1_indirect = struct_init_ptr->val1;
param2_init_const_indirect = struct_init_const_ptr->val1;
param1_not_init_indirect = struct_not_init_ptr->val1;
while(1){
// use all variables we init above
// usage of variables may also occure in some functions or methods
// directly or indirectly called from this loop
}
}
I wanna be sure that initialization of param1_ variables will lead to fetch data from flash. Because data in flash section20 can be changed using bootloader(at the moment when main firmware is not running).
The question is: Can LTO(and other optimizations) throw away fetches from flash and just substitute known values because they are known at link time because of initialization.
What approach is better?
If LTO can substitute values - then initialization should be avoided?
I know volatile can help, but is it really needed in this situation?
Code exampe shows different approaches of accessing and initializing data.
not_init version seems to be the best, because compiler can't substitute anything. But it will be a good idea to have some default parameters, so i'd prefer init version if it can be used.
What approach should be chosen?
Currently i am using GCC 4.9.3 but this is general question about any C/C++ compiler.
C and C++ both feature extern variables, which lets you define constants without immediately giving away their values:
// .h
extern int const param1;
extern char const* const param2;
// ...
In general you would define them in a (single) source file, which would hide them away from anything not in this source file. This is not LTO resilient, of course, but if you can disable LTO it is an easy enough strategy.
If disabling LTO is not an option, another solution is to not define them, let LTO produce a binary, and then use a script to splice the definitions in the produced binary in the right section (the one that can be flashed).
With the value not available at LTO time, you are guaranteed that it will not be substituted.
As for the solutions you presented, while volatile is indeed a standard compliant solution, it implies that the value is not constant, which prevents caching it during run-time. Whether this is acceptable or not is for you to know, just be aware it might have a performance impact, which as you are using LTO I surmised you would like to avoid.
In the C language, in order to initialize a static local variable to a value unknown during compilation, I would normally do something like this (for example):
void func()
{
static int var = INVALID_VALUE;
if (var == INVALID_VALUE)
var = some_other_func();
...
}
In the C++ language, I can simply do:
void func()
{
static int i = some_other_func();
...
}
The only way (that I can think of) for a C++ compiler to resolve it properly, is by replacing this code with a mechanism similar to the C example above.
But how would the compiler determine a "proper" invalid value? Or is there another way which I haven't taken into consideration?
Thanks
Clarification:
INVALID_VALUE is a value which function some_other_func never returns.
It is used in order to ensure that this function is never invoked more than once.
The compiler will not generate code to do it based on its value but on a thread safe flag that ensure that the code is only executed once.
Something like that:
void func()
{
static int i;
static bool i_initialized;
if (!i_initialized) {
i = some_other_func();
i_initialized = true;
}
}
Except that generally it is not a bool but a thread safe way of testing it.
According to code seen by disassembling and debugging the g++ compiled code, there is a hidden variable that is initialized to 0 and when the initialization is run it is set to 1.
So the next time the initialization code isn't executed.
Learning C++ with help of "Thinking in C++" by Bruce Eckel,stuck in exercise 32, chapter 10.
The question is how to change link order, that Mirror::test() called for object m5 return false.
Here is my code.
mirror.h:
#ifndef MIRROR_H_
#define MIRROR_H_
class Mirror {
public:
Mirror() {logic_ = true; self_ = 0;};
Mirror(Mirror *ptr) {self_ = ptr; logic_ = false;};
bool test() {
if (self_ != 0) {
return self_->test();
} else {
return logic_;
}
};
private:
bool logic_;
Mirror *self_;
};
#endif // MIRROR_H_
task
one.cpp
#include "mirror.h"
Mirror m1;
two.cpp
#include "mirror.h"
extern Mirror m1;
Mirror m2 (&m1);
three.cpp
#include "mirror.h"
extern Mirror m2;
Mirror m3 (&m2);
and so on. Finally,
five.cpp
#include "mirror.h"
#include <iostream>
extern Mirror m4;
Mirror m5 (&m4);
int main(int argc, char* argv[]) {
std::cout << m5.test() << std::endl;
}
m5.test() returns true. The task says, that I should change linking order, that m5.test() returns false. I have tried to use:
init_priority (priority)
In Standard C++, objects defined at namespace scope are guaranteed to be initialized in an order in strict accordance with that of their
definitions in a given translation unit. No guarantee is made for
initializations across translation units. However, GNU C++ allows
users to control the order of initialization of objects defined at
namespace scope with the init_priority attribute by specifying a
relative priority, a constant integral expression currently bounded
between 101 and 65535 inclusive. Lower numbers indicate a higher
priority.
but no luck.
Full exercise text:
In a header file, create a class Mirror that contains two data
members: a pointer to a Mirror object and a bool. Give it two
constructors: the default constructor initializes the bool to true and
the Mirror pointer to zero. The second constructor takes as an
argument a pointer to a Mirror object, which it assigns to the
object’s internal pointer; it sets the bool to false. Add a member
function test( ): if the object’s pointer is nonzero, it returns the
value of test( ) called through the pointer. If the pointer is zero,
it returns the bool. Now create five cpp files, each of which includes
the Mirror header. The first cpp file defines a global Mirror object
using the default constructor. The second file declares the object in
the first file as extern, and defines a global Mirror object using the
second constructor, with a pointer to the first object. Keep doing
this until you reach the last file, which will also contain a global
object definition. In that file, main( ) should call the test( )
function and report the result. If the result is true, find out how to
change the linking order for your linker and change it until the
result is false.
You'll need to change the order of the object files when passing them to the linker. This works reasonable for the toplevel code although different compilers use different approaches, i.e., it isn't portable. Also, for libraries you generally can't control the order in which the objects are included. For example, if you have
// file1.cpp
int main() {
}
// file2.cpp
#include <iostream>
static bool value = std::cout << "file2.cpp\n";
// file3.cpp
#include <iostream>
static bool value = std::cout << "file3.cpp\n";
... and you link two programs like this:
g++ -o tst1 file1.cpp file2.cpp file3.cpp
g++ -o tst2 file1.cpp file3.cpp file2.cpp
you will get different output for tst1 and tst2, e.g.:
$ ./tst1
file2.cpp
file3.cpp
$ ./tst2
file3.cpp
file2.cpp
The overall moral is: don't do it. That is: don't use global objects. If you feel you absolutely need to use global objects, encapsulate them into functions, e.g.:
Type& global_value() {
static Type value; // possibly with constructor arguments
return value;
}
This way, value is initialized the first time it is accessed and there is no way to access it while it isn't constructed, yet. If you encapsulate all objects like this, you can guarantee that they are constructed in an appropriate order (unless you have a cyclic dependency in which case it can't be made to work and you should seriously rethink your design). The above approach encapsulating objects into function is, unfortunately, not thread-safe in C++ 2003. It is thread-safe in C++ 2011, however. Still, use of global variable is generally problematic and you definitely want to minimize their use.
I was struggling with this exercise too.
I managed to write a small Python script to prepare makefile entries that link and test final executable using all possible permutation of object files:
import itertools
for perm in itertools.permutations([1, 2, 3, 4, 5]):
print '\tg++ u0{0}.o u0{1}.o u0{2}.o u0{3}.o u0{4}.o -o $# && ./main.exe'.format(*perm)
After executing my make process it turned out, that all of the possible configurations yielded true value.
This is due to the fact, that all global (i.e. static) variables are guaranteed to be initialized before entering main function.
I defined a global bool variable that holds result from a test() function before main, something like this:
#include "mirror.h"
#include <iostream>
extern Mirror m4;
Mirror m5 (&m4);
bool result = m5.test();
int main(int argc, char* argv[]) {
std::cout << result << std::endl;
}
Bingo! Some of the objects' permutations yielded false at the progam's output.
All static variables are initialized with zeroes before any of their possible constructors are called. In this exercise, the order in which constructors are called is the clue.
If any object in the depencence chain has not been initialized by constructor when a value of result variable is established, the result is false value (self_ value is 0 and logic_ value is false, so test function returns false).
When a result variable is evaluated before entering main function, there is such possibility and order of object files in a linker command has something to do with the result.