I'm working on a huge project which uses a system that, when running make, generates some header files that contain some constants that are used everywhere in the code. Due to the size of this project if one of those headers is changed (a constant is removed or another is added) almost the whole project must be recompiled (which sometimes takes several hours).
My initial solution was to write some sort of ConstantManager class that has a map with each code-value pair and a getter that returns that given the code as a string returns its value (pretty straight forward) and also change the header generator so it would define the constants as #defines that expand to an instance of the ConstantManager and a call to the getter. The problem with this is that it will not work with switch statements (well... case statements actually) because the return values are not constant expressions.
My question is: are there any alternative solutions to this problem or some trick to make mine work with switches?
You could split the huge header into smaller ones and include those. This might be a lot of initial work but is the most straight forward and probably compatible with your current solution.
Another option is to create to make your ConstManager class have constexpr members. For this you would not need much but you cannot use map.
ConstManager.h
namespace ConstManager {
namespace detail {
struct Constant {
char const * const name;
char const * const value;
};
Constant const * const constants;
unsigned int total_variables;
}
inline char const * const ConstManager::get(char const * needle) constexpr {
using namespace ConstManager::detail;
/* i was not able to quickly find a constexpr compatible binary search function*/
if(int i = 0; i < total_variables; ++i){
Constant & const c = constants[i];
if(strcmp(c.name, needle) == 0){
return c.value;
}
}
return nullptr;
}
}
ConstManager.c should be generated
ConstManager::detail::Constant ConstManager::detail::constants [] = {
{"first var", "first value"},
{"second var", "second value"}
};
unsigned int ConstManager::detail::total_variables = 2;
First solution proposed (this was before the first edit):
I think you should replace all your defines by regular external variables.
Although two caveats:
It will not work if those defines are used to concatenate strings
It might make inlining harder if not impossible for the compiler but if this matters only a profiler can tell.
constmgr.h
// make all your needed constansts externs
namespace Constants {
extern int const theFirstConstant;
extern char const * const someStringConstant;
}
And the source file is then generated at build time.
constexpr.cpp
int const Constants::theFirstConstant = 1000;
char const * const Constants::someStringConstant = "hihihoho";
Related
I've a big file of Constants.h file where about 200 variables(mostly arrays) are declared and initialised. I'm using namespace.
METHOD 1:
//Constants.h
#ifndef CONSTANTS_H_
#define CONSTANTS_H_
namespace LibConstants
{
const int a = 12;
const std::string theme[2]; = {"themeA", "themeB"};
const int arr[2] = {1, 2};
// and around 200 more declarations with initialization
}
#endif
This .h file is #include in almost every .cpp file but each time only very minimal variables are being used like LibConstants::theme[0].
My ways works fine but doesn't it allocate memory unnecessarily?
Shall I follow the approach, to only define variables in .h file and initialize in .cpp?
Like in below code:
METHOD 2:
//Constants.h
#ifndef CONSTANTS_H_
#define CONSTANTS_H_
namespace LibConstants {
std::string getMyTheme(int arg);
std::string getMyThemeName(int arg);
const int a;
const std::string theme[2];
const int arr[2];
// and around 200 more declarations with initialisation
};
#endif
Initialising in cpp file
//Constants.cpp
#include LibConstants.h
using namespace LibConstants {
std::string getMyTheme(int arg) {
theme[2] = {"themeA", "themeB"};
return theme[arg];
}
std::string getMyThemeName(int arg) {
...
}
}
Using it like
//HelloWorld.cpp
#include Constants.h
int main() {
//METHOD 1:
std::string a = LibConstants::theme[0]; // arg=0 is dynamic and is being read from localStorage in my case.
//METHOD 2:
std::string a = LibConstants::getMyTheme(0); //Here also arg=0 is dynamic.
...
}
Here, unnecessary allocation of memory for unneeded variables won't happen except for arrays which are declared as const std::string st[2]; in the header file.
Here "arg=0" is run time involvement. Does it matter if some variable is not dependent on run-time but only compile time in which case it will simply replace the value of placeholder in corresponding .cpp file?
Please correct me wherever I am wrong.
Take the std::string example. Consider,
namespace Constants {
const std::string str = "A not very short string";
}
str is a class type, it has a constructor, and it has to allocate memory to store its contents (unless short-string optimization is used in its implementation. Even in that case, it only applies to, well, "short string"). It is declared a namespace scope variable. So the program has to construct this variable at launch. Every program that has this variable declared, will need to allocate memory, and initialize its contents. That is an overhead you probably don't want, depending on how much that will effect your performance.
On the other hand, if you have,
const std::string &str()
{
const static std::string s = "A not very short string";
return s;
}
Now, you have a static local variable. It will be initialized at the first entry of the function. And it will be initialized only once. If str is never called, it will not be initialized at all.
However, note that the string literal "A not very short string" is still going to occupy some memory. Where and how it is store is implementation defined. Usually in the data segments, and the impact is usually minimal.
In contrast to class types. It is preferable to define fundamental types, especially integral types in the header.
For example,
namespace Constants {
constexpr int x = 10;
}
An optimizing compiler will most likely not to store the variable x and its contents 10 at all. Instead, wherever x is used, it will be replaced by 10, and in some cases, coded into the instruction op-code (so called immediate operands). Of course, this is again an implementation detail. And such optimization cannot be relied on with all compilers. And if you take x's address or otherwise ODR used, compiler will be compelled to make room for this variable anyway. But the bottomline is that, it will be very unlikely that you will be worse off by declaring this constants in header than defining it in an external source file.
This code will not compile:
#ifndef RemoteControl_h
#define RemoteControl_h
#include "Arduino.h"
class RemoteControl
{
public:
RemoteControl();
~RemoteControl();
static void prev_track();
static void next_track();
static void play_pause_track();
static void mute();
static void vol_up();
static void vol_down();
void respond(int code);
void add_code(int code, void (*func)());
private:
boolean active = true;
struct pair {
int _code;
void (*_func)();
};
const int max = 1000;
int database_length = 0;
pair database[max]; //This line doesn't compile unless I use a literal constant instead of "max"
};
#endif
But if I put the section below in the constructor for the class instead it works fine.
const int max = 1000;
int database_length = 0;
pair database[max];
Am I not allowed to declare an array within a class in c++ and use a virtual constant as the length? I am working in arduino if that makes a difference, but I expect that I am not understanding something with the c++ language since this is a standard .h file. Oh and the problem isn't the .cpp file because I completely removed it with the same results: compiles with literal constant length but not virtual constant length.
In C or C++,try using malloc() in stdlib.h, cstdlib for c++. Don't forget free()
const int max = 1000;
struct pair *ptr = malloc(sizeof(pair) * max); // allocated 1000 pairs
free(ptr); // when the amount of memory is not needed anymore
Let me first clear a few things up for you.
In C, a const variable is considered as const-qualified, it is not a compile-time constant value (unlike an integer literal, which is a compile time constant value). So, as per the rules for normal array size specification, you cannot even use a const variable in this case.
In C, we may have the provision to use VLA which enables us to use syntax like pair database[max] even if max is not a const variable but that is again some optional feature of the compiler (as per C11).
In C++, we can use a const variable as the size of array, as in C++, a const variable is a compile time constant.
So, to answer your question:
In C, your code will be ok if your compiler supports VLA. and even if max is not const.
In C++, there is no VLA, but it maybe supported as a gnu extension. If max is const, it will be ok.
The easiest fix is to just take the
const int max = 1000;
out of the class and put it above the class.
Even better would be to ensure that it is a compile-time constant like so:
constexpr int max = 1000;
Basically I want in my code to be able to do this:
Engine.getById(WSID('some-id'));
Which should get transformed by
Engine.getById('1a61bc96');
just before being compiled into asm. So at compile-time.
This is my try
constexpr int WSID(const char* str) {
boost::crc_32_type result;
result.process_bytes(str,sizeof(str));
return result.checksum();
}
But I get this when trying to compile with MSVC 18 (CTP November 2013)
error C3249: illegal statement or sub-expression for 'constexpr' function
How can I get the WSID function, using this way or any, as long as it is done during compile time?
Tried this: Compile time string hashing
warning C4592: 'crc32': 'constexpr' call evaluation failed; function will be called at run-time
EDIT:
I first heard about this technique in Game Engine Architecture by Jason Gregory. I contacted the author who obligingly answer to me this :
What we do is to pass our source code through a custom little pre-processor that searches for text of the form SID('xxxxxx') and converts whatever is between the single quotes into its hashed equivalent as a hex literal (0xNNNNNNNN). [...]
You could conceivably do it via a macro and/or some template metaprogramming, too, although as you say it's tricky to get the compiler to do this kind of work for you. It's not impossible, but writing a custom tool is easier and much more flexible. [...]
Note also that we chose single quotes for SID('xxxx') literals. This was done so that we'd get some reasonable syntax highlighting in our code editors, yet if something went wrong and some un-preprocessed code ever made it thru to the compiler, it would throw a syntax error because single quotes are normally reserved for single-character literals.
Note also that it's crucial to have your little pre-processing tool cache the strings in a database of some sort, so that the original strings can be looked up given the hash code. When you are debugging your code and you inspect a StringId variable, the debugger will normally show you the rather unintelligible hash code. But with a SID database, you can write a plug-in that converts these hash codes back to their string equivalents. That way, you'll see SID('foo') in your watch window, not 0x75AE3080 [...]. Also, the game should be able to load this same database, so that it can print strings instead of hex hash codes on the screen for debugging purposes [...].
But while preprocess has some main advantages, it means that I have to prepare some kind of output system of modified files (those will be stored elsewhere, and then we need to tell MSVC). So it might complicate the compiling task. Is there a way to preprocess file with python for instance without headaches? But this is not the question, and I'm still interested about using compile-time function (about cache I could use an ID index)
Here is a solution that works entirely at compile time, but may also be used at runtime. It is a mix of constexpr, templates and macros. You may want to change some of the names or put them in a separate file since they are quite short.
Note that I reused code from this answer for the CRC table generation and I based myself off of code from this page for the implementation.
I have not tested it on MSVC since I don't currently have it installed in my Windows VM, but I believe it should work, or at least be made to work with trivial changes.
Here is the code, you may use the crc32 function directly, or the WSID function that more closely matches your question :
#include <cstring>
#include <cstdint>
#include <iostream>
// Generate CRC lookup table
template <unsigned c, int k = 8>
struct f : f<((c & 1) ? 0xedb88320 : 0) ^ (c >> 1), k - 1> {};
template <unsigned c> struct f<c, 0>{enum {value = c};};
#define A(x) B(x) B(x + 128)
#define B(x) C(x) C(x + 64)
#define C(x) D(x) D(x + 32)
#define D(x) E(x) E(x + 16)
#define E(x) F(x) F(x + 8)
#define F(x) G(x) G(x + 4)
#define G(x) H(x) H(x + 2)
#define H(x) I(x) I(x + 1)
#define I(x) f<x>::value ,
constexpr unsigned crc_table[] = { A(0) };
// Constexpr implementation and helpers
constexpr uint32_t crc32_impl(const uint8_t* p, size_t len, uint32_t crc) {
return len ?
crc32_impl(p+1,len-1,(crc>>8)^crc_table[(crc&0xFF)^*p])
: crc;
}
constexpr uint32_t crc32(const uint8_t* data, size_t length) {
return ~crc32_impl(data, length, ~0);
}
constexpr size_t strlen_c(const char* str) {
return *str ? 1+strlen_c(str+1) : 0;
}
constexpr int WSID(const char* str) {
return crc32((uint8_t*)str, strlen_c(str));
}
// Example usage
using namespace std;
int main() {
cout << "The CRC32 is: " << hex << WSID("some-id") << endl;
}
The first part takes care of generating the table of constants, while crc32_impl is a standard CRC32 implementation converted to a recursive style that works with a C++11 constexpr.
Then crc32 and WSID are just simple wrappers for convenience.
If anyone is interested, I coded up a CRC-32 table generator function and code generator function using C++14 style constexpr functions. The result is, in my opinion, much more maintainable code than many other attempts I have seen on the internet and it stays far, far away from the preprocessor.
Now, it does use a custom std::array 'clone' called cexp::array, because G++ seems to not have not added the constexpr keyword to their non-const reference index access/write operator.
However, it is quite light-weight, and hopefully the keyword will be added to std::array in the close future. But for now, the very simple array implementation is as follows:
namespace cexp
{
// Small implementation of std::array, needed until constexpr
// is added to the function 'reference operator[](size_type)'
template <typename T, std::size_t N>
struct array {
T m_data[N];
using value_type = T;
using reference = value_type &;
using const_reference = const value_type &;
using size_type = std::size_t;
// This is NOT constexpr in std::array until C++17
constexpr reference operator[](size_type i) noexcept {
return m_data[i];
}
constexpr const_reference operator[](size_type i) const noexcept {
return m_data[i];
}
constexpr size_type size() const noexcept {
return N;
}
};
}
Now, we need to generate the CRC-32 table. I based the algorithm off some Hacker's Delight code, and it can probably be extended to support the many other CRC algorithms out there. But alas, I only required the standard implementation, so here it is:
// Generates CRC-32 table, algorithm based from this link:
// http://www.hackersdelight.org/hdcodetxt/crc.c.txt
constexpr auto gen_crc32_table() {
constexpr auto num_bytes = 256;
constexpr auto num_iterations = 8;
constexpr auto polynomial = 0xEDB88320;
auto crc32_table = cexp::array<uint32_t, num_bytes>{};
for (auto byte = 0u; byte < num_bytes; ++byte) {
auto crc = byte;
for (auto i = 0; i < num_iterations; ++i) {
auto mask = -(crc & 1);
crc = (crc >> 1) ^ (polynomial & mask);
}
crc32_table[byte] = crc;
}
return crc32_table;
}
Next, we store the table in a global and perform rudimentary static checking on it. This checking could most likely be improved, and it is not necessary to store it in a global.
// Stores CRC-32 table and softly validates it.
static constexpr auto crc32_table = gen_crc32_table();
static_assert(
crc32_table.size() == 256 &&
crc32_table[1] == 0x77073096 &&
crc32_table[255] == 0x2D02EF8D,
"gen_crc32_table generated unexpected result."
);
Now that the table is generated, it's time to generate the CRC-32 codes. I again based the algorithm off the Hacker's Delight link, and at the moment it only supports input from a c-string.
// Generates CRC-32 code from null-terminated, c-string,
// algorithm based from this link:
// http://www.hackersdelight.org/hdcodetxt/crc.c.txt
constexpr auto crc32(const char *in) {
auto crc = 0xFFFFFFFFu;
for (auto i = 0u; auto c = in[i]; ++i) {
crc = crc32_table[(crc ^ c) & 0xFF] ^ (crc >> 8);
}
return ~crc;
}
For sake of completion, I generate one CRC-32 code below and statically check if it has the expected output, and then print it to the output stream.
int main() {
constexpr auto crc_code = crc32("some-id");
static_assert(crc_code == 0x1A61BC96, "crc32 generated unexpected result.");
std::cout << std::hex << crc_code << std::endl;
}
Hopefully this helps anyone else that was looking to achieve compile time generation of CRC-32, or even in general.
#tux3's answer is pretty slick! Hard to maintain, though, because you are basically writing your own implementation of CRC32 in preprocessor commands.
Another way to solve your question is to go back and understand the need for the requirement first. If I understand you right, the concern seems to be performance. In that case, there is a second point of time you can call your function without performance impact: at program load time. In that case, you would be accessing a global variable instead of passing a constant. Performance-wise, after initialization both should be identical (a const fetches 32 bits from your code, a global variable fetches 32 bits from a regular memory location).
You could do something like this:
static int myWSID = 0;
// don't call this directly
static int WSID(const char* str) {
boost::crc_32_type result;
result.process_bytes(str,sizeof(str));
return result.checksum();
}
// Put this early into your program into the
// initialization code.
...
myWSID = WSID('some-id');
Depending on your overall program, you may want to have an inline accessor to retrieve the value.
If a minor performance impact is acceptable, you would also write your function like this, basically using the singleton pattern.
// don't call this directly
int WSID(const char* str) {
boost::crc_32_type result;
result.process_bytes(str,sizeof(str));
return result.checksum();
}
// call this instead. Note the hard-coded ID string.
// Create one such function for each ID you need to
// have available.
static int myWSID() {
// Note: not thread safe!
static int computedId = 0;
if (computedId == 0)
computedId = WSID('some-id');
return computedId;
}
Of course, if the reason for asking for compile-time evaluation is something different (such as, not wanting some-id to appear in the compiled code), these techniques won't help.
The other option is to use Jason Gregory's suggestion of a custom preprocessor. It can be done fairly cleanly if you collect all the IDS into a separate file. This file doesn't need to have C syntax. I'd give it an extension such as .wsid. The custom preprocessor generates a .H file from it.
Here is how this could look:
idcollection.wsid (before custom preprocessor):
some_id1
some_id2
some_id3
Your preprocessor would generate the following idcollection.h:
#define WSID_some_id1 0xabcdef12
#define WSID_some_id2 0xbcdef123
#define WSID_some_id3 0xcdef1234
And in your code, you'd call
Engine.getById(WSID_some_id1);
A few notes about this:
This assumes that all the original IDs can be converted into valid identifiers. If they contain special characters, your preprocessor may need to do additional munging.
I notice a mismatch in your original question. Your function returns an int, but Engine.getById seems to take a string. My proposed code would always use int (easy to change if you want always string).
Hey guys! When I try to do the following in a header file
static const char FOOT[] = "Foot";
static const char FEET[] = FOOT;
I get a compiler error of error: initializer fails to determine size of FEET. I was wondering what the cause of this is, and if there is a way to rectify it. Thanks!
Even though why you get this error has been answered, there's more to the story. If you really need for FEET to be an array, then you can make it a reference instead of a pointer:
char const foot[] = "foot";
char const (&feet)[sizeof foot] = foot;
// reference to array (length 5) of constant char
// (read the declarations "from the inside-out")
char const* const smelly_feet = foot;
// constant pointer to const char
int main() {
cout << sizeof feet << ' ' << feet << '\n';
cout << sizeof smelly_feet << ' ' << smelly_feet << '\n';
cout << sizeof(void*) << " - compare to the above size\n";
return 0;
}
(More examples of the inside-out rule.)
Secondly, static at file and namespace scope means internal linkage; so when you use it in a header, you'll get duplicate objects in every TU using that header. There are cases when you want this, but I see no reason for it in your code, and it's a common error.
Regarding array size: The sizeof operator returns the memory-size of an object or instance of a type. For arrays this means the total memory-size of all of its items. Since C++ guarantees that sizeof(char) is 1, the memory-size of a char array is the same as its length. For other array types, you can divide by the memory-size of one item to get the length of an array:
void f() {
int array[5];
assert((sizeof array / sizeof *array) == 5);
}
And you can generalize it to a function template:
template<class T, int N>
int len(T (&)[N]) {
return N;
}
// use std::size_t instead of int if you prefer
This exists in boost as boost::size.
You may see code that uses sizeof array / sizeof *array, either through a macro or directly, either because it's old or doesn't want to complicate matters.
The cause is that you are saying that FEET[] is an array of chars but you're initializing it to a ptr;
To rectify, you can change FEET from FEET[] to *FEET
Could also do the same with FOOT
There is no '=' operator for arrays in C++, you need to use strcpy or something similar (which you can't do with a static const). Depending on your requirements the following might be useful:
#define contentsOfFoot "FOOT"
static const char FOOT[5] = contentsOfFoot;
static const char FEET[5] = contentsOfFoot;
static const char *feeet = FOOT;
static is deprecated, and const variables at namespace scope are effectively static, anyway. Also, please reserve ALL_CAPS for preprocessor identifiers.
A bigger question is why you are using raw arrays at all. You very likely want something more like this:
std::string const foot = "foot";
std::string const feet = foot;
Or, more likely:
#include <string>
namespace spbots {
typedef std::string string_t;
string_t const foot = "foot";
string_t const feet = foot;
}
One way would be this
static const char FOOT[] = "Foot";
static const char *FEET = FOOT;
I want to emphasise that you shouldn't put that stuff in a header file. Variables should only be put in header files if you want to make them globally visible (generally not a great idea but it's a legitimate technique).
feet.h
extern char feet[];
feet.cpp
char feet[] = "FEET";
Now when you include feet.h in a different .cpp file, say leg.cpp, the declaration in feet.h means leg.cpp can see and use the array defined in feet.cpp
The extern keyword has the opposite meaning to the static keyword you use. A typical use of static is as follows;
feet.cpp
static char feet[] = "FEET";
Now feet[] is explicitly not visible outside feet.cpp. The static keyword is responsible for this lack of visibility. So static=local, extern=global and (perhaps unfortunately?) extern is the default, so if you don't use either keyword you get extern whether you like it or not.
Because static=local, there's nothing to communicate to other modules, so no need to have a declaration in a header file for other modules to see. Everything is kept within feet.cpp, probably a better idea than a global variable.
Another point I am implicitly trying to get through with this explanation is that declarations go into header files, definitions go into .cpp files. In your question you have a definition in a header file. Not illegal, but if you include it in more than one file (and if you don't, why would you put it in a header?), you will get a single variable defined multiple times, which will give you a link error.
All of this is actually old school C rather than C++ in the modern sense of the word. In other words you are really writing C, but since C is (almost) a subset of C++, you can still use C++ to do this stuff.
I am working on a game and have an interesting question. I have some game-wide constant values that I want to implement in one file. Right now I have something like this:
constants.cpp
extern const int BEGINNING_HEALTH = 10;
extern const int BEGINNING_MANA = 5;
constants.hpp
extern const int BEGINNING_HEALTH;
extern const int BEGINNING_MANA;
And then files just #include "constants.hpp"
This was working great, until I needed to use one of the constants as a template parameter, because externally-linked constants are not valid template parameters.
So my question is, what is the best way to implement these constants? I am afraid that simply putting the constants in a header file will cause them to be defined in each translation unit. And I don't want to use macros.
Thanks
Get rid of the extern and you're set.
This code works perfectly fine in a header, because everything is "truly constant" and therefore has internal linkage:
const int BEGINNING_HEALTH = 10;
const int BEGINNING_MANA = 5;
const char BEGINNING_NAME[] = "Fred";
const char *const BEGINNING_NAME2 = "Barney";
This code cannot safely be put in a header file because each line has external linkage (either explicitly or because of not being truly constant):
extern const int BEGINNING_HEALTH = 10;
extern const int BEGINNING_MANA = 5;
const char *BEGINNING_NAME = "Wilma"; // the characters are const, but the pointer isn't
How about enums?
constants.hpp
enum {
BEGINNING_HEALTH = 10,
BEGINNING_MANA = 5
}
Use "static const int" in your .hpp file, and put nothing in the .cpp file (except whatever other code you have there of course).
make use of namespaces:
namespace GameBeginning {
const int HEALTH = 10;
const int MANA = 5;
};
then u can use as player.health = GameBeginning::HEALTH;
Most compilers simply don't allocate space for const POD values. They optimize them out and treat them as if they had been #defined, don't they?
What ever happened to a simple:
#define BEGINNING_HEALTH 10
Man, those were the days.
Oh wait, those still are the days!
perhaps something along the lines of a static class?
class CONSTANTS {
public:
static inline int getMana() { return 10;};
};
As a quick answer to the title question, a singleton pattern is a possible best, C++ way to define cross-file constants and insure only one instance of the object.
As far as the template parameter problem, you need to pass a type not a value. Your type is "int".