The best way to handle config in large C++ project - c++

In order to start my C++ program, I need to read some configs, e.g. ip address, port number, file paths... These settings may change quite frequently (every week or everyday!), so hardcoding them into source files is not a good idea.
After some research, I'm confused about whether there is a best practice to load config settings from a file and made those configs available to other class/module/*.cpp in the same project.
static is bad; singleton is bad (an anti-pattern?) So, what other options do we have? Or, maybe the idea of "config file" is wrong?
EDIT: I have no problem of loading the config file. I'm worried about, after loading all those settings into a std::map< string, string > in memory, how to let other classes, functions access those settings.
EDIT 2: Thanks for everybody's input. I know these patterns that I listed here are FINE, and they are used by lots of programs. I'm curious about whether there is a (sort of) BEST pattern to handle configurations of a program.


Arguably, a configuration file is a legitimate use for a Singleton. The Singleton pattern is usually frowned upon because Singletons cause problems with race conditions in a multi-threaded environment, and since they're globally accessible, you run into the same problems you have with globals. But if your Singleton object is initialized once when you read in the config file, and never altered after that, I can't think of a legitimate reason to call it an "anti-pattern" other than some sort of cargo-cult mentality.
That being said, when I need to make a configuration file available as an object to my application, I don't use a Singleton. Usually I pass the configuration object around to those objects/functions which need it.


The best pattern I know of solving this is through an options class, that gets injected into your code on creation/configuration.
Steps:
create an options parser class
configure the parser on what parameters and options it should accept, and their default values (default values can be your "most probable" defaults)
write client code to accept options as parameters (instead of singleton and/or static stuff).
inject options when creating objects.
Have a look at boost.program_options for an already mature module for program options.
If you're familiar with python, have a look at the examples in the doc of argparse (same concept, implemented in python library). They are very easy to get the concept and interactions from.

Related

Can I have configurable static linkage in C++?

I know this sounds like "can I have dynamic static linkage" which makes little sense, but let me explain.
I am looking for options to explore and I am aware that there might be something out there that I'm not aware of.
My goal is to have a modular code base where plugins would be provided as static libs, to avoid having exposed dll and result in a single library which clients could swallow in their code.
I imagine having a config file listing all the desired plugins, feed that to a script and boom : a magic sln file with everything ready to be built.
My initial idea is to have a 'main deck' that would know every possible plugin interface and link with them all. Plugins not yet implemented or required by the client would be dead-end/no-op implementations, while required plugins would be implementations realizing the interfaces called by the 'main deck'.
I think that would work, but I find conceptually horrible the idea of linking dead implementations for the sake of modularity.
The main issue I see is at that 'main deck' level : how could I remove useless headers to prevent useless linking or add newly developed ones without editing the code each time? I cannot figure this out without a ton of macros or generating some source files.
Could other patterns solve that issue?

I think there is no possible way that doesn't involve macro magic and a complex build system.
If I understand correctly, what you want to do is similar to a library I have used, rocksdb. At build time you can specify what modules/packages you want and it will build them into the static library for you. Check out what they do and see if it is along the lines of what you want.

Restricting header inclusions in C++

What is the best way to restrict use of certain headers(features of the library itself) in certain Cpp files. And if it fails to follow the set rules, compilation should halt.
This is not about finding out superfluous includes. This is about restricting the developers to the applicaiton framework.
For example if there exists a osUtils class as osUtils.h and if as per this, this application's development framework mandates use of osUtils.h for filesystem operation like to make a folder. but there are always a chance that individual module finds it convenience to break this rule by including sys/stat and use a mkdir() method. But if the intention of providing a framework here lets say for cross-platform abstraction or special path handling logic, the objective is lost by doing it out of framework. Is there a way to restrict this? like restricting the usage of sys/stat.h in certain files (except for osUtils.h file in this case) can help solve the problem. but how to implement it so it will not compile if the rule is broken.

I don't know how to do this by breaking compilation - the idea of compilation failure because of a valid code don't appeal to me. I've got some other ideas:
Code review. If done right this should prevent such errors.
I am pretty sure that some static code analysis tool can help detect
those things (they can check things like 'include what you use', so
a rule 'don't include 'XYZ' should be there)
If you have this static analysis tool ready there is a problem with getting people to use it and fix errors shown by it. One option that you can use is git hook. If the new code don't pass the static analysis - reject the commit. If you cannot use hooks, or don't want to - make a separate CI job that will check for the violations of the static checking. Then you'll see who and when pushed some bad code.

Is there a way to implement dynamic factory pattern in c++?

The DYNAMIC FACTORY pattern describes how to create a factory that
allows the creation of unanticipated products derived from the same
abstraction by storing the information about their concrete type in
external metadata
from : http://www.wirfs-brock.com/PDFs/TheDynamicFactoryPattern.pdf

The PDF says:
Configurability
. We can change the behavior of an application by just changing its configuration
information. This can be done without the need to change any source code (just change the descriptive information about the type in the metadata repository) or to restart the application (if caching is not used – if caching is used the cache will need to be flushed).
It is not possible to introduce new types to a running C++ program without modifying source code. At the very least, you'd need to write a shared library containing a factory to generate instances of the new type: but doing so is expressly rules out by the PDF:
Extensibility / Evolvability
. New product types should be easily
added without requiring neither a
new factory class nor modifying
any existing one.
This is not practical in C++.
Still, the functionality can be achieved by using metadata to guide some code writing function, then invoking the compiler (whether as a subprocess or a library) to create a shared library. This is pretty much what the languages mentioned in the PDF are doing when they use reflection and metadata to ask the virtual machine to create new class instances: it's just more normal in those language environments to need bits of the compiler/interpreter hanging around in memory, so it doesn't seem such a big step.

Yes...
Look at the Factories classes in the Qtilities Qt library.
#TonyD regarding
We can change the behavior of an application by just changing its configuration information.
It is 100% possible if you interpret the sentence in another way. What I read and understand is you change a configuration file (xml in the doc) that gets loaded to change the behaviour of the application. So perhaps your application has 2 loggers, one to file and one to a GUI. So the config file can be edited to choose one or both to be used. Thus no change of the application but the behaviour is changed. The requirement is that anything that you can configure in the file is available in the code, so to say log using network will not work since it is not implemented.
New product types should be easily added without requiring neither a new factory class nor modifying any existing one.
Yes that sounds a bit impossible. I will accept the ability to add ones without having to change the original application. Thus one should be able to add using plugins or another method and leave the application/factory/existing classes in tact and unchanged.
All of the above is supported by the example provided. Although Qtilities is a Qt library, the factories are not Qt specific.

C++: Any way to 'jail function'?

Well, it's a kind of a web server.
I load .dll(.a) files and use them as program modules.
I recursively go through directories and put '_main' functors from these libraries into std::map under name, which is membered in special '.m' files.
The main directory has few directories for each host.
The problem is that I need to prevent usage of 'fopen' or any other filesystem functions working with directory outside of this host directory.
The only way I can see for that - write a warp for stdio.h (I mean, write s_stdio.h that has a filename check).
May be it could be a deamon, catching system calls and identifying something?
add
Well, and what about such kind of situation: I upload only souses and then compile it directly on my server after checking up? Well, that's the only way I found (having everything inside one address space still).

As C++ is low level language and the DLLs are compiled to machine code they can do anything. Even if you wrap the standard library functions the code can do the system calls directly, reimplementing the functionality you have wrapped.
Probably the only way to effectively sandbox such a DLL is some kind of virtualisation, so the code is not run directly but in a virtual machine.
The simpler solution is to use some higher level language for the loadable modules that should be sandboxed. Some high level languages are better at sandboxing (Lua, Java), other are not so good (e.g. AFAIK currently there is no official restricted environment implemented for Python).

If you are the one loading the module, you can perform a static analysis on the code to verify what APIs it calls, and refuse to link it if it doesn't check out (i.e. if it makes any kind of suspicious call at all).
Having said that, it's a lot of work to do this, and not very portable.

How do you handle command line options and config files?

What packages do you use to handle command line options, settings and config files?
I'm looking for something that reads user-defined options from the command line and/or from config files.
The options (settings) should be dividable into different groups, so that I can pass different (subsets of) options to different objects in my code.
I know of boost::program_options, but I can't quite get used to the API. Are there light-weight alternatives?
(BTW, do you ever use a global options object in your code that can be read from anywhere? Or would you consider that evil?)

At Google, we use gflags. It doesn't do configuration files, but for flags, it's a lot less painful than using getopt.
#include <gflags/gflags.h>
DEFINE_string(server, "foo", "What server to connect to");
int main(int argc, char* argv[]) {
google::ParseCommandLineFlags(&argc, &argv, true);
if (!server.empty()) {
Connect(server);
}
}
You put the DEFINE_foo at the top of the file that needs to know the value of the flag. If other files also need to know the value, you use DECLARE_foo in them. There's also pretty good support for testing, so unit tests can set different flags independently.

For command lines and C++, I've been a fan of TCLAP: Templatized Command Line Argument Parser.
http://sourceforge.net/projects/tclap/

Well, you're not going to like my answer. I use boost::program_options. The interface takes some getting used to, but once you have it down, it's amazing. Just make sure to do boatloads of unit testing, because if you get the syntax wrong you will get runtime errors.
And, yes, I store them in a singleton object (read-only). I don't think it's evil in that case. It's one of the few cases I can think of where a singleton is acceptable.

If Boost is overkill for you, GNU Gengetopt is probably, too, but IMHO, it's a fun tool to mess around with.
And, I try to stay away from global options objects, I prefer to have each class read its own config. Besides the whole "Globals are evil" philosophy, it tends to end up becoming an ever-growing mess to have all of your configuration in one place, and also it's harder to tell what configuration variables are being used where. If you keep the configuration closer to where it's being used, it's more obvious what each one is for, and easier to keep clean.
(As to what I use, personally, for everything recently it's been a proprietary command line parsing library that somebody else at my company wrote, but that doesn't help you much, unfortunately)

I've been using TCLAP for a year or two now, but randomly I stumbled across ezOptionParser. ezOptionParser doesn't suffer from "it shouldn't have to be this complex"-syndrome the same way that other option parsers do.
I'm pretty impressed so far and I'll likely be using it going forward, specifically because it supports config files. TCLAP is a more sophisticated library, but the simplicity and extra features from ezOptionParser is very compelling.
Other perks from its website include (as of 0.2.0):
Pretty printing of parsed inputs for debugging.
Auto usage message creation in three layouts (aligned, interleaved or staggered).
Single header file implementation.
Dependent only on STL.
Arbitrary short and long option names (dash '-' or plus '+' prefixes not required).
Arbitrary argument list delimiters.
Multiple flag instances allowed.
Validation of required options, number of expected arguments per flag, datatype ranges, user defined ranges, membership in lists and case for string lists.
Validation criteria definable by strings or constants.
Multiple file import with comments.
Exports to file, either set options or all options including defaults when available.
Option parse index for order dependent contexts.

GNU getopt is pretty nice. If you want a C++ feel, consider getoptpp which is a wrapper around the native getopt.
As far as configuration file is concerned, you should try to make it as stupid as possible so that parsing is easy. If you are bit considerate, you might want to use yaac&lex but that would be really a big bucks for small apps.
I also would like to suggest that you should support both config files and command line options in your application. Config files are better for those options which are to be changed less frequently. Command-line options are good when you want to pass the immediate changing arguments (typically when you are creating a app, which would be called by some other program.)

If you are working with Visual Studio 2005 on x86 and x64 Windows there is some good Command Line Parsing utilities in the SimpleLibPlus library. I have used it and found it very useful.

Not sure about command line argument parsing. I have not needed very rich capabilities in that area and have generally rolled my own to save adding more dependencies to my software. Depending upon what your needs are you may or may not want to try this route. The C++ programs I have written are generally not invoked from the command line.
On the other hand, for a config file you really can't beat an XML based format. It's readable, extensible, structured, etc... :) Plus there are lots of XML parsers out there. Despite the fact it is a C library, I tend to use libxml2 from xmlsoft.org.

Try Apache Ant. Its primary usage is Java projects, but there isn't anything Java about it, and its usable for almost anything.
Usage is fairly simple and you've got a lot of community support too. It's really good at doing things the way you're asking.
As for global options in code, I think they're quite necessary and useful. Don't misuse them, though.