I am including a complicated project as a library in C++ using Visual Studio 2008.
I have a set of include files that are scattered throughout a very complicated directory tree structure. The root of the tree has around ten directories, and then each directory could have multiple subdirectories, subsubdirectories, etc.
I know that all the header files in that structure are necessary, and they are hopelessly interlinked; I can't just include one directory, because then dependencies in another directory will feel left out and cause the compiler to crash in their annoyance at not being invited to the party. So, everyone has to be included.
I can do this by adding the directories one at a time to the project (right click->properties->additional include directories), but that can be fraught with pain, especially when one of the dependencies has children and makes a brand new subsubsubdirectory.
Is there a way to specify an include directory in a header file itself, so that I can just include that header whenever I need to use the functions it contains? That way, I get an easier way to edit the include files, and I don't have to make sure that the debug and release versions agree with each other (since the properties right click defaults to the current build, not all builds, a feature that has led to much crashing when switching from debug to release). Even better, is there a way to point to the directory root and force everything to be recursively included?
EDIT for all those replies so far:
I cannot edit the structure of this project. I can only link to it. I don't like the way the code is organized anymore than anyone else seems to, but I have to work within this constraint. Rather than spending potentially hours in the error-prone process of finding all the interdependencies and putting them in a project file, is there a way to do this programmatically?
That's clearly not a good idea, really.
These directories are a way to organize the code in logical groups.
/web
/include
/web
/stackoverflow
/language-agnostic
/algorithm
/database
/meta
/bug
/feature-request
/src
/local/
/include
/local
/my-favorites
/src
Now if I type
#include "exception.h"
What the heck am I trying to include ? Where's that file ? How can I see its content ?
On the other hand if I type
#include "local/my-favorites/exception.h"
Then it's perfectly clear. (and I just have two includes -Iweb/include -Ilocal/include)
And this way, I can have multiple files that have the exact same name and there would be no ambiguity, nifty when you wish to integrate two different 3rd party libraries which both have such a 'exception.h'.
Also note that for clarity, the namespace nesting should reflect the directories organization. So that
file: "web/include/web/meta/bug/exception.h"
namespace web { namespace meta { namespace bug {
struct exception: std::runtime_error {};
} } } // namespace bug, namespace meta, namespace web
This way it's easy to think of what header you have to include when you want a class.
Also note that, for example if you look at boost, they put headers for 'lazy' programmers, in each directory, which include the headers of all subdirectories
file: "web/include/web/meta/bug.h"
#include "web/meta/bug/file1.h"
#include "web/meta/bug/file2.h"
#include "web/meta/bug/file3.h"
#include "web/meta/bug/file4.h"
#include "web/meta/bug/file5.h"
file: "web/include/web/meta.h"
#include "web/meta/bug.h"
#include "web/meta/feature-request.h"
These includes might also 'pull' names into a more generic namespace with a using directive:
namespace web { namespace meta {
using ::web::meta::bug::bug;
} } // namespace meta, namespace web
To make it less painful for developers.
So as you can see, the language already provide you with a very good way of organizing your code cleanly, if you go with the 'all includes' options, you'll just end up with an unmaintainable mess:
#include "exception.h"
#include "bug.h"
#include "myString.h"
#include "database_connect.h"
#include "helper.h" // really love this one...
#include "file.h" // not bad either eh ?
I've had some of these at work... think 20 unqualified includes when you depend on 25+ components... now, do you think that it would be possible to remove a dependency on component X ? ;)
EDIT: How to deal with 3rd party library ?
Sometimes a 3rd party library does not live up to your expectations, whether it is:
not self-sufficient headers (ie you need to include 3 files to use 1 object)
warnings at compilation
header organization problem
you always have the opportunity to wrap them in headers of your own.
For example, say I have:
/include
/file1.h
/file2.h
/detail
/mustInclude.h
/exception.h
And anytime you wish to include a file, you have to include 'exception.h' BEFORE and 'mustInclude.h', and of course you have the problem that it is difficult to spot that the files included come from this 3rd party library and not your own (current) project.
Well, just wrap:
/include
/3rdParty
/file1.h (same name as the one you would like to include, it's easier)
file: "/include/3rdParty/file1.h"
#pragma push
// ignore warnings caused
#include "detail/exception.h" // necessary to include it before anything
#include "file1.h"
#include "detail/mustInclude.h"
#pragma pop
And then in your code:
#include "3rdParty/file1.h"
You have just isolated the problem, and all the difficulty now lies within your wrappers files.
Note: I just realize that you may have the problem that the 3rd party headers reference each others without taking the 'relative path' into account, in this case, you can still avoid the 'multiple include' syndroms (even without edition), but that might be ill-fated.
I suppose you don't have the opportunity not to use such crap :x ?
If in any way possible you really should take the time to rework that mess. It will only get worse.
If you are including it as a library then you probably only need a subset of its functionality anyway - make it slowly accessible and usable again step by step.
edit:
And no, there is no recursive include - for good reason. It would be completely uncontrollable from a certain size on and you'd fall over file-name-collisions all the time.
You could hack around the limitation via scripting and the project files, but you really would regret it in the long run.
edit2:
Ok, emergency procedure then. Put the library in its own project and use a short script to generate a line containing the neccessary paths seperated by ; which to put in the project file into the projects .vcproj (which is just an xml-file) under:
VisualStudioProject -> Configurations -> Configuration -> Tool -> AdditionalIncludeDirectories
(at least in VS2005, might differ a bit in later versions).
If you need all the includes in your calling code, you might want to consider wrapping that part in a static library in a seperate project to avoid polluting the rest.
Q: I cannot edit the structure of this
project. I can only link to it. I
don't like the way the code is
organized anymore than anyone else
seems to, but I have to work within
this constraint. Rather than spending
potentially hours in the error-prone
process of finding all the
interdependencies and putting them in
a project file, is there a way to do
this programmatically?
This should get you started:
write a script (in your favorite "quick" language) to enumerate all the [sub-...]directories in the library directory
filter the directories (for start, remove the ones that don't contain any .h files)
output them either to header file as you described or directly to the vcproj file. The
vs2008 vcproj file is relatively simple XML. It might even be documented, but if you just view it in a text editor, you'll figure out where and how the include paths are defined
try to compile
if errors, try to figure out why, adjust directory filter (blacklist specific directories etc.) and go back to 2.
success
Anyway, don't be afraid to write code generation scripts
Related
In the following files:
app/main.cpp
app/config.hpp
app/show.hpp
app/file.hpp
lib/append.hpp
lib/clear.hpp
lib/reopen.hpp
lib/crypt.hpp
I have a problem. Imagine a few of my files use lib/crypt.hpp. For example in app/file.hpp I should write:
#include "../lib/crypt.hpp"
If I take this file to lib folder it does not work anymore. So I need a something like:
#include "LIB/crypt.hpp"
So, this LIB/ has meaning of (empty) in lib directory while it has meaning of "../lib/" in app directory.
Then I will have no worry about moving file.hpp from app to lib. I just need to fix the callers of it. But not what it calls.
Similar to concept of web programming frameworks. Is it possible in C/C++?
According to what you wrote you're searching for a way to move your sources around without worrying for hard-coded paths to your headers.
You didn't specify the compiler you're using so I'm not focusing on a specific one, anyway most C++ compilers have an option to specify one or more header search paths (on gcc you would just use something like -Ilib_parent_dir).
Once you've included your lib's parent path to the search list, you can move your sources around (as long as you don't move lib's headers) and have them include the right headers with something like #include <lib/crypt.hpp> (and keep include paths priority in mind)
This should be a cleaner and simpler way to achieve what you asked rather than using a macro.
Is there an automated way to take a large amount of C++ header files and combine them in a single one?
This operation must, of course, concatenate the files in the right order so that no types, etc. are defined before they are used in upcoming classes and functions.
Basically, I'm looking for something that allows me to distribute my library in two files (libfoo.h, libfoo.a), instead of the current bunch of include files + the binary library.
As your comment says:
.. I want to make it easier for library users, so they can just do one single #include and have it all.
Then you could just spend some time, including all your headers in a "wrapper" header, in the right order. 50 headers are not that much. Just do something like:
// libfoo.h
#include "header1.h"
#include "header2.h"
// ..
#include "headerN.h"
This will not take that much time, if you do this manually.
Also, adding new headers later - a matter of seconds, to add them in this "wrapper header".
In my opinion, this is the most simple, clean and working solution.
A little bit late, but here it is. I just recently stumbled into this same problem myself and coded this solution: https://github.com/rpvelloso/oneheader
How does it works?
Your project's folder is scanned for C/C++ headers and a list of headers found is created;
For every header in the list it analyzes its #include directives and assemble a dependency graph in the following way:
If the included header is not located inside the project's folder then it is ignored (e.g., if it is a system header);
If the included header is located inside the project's folder then an edge is create in the dependency graph, linking the included header to the current header being analyzed;
The dependency graph is topologically sorted to determine the correct order to concatenate the headers into a single file. If a cycle is found in the graph, the process is interrupted (i.e., if it is not a DAG);
Limitations:
It currently only detects single line #include directives (e.g., #include );
It does not handles headers with the same name in different paths;
It only gives you a correct order to combine all the headers, you still need to concatenate them (maybe you want remove or modify some of them prior to merging).
Compiling:
g++ -Wall -ggdb -std=c++1y -lstdc++fs oneheader.cpp -o oneheader[.exe]
Usage:
./oneheader[.exe] project_folder/ > file_sequence.txt
(Adapting an answer to my dupe question:)
There are several other libraries which aim for a single-header form of distribution, but are developed using multiple files; and they too need such a mechanism. For some (most?) it is opaque and not part of the distributed code. Luckily, there is at least one exception: Lyra, a command-line argument parsing library; it uses a Python-based include file fuser/joiner script, which you can find here.
The script is not well-documented, but they way you use it is with 3 command-line arguments:
--src-include - The include file to convert, i.e. to merge its include directives into its body. In your case it's libfoo.h which includes the other files.
--dst-include - The output file to write - the result of the merging.
--src-include-dir - The directory relative to which include files are specified (i.e. an "include search path" of one directory; the script doesn't support the complex mechanism of multiple include paths and search priorities which the C++ compiler offers)
The script acts recursively, so if file1.h includes another file under the --src-include-dir, that should be merged in as well.
Now, I could nitpick at the code of that script, but - hey, it works and it's FOSS - distributed with the Boost license.
If your library is so big that you cannot build and maintain a single wrapping header file like Kiril suggested, this may mean that it is not architectured well enough.
So if your library is really huge (above a million lines of source code), you might consider automating that, with tools like
GCC make dependency generator preprocessor options like -M -MD -MF etc, with another hand made script sorting them
expensive commercial static analysis tools like coverity
customizing a compiler thru plugins or (for GCC 4.6) MELT extensions
But I don't understand why you want an automated way of doing this. If the library is of reasonable size, you should understand it and be able to write and maintain a wrapping header by hand. Automating that task will take you some efforts (probably weeks, not minutes) so is worthwhile only for very large libraries.
If you have a master include file that includes all others available, you could simply hack a C preprocessor re-implementation in Perl. Process only ""-style includes and recursively paste the contents of these files. Should be a twenty-liner.
If not, you have to write one up yourself or try at random. Automatic dependency tracking in C++ is hard. Like in "let's see if this template instantiation causes an implicit instantiation of the argument class" hard. The only automated way I see is to shuffle your include files into a random order, see if the whole bunch compiles, and re-shuffle them until it compiles. Which will take n! time, you might be better off writing that include file by hand.
While the first variant is easy enough to hack, I doubt the sensibility of this hack, because you want to distribute on a package level (source tarball, deb package, Windows installer) instead of a file level.
You really need a build script to generate this as you work, and a preprocessor flag to disable use of the amalgamate (that could be for your uses).
To simplify this script/program, it helps to have your header structures and include hygiene in top form.
Your program/script will need to know your discovery paths (hint: minimise the count of search paths to one if possible).
Run the script or program (which you create) to replace include directives with header file contents.
Assuming your headers are all guarded as is typical, you can keep track of what files you have already physically included and perform no action if there is another request to include them. If a header is not found, leave it as-is (as an include directive) -- this is required for system/third party headers -- unless you use a separate header for external includes (which is not at all a bad idea).
It's good to have a build phase/translation that includes header alone and produces zero warnings or errors (warnings as errors).
Alternatively, you can create a special distribution repository so they never need to do more than pull from it occasionally.
What you want to do sounds "javascriptish" to me :-) . But if you insist, there is always "cat" (or the equivalent in Windows):
$ cat file1.h file2.h file3.h > my_big_file.h
Or if you are using gcc, create a file my_decent_lib_header.h with the following contents:
#include "file1.h"
#include "file2.h"
#include "file3.h"
and then use
$ gcc -C -E my_decent_lib_header.h -o my_big_file.h
and this way you even get file/line directives that will refer to the original files (although that can be disabled, if you wish).
As for how automatic is this for your file order, well, it is not at all; you have to decide the order yourself. In fact, I would be surprised to hear that a tool that orders header dependencies correctly in all cases for C/C++ can be built.
usually you don't want to include every bit of information from all your headers into the special header that enables the potential user to actually use your library. The non-trivial removal of type definitions, further includes or defines, that are not necessary for the user of your interface to know can not be automatedly done. As far as I know.
Short answer to your main question:
No.
My suggestions:
manually make a new header, that contains all relevant information (nothing more, nothing less) for the user of your library interface. Add nice documentation comments for each component it contains.
use forward declarations where possible, instead of full-fledged included definitions. Put the actual includes in your implementation files. The less include statements you have in your headers, the better.
don't build a deeply nested hierarchy of includes. This makes it extremely hard to keep an overview on the contents of every bit you include. The user of your library will look into the header to learn how to use it. And he will probably not be able to distinguish relevant code from irrelevant on the first sight. You want to maximize the ratio of relevant code per total code in the main header for your library.
EDIT
If you really do have a toolkit library, and the order of inclusion really does not matter, and you have a bunch of independent headers, that you want to enumerate just for convenience into a single header, then you can use a simple script. Like the following Python (untested):
import glob
with open("convenience_header.h", 'w') as f:
for header in glob.glob("*.h"):
f.write("#include \"%s\"\n" % header)
I come from a Java/AS3/Javascript background where all of my classes are organized into packages that help denote their functionality.
In starting a C++ project I sought to mimic this file system structure in mostly the same way but I've been running into issues with the includes.
Currently I have an src directory with the main.cpp file inside it. Then I have some root directories and with other files inside. Here's an example:
src
->main.cpp
->window
---->Window.h
---->Window.cpp
main.cpp includes Window.h with the statement #include "Window.h" and everything builds just fine. But if i restart Visual Studio, it complains that it can't find "Window.h".
In looking a open source projects, I've seen some that just have all the source files in one directory with no nesting for easy includes I suppose. Some have the headers and cpp files separated.
What is the correct way (or at least a way that will cause less headaches) to organize a large-ish C++ project on the file system?
Thanks!
Breaking it out like you've tried to do is reasonable and easy enough to accomplish.
You just need to set your project's include paths. From Visual Studio, right click on the project name and click "Properties". From there, in the tree control on the left hand side, expand "C/C++", and then select "General" in the tree. The first option on the right hand side should then be "Additional Include Directories".
There you have several options:
You can specify specific include directories (separated by semicolons). For instance, if you had folders "Window" and "Printing" you could put in:
..\Window;..\Printing
Which would allow you to include the files from window and printing easily, like this:
#include <Window.h> // from src/window
#include <Printing.h> // from src/printing
The above approach has some drawbacks, as you can easily collide with names from other libraries you may be using, making the include ORDER very important.
A better approach (in my opinion) is to add the following as an include path:
..\
This will make it search the parent directory when looking for includes. This allows you to be more verbose in your include paths, like this:
#include <Window/Window.h> // it's more clear where these are coming from
#include <Printing/Printing.h> // and much less likely to collide with other library
// header files
It makes sense to follow the Java example and arrange source files by C++ namespace. Create sub-folders under your /src directory that correspond to the namespaces.
Are there any articles or recommendations how to organize file hierarchy in your project? I am interested in how to name folders, to separate sources and headers or not.
I have project written in C++, a library and project using it. Library has many components, they are separated from each other but some of them use common files. Should I create directories for them?
I will be glad to hear all recommendations.
Do not split headers and source files into separate folders. It does nothing more than add an extra folder level.
At best it is completely useless; if you're looking for "widget.h" you can trivially find it even if there's a "widget.cpp" right next to it. At worst it's rather counter-productive - e.g. when you're editing "widget.h" and find that you also need to update "widget.cpp".
It's good to keep your namespaces in separate folders. Nest namespaces in the same manner as they are nested within your project. For instance, if you have:
namespace Foo{ namespace Bar{ } }
then you'd want any objects in the Bar namespace to be found in
{Foo's parent folder}\Foo\Bar\{how you're organizing code at this level}
We use an include folder for headers, a source for .cpp, a test folder for unit tests, and an object folder for compiled code bits. The reason we separate them is that it makes it easier to package the code in our scripts. You're always going to pass around the headers, you won't be passing around the source. (Here is another SO thread discussing separating header/source files. It's a preference thing.)
Here is a link to Google's style guidelines, if that helps.
I don't separate headers from source : it's a pain to browse
I usually have the subdirectories match my namespaces
+ Project root
+ <project_name> // namespace project
- sub_dir_1 // namespace project::sub_dir_1
- sub_dir_2 // namespace project::sub_dir_2
I only add "Project root" as an additional include path, so includes have the form :
#include "project/sub_dir_1/a.h"
#include "project/sub_dir_2/b.h"
Because source and header are usually named according to the class they contain, the whole qualified named can be deduced from the include path :
project::sub_dir_1::a is found in project/sub_dir_1/a.h
a.c trivially includes a.h (relative path provided)
If I must include a.h from b.h, I use absolute path (starting from the project/ root)
One of the popular way to organize project directory is more or less like this:
MyLib
+--mylib_class_a.h
mylib_class_a.cpp
mylib_library_private_helpers.h
mylib_library_private_helpers.cpp
MyApp
+--other_class.h
other_class.cpp
app.cpp
app.cpp:
#include "other_class.h"
#include <mylib_class_a.h> // using library MyLib
All .h and .cpp files for the same library are in the same directory. To avoid name collision, file names are often prefix with company name and/or library name. MyLib will be in MyApp's header search path, etc. I'm not a fan of prefixing filenames, but I like the idea of looking at the #include and know exactly where that header file belongs. I don't hate this approach of organizing files, but I think there should be a better way.
Since I'm starting a new project, I want to solicit some directory organization ideas. Currently I like this directory structure:
ProjA
+--include
+--ProjA
+--mylib
+--class_a.h
+--app
+--other_class.h
+--src
+--mylib
+--class_a.cpp
library_private_helpers.h
library_private_helpers.cpp
+--app
+--other_class.cpp
app.cpp
util.h
app.cpp:
#include "util.h" // private util.h file
#include <ProjA/app/other_class.h> // public header file
#include <ProjA/mylib/class_a.h> // using class_a.h of mylib
#include <other3rdptylib/class_a.h> // class_a.h of other3rdptylib, no name collision
#include <class_a.h> // not ProjA/mylib/class_a.h
#include <ProjA/mylib/library_private_helpers.h> // error can't find .h
.cpp files and private (only visible to immediate library) .h files are stored under the src directory (src is sometimes called lib). Public header files are organized into a project/lib directory structure and included via <ProjectName/LibraryName/headerName.h>. File names are not prefixed with anything. If I ever needed to package up MyLib to be used by other teams, I could simply change my makefile to copy the appropriate binary files and the whole include/ProjA directory.
Once files are checked into source control and people start working on them it will be hard to change directory structure. It is better to get it right at the get-go.
Anyone with experience organizing source code like this? Anything you don't like about it? If you have a better way to do it, I would very much like to hear about it.
Well, it all depends on how big these projects are. If you've only got a few files, then whack them all in one folder.
Too many folders when you haven't got many files to manage is in my opinion overkill. It gets annoying digging in and out of folders when you've only got a few files in them.
Also, it depends on who's using this stuff. If you're writing a library and its going to be used by other programmers, then it's good to organize the headers they want to use into an include folder. If you're creating a number of libraries and publishing them all, then your structure might work. But, if they're independent libraries, and the development isn't all done together and they get versioned and released at different times, you'd be better off sticking with having all files for one project locatable within one folder.
In fact, I would say keep everything in one folder, until you get to a point where you find its unmanagable, then reorganize into a clever scheme of dividing the source up into folders like you've done. You'll probably know how it needs to be organized from the problems you run into.
KISS is usually always the solution in programming -> keep everything as simple as possible.
Why not do something like the first, only use the directory that MyLib resides in as a part of the include directive, which reduces the silly prefixing:
#include <MyLib/ClassA.h>
That tells you where they are from. As for the second choice, I personally get really annoyed when I have a header or source file open, and have to navigate around through the directory structure to find the other and open it. With your second example, if you had src/mylib/class_a.cpp open, and wanted to edit the header, in many editors you'd have to go back two levels, then into include/ProjA before finding the header. And how are we to know that the header is in the ProjA subdirectory without some other external clue? Plus, it's too easy for one file or the other to get moved into a different place that "better" represents how it is used, without the alternate file being moved. It just gives me headaches when I encounter it at my job (and we do have some parts of our codebase where people did every potential problem I've just mentioned).
I have tried both methods. Personally, I like the first better. I understand the urge to put everything in more specific directories, but it causes a lot of over-complication.
I usually use this rule: applications and in-house libraries use the first method. Public open source libraries use the second method. When you are releasing the code, it helps a lot if the include files are in a separate directory.