My C++ project is very large and results in 5 different binaries to be generated. In VStudio for example, my single solution has 5 different "projects". In XCode for example, my single project has 5 different targets.
The code is organized in a very deep "src" folder with many many levels of nested sub folders. This src folder is common to all 5 binaries because there is heavy reuse of much of the source, but each binary requires some of the source but not all of it.
I'd like to know how to efficiently create a CMakeList.txt that can create what I need here.
Notes:
Reorganizing the code into a different structure is not an option nor is making the code a bunch of static libraries.
A CMakeList.txt for each subfolder is not an option. There's too many of them and maintenance would be a nightmare.
A file(GLOB_RECURSE is not a great option either because it's going to pick up a ton of source files for each binary that are unnecessary to compile for that particular binary.
Ideally, ONE XCode project (with 5 targets) or ONE VStudio Solution (with 5 projects) would be generated. I don't want 5 different projects to open.
I would be completely content with having to manually add/remove source files from a giant list somewhere...ideally in an external file that could be sucked up by CMake. E.g. SourceFilesForBinary1.txt, SourceFilesForBinary2.txt etc. but i'm not sure how to do that or if that's insane.
Any advice would be appreciated.
CMake has an include function. You can use that to implement your "giant manually managed list somewhere" solution.
You know that GLOB_RECURSE an be given a pattern, right, so it excludes uninteresting files? Even if not, everywhere you can use a GLOB_RECURSE you could also use an include and an evil manually-managed list.
I'm not sure why you don't want static libraries. Those are a good solution to this problem. For a large pile of shared code like this, if you compile it once into a relocatable static library and then link that with LTO into your various uses, you avoid recompiling the source many times. If your use is a shared library (so the static library approach would make all your unused symbols disappear), you can use the --whole-archive compiler switch to preserve them.
Related
I'm working with a very old and large VC6++ project and it's all messed up. There are unused files and folders everywhere, copies of folders and it's just a mess to clean it up by hand in its current state.
It will be done eventually, but is there any simple way to check what files and folders are used when it does a clean compile?
The project settings doesnt help me at all because it simply uses copies of folders and additional include directories.
Any suggestions?
Well, if you want to parse the compiler output you can get which files are actually used. I also find this when googling around, you might want to try (I haven't tried it myself). My way would be to clean the build, list all source files, build, and for each source find its corresponding .obj. The ones without .obj are not used. Note that this only works for source files, unused header files stay undetected.
VC6 will produce a makefile for you:
http://msdn.microsoft.com/en-us/library/aa233950%28v=vs.60%29.aspx
You can use the generated makefile (and the associated .dep file) as a starting point and edit it down to the list of files that get used in a build.
This will let you see the header files that the project depends on in addition to the .c/.cpp/.lib files that might show in the build log. One thing to keep in mind is that you'll probably also want to make sure you track the .dsw and .dsp workspace and project files.
If you're a bit adventurous, you might be able to convince the makefile to actually copy the source files to some other location for you with an appropriate override of the certain macros and/or dependencies. But that would probably be more trouble than it's worth for a one-time effort.
Finally, there's a commercial product, CopyWiz by Kinook Software, that seems to have features that might do what you're looking for (and it supports VC++ 6). Note: I'm not sure if it will do what you want, but it may be worth a look.
Yes. Run Process Monitor from SysInternals. It can capture all file system events and filter them based on the path and other factors.
So, set the filter to the root of your source tree, only succesfull file reads (VC looks for headers in many places), and build your project. You'll probably still see several thousand events. So, save them to file, sort by path, and remove duplicate paths (headers especially will have many duplicate entries)
This link mentions wildcards as a way to automatically list the SOURCES and HEADERS in the Makefile.am file. It also mentions that some people write external scripts to generate those files.
Do you know of the standard way of automatically including all the *.h *.cpp here, or should I just write my own Perl script to generate them. Do you have such a script already that you use?
PS: I organize the source files in my project according to the following purely-logical separation of directories:
src/dog/woof.h
src/dog/woof.cpp
src/cow/moo.h
src/cow/moo.cpp
Automake won't add this feature. It makes assumptions that a particular .h or .cpp file is associated with a particular project. That assumption holds for a number of common project layouts and fails for any layout that differs.
For example, I've had projects that were laid out as
src/module/code
src/app/code
src/library/code
include/headers
built from one central makefile in the root. Other times, I've had the same layout built from four makefiles in the appropriate local directories.
There's a lot of variability in projects. Some keep public header files mixed with private headers files in the code directories, some keep them separate. Some build shared object libraries, some don't. Some ship code that's not SUPPOSED to compile on incompatible platforms.
To put in a wild card inclusion would actually pose a great risk of limiting the functionality, and for those odd people who do things like 'file.template.c' and such it would be fatal.
If you consider it a flaw of automake, that's fine; however, it's one of those flaws that automake embraces as it's preserved in the effort to make things more flexible. Automake doesn't impose the "how" you do things, it provides a lot of enabling tools, but it goes out of it's way to ensure that you aren't forced into one "method" of laying out or building your code.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 3 years ago.
Improve this question
This questions concerns mostly Unix/Linux style C++ development. I see that many C++ libraries store their header files in a "include" folder and source files in an "src" folder. For the sake of conformance I adopted this in my own code. But it is not clear to me whether this should be done for application code as well. I've seen a few cases where a flat directory structure is used for that. What would be the recommended approach?
I also separate them, but not strictly on the extension, but on the access of the file.
Suppose you have a module that manages customer information and uses 2 classes to do this: Customer, CustomerValidityChecker.
Also suppose that other parts in your application only need to know about the Customer class, and that the CustomerValidityChecker is only used by the Customer class to perform some checking.
Based on these assumptions I store the files like this:
Public folder (or include folder):
customer.h
Private folder (or source folder):
customer.cpp
customervaliditychecker.h
customervaliditychecker.cpp
That way, it becomes immediately clear for callers of your module which parts are accessible (public) and which parts aren't.
We have a build system that auto-generates our makefiles. One thing it does is recursively descend any subdirectories and build them as libraries, linking them together with the main directory's objects to make the application. (In practice, these "subdirectories" are usually symbolic links.) Libraries are static unless the directory name ends in ".so". One thing that's nice about this is that a full build of our system, which has many executables, doesn't have to repeatedly compile the common libraries.
However, as a result of this, there's no separation of headers and sources. And it has never been a problem. Honestly, I think it's better this way because headers and source files have commonality of location, and you can grab a directory and know you got everything you need to use it. It also works great with Subversion's "externals" feature, and similar features in other VCSs.
One last place where an include/src separation fails is if you use any code generators, such as flex, bison, or gengetopts. Figuring out where these tools should put their outputs so they get built is tricky if you've spread things out.
It makes sense to separate them for shared libraries because they may be distributed in a compiled form without the source. I've seen projects that separate out "public" headers (headers that may be accessed from code outside your project or library) while leaving "private" headers and source files in the same directory. I think it's good to use a consistent approach whether you're writing shared library or application level code because you never know when you may want to turn something that you've written at the application level into a lower level library that is shared by multiple projects.
A lot depends on the size of project involved. Up to a few dozen files or so, keeping them in one directory tends to be more convenient. For a bigger application that includes hundreds or thousands of files, you start to look for ways to separate them (though in the projects I've worked on, it was done more on functional lines than src/include). In between those, it's probably open to question.
I don't do this; there seems little advantage in it. Since headers tend to have a different extension from source files, you can have your editor show them separately if you really feel the need -- Visual Studio does this by default, but I disable it since I prefer seeing them together
Bottom Line: sources and headers that are still changing go in /src. Code that has crystallised should go in /lib & /include (actually you could keep all .libs and their .hs in /lib).
Keep own sources and headers together, provided they are (a) specific to this project or (b) have not yet been factored out as a shared library.
Once certain sources in the main project have been factored out as a (relatively stable) library, place the .a or .lib into /lib, and its public interface header into /include.
All third party libraries and their public interface headers also go into /lib & /include.
As others note, it is often more compatible for tools / IDEs to access .h/.c from one folder. But from an organisational view it can be useful to separate changing local code from stable lib code.
There is no clear advantage to either in my view. I finally decided to keep program and header files together because my editor (Visual SlickEdit) happens to provide additional referential features when they are not separated.
I almost always create include and src folders to split up my source code. I think it makes the folder less cluttered and files are easier to find in my IDE. But I think this is just a matter of taste.
Either method is valid. It depends on the coding style you want to follow how you do this.
I place include (header) and source files in the same directory (folder). I create different folders for different themes. I get frustrated when trying to find header files (while debugging and also for researching). In some shops, there are only two folders: source and includes. These directories tend to grow exponentially. Reusing code becomes a nightmare at best.
IMHO, I believe organizing by theme is better. Each theme folder should build into at least one library. Different projects can easily include the themes by searching or including the folders. The projects only need to include the libraries. Smart build engines can list the theme folders as dependencies. This speeds up the build process.
The theme organization also adds a bit of safety to the project. Accidental damage to files (such as removing the wrong ones or replacing with different versions) is reduced since files are located in different directories. Deletion of files in the "Person" folder will not affect files in the "Shape" folder.
This is just my opinion, Your Mileage May Vary.
We have a build system which use this rule. This build system is sconspiracy a set of scripts to configure SCons and dedicated to the C++ world. You can see an example which use this tools : fw4spl
Currently, I program in Java and use Maven quite a bit. As so I've become accustom to the naming schemes and folder structures that I've used over the past 4 or 5 years.
As I have recently started to learn C++, I'm realizing that I have no idea where to put all my files. Should I keep everything broken down by namespace, or by what tier it is in? Where, for example, would I keep a series of files devoted to UI, as apposed to files meant to help store data?
Are there any standards for this sort of thing?
Clearly, there is no definitive answer to this question. I'm simply looking for a good guide. I do not want to start learning C++ by spending too much time worrying about how my files are laid out. I'd rather have some good models, and just get to the coding.
The following is fairly typical...
third-party library
release
obj
debug
obj
include
src
sublib 1
sublib 2
mylibrary
release
obj
debug
obj
include
src
sublib 1
sublib 2
myapp
release
obj
debug
obj
subapp 1
subapp 2
mylittleapp
release
obj
debug
obj
Basically, subfolders for subprojects is common for larger projects, but mostly a particular project has folders for src, include etc. A folder for each build configuration is common, and keeping the obj files and other intermediates in a subfolder of that is a good idea. It may be tempting to put subproject folders in obj folders, but usually that's unnecessary - the obj folders don't need to be well organised, so the only concern is a filename conflict, and the best fix for that is to have unique source filenames within (at least) each project.
The "include" folders should IMO only contain headers that will be #included by other projects - internal headers belong in the "src" folder.
Putting UI stuff in a separate folder isn't a bad idea, if it's big enough. I've seen UI stuff done as a separate static-linked top-level project, and I do mean app-specific here, not (e.g.) wxWidgets. Usually, though, that level of division is sub-project if it's worth separating at all. How you divide subprojects is more a matter of application-specific blocks in general, so it depends on whether UI stuff is best handled as a separate block or as separate chunks mixed in with task-specific logic.
Namespaces aren't the most used language feature, possibly because a lot of people use "using" so much they don't make much difference. A namespace for a main library project makes sense, but associating subfolders to namespaces 1:1 isn't something I've seen. I personally have a namespace that encompasses most of my library code, with a couple of sub-namespaces for things rarely used in general, but used a lot in a few places (e.g. a "bitwise" namespaces). The sub-namespaces are limited to single source/header pairs, so no need for subfolders. Most of the library-specific selection is done by including the right header - except that I usually include the lot through a main-project top-level header anyway.
Basically, namespaces are a way of avoiding naming conflicts. They don't necessarily associate with abstractions or functional blocks or anything. Within a particular project, you're probably better off just making sure the names don't conflict. As with the "std" namespace, it's fine to put a lot of stuff in one namespace.
As you say, though, this isn't a definitive answer - there are of course minor variations and quite different approaches.
On small projects my team groups all the files together by a link unit ie library, DLL, EXE. If the unit is very large we will sometimes breakup the files by functional unit or subsystem so that if you need to edit a component they are generally in the same place.
I break my projects by theme, one directory for theme:
menu_planner
src
recipes
debug -- contains debug object files and libraries
release -- contains release object files and libraries
obsolete -- contains obsolete source files
ingredients
debug -- contains debug object files and libraries
release -- contains release object files and libraries
obsolete -- contains obsolete source files
references
debug -- contains debug object files and libraries
release -- contains release object files and libraries
obsolete -- contains obsolete source files
meals
debug -- contains debug object files and libraries
release -- contains release object files and libraries
obsolete -- contains obsolete source files
menus
debug -- contains debug object files and libraries
release -- contains release object files and libraries
obsolete -- contains obsolete source files
docs
designs
My experience with C and C++ has shown to me that header and source files should be in the same directory. Often, finding a header file is more difficult when it is not in the same directory as the source file.
One directory (folder) per concept is a good idea. Any concept that is complex or compound should be split into multiple folders or concepts.
I've also learned to make libraries. I use libraries to contain code that doesn't change much. The linking step performs faster with libraries than directories of object files.
However, work places (a.k.a. shops) may have different style rules that must be followed.
It is not necessary to have your header files and cpp files in the same folder. I have done this many times. You can have the in different folders and use another file to fetch/include both files on file, which you will use as your include.
Let's say you have several bespoke C++ projects in separate repositories or top-level directories in the same repository. Maybe 10 are library projects for stuff like graphics, database, maths, etc and 2 are actual applications using those libraries.
What's the best way to organise those 2 application projects to have the .libs they need?
Each lib project builds the .lib in its own directory, developers have to copy these across to the application area manually and make sure to get the right version
Application projects expect lib projects to be in particular paths and look for .libs inside those locations
A common /libs directory is used by all projects
Something else
This is focused on C++, but I think it's pretty similar with other languages, for instance organising JARs in a Java project.
I'd suggest this approach:
Organise your code in a root folder. Let's call it code.
Now put your projects and libraries as subfolders (e.g. Projects and Libraries).
Build your libraries as normal and add a post-build step that copies the resulting headers and .lib files into a set of shared folders. For example, Libraries\include and Libraries\lib. It's a good idea to use subfolders or a naming convention (myLib.lib, myLib_d.lib) to differentiate different builds (e.g. debug and release) so that any lib reference explicitly targets a single file that can never be mixed up. It sucks when you accidentally link against the wrong variant of a lib!
You can also copy third-party libraries that you use into these folders as well.
Note: To keep them organised, include your files with #include "Math\Utils.h" rather than just "Utils.h". And put the headers for the whole Math library into include\Math, rather than dropping them all in the root of the include folder. This way you can have many libraries without name clashes. It also lets you have different versions of libraries (e.g. Photoshop 7, Photoshop 8) which allows you to multi-target your code at different runtime environments.
Then set up your projects to reference the libraries in one of two ways:
1) Tell your IDE/compiler where the libs are using its global lib/include paths. This means you set up the IDE once on each PC and never have to specify where the libs are for any projects.
2) Or, set each project to reference the libs with its own lib/include paths. This gives you more flexibility and avoids the need to set up every PC, but means you have to set the same paths in every new project.
(Which is best depends on the number of projects versus the number of developer PCs)
And the most important part: When you reference the includes/libs, use relative paths. e.g. from Projects\WebApp\WebApp.proj, use "..\..\Libraries\include" rather than "C:\Code\Libraries\Include". This will allow other developers and your buildserver to have the source code elsewhere (D:\MyWork instead of C:\Code) for convenience. If you don't do this, it'll bite you one day when you find a developer without enough disk space on C:\ or if you want to branch your source control.