Is it normal to generate the list of source files (.c, .cpp etc, not headers) automatically?
How would you do it? I am thinking to use find and sed.
EDIT: The project is a library. All source files are inside the project directory. No unrelated source files are there. All object files are generated using the same compiler options. I am thinking to generate the list of all source files and then a dependency file for each source file following Tromey's way. It is a viable approach?
MORE EDIT: It is a fairly large C++ library. The project is being developed. Minimising recompilation is highly desired.
Thanks.
With GNU make you can use wildcards. See this question for an example.
Normal? It is common, but not wise.
Many people use Make wildcards or find or something similar to generate a list of all the source files that exist in a certain directory tree, then feed them to the compiler and link the objects together. This is a brittle solution that will get you into trouble. If a conflict appears among the source files (e.g. two separate definitions of void foo()) the linker will complain and it may not be obvious how to fix the problem. You may find yourself with a forest of source files, many of them unnecessary to your project, slowing down your builds and causing conflicts. And if you want to make use of some of these sources (but not all) in another executable, you'll have to resort to symbolic links or some other kludgery.
A better approach is to specify in the makefile which objects are necessary to a given target, then let Make figure out which sources to use. This is what Make is good at. There is no reliable way to maintain the object lists automatically, you just have to do it by hand, but it's not that much work; if you're changing them often enough that this is a real chore, then you're doing something wrong.
EDIT:
If the project is a library as you describe, then yes, this is a viable method, and a pretty good one. And Tromey's method will work quite nicely to prevent unnecessary recompilation.
Related
I'm working with a very old and large VC6++ project and it's all messed up. There are unused files and folders everywhere, copies of folders and it's just a mess to clean it up by hand in its current state.
It will be done eventually, but is there any simple way to check what files and folders are used when it does a clean compile?
The project settings doesnt help me at all because it simply uses copies of folders and additional include directories.
Any suggestions?
Well, if you want to parse the compiler output you can get which files are actually used. I also find this when googling around, you might want to try (I haven't tried it myself). My way would be to clean the build, list all source files, build, and for each source find its corresponding .obj. The ones without .obj are not used. Note that this only works for source files, unused header files stay undetected.
VC6 will produce a makefile for you:
http://msdn.microsoft.com/en-us/library/aa233950%28v=vs.60%29.aspx
You can use the generated makefile (and the associated .dep file) as a starting point and edit it down to the list of files that get used in a build.
This will let you see the header files that the project depends on in addition to the .c/.cpp/.lib files that might show in the build log. One thing to keep in mind is that you'll probably also want to make sure you track the .dsw and .dsp workspace and project files.
If you're a bit adventurous, you might be able to convince the makefile to actually copy the source files to some other location for you with an appropriate override of the certain macros and/or dependencies. But that would probably be more trouble than it's worth for a one-time effort.
Finally, there's a commercial product, CopyWiz by Kinook Software, that seems to have features that might do what you're looking for (and it supports VC++ 6). Note: I'm not sure if it will do what you want, but it may be worth a look.
Yes. Run Process Monitor from SysInternals. It can capture all file system events and filter them based on the path and other factors.
So, set the filter to the root of your source tree, only succesfull file reads (VC looks for headers in many places), and build your project. You'll probably still see several thousand events. So, save them to file, sort by path, and remove duplicate paths (headers especially will have many duplicate entries)
This link mentions wildcards as a way to automatically list the SOURCES and HEADERS in the Makefile.am file. It also mentions that some people write external scripts to generate those files.
Do you know of the standard way of automatically including all the *.h *.cpp here, or should I just write my own Perl script to generate them. Do you have such a script already that you use?
PS: I organize the source files in my project according to the following purely-logical separation of directories:
src/dog/woof.h
src/dog/woof.cpp
src/cow/moo.h
src/cow/moo.cpp
Automake won't add this feature. It makes assumptions that a particular .h or .cpp file is associated with a particular project. That assumption holds for a number of common project layouts and fails for any layout that differs.
For example, I've had projects that were laid out as
src/module/code
src/app/code
src/library/code
include/headers
built from one central makefile in the root. Other times, I've had the same layout built from four makefiles in the appropriate local directories.
There's a lot of variability in projects. Some keep public header files mixed with private headers files in the code directories, some keep them separate. Some build shared object libraries, some don't. Some ship code that's not SUPPOSED to compile on incompatible platforms.
To put in a wild card inclusion would actually pose a great risk of limiting the functionality, and for those odd people who do things like 'file.template.c' and such it would be fatal.
If you consider it a flaw of automake, that's fine; however, it's one of those flaws that automake embraces as it's preserved in the effort to make things more flexible. Automake doesn't impose the "how" you do things, it provides a lot of enabling tools, but it goes out of it's way to ensure that you aren't forced into one "method" of laying out or building your code.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 3 years ago.
Improve this question
This questions concerns mostly Unix/Linux style C++ development. I see that many C++ libraries store their header files in a "include" folder and source files in an "src" folder. For the sake of conformance I adopted this in my own code. But it is not clear to me whether this should be done for application code as well. I've seen a few cases where a flat directory structure is used for that. What would be the recommended approach?
I also separate them, but not strictly on the extension, but on the access of the file.
Suppose you have a module that manages customer information and uses 2 classes to do this: Customer, CustomerValidityChecker.
Also suppose that other parts in your application only need to know about the Customer class, and that the CustomerValidityChecker is only used by the Customer class to perform some checking.
Based on these assumptions I store the files like this:
Public folder (or include folder):
customer.h
Private folder (or source folder):
customer.cpp
customervaliditychecker.h
customervaliditychecker.cpp
That way, it becomes immediately clear for callers of your module which parts are accessible (public) and which parts aren't.
We have a build system that auto-generates our makefiles. One thing it does is recursively descend any subdirectories and build them as libraries, linking them together with the main directory's objects to make the application. (In practice, these "subdirectories" are usually symbolic links.) Libraries are static unless the directory name ends in ".so". One thing that's nice about this is that a full build of our system, which has many executables, doesn't have to repeatedly compile the common libraries.
However, as a result of this, there's no separation of headers and sources. And it has never been a problem. Honestly, I think it's better this way because headers and source files have commonality of location, and you can grab a directory and know you got everything you need to use it. It also works great with Subversion's "externals" feature, and similar features in other VCSs.
One last place where an include/src separation fails is if you use any code generators, such as flex, bison, or gengetopts. Figuring out where these tools should put their outputs so they get built is tricky if you've spread things out.
It makes sense to separate them for shared libraries because they may be distributed in a compiled form without the source. I've seen projects that separate out "public" headers (headers that may be accessed from code outside your project or library) while leaving "private" headers and source files in the same directory. I think it's good to use a consistent approach whether you're writing shared library or application level code because you never know when you may want to turn something that you've written at the application level into a lower level library that is shared by multiple projects.
A lot depends on the size of project involved. Up to a few dozen files or so, keeping them in one directory tends to be more convenient. For a bigger application that includes hundreds or thousands of files, you start to look for ways to separate them (though in the projects I've worked on, it was done more on functional lines than src/include). In between those, it's probably open to question.
I don't do this; there seems little advantage in it. Since headers tend to have a different extension from source files, you can have your editor show them separately if you really feel the need -- Visual Studio does this by default, but I disable it since I prefer seeing them together
Bottom Line: sources and headers that are still changing go in /src. Code that has crystallised should go in /lib & /include (actually you could keep all .libs and their .hs in /lib).
Keep own sources and headers together, provided they are (a) specific to this project or (b) have not yet been factored out as a shared library.
Once certain sources in the main project have been factored out as a (relatively stable) library, place the .a or .lib into /lib, and its public interface header into /include.
All third party libraries and their public interface headers also go into /lib & /include.
As others note, it is often more compatible for tools / IDEs to access .h/.c from one folder. But from an organisational view it can be useful to separate changing local code from stable lib code.
There is no clear advantage to either in my view. I finally decided to keep program and header files together because my editor (Visual SlickEdit) happens to provide additional referential features when they are not separated.
I almost always create include and src folders to split up my source code. I think it makes the folder less cluttered and files are easier to find in my IDE. But I think this is just a matter of taste.
Either method is valid. It depends on the coding style you want to follow how you do this.
I place include (header) and source files in the same directory (folder). I create different folders for different themes. I get frustrated when trying to find header files (while debugging and also for researching). In some shops, there are only two folders: source and includes. These directories tend to grow exponentially. Reusing code becomes a nightmare at best.
IMHO, I believe organizing by theme is better. Each theme folder should build into at least one library. Different projects can easily include the themes by searching or including the folders. The projects only need to include the libraries. Smart build engines can list the theme folders as dependencies. This speeds up the build process.
The theme organization also adds a bit of safety to the project. Accidental damage to files (such as removing the wrong ones or replacing with different versions) is reduced since files are located in different directories. Deletion of files in the "Person" folder will not affect files in the "Shape" folder.
This is just my opinion, Your Mileage May Vary.
We have a build system which use this rule. This build system is sconspiracy a set of scripts to configure SCons and dedicated to the C++ world. You can see an example which use this tools : fw4spl
I have a very large project with tons of convoluted header files that all include each other. There's also a massive number of third-party libraries that it depends on. I'm trying to straighten out the mess, but I'm having some trouble, since a lot of the time I'll remove one #include directive only to find that the stuff it was including is still included through one of the other files. Is there any tool that can help me understand this? I'd really like to be able to click on a .h file and ask it which CPP files it's included in (directly or indirectly), and the paths through which it is included, and likewise click a cpp file and ask it which .h files are included (directly and indirectly). I've never heard of a tool that does this, and a bit of quick googling hasn't turned anything up, but maybe I don't know what to search for.
http://www.profactor.co.uk/includemanager.php
For VS2003 there is /showIncludes flag (in C/C++/Advanced properties). This will print all headers each .cpp file includes and what they include, so you can go from there.
I'm sure there is same option in same place for VS2008.
if you use GCC compilers, try this
g++ -M abc.cpp
it will show all include dependencies for the file abc.cpp
Your situation reminds me of my own. I have a bunch of headers that I have created that I use as a library instead of bothering with a DLL.
Of course the cyclic-includes can become troublesome, so I find that a tool like Visual Assist X (1) helps with this sort of thing. It has a function that can find references to stuff, so that you can easily weed out where something is being defined/declared/included etc. It also has a lot of other useful features, so I consider it to be pretty useful.
There’s probably other tools/plugins that have a referencing function, but usually as one feature among the other refactoring and productivity functions of the utility.
HTH
It's pretty tedious, but you can binary-search your way to where an #include happens by using #error (and #pragma message) to narrow down which include line is pulling in the third party. I've done this in the case of a single file I was trying to track down, but it sounds like your problem is bigger so probably one of the tools others have mentioned would be more effective.
We are migrating our works repository so I want to do a cull of all the unreferenced files that exist in the source tree before moving it into the nice fresh (empty) repository.
So far I have gone through by hand and found all the unreferenced files that I know about but I want to find out if I have caught them all. One way would be to manually move the project file by file to a new folder and see what sticks when compiling. That will take all week, so I need an automated tool.
What do people suggest?
Clarifications:
1) It is C++.
2) The files are mixed. I am looking for files that have been superseded by others but have left to rot in the repository - for instance file_iter.h is not referenced by any other file in the program but remains in the repository just in case someone wants to compile a version from 1996! Now we are moving to a fresh repository we can safely junk all the files that are no longer used.
3) Lint only finds unused includes - not unused files (I have the 7.5 manual in front of me).
You've tagged this post with c++, so I'm assuming that's the language in question. If that's the only thing that's in the repository then it shouldn't be too hard to grep all files in the repository for each filename to give you a good starting point. If the repository contains other files (metadata, support files, resources, etc) then you're probably going to need to do it manually.
I can't offer an existing tool for it, but I would expect that you can get a lot of this information from you build tools (with some effort, probably). Typically you can at least let the build tool print the commands it would run, without actually running them. (E.g. the -n option of make and bjam does this.) From it you should be able to extract at least the used source files.
With the -MM of g++ you can get all the non-system header files for the given source files. The output is in the form of a make rule, but with some filtering this shouldn't be a problem.
I don't know if this helps; it's just what I would try in your situation.
You can actually do this indirectly with Lint by running a "whole project analysis" (in which all files are analysed together rather than individually).
Configure it to ignore everything but unreferenced variable/enum/function etc warnings and it should give you a reasonable indicator of where the deadwood lies without those issues being obscured by any others in the codebase.
A static source code analysis tool like lint might do the job. They will tell you if a piece of code will never be called.
Have you taken a look at Source-Navigator? It can be used as an IDE but I found to be very good at analyzing source code structure. For example, it can find out where and if a certain method is used in your source code.
I don't know if it's scriptable but it might be a good starting point for you.