Firstly, this is old code that I inherited and I would love to clean it up and make it perfect someday, but I cannot do that today. Therefore, I ask that answers like "rewrite the whole thing so that there are no circular dependencies" not be given.
I have a solution with 44 projects. Most of the projects build DLLs and a few build executables. There are multiple projects that depend on another project that, in one way or another, requires the first. Some are as simple as A requires B.lib and B requires A.lib and others are longer chains that loop back around.
When I inherited the code, it was not clean and there were old build files still amongst the source, namely .LIB files. Without removing those .LIB files all 44 projects are able to build and all of the old .LIB files are replaced with newly built files. This tells me that if I build a second time, none of the old .LIBs are being used and that all of my deliverables are now entirely generated from current source.
This leaves me with a problem: I wanted to clean out all none source files from version control, but if I do so, then I can no longer build the projects.
To further clearify, the projects that build DLLs create a .LIB file as an intermidiate step and these files are copied into a common folder that is listed in the "Additional Library Directories" property of each project and the individual .LIB files that are needed for a give project are in the "Additional Dependencies" property for that project.
Edit: I am not looking to do a major overhaul of the code. I know that all of the object files that all of the projects need are created. I also know that this structure is not optimal. I do, however, believe that if the linker knew where to look, it could find everything that is needed.
Extract just the parts from A that B requires into a new DLL 'C'. Then make both A depend on both B & C, whereas B just depends on C.
It's not going to be easy, but just keep extracting little bits at a time.
Related
0. Disclaimer
This question is only about Visual Studio C++ project/solution configuration and may involve subjectivity.
However, the idea behind this post is to share our approaches to configure a large Visual Studio solution.
I'm not considering any tool like CMake/Premake here.
1. Problem
How do you handle a large scaled C++ application architecture and configuration using Visual Studio?
What is, for you, the best way to setup a new Visual Studio solution composed of multiple projects?
What Visual Studio project/solution configuration feature are you trying to avoid? (Ex: Filters instead of folders)
2. Personnal approach
2.1. Context
I'm a software developer for a video game company, so I will take a very simplified game engine architecture to illustrate my words:
2.2. File Structure
My Visual Studio solution would probably look something like this:
Where Application is an executable and every other projects are libraries (Dynamically linked).
My approach would be to separate each project into two folders: include, src
And the inner structure would be separated into folders following my namespaces:
2.3. Project Configuration
The following lines will assume there is only one $(Configuration) and $(Platform) available (Ex: Release-x64) and that referencing .lib files into Linker/Input/Additional Dependencies is done for each project.
I would here define a Bin (Output), Bin-Int (Intermediate output) and Build (Organized output) folder, let's say they are located in $(SolutionDir):
$(SolutionDir)Bin\
$(SolutionDir)Bin-Int\
$(SolutionDir)Build\
The Bin and Bin-Int folders are playgrounds for the compiler, while the Build folder is populated by each project post-build event:
$(SolutionDir)Build\$(ProjectName)\include\ (Project includes)
$(SolutionDir)Build\$(ProjectName)\lib\ (.lib files)
$(SolutionDir)Build\$(ProjectName)\bin\ (.dll files)
This way, each $(SolutionDir)Build\$(ProjectName)\ can be shared as an independent library.
Note: Following explainations may skip $(SolutionDir) from the Build folder path to simplify reading.
If B is dependent of A, Build\B\include\ will contain B and A includes. The same way, Build\B\bin\ will contain B and A binaries and Build\B\lib\ will contain B and A .lib files (If and only if B is ok to expose A to its user, otherwise, only B .lib files will be added to Build\B\lib\).
Projects reference themselves relatively to Build\ folders. Thus, if B is dependent of A, B include path will reference $(SolutionDir)Build\A\include\ (And not $(SolutionDir)A\include\), so any include used by A will be available for B without specifying it explicitly. (But result to sections 2.6.2., 2.6.3. and 2.6.4. technical limitations).
After that, I make sure that my solution has a proper Project Dependencies configuration so the Build Order, when building the whole solution, will consider and respect my project dependencies.
2.4. User Project Configuration
Our EngineSDK user (Working onto Application) will only have to setup Application such as:
Include Directories: $(SolutionDir)Build\Engine\include\
Library Directory: $(SolutionDir)Build\Engine\lib\
Post-build: Copy $(SolutionDir)Build\Engine\bin\* to $(OutDir)
Additional Dependencies: Any .lib file upstream in the dependency hierarchy is listed here
This is the typical Visual Studio configuration flow of a lot of C++ library.
Common library folder architecture that I try to preserve:
lib\
include\
bin\
Here are some example of libraries using this folder architecture model (Do note that bin is exclusively for dynamically linked libraries as statically linked libraries don't bother with DLLs):
SFML: https://www.sfml-dev.org/
SDL: https://www.libsdl.org/
2.5. Advantages
Clear folder architecture
Ability to export a library directly by copy-pasting or zipping a sub-folder of $(SolutionDir)Build\
2.6. Technical limitations
The approach I wrote here has some drawbacks. These limitations are the reason of this post as I want to improve myself:
2.6.1. Tedious configuration
Dealing with 10 or less projects is fine, however, with bigger solutions (15+ projects), it can quickly become a mess. Project configurations are very rigid, and a small change in project architecture can result into hours of project configuration and debugging.
2.6.2. Post-build limitation
Let's consider a simple dependency case:
C is dependent of B and B is dependent of A.
C is an executable, and B and A are libraries
B and A post-build events update their Build\$(ProjectName)\ directory
When changing the source code of A, then compiling it, Build\A\ will get updated. However, as B has been previously compiled (Before A changes), its Build\B\ folder contains a copy of previous A binaries and includes. Thus, executing C (Which is only aware of B as a dependency), will use old A binaries/includes. A workaround I found for this problem is to manually trigger B post-build event before executing C. However, forgetting to trigger an intermediate project post-build can result into headaches during debugging (Not loaded symbols, wrong behaviour...).
2.6.3. Multiple times single header reference
Another limitation for this approach is "Multiple times single header reference".
This problem can be explained by considering the project dependency image at section 2.1..
Considering that Graphics and Physics are both including Maths headers, and that Engine is including Build\Graphics\include\ and Build\Physics\include\, typing a header name will show multiple identical results:
2.6.4. De-synchronized symbol referencing
If B is dependent of A and any header changes in A (for instance, we add a new function), Rescan File/Rescan Solution will be needed to access the new symbol from B.
Also, navigating to files or symbol can make us move to the wrong header (Copied header instead of the original one).
3. Interrogations and learning perspectives
3.1. Project Reference
During my Visual Studio software developer journey, I came through the project Reference concept, but I can't find how it can solve the technical limitations of my current approach, nor how it can helps me to re-think it.
3.2. Property sheets
As every project configuration of my solution is following the same principle but the content (Include dirs, library dirs...) for each one is different, I'm not sure how to make a great usage of property sheets.
3.3. Exploring GitHub
Currently I'm struggling finding some good project architecture references. I would be pleased finding some Visual Studio configured solution on GitHub or any code sharing platform. (I know that CMake and Premake are prefered in most case when sharing code, however, learning more about Visual Studio project configuration is my actual goal).
Thanks for reading my words, I hope that you are also interested into discussing about this subject and maybe we can share our approaches.
I'm working at an organization with a product suite based on several hundred Visual Studio solutions (mostly C++). Some of these solutions generate libraries that are used by other solutions and there's also a common "include" folder containing headers that shared by multiple modules.
The issue is that the dependencies are not explicitly stated anywhere, and the build system resolves dependencies by specifying a linear build order that makes sure the dependent modules get built at the right time. This works well for the build system but leaves developers at a disadvantage when trying to work on components with many direct and indirect external dependencies. For example, I might want to edit one of the library projects or shared headers and then build all the affected modules without necessarily knowing ahead of time which ones are affected. Another use case involves building a module after doing a fresh pull from TFS and having the modules it depends on built first without having to build the entire system.
I am wondering if there is/are any tool(s) available that can automate dependency generation for building large projects. I have considered creating a few really big solutions that encapsulate the other solutions but that seems really awkward and clumsy. Also, I don't like the idea of having developers manually specify dependencies as it can error prone, especially with such a large code base. I worked with scons a few years ago and really liked the way it could parse source files and automatically discover all the dependencies dependencies. Is there anything available today that can do the same thing with Visual Studio solutions?
This is not a duplicate of Visual Studio: how to handle project dependencies right?
I need to emphasize the magnitude of the problem I am trying to solve. This is a very large existing code base. In the main directory there are several hundred sub-folders, each one containing one of more VS solutions (not projects). Each solution, in turn, contains one or more projects. As I said before, I'm not trying to establish dependencies among a few projects in a solution. The problem is much bigger than that. I'm trying to find a way to establish dependencies among the solutions themselves (several hundred of them). For example, one solution may contain some projects that generate libraries for security, others for communications, etc. There may be, for example, dozens of solutions that use the communications libraries. So essentially I'm trying to create a directed a cyclic graph with hundreds of nodes and potentially tens of thousands of edges.
You could use cmake (https://cmake.org/). With it, you can specify several libraries and apps to be built. Once configured, you can modify a project and the build will just update the dependent projects. Cmake also provides a visual studio generator, so that you can continue using that IDE.
A possible disavantage to you is that, to configure, you must explictly specify, for each project (library or executable), with what projects it must be linked and what folders it must include. There are ways to define some global includes and links, but the use will depends on your problem.
VS does track dependencies (by parsing source files). It doesn't make sense that something could automatically set dependencies of your VS projects, in any other build tools you'd still have to specify in some way that for linking project A.exe you need to use B.lib.
If you use newer VS versions you should simply add references to lib to your exe/dll projects. If you manually added project dependencies, most likely you should remove them all, especially make sure you don't make static lib projects dependent on each other. VS allows you to do that (for example, if build of one library generates some source files that another static lib uses), but in general these shouldn't have any dependencies and this allows VS to optimize builds by building them in parallel.
For example, commonly you could have some kind of Base.lib, then System.lib and Graphics.lib. All of these are user by your App.exe. System.lib uses code from Base.lib, Graphics.lib uses code from System.lib and Base.lib. So, naturally the dependency chain is clear and you go and set them in VS, and that's a mistake! In cases like this in VS you should make these 4 libs independent and only App.exe should be dependent on all these libs (e.g. it should have references to all of these). VS will figure out what is the the correct dependency of these projects.
Regarding Cmake case: it simply generates VS projects and solutions, if you use VS then cmake cannot do more than VS itself can.
I am looking for detailed steps to create a second Static Lib Project in Visual Studio 2010
that my first project will reference.
This project will be in source control and used by others so the referencing needs to be able to work on all folder structures. (if possible)
I have done this before but have had problems recently. I mostly end up adding random references to everything and every folder in my project until it works as I do not know the correct steps to accomplish it.
This will be my projects folder structure
<Whatever Structure>/MyProject/MainProject
<Whatever Structure>/MyProject/SecondProject
<Whatever Structure>/MyProject/MyProject.sln
I need my SecondProject to be built as a Static Lib library.
Inside my FirstProject I would like to reference files from my SecondProject as
#include <SecondProject/<filename or class or namespace>
As I said above Detailed Steps to accomplish this would be greatly appreciated.
I have searched many other posts but most just pertain to Header Files or they are half the steps.
Thank you.
#include is solely used for headers. This is parsed at compile time. Since you want to use headers from <Whatever>/MyProject/SecondProject as just SecondProject/, obviously <Whatever>/MyProject/ must be among the include directories. Probably the best way to specify it would be as just ../, because that means you don't have to hardcode <Whatever>
After compiling, the next step is linking. The easiest solution here is to go the the property pages of MainProject, Common Properties > Frameworks & References, and use [Add New Reference...] button. Linking will make the compiled functions inside the .lib available.
I'm writing a program (in C++), which requires several VS projects, that I would like to put in the same VS solution. I'm using Visual Studio 2010.
Here is simply the wanted architecture : I'm using a 3rd party library (A) for my project, I have all the headers and .lib files, that I compiled with the source code.
With this library, I'm writing my own classes and function. That is my project (B).
Then I would like to develop two interfaces for the users: A command line interface (C1) and a GUI interface (C2), that are using the classes and functions defined in (B).
A <-- B <-- C1
<-- C2
I'm new to Visual Studio, and I don't know how to handle these dependencies properly.
Shall I use project dependencies (in the solution properties) or references (in the project properties) ? In fact, I'm not sure what dependencies and references are doing exactly.
Shall I compile B into some .lib library, or do something else ? If I do so, have to link only B.lib to my C1 and C2 projects, or should I also link A.lib (in other words, is the content of A.lib included somehow in B.lib ?).
And of course I would want the dependencies to be handle well, in order to always work with the up-to-date version of each project.
Is there a good way of doing it ?
Thank's in advance, and have a nice week-end :)
Yes. Use Project References.
Here is the official answer from Microsoft. While the page talks about .NET, its almost the same for native projects too.
TL;DR version:
Advantages of Project References:
They work on all development workstations where the solution and project set are loaded. This is because a project Globally Unique Identifier (GUID) is placed in the project file, which uniquely identifies the referenced project in the context of the current solution.
They enable the Visual Studio .NET build system to track project dependencies and determine the correct project build orders.
They avoid the potential for referenced assemblies to be missing on a particular computer.
They automatically track project configuration changes. For example, when you build using a debug configuration, any project references refer to debug assemblies generated by the referenced projects, while they refer to release assemblies in a release configuration. This means that you can automatically switch from debug to release builds across projects without having to reset references.
They enable Visual Studio .NET to detect and prevent circular dependencies.
Here is another nice article on Project Settings Changes in VS 2010 that also states that references are to be preferred over Project Dependencies. In fact, the article also says that the VS2010 solution converter automatically detects Project dependencies and changes them to Project references.
The policy at the company I work for is to use project references.
The project references are more useful since they keep the information of what projects a given project depends on with the project. If you then have to add the project to a new solution you do not have to go back to the old solution file to find out what projects a given project depends on.
Okay so i am trying to convert some projects into DLL's. These are all visual studio 2008 projects, and are all written in C++.
We have 9 different projects(all under 1 solution in Visual Studio 2008). E.g.:
RasterTools\
RasterDataset.h
RasterDataset.cpp
Point.h
Point.cpp
<... insert like 50 more cpp files ...>
CoreTools\
Image.h
Image.cpp
CoreUtilities.h
CoreUtilities.cpp
<... insert more ...>
<... insert 7 or more projects ...>
Now the thing is certain projects use other project's classes. For example Image.h uses Point.h, and CoreUtilities.h has some functions for dealing with Points. What is the proper way to build a DLL when you have multiple projects that are all intertwined.
Basically we want to have the following:
CoreTools.dll
RasterTools.dll
<other projects.dll>
I understand how to handle the __declspec(dllexport) and dllimport fine, through macros. But when i attempt to build for example the CoreTools.dll i get linker errors where ever i use a class or functionality not contained within that project. For example when building coreTools.dll i get linker errors for any of the Point Class functions.
How can i deal with this?
EDIT
I tried this, and basically i noticed that almost all of the checkboxes were grayed out since its results in circular dependencies. :(
If you right-click on the project which throw ups the linker errors and choose 'Project Dependencies...' you'll see a list of all other projects in your solution.
Check the boxes next to the (DLL) projects you need to link with and Visual Studio will automatically link to it (and ensure the dependencies are built before the clients).
For each DLL you build you should also build a LIB. If Project B (B.DLL) references entries in Project A (A.DLL), then Project A is built first, it creates A.DLL and A.LIB, then Project B can add A.LIB to optional link libraries to resolve its references. Then C.DLL that depends on B.DLL can include in the link step B.LIB and so on and so forth.
At least this is what happens at the very low, build/make level. The dev environemnt may do all these under the covers for you if you choose the project dependencies right, as leegent suggests.
If you have circular dependencies between your DLLs, you'll have to resolve them by moving stuff around so that you get layered DLLs, with each DLL at a given level only using code from levels below.