When creating new projects I've always stumbled upon the issue that my final executable or dll does not properly run because it is missing dependencies I was using.
In my live as a developer I've seen several approaches to handle that (of which I don't like any)
Set environment path to all those dependencies so that the OS can find them (bad, since very unportable, requires environment, won't work out of the box)
Copy every dependency of a potentially large package into the binary output directory (e.g. dozens of dlls regardless of need). For dependencies like OSG, Qt, etc this is quite odd as you typically won't link all dlls provided by larger packages and you may end up copying much more data than necessary.
Hand-Pick single dependencies (and optionally their pdbs) with a fine-grained xcopy/robocopy or whatever task. (Don't like that as it needs attention, I add a dependency in Visual Studio and then I need to adjust some script). There are tools like dependency walker which help this, but still, this might still be unportable because your include path in VS may have a version in it, if you change that, you need to change it also in the script, which is too much Department of Redundancy Department
In larger projects or companies I would expect this issue to be quite common. Is their nothing in Visual Studio or in Windows that aids that? I need to pick my include paths and import lib paths properly in order to compile a binary but then I need some other mechanism to actually make things work.
I am thinking about solving this problem in a more general way for more projects but really wonder if I am missing something.
Related
Question:
Once my code is working how should I prepare my files so that a stranger on a different computer can compile it without difficulty?
Additional Details:
I am sending a code sample to a company as part of an application so obviously an elegant solution would be better (i.e. minimise number of files required etc) and no work should be necessary by the stranger at the other end.
Although I am only using one simple library, even so I need to set include directories, include lib files, images, dll files etc so that it all compiles correctly.
If it matters, I am using Visual Studio 2015 and the simple library is SDL.
Sorry if this is a duplicate, I was sure that this question would have been asked before but if it exists I just don't know the correct terminology to find it amongst the noise.
Apologies if this is overly simplistic, but you might want to bound the scope of your project by deciding which computers you want to support, and build your code yourself on those platforms, in advance, just to be sure.
List the supported platforms in your release notes, including any platform-specific instructions or information (which VC++ versions, which C++ versions, which OS versions, which DLLs, directory structure, etc.).
You may have to stick some "#ifdef"s and such in your code, but only by building on a particular platform/configuration will you really know for sure.
You can use properties/props files in your VS solution which sets the paths to includes and precompiled libs, then reference the build variables in your project files.
To compile on another machine, you just need to change the values in the properties files.
I'm working at an organization with a product suite based on several hundred Visual Studio solutions (mostly C++). Some of these solutions generate libraries that are used by other solutions and there's also a common "include" folder containing headers that shared by multiple modules.
The issue is that the dependencies are not explicitly stated anywhere, and the build system resolves dependencies by specifying a linear build order that makes sure the dependent modules get built at the right time. This works well for the build system but leaves developers at a disadvantage when trying to work on components with many direct and indirect external dependencies. For example, I might want to edit one of the library projects or shared headers and then build all the affected modules without necessarily knowing ahead of time which ones are affected. Another use case involves building a module after doing a fresh pull from TFS and having the modules it depends on built first without having to build the entire system.
I am wondering if there is/are any tool(s) available that can automate dependency generation for building large projects. I have considered creating a few really big solutions that encapsulate the other solutions but that seems really awkward and clumsy. Also, I don't like the idea of having developers manually specify dependencies as it can error prone, especially with such a large code base. I worked with scons a few years ago and really liked the way it could parse source files and automatically discover all the dependencies dependencies. Is there anything available today that can do the same thing with Visual Studio solutions?
This is not a duplicate of Visual Studio: how to handle project dependencies right?
I need to emphasize the magnitude of the problem I am trying to solve. This is a very large existing code base. In the main directory there are several hundred sub-folders, each one containing one of more VS solutions (not projects). Each solution, in turn, contains one or more projects. As I said before, I'm not trying to establish dependencies among a few projects in a solution. The problem is much bigger than that. I'm trying to find a way to establish dependencies among the solutions themselves (several hundred of them). For example, one solution may contain some projects that generate libraries for security, others for communications, etc. There may be, for example, dozens of solutions that use the communications libraries. So essentially I'm trying to create a directed a cyclic graph with hundreds of nodes and potentially tens of thousands of edges.
You could use cmake (https://cmake.org/). With it, you can specify several libraries and apps to be built. Once configured, you can modify a project and the build will just update the dependent projects. Cmake also provides a visual studio generator, so that you can continue using that IDE.
A possible disavantage to you is that, to configure, you must explictly specify, for each project (library or executable), with what projects it must be linked and what folders it must include. There are ways to define some global includes and links, but the use will depends on your problem.
VS does track dependencies (by parsing source files). It doesn't make sense that something could automatically set dependencies of your VS projects, in any other build tools you'd still have to specify in some way that for linking project A.exe you need to use B.lib.
If you use newer VS versions you should simply add references to lib to your exe/dll projects. If you manually added project dependencies, most likely you should remove them all, especially make sure you don't make static lib projects dependent on each other. VS allows you to do that (for example, if build of one library generates some source files that another static lib uses), but in general these shouldn't have any dependencies and this allows VS to optimize builds by building them in parallel.
For example, commonly you could have some kind of Base.lib, then System.lib and Graphics.lib. All of these are user by your App.exe. System.lib uses code from Base.lib, Graphics.lib uses code from System.lib and Base.lib. So, naturally the dependency chain is clear and you go and set them in VS, and that's a mistake! In cases like this in VS you should make these 4 libs independent and only App.exe should be dependent on all these libs (e.g. it should have references to all of these). VS will figure out what is the the correct dependency of these projects.
Regarding Cmake case: it simply generates VS projects and solutions, if you use VS then cmake cannot do more than VS itself can.
I'm lost here and I have no clue how to proceed. This is not a question about how to make my program work, this is a question about how to stop wasting my time.
My programming environment is Visual Studio 2013 on windows, in C++.
I use 3 libraries extensively, namely: boost (using dynamic linking), OpenCV, and Qt.
During the development, I have configured VS to look at those 3 libraries by default for include and .lib. I have also added the 3 folders containing all the dlls to my PATH environment variable.
It works, but it is sometime painful, let me explain you when.
First hassle: Anytime I have a LNK error telling me I miss a function, it is usually on OpenCV since it has only one include file referencing all the functions. I have to look at OpenCV's source code to see what module this function belongs to and to know what I must link my program to.
Second Hassle: When comes the time to deploy my application, I have to ship it with all the relevant dlls. To know which one I need, I open dependency walker and try to forget nothing, I have then to test it on a different computer because 102% of the time I have missed a couple, and then I have to configure my Installer generator to include all those one by one.
Third Hassle: To ease a little bit the process of configuring a new development machine, I have recently switched to NuGet. It is great, I add boost with a couple of clicks to any project. But now my boost DLLs are everywhere, I have one folder per boost library, and since there are dozens of those I can't even add them all at once to my PATH now, so I have to move them manually to the appropriate folder, and that is really not what I want to do with my not-so-precious-but-who-are-you-to-judge time
I have looked around and couldn't find any good practice regarding this issue, maybe because they are too obvious, or too specific to a particular setup.
How do you do? How would you do if you were me?
We put all our external dependencies in version control along with the code. This ensures that all code can build "out of the box" on any of our development machines and also ensures that for any given version of the code, we know exactly which dependencies is has.
The best way to check for missing dependencies is how have a good automated test suite, if you've got comprehensive converge then if your tests pass you must have deployed the required libraries.
In terms of linking to the appropriate libraries, unfortunately, that just sounds like an issue with the structure of OpenCV (I'm not familiar with OpenCV). I tend to use dumpbin under Windows and nm under Linux to easily grep for symbols when I get link errors with an unfamiliar library.
I have created a simple application in Qt5 but, when run that application in for example windows 7, tells me the dll file something is missing, and another dll and another dll.
Now I want a package that contain all dll files that any Qt application needed.
Or, What's the Important dll files that needed to work any Qt application ?
Or, What's the Important dll files that needed to work any Qt application ?
In general, the Qt modules that you are using. If you use QtSql, you need to have the corresponding dll, but it goes the same way for any module. As for a simple core application, you would need to have the QtCore dll, respectively. I could continue the enumeration, but I believe you see the pattern how it goes.
Going even a bit further, you would need to specify your dependencies in the LIBS variable if you happen to use qmake. That is also a place where you would already need to be aware of your direct dependencies.
It would be usually the single library name, but in complex cases, you can always read the documentation of the project. The main point is the fact that you need to get aware of this without debugging. The latter would be more like an after-thought.
There are no "hidden dependencies". Since you are coding the project, you know the dependencies that you use. You will need to ship them.
There are some odd and rare cases when issues come that you would not be aware of, like missing dll for a different machine having a distinct VS/MSVC setup. In that case, it is better not to ship all the dlls for each possible end machine, but the end machine is supposed to install redistributable packages.
In rare cases when the above does not suffice, you can check the error message. If that is still unclear, you can use introspection tools for debugging the issue, like dependency walker.
Disclaimer: this answer goes as broad as the question is.
I would like some help in setting up a project in SVN with regards to directory structure. I have read several answers regarding this on SO, but as I am new to this, most of them are difficult to understand.
I am building a single library, on which several other distinct project depends on:
I need the ability to export MyLibrary (headers and .lib only) easily for use by third parties
MyLibrary1
Depends on external libraries, should be able to manage different versions of these libraries!
MyLibrary2
Depends on External Libraries fmod, glew, ...
Project 1, 2, 4, 5, 6 ...
Depends on MyLibrary1, 2, or both
Each project could need versions for multiple platforms (osx, windows ...)
I would like to know of a good way to organize this, do keep in mind that I am rather new to this - a more pedantic answer would be helpful. For example if you write something like /src, do explain what is supposed to go into it! I would be able to guess, but I wont be sure =)
////////////////////////////////////////////////////////////////////////////////////////////////////////////
// Edit
I cant put this into a comment, so here goes:
#J.N, thanks for the extensive reply, I would like to clarify some stuff, I hope I understood what you meant properly:
root
library foo
/branches // old versions of foo
/tags // releases of foo
/trunk // current version
/build // stuff required by makefiles
/tools // scripts to launch tests ect
/data // test data needed when running
/output // binaries, .exe files
/dependencies // libraries that foo needs
/lib name
include
lib
/docs // documentation
/releases // generated archives
/sample // sample project that shows how to use foo
/source // *.h, *.cpp
program bar
/branches // old versions of bar
/tags // releases of bar
/trunk // current version
/build // stuff required by makefiles
/tools // scripts to launch tests ect
/data // test data needed when running
/output // binaries, .exe files
/dependencies // libraries that bar needs
/lib name
include
lib
/docs // documentation
/releases // generated archives
/sample // sample project that shows how to use bar
/source // *.h, *.cpp
1) Where do the *.sln files go? In /build?
2) do I need to copy foo/source into bar/dependencies/foo/include? After all, bar depends on foo
3) Where do *.dll files go? If foo has dependencies on dll files, then all programs using foo need access to the same dll files. Should this go into root/dlls?
There are several levels to your questions: how to organize a single project source tree, how to maintain the different projects together, how to maintain the dependencies of those project, how to maintain different variants of each projects and how to package them.
Please keep in mind that whatever you do, your project will eventually grow large enough to make it unadapted. It's normal to change the structure several times in the lifetime of a project. You'll get the feeling that it isn't right anymore when that will happen: it's usually when the setup is bothering you more than it helps.
1 - Maintaining the different variants of each project
Don't have variants for each project, you won't solve several variants by maintaining parralel versions or branches. Have a single source tree for every project/library that can be used for all variants. Don't manage different "OSes", manage different features. That is, have variants on things like "support posix sockets" or "support UI". That means that if a new OS come along, then you just need to choose the set of features it supports rather than starting a new version.
When specific code is needed, create an interface (abstract class in C++), and implement the behaviour with respect to it. That will isolate the problematic code and will help adding new variants in the future. Use a macro to choose the proper one at compile time.
2 - Maintaining the dependencies of each project
Have a specific "dependencies" folder in which each subfolder contains everything needed for one dependency (that is includes and sub dependencies). At the beggining when the codebase is not too large, you don't care too much about ensuring automatically that all the dependencies are compatible with each other, save it for later.
Don't try to merge the dependencies from their root location higher in the svn hierarchy. Formally deliver each new version to the teams needing it, up to them to update their own part of the SVN with it.
Don't try to use several versions of the same dependency at once. That will end badly. If you really need to (but try avoiding it as much as you can), branch your project for each version.
3 - Maintain the different projects
I'd advise to maintain each projects repository independently (with SVN they still could be the same repo, but in separated folders). Branches and tags should be specific to one project, not all of them. Try to limit to a maximum the number of branches, they don't solve problems (even with git). Use branches when you have to maintain different chronoligical versions in parallel (not variants) and fight back as much as you can before you actually do it, everybody will benefit from the use of the newer code.
That will allow to impose security restrictions (not sure if feasible with vanilla SVN, but there are some freely available servers that support it).
I'd recommend sending emails notifications whenever someone commits on a project to everybody potentially interested.
4 - Project source tree organization
Each project should have the following SVN structures:
trunk (current version)
branches (older versions, still in use)
tags (releases, used to create branches without thinking too much when patches are required)
When the project gets bigger, organize branches and tags in sub folders (for instance branches/V1.0/V1.1 and branches/V2.0/V2.1).
Have a root folder with the following subfolders: (some of this may be created by VC itself)
Build system (stuff required by your makefiles or others)
Tools (if any, like an XSLT tool or SOAP compiler, scripts to launch the tests)
Data (test data you need while running)
Output (where the build system put the binaries)
Temp Output (temporary files created by the compilation, optional)
Dependencies
Docs (if any ;) or generated docs)
Releases (the generated archives see later)
Sample (a small project that demonstrate how to use the project / library)
Source ( I don't like to split headers and .cpp, but that's my way )
Avoid too many levels of subfolders, it's hard to search trees, lists are easier
Define properly the build order of each folder (less necessary for VC but still)
I make my namespaces match my folders names (old Java habits, but works)
Clearly define the "public" part that you need to export
If the project is large enough to hold several binaries / dlls each should have its own folder
Don't commit any binaries you generate, only the releases. Binaries like to conflict with each other and cause pain to the other people in the team.
5 - Packaging the projects
First, make sure to include a text file with the SVN revision and the date, there's an automated way to do that with auto props.
You should have a script to generate releases (if time allows). It will check that everything is commited, generate a new version number .... Create a zip/tar.gz archive you must commit/archive, whose name contains the SVN revision, branch and the current date (the format should be normalized accross projects). The archive should have everything that is needed to run the app / use the library in a file structure. Create a tag so that you can start from it for emergency bug fixing.