I want to ease adoption of dependency management in my organization, and it does not seem to be as easy of a task with C/C++ as it is with Java.
I want to use an internal Artifactory repository (Maven, Ivy, Gradle or whatever is suitable) to essentially be able to download and publish external libraries which have been precompiled to then statically link them (catch: we are using a custom compiler for an embedded platform)
I have read what it seems to be the basic guide from Gradle's website, however there is no mention of external repositories:
https://docs.gradle.org/current/userguide/native_software.html
http://gradle.monochromeroad.com/docs/userguide/nativeBinaries.html
These two threads seem to touch on the subject, but I'm confused as to where the linking is happening and in what order:
https://discuss.gradle.org/t/external-dependencies-in-cpp-projects/7135
https://discuss.gradle.org/t/right-way-to-copy-contents-from-dependency-archives/7449
So far I cannot wrap my mind around some of the closures/syntax and correct usage of stuff like "configurations", "dependencies", "repositories" because they seem to be used in different ways.
With all that said, what would be a minimal example to get the following done?:
Go to Artifactory
Fetch a dependency (let's assume it is a .a or .o file)
Put that individual dependency in a specific location within the project
Build (specifying the linking order and compiler)
Related
I am very much confused between these two terms Build tools and Package Manager. According to my current knowledge, Package managers are the ones use to install dependencies required for the code to execute while Build tools are used to Package the code plus dependencies into single file i.e. building the code. Building our application will enable to make it production ready.
Am I right???
Short answer
Build systems/tools manage your compilation requirements.
Package managers/tools manager your library requirements.
A build tool may have integrated package management.
For example, in both C++ and Java you can directly call the compiler and provide all the include, source and library paths manually or you can use a build system (make/cmake... for c++, maven/gradle/ant... for java).
When you link external libraries with your build system it will do its best to find them in its search path, and will link with the first version that meets its requirements or tell you that it couldn't find it. Adding libraries manually is fairly easy, but sometimes each library you add will require another library with it.
A package manager would make sure that your libraries are downloaded, are the right version, and all the libraries they depend on are downloaded. some examples are maven and gradle which have integrated package managment for java, and conan is a fairly popular option to combine with cmake.
So ideally you would use both, but it can be more work setting them up than you save not doing things manually. It depends on your programming language, if you need multiple versions of something, and your OS.
In the past I have created a jar file that contained many "helper functions" that I used and made common to many different applications. I felt this was important as anytime I used my "helper" jar file in any new applications or when making changes to any existing ones, the latest and most up to date version of my "helper" jar was always used. It was developed separately and had it's own version control.
I'm looking to do something similar with C/C++
At the moment I have a collection of headers, doing something similar to my "helper" jar in java but finding it cumbersome managing changes, ensuring the most up to date collections are used. So for example, if I made some changes to these "helper" headers, I need to copy them into each project and rebuild.
If we take the below as an example of what I do in Java;
and the below is the structure that I'd like to do something similar with in C++;
I'd like some way of keeping my_includes separate so that any changes I made to my_includes are automatically included in any existing or new applications, in the way Utilities.jar is in the above Java example
I accept that I cant build a library or such as it won't then be as portable, right?
I suspect I'm missing something quite obvious, just not to me.
All helpful comments appreciated, thanks in advance...
At first you could create a library from your utilitis.cpp and all include files independently and add this library to any project. I just provide URL for sample generate the static and shared library.
Create static and shared library (GCC)
And then you can add custom include files to any project in c++ just need to add the specific directories to your include Path in compile time base your platform or if you use cmake you can edit the "include_directories". And also you should link the generated library to your project as described in provided URL.
I want to use google-url in my project as a shared library on Linux\Mac OS, but can not figure out the right way to build it...
Question: what is the way you suggest to build it from scratch form official sources?
Requirements - be able to stay in sync with official repo and use standard(make) tools.
As far as i can see, right now there are few ways to build it:
in the official repo itself only Visual Studio 2005 build files are included
it is in use at Chromium and so there is .gyp available for it but looks like it is tight integrated with Chromium build structure, so there is no easy way to generate Makefile for the standalone library build.
Although it has a comment inside "TODO(mark): Upstream this file to googleurl."
So at list this considered to be possible.
Googleurl is also integrated with PageSpeed project in .gyp form (thought no the same one as above) and so it is somehow built there
third-party bindings for python are available and also contain some build instructions, but with SCons this time, and AFAIK it is kind of obsolete system to rely on.
Looks like i'm not the only one with this trouble, so other people i found both just implemented their own build files using autotools:
https://github.com/artemg/Googleurl-separate-library
https://github.com/commoncrawl/commoncrawl-crawler/blob/master/src/native/src/libGoogleURL/googleurl/README.google
It could work but the filesystem layout is not the same as in official repo/they have local modification so there are no easy way to downstream changes and stay in sync.
The most tempting way would be to use GYP to generate platform-specific build files for the oficial repo once: make/xcode/visual studio, then just save and use them later as needed..but i have no idea how to approach this and where to start from.
Imagine an overall project with several components:
basic
io
web
app-a
app-b
app-c
Now, let's say web depends on io which depends on basic, and all those things are in one repo and have a CMakeLists.txt to build them as shared libraries.
How should I set things up so that I can build the three apps, if each of them is optional and may not be present at build time?
One idea is to have an empty "apps" directory in the main repo and we can clone whichever app repos we want into that. Our main CMakeLists.txt file can use GLOB to find all the app directories and build them (not knowing in advance how many there will be). Issues with this approach include:
Apparently CMake doesn't re-glob when you just say make, so if you add a new app you must run cmake again.
It imposes a specific structure on the person doing the build.
It's not obvious how one could make two clones of a single app and build them both separately against the same library build.
The general concept is like a traditional recursive CMake project, but where the lower-level modules don't necessarily know in advance which higher-level ones will be using them. Yet, I don't want to require the user to install the lower-level libraries in a fixed location (e.g. /usr/local/lib). I do however want a single invocation of make to notice changed dependencies across the entire project, so that if I'm building an app but have changed one of the low-level libraries, everything will recompile appropriately.
My first thought was to use the CMake import/export target feature.
Have a CMakeLists.txt for basic, io and web and one CMakeLists.txt that references those. You could then use the CMake export feature to export those targets and the application projects could then import the CMake targets.
When you build the library project first the application projects should be able to find the compiled libraries automatically (without the libraries having to be installed to /usr/local/lib) otherwise one can always set up the proper CMake variable to indicate the correct directory.
When doing it this way a make in the application project won't do a make in the library project, you would have to take care of this yourself.
Have multiple CMakeLists.txt.
Many open-source projects take this appraoch (LibOpenJPEG, LibPNG, poppler &etc). Take a look at their CMakeLists.txt to find out how they've done this.
Basically allowing you to just toggle features as required.
I see two additional approaches. One is to simply have basic, io, and web be submodules of each app. Yes, there is duplication of code and wasted disk space, but it is very simple to implement and guarantees that different compiler settings for each app will not interfere with each other across the shared libraries. I suppose this makes the libraries not be shared anymore, but maybe that doesn't need to be a big deal in 2011. RAM and disk have gotten cheaper, but engineering time has not, and sharing of source is arguably more portable than sharing of binaries.
Another approach is to have the layout specified in the question, and have CMakeLists.txt files in each subdirectory. The CMakeLists.txt files in basic, io, and web generate standalone shared libraries. The CMakeLists.txt files in each app directory pull in each shared library with the add_subdirectory() command. You could then pull down all the library directories and whichever app(s) you wanted and initiate the build from within each app directory.
You can use ADD_SUBDIRECTORY for this!
https://cmake.org/cmake/help/v3.11/command/add_subdirectory.html
I ended up doing what I outlined in my question, which is to check in an empty directory (containing a .gitignore file which ignores everything) and tell CMake to GLOB any directories (which are put in there by the user). Then I can just say cmake myrootdir and it does find all the various components. This works more or less OK. It does have some side drawbacks though, such as that some third-party tools like BuildBot expect a more traditional project structure which makes integrating other tools with this sort of arrangement a little more work.
The CMake BASIS tool provides utilities where you can create independent modules of a project and selectively enable and disable them using the ccmake command.
Full disclosure: I'm a developer for the project.
There are already some questions about dependency managers here, but it seems to me that they are mostly about build systems, while I am looking for something targeted purely at making dependency tracking and resolution simpler (and I'm not necessarily interested in learning a new build system).
So, typically we have a project and some common code with another project. This common code is organized as a library, so when I want to get the latest code version for a project, I should also go get all the libraries from the source control. To do this, I need a list of dependencies. Then, to build the project I can reuse this list too.
I've looked at Maven and Ivy, but I'm not sure if they would be appropriate for C++, as they look quite heavily java-targeted (even though there might be plugins for C++, I haven't found people recommending them).
I see it as a GUI tool producing some standardized dependency list which can then be parsed by different scripts etc. It would be nice if it could integrate with source control (tag, get a tagged version with dependencies etc), but that's optional.
Would you have any suggestions? Maybe I'm just missing something, and usually it's done some other way with no need for such a tool? Thanks.
You can use Maven in relationship with C++ in two ways. First you can use it for dependency management of components between each other. Second you can use Maven-nar-plugin for creating shared libraries and unit tests in relationship with boost library (my experience). In the end you can create RPM's (maven-rpm-plugin) out of it to have adequate installation medium. Furthermore i have created the installation for CI environment via Maven (RPM's for Hudson, Nexus installation in RPM's).
I'm not sure if you would see an version control system (VCS) as build tool but Mercurial and Git support sub-repositories. In your case a sub-repository would be your dependencies:
Join multiple subrepos into one and preserve history in Mercurial
Multiple git repo in one project
Use your VCS to archive the build results -- needed anyway for maintenance -- and refer to the libs and header files in your build environment.
If you are looking for a reference take a look at https://android.googlesource.com/platform/manifest.