Better way to structure custom compiled libs - c++

I am custom compiling libcurl , libssl and some other library. I don't want to replace system library, because if I am changing it system wise, it is going to create lib conflict and I need to compile all other component depending on these libs.
So I started using RPATH and started structuring like this:
|-- bin
| |-- app.out
|-- lib
| |-- libboost_program_options.so -> libboost_program_options.so.1.49.0
| |-- libboost_program_options.so.1.49.0
| |-- libboost_system.so -> libboost_system.so.1.49.0
| |-- libboost_system.so.1.49.0
| |-- libboost_thread.so -> libboost_thread.so.1.49.0
| |-- libboost_thread.so.1.49.0
| |-- libcares.so -> libcares.so.2.0.0
| |-- libcares.so.2 -> libcares.so.2.0.0
| `-- pkgconfig
`-- sbin
`-- nginx
This approach worked. Now problem is that , We started using PHP and node which require same application version.
|-- bin
| |-- a.out
|-- lib
| |-- libboost_program_options.so -> libboost_program_options.so.1.49.0
| |-- libboost_program_options.so.1.49.0
| |-- libboost_system.so -> libboost_system.so.1.49.0
| |-- libboost_system.so.1.49.0
| |-- libboost_thread.so -> libboost_thread.so.1.49.0
| |-- libboost_thread.so.1.49.0
| |-- libcares.so -> libcares.so.2.0.0
| |-- libcares.so.2 -> libcares.so.2.0.0
| `-- pkgconfig
|-- php_ext
| `-- sqlite3.so
|-- node
| `-- node_modules
| |-- bin
| | |-- node
`-- sbin
`-- nginx
Now , this svn repo is becoming bigger and bigger after every release. Is there a better way to structure this ? without duplicating lib folder in each app ?

As someone who has used both git and svn extensively for years, I'd seriously consider moving to git and using git submodules. Git is hugely more space-efficient (among many, many other benefits). There are also git-svn bridges that you can make if you're stuck using svn at your company.
Failing that, I'd make an svn externals for each group of shared libraries. If you have something that changes often or is logically grouped together, it can go in one svn repo while other libraries that may not change very often.
One of the advantages of git over svn is that git protects you from file corruption. I can painfully remember several occurrences of svn corrupting files (something that wasn't noticed until the client submitted a bug report).
Seriously, save yourself a world of headaches, ditch svn in favour of git.

I was taking a totally different approach when I was working with C++.
I don't treat libs as part of source code. I only want the "dependency" to exist with my source. (It is not appropriate to have "version control" for the lib binary anyway)
I have a separate directory to store all libs in a organized manner, something like libname/version/arch.
In the build script, I am referring to something like $LIB_DIR/libname/version/arch/lib-ver.so.
You can have different way to store/distribute the Lib directory, either put it in a network volume, put that in SVN etc.

Related

How to change Bazel build output directory location

I have small project and after building it he layout looks like this.
|-- BUILD
|-- README.md
|-- VERSION
|-- WORKSPACE
|-- bazel-bin -> /home/bkch/.cache/bazel/_bazel_bkch/172376ca1288bc6e93208fc2d53c0b74/execroot/distroless/bazel-out/k8-fastbuild/bin
|-- bazel-distroless -> /home/bkch/.cache/bazel/_bazel_bkch/172376ca1288bc6e93208fc2d53c0b74/execroot/distroless
|-- bazel-out -> /home/bkch/.cache/bazel/_bazel_bkch/172376ca1288bc6e93208fc2d53c0b74/execroot/distroless/bazel-out
|-- bazel-testlogs -> /home/bkch/.cache/bazel/_bazel_bkch/172376ca1288bc6e93208fc2d53c0b74/execroot/distroless/bazel-out/k8-fastbuild/testlogs
|-- debian_archives.bzl
bazel-bin, bazel-out, bazel-distroless, bazel-testlogs are symbolic links to .cache in user root folder. Instead of the symbolic links I would like to build them in current workspace directory.
From the documentation I tried using --output_base=$PWD/output or --output_user_root=$PWD/output . The moment I use these options the bazel build is failing of various random reasons. But when I use bazel build //... with out these options the build succeeds.
I guess the answer might lie here... github.com/bazelbuild/bazel/issues/13601 we just need to have folder outside the project folder --output_base=$PWD/../output

CMake: best structure when migrating microrepositories to a monorepo?

At the early stage of our development, we have created micro-repositories, to be able to deeply modify repositories while minimizing interference with each other's work.
Now that we have a fairly stable version of our codebase, we want to merge some of the micro-repositories into a monorepo. That will ease the users' experience while building from source, as well as the deployment phase.
All our micro-repositories are built with CMake, and each of them obviously defines a project(). They also have 'internal' dependencies: some of the micro-repositories is built first and then used by the others with find_package().
RepoA
|-- CMakeLists.txt
| project(RepoA)
RepoB
|-- CMakeLists.txt
| project(RepoB)
| find_package(RepoA)
RepoC
|-- CMakeLists.txt
| project(RepoC)
| find_package(RepoA)
| find_package(RepoB)
While migrating them into a monorepo, the first option that came to my mind is moving each of them into a subdirectory, then adding add_subdirectory() to the root CMakeLists.txt file to build them. In the first instance, I would define a global project by using project() in the root CMakeLists.txt file, and I would also keep the project() definition in subdirectories.
Monorepo/
|-- CMakeLists.txt
| project(Monorepo)
| add_subdirectory(RepoA)
| add_subdirectory(RepoB)
| add_subdirectory(RepoC)
|-- RepoA/
|-- CMakeLists.txt
| project(RepoA)
|-- RepoB/
|-- CMakeLists.txt
| project(RepoB)
| find_package(RepoA)
|-- RepoC/
|-- CMakeLists.txt
| project(RepoC)
| find_package(RepoA)
| find_package(RepoB)
Now the question is: is that a good choice? Would you advise having a global project and multiple sub-projects, all of them defined through the use of the CMake project() command? Are there better strategies?
And also, would this strategy have drawbacks?
Many thanks for your kind help!

CMake - Module + Library confusion

I've started a new c++ project , and I am confused with all the CMake capabilities. I have tried to understand better by looking at examples and CMake tutorials
I should create a new project composed of:
Library: It contains some common classes that will be used by the following module(s) (e.g., vector, matrix, image, etc..)
Module (possibly more than 1 in the future): It contains some module-specific classes (e.g., classifier, estimator, etc.) and a main.
My proposed folder structure is as below:
|-- Root Project
|-- CMakeLists.txt
|
|-- Library
| |-- CMakeLists.txt
| |-- include
| | |-- CMakeLists.txt (?)
| | `-- Lib_Class.h
| `-- src
| |-- CMakeLists.txt (?)
| `-- Lib_Class.h
|
|-- Application 1
| |-- CMakeLists.txt
| |-- include
| | |-- CMakeLists.txt (?)
| | `-- Method.h
| `-- src
| |-- CMakeLists.txt (?)
| |-- Method.cpp
| `-- main.cpp
|
|-- Application 2
| |-- CMakeLists.txt
| |
`
The problem arises when I have to actually add the code to the different CMakeLists.txt files. According to my reasoning, I would have:
Root/CMakeLists.txt: For creating the project and adding the subdirectories of the Library and the Module(s).
Library/CMakeLists.txt: This creates the library with the header (from include folder) and source (from src folder) files.
Module/CMakeLists.txt: This creates an executable from the src/main.cpp file using the Library and the module-specific classes with header files in include folder and source files in src folder.
I have 2 questions:
First, I also found solutions in other replies with CMakeLists.txt files in the Library/src and Module/src folders. But I really don't understand how to use them and what to write inside them, because I would have used only the CMakeLists.txt file in the parent folder.
Second, in case I want to link an external library (e.g., OpenCV or dlib) should I link it in the modules and library, individually, or should I link it in the root CMakeLists.txt file (provided that the library is used everywhere)?
I really need some assistance to try to understand CMAKE. Can someone explain or please direct me to a suitable tutorial on this subject.
Matthieu, thank you very much for your help. According to the explanation you provided me, I came out with the following CMakeLists.txt files:
Root/CMakeLists.txt
cmake_minimum_required(VERSION 2.8)
project(Project_Name)
add_subdirectory(Library)
add_subdirectory(Application)
Library/CMakeLists.txt
project(Library)
set(LIB_HEADERS
include/Lib_Class.h
)
set(LIB_SOURCES
src/Lib_Class.cpp
)
add_library(Library_Name SHARED ${LIB_SOURCES} ${LIB_HEADERS})
Application/CMakeLists.txt
project(Application)
set(APP_HEADERS
include/Method.h
)
set(APP_SOURCES
src/Method.cpp
src/main.cpp
)
add_executable(Application_Name ${APP_SOURCES} ${APP_HEADERS})
target_link_libraries(Application_Name Library_Name)
Now everything seems to work grat! Thank you again and sorry again for being confusing somethimes!
The root cmakelists should set up all the variables, checking compiler support and library presence.
Then you go to each subfolder and create the libraries and executables based on the source code and the detected libraries. You should also set up all the linked libraries there.
Then cmake will figure out what depends on what.

How do I share headers and libraries in subdirectory by cmake?

I want to use my headers and libs as common libraries for app1 and app2. My project tree are below. image/ and math/ are library directories used by app1 and app2. In this case, should I set same settings to both CmakeLists.txt under app1 and app2? Of course I know it works, but are there any smarter ways to set common libraries?
|-- CMakeLists.txt
|-- app1
| |-- CMakeLists.txt
| `-- main.cc
|-- app2
| |-- CMakeLists.txt
| `-- main.cc
|-- image
| |-- CMakeLists.txt
| |-- include
| | `-- image_func.h
| `-- src
| `-- image_func.cc
`-- math
|-- CMakeLists.txt
|-- include
| `-- math_util.h
`-- src
`-- math_util.cc
Roots CMakelists.txt is below. Is it possible to set math and image parameters for app1 and app2? My actual project has many applications which uses multiple libraries.
cmake_minimum_required(VERSION 2.8)
add_subdirectory("./image")
add_subdirectory("./math")
add_subdirectory("./app1")
add_subdirectory("./app2")
With newer versions oft CMake (since 2.8.12) you can use target_link_libraries and related functions to manage dependencies. By specifying PUBLIC the includes and libraries are also applied to all targets using the library.
This will reduce duplication to a minimum.
For math and image you need to specify that the respective include directory is used and any libraries you might require.
math/CMakeLists.txt
add_library(math ...)
target_include_directories(math PUBLIC include ...)
target_link_libraries(math PUBLIC ...)
image/CMakeLists.txt
add_library(image ...)
target_include_directories(image PUBLIC include ...)
target_link_libraries(image PUBLIC ...)
app1/CMakeLists.txt
add_executabke(app1 ...)
target_link_libraries(app1 PUBLIC image math)
app2/CMakeLists.txt
add_executabke(app2 ...)
target_link_libraries(app2 PUBLIC image math)

How to logically organize source files in C++ [closed]

As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 9 years ago.
My source file pane is quickly growing (in terms of the number of files in my project) and it is getting a bit cumbersome to quickly locate the specific source file I need to access at any given time. I'm using Embarcadero's C++Builder, but I have encountered this issue in other C++ IDEs as well.
In Java, I often utilize packages for creating logical divisions of my source code, especially when dealing with a large number of source files in a single project. While this, of course, isn't the only purpose of Java packages, they are very handy in this regard.
Does anyone have any ideas on how I can achieve similar functionality in C++? Should I separate my source into physical folders? Does C++Builder offer some kind of virtual folder/grouping functionality that I'm just not seeing? Any ideas are appreciated and thank you.
I generally recommend not to use (only) the IDE or the language syntax for organizing your source code. For one, you tie yourself to the environment: well organized in the IDE, unorganized on file, and then comes the day when you might want to use a different environment...
Because of this, I usually use all three ways of organizing my source at the same time: I separate my source into functional modules, i.e. related classes. Each module gets its own namespace, physical folder, and IDE folder. (In my case, using CMake and source_group() to generate IDE project files if needed -- personally preferring the command line, Vim, and "make".)
Hence, whether I look at the project from within the IDE, from the command line, or from a compiler log, foo/some_class.hpp is foo/some_class.cpp is foo::some_class, minimizing confusion all around.
Actually, my currently preferred setup further subdivides each module directory into <project>/<module>/<class>.hpp or <project>/<module>/src/<class>.hpp depending on whether the class is used outside its own module or not, <project>/<module>/src/<class>.cpp, and <project>/<module>/test/<class>_tu.cpp. Namespace is <project>::<module>::<class>, of course.
project
|-- foo
| |-- some_class.hpp
| |-- src
| | |-- internal_class.hpp
| | |-- internal_class.cpp
| | `-- some_class.cpp
| `-- test
| |-- internal_class_tu.cpp
| `-- some_class_tu.cpp
|-- bar
| |-- ...
The idea here is that the "external" interface of each module (foo) is documented by the headers in that subfolder, with implementation details and tests "hidden" in the respective subfolders.
But in the end, it very much depends -- on your taste, on that of your co-developers, and the scope of your project.
This is how I roll:
PROJECT_NAME
|-- build // This is DVCS ignored but has all the built intermediates and final binaries
| |-- release // These are the different build profiles
| |-- debug
| |-- profile
| `-- coverage
|-- bin // For binary source code
| `-- hello_world
| |-- doc
| |-- inc
| |-- src
| |-- tests
| `-- build_script // Builds binary into the build folder
|-- include // Public headers for the library
| `-- these
| `-- folders
| `-- represent
| `-- namespaces
| `-- my_awesome_class.hpp
|-- lib // library source code
| |-- these
| | `-- folders
| | `-- represent
| | `-- namespaces
| | |-- inc // Private headers
| | | `-- my_private_class.hpp // internal class
| | |-- src // Source code for this namespace
| | | |-- posix
| | | | `-- my_awesome_class.cpp // posix specific source code
| | | |-- nt
| | | | `-- my_awesome_class.cpp // nt specific source code
| | | |-- my_private_class.cpp // non-visibile class
| | | `-- my_awesome_class.cpp // cross platform source code
| | |-- tests // Unit tests
| | | `-- my_awesome_class.cpp // builds a test executable for the library
| | `-- doc // Documentation for this namespace
| | `-- namespace.dox
| `-- build_script // Builds binary into the build folder
|-- doc // Documentation files
| |-- main_page.dox
| `-- namespace.dox
`-- build_script // Builds the source code into the build folder
This represents the these::folders::represent::namespaces::MyAwesomeClass class which has posix and NT specific source code (as well as generic source code) plus there is a private these::folders::represent::namespaces::MyPrivateClass that is used internally in the library, the headers are not public and the visibility of the class symbols are hidden.
This has scaled very well and provides easy locating of files.
I make projects in order to keep all my files easily accesible. It is the easiest way to organize, along with clear class/file names.