Keeping all libraries in the Arduino sketch directory - c++

I know that you are supposed to place any external libraries under the "libraries" folder of the arduino install directory, but I have a project that uses several libraries that I have created for the project and mainly to keep all that code self contained and out of the main pde file. However, I have tried to place the libraries in the same directory as the main PDE file so that I can more easily keep everything synced up in subversion (I work on this on multiple computers) and I don't want to have to keep going back and syncing up the libraries separately. Also, just for the sake of being able to easily zip of the sketch folder and know that it contains everything it needs.
I've tried adding the header files to the sketch as a new tab, but that doesn't seem to work at all... don't even care if they should up in the arduino IDE.
I've also tried adding the libraries to the sketch directory in subdirectories (what I would greatly prefer) and then linking to them as:
#include "mylib/mylib.h"
and
#include <mylib/mylib.h>
But both of these result in file not found errors.
Is this possible? And, if so, how do I include them in the main file for building? Preferably in their own subdirectories.

I had the same issue. Solved it for Arduino IDE > 1.8. Seems a specialty in newer IDEs (?) according to the reference (see bottom link).
You have to add a "src" Subdirectory before creating a library folder. So essentially your project should look like this:
/SketchDir (with *.ino file)
/SketchDir/src
/SketchDir/src/yourLib (with .h and .cpp file)
and finally in your sketch you reference:
#include "src/yourLib/yourLib.h"
otherwise in my case - if I am missing the "src" folder - I get the error message that it cannot find the yourLib.cpp file.
Note: I am using a windows system in case it differs and actually VS Code as wrapper for Arduino IDE. But both IDE compile it with this structure.
References:
https://forum.arduino.cc/index.php?topic=445230.0

For the sketches I have, the "*.h" and "*.cpp" library files actually reside in the same folder as the sketch, and I call them like "someheader.h". I also noticed that if I go into sketch menu and add file... that the file doesn't appear until I close and reopen the sketch.

I agree with you; this is an intolerable way to develop software: it requires every file that you need to be in the same directory as the main program!
To get around this, I use make to put together a single .h file from my .h and .cpp sources - you can see this used in this Makefile:
PREPROCESS=gcc -E -C -x c -iquote ./src
# -E : Stop after preprocessing.
# -C : Don't discard comments.
# -x c : Treat the file as C code.
# -iquote ./src : Use ./src for the non-system include path.
TARGETS=sketches/morse/morse.h
all: $(TARGETS)
clean:
rm $(TARGETS)
%.h: %.h.in
$(PREPROCESS) $< -o $#
Arduino is very picky about file endings - if you put a .cpp or .cc file in its directory it automatically uses it in the source, and you can't include anything that's not a .cpp, .cc or .h - so this is about the only way to do it.
I use a similar trick also to put together JavaScript files here.
This requires that you run make after editing your files, but since I'm using an external editor (Emacs) anyway, this is zero hassle for me.

Unfortunately the Arduino IDE is awful and shows no signs of improving. There is no real build system so it only lets you build programs that reside in a single directory.
The only real solution is to write a makefile, then you can use a real IDE. I'm hopeful that one day someone will write an Arduino plugin for QtCreator.
Here's an example makefile:
http://volker.top.geek.nz/arduino/Makefile-Arduino-v1.8

I just had this same problem (I also like to keep the code self-contained), so I'll just jot down some notes; say I have a MyPdeSketch.pde using MyLibClass.cpp; then I have it organized like this
/path/to/skdir/MyPdeSketch/MyPdeSketch.pde
/path/to/skdir/MyPdeSketch/MyLibClass/MyLibClass.cpp
/path/to/skdir/MyPdeSketch/MyLibClass/MyLibClass.h
(In principle, /path/to/skdir/ here is equivalent to ~/sketchbook/)
What worked for me is something like:
mkdir /path/to/arduino-0022/libraries/MyLibClass
ln -s /path/to/skdir/MyPdeSketch/MyLibClass/MyLibClass.* /path/to/arduino-0022/libraries/MyLibClass/
After restart of the IDE, MyLibClass should show under ''Sketch/Import Library''.
Note that the only way I can see so far for a library class file to refer to other library files is to include them relatively (from 'current location'), assuming they are all in the same main arduino-0022/libraries folder (possibly related Stack Overflow question: Is it possible to include a library from another library using the Arduino IDE?).
Otherwise, it should also be possible to symlink the MyLibClass directory directly into arduino-0022/libraries (instead of manually making a directory, and then symlinking the files). For the same reason, symlinking to the alternate location ~/sketchbook/libraries could also be problematic.
Finally, a possibly better organization could be:
/path/to/skdir/MyLibClass/MyLibClass.cpp
/path/to/skdir/MyLibClass/MyLibClass.h
/path/to/skdir/MyLibClass/MyPdeSketch/MyPdeSketch.pde
... which, after symlinking to libraries, would force MyPdeSketch to show under the examples for the MyLibClass library in Arduino IDE (however, it may not be applicable if you want to self-contain multiple class folders under a single directory).
EDIT: or just use a Makefile - which would work directly with avr-gcc, bypassing the Arduino IDE (in which case, the sketchbook file organization can be somewhat loosened)..

Think I know what do u need exactly.
you have a project folder say MYPROJ_FOLDER and you want to include a Libraries folder that contains more children folders for your custom libraries.
you need to do the following:
1- create folders as follows:
-MyProjFolder
-MyProjFolder/MyProjFolder
and then create a file with the folder name in .ino extension
-MyProjFolder/MyProjFolder/MyProjFolder.ino
2- create libraries folder:
-MyProjFolder/libraries <<<<< name is not an option should be called like that.
3- then create your own libraries
-MyProjFolder/libraries/lib1
-MyProjFolder/libraries/lib1/lib1.cpp
-MyProjFolder/libraries/lib1/examples <<<< this is a folder
-MyProjFolder/libraries/lib1/examples/example1
repeat step 3 as much as you want
also check http://arduino.cc/en/Guide/Libraries

I did it a little differently. Here is my setup.
Visually this is the directory layout
~/Arduino/Testy_app/ <- sketch dir
/Testy_app.ino <- has a #include "foo.h"
/foo <- a git repo
/foo/foo.h
/foo/foo.cpp
Here is how I build:
~/Arduino/Testy_App/$ arduino-cli compile --library "/home/davis/Arduino/Testy_app/foo/" --fqbn arduino:samd:mkrwan1310 Testy_app
If you wish to be more elaborate and specify libs and src dirs, this also works
~/Arduino/Testy_app/ <- sketch dir
/Testy_app.ino <- has a #include "foo.h"
/lib <- a git repo
/lib/foo/src/foo.h
/lib/foo/src/foo.cpp
and the build method is:
~/Arduino/Testy_App/$ arduino-cli compile --library "/home/davis/Arduino/Testy_app/lib/foo/src" --fqbn arduino:samd:mkrwan1310 Testy_app
One more bit of tweaking needs to be done to include files from the lib dirs to main dir. If you need to do that, this is the work around:
~/Arduino/Testy_app/ <- sketch dir
/Testy_app.ino <- has a #include
"foo.h"
/inc/Testy_app.h
/foo <- a git repo
/foo/foo.h
/foo/foo.cpp < has a "include testy_app.h"
Then do the compile like this
~/Arduino/Testy_App/$ arduino-cli compile \
--library "/home/davis/Arduino/Testy_app/inc" \
--library "/home/davis/Arduino/Testy_app/foo/src" \
--fqbn arduino:samd:mkrwan1310 Testy_app

What has worked for me is to create a dir, for example "src" under the sketch dir, and under that a dir for each personal library.
Example:
I have a project called ObstacleRobot, under that a folder for my sketch, named obstaclerobot (automatically created by the IDE) and there my sketch "obstacleRobot.ino"
Up to now we have:
/ObstacleRobot
/obstaclerobot
obstacleRobot.ino
Then I wanted to include a personal library that was fully related with this project and made no sense in including it in the IDE libraries, well in fact I want to do this for each part of the robot but I'm still working on it.
What in the end worked for me was:
/ObstacleRobot
/obstaclerobot
obstacleRobot.ino
/src
/Sonar
Sonar.h
Sonar.cpp
Then what you have to do in the main sketch is to write the include as follows:
#include "src/Sonar/Sonar.h"
And thats all.

Following the lines of Hefny, make your project an example for your library.
For example (Unix env), let's say the libraries are in ~arduino/libraries
Your create your project ~arduino/libraries/MyProject, your libraries go there (for example ~/arduino/libraries/MyProject/module1.h ~/arduino/libraries/MyProject/module1.cpp ~/arduino/libraries/MyProject/module2.h ~/arduino/libraries/MyProject/module2.cpp
Now:
mkdir -p ~arduino/libraries/MyProject/examples/myproj
edit ~arduino/libraries/MyProject/examples/myproj/myproj.ino
(note that this is not examples/myproj.ino but examples/myproj/myproj.ino)
Restart the IDE, you should find your project in the menu File/Example/MyProject.
Also note that you do the include with #include

Why dont we just write a script with a single copy command, copying our libs from wherever our library is located into the arduino IDE library folder?
This way we keep the file structure we want and use the IDE library requirements without fuss.
Something like this works for me:
cp -r mylibs/* ~/Documents/programs/arduino-1.5.8/libraries/.
Note that the paths are relative to my own file structure.
Hope this helps someone. This includes my future self that I bet will be reading this in a near future... as usual!
J

Related

How to run a makefile that takes in all header files within a subdirectory

I'm extremely confused over how to make in c++. I'm working on a school project where we downloaded several files -- some starter code, and some libraries. File structure is something along the lines of
main --- lib/ severalLibraryFolders
\--- src/ starterCode
I cannot seem to get the starter code to make within the command line. There's a vector library within one of the library folders and when I run make on the starter file (a .cpp and .h) I get a fatal error: 'vector.h' not found. I managed to use gcc and the -I flag to include the folder that vector.h was in which fixed it -- however other similar problems popped up even after I used the -I flag on all possible library folders.
I'm fairly certain I'm doing something wrong/need a makefile but all the examples I've found are not clear on how to do what I need to do, which is essentially to find all header files below the parent directory. I'd love some direction here, as I come from a more scripty background where this sort of thing is much easier!

Compile .qm (i18n) files into binary

I have multiple i18n qm files and translation works like a charm right now.
The problem I'm facing is, that I have the following folder structure:
project
-build
--debug
--release
-src
-i18n
Now my colleagues have different build folder structures, hence:
translator.load("../../i18n/{translationfile}");
won't work for them.
One solution might be to settle on a specific build-folder structure, but this is not what I want. I'd prefer to export all the translations to the application itself so it may be shipped without any special folder structure. Just give the customer the binary and go, like it should be.
Anyway, I have no idea how to do this in qmake (nor in cmake for that matter, but qmake is what I need to use here.)
I had a look at: How to Integrate Qt4 qm files into binary using cmake and QRC? but still not sure.
Any help appreciated.
Looks like I fixed it.
Instead of
translator.load("../../i18n/{translationfile}");
I use
QDir dir(":/i18n");
translator.load(dir.absoluteFilePath({translationFile}));
I also added a i18n.qrc file to my pro file:
RESOURCES += images.qrc \
i18n.qrc
It works if I move the binary/executable to another folder and execute it from there, so I guess that's all that is needed to compile translation files into a QApplication.

Different paths used for #include and other files

I'm quite confused about this weird behaviour of my .cpp project. I've got the following folder structure:
include/mylib.h
myproject/src/eval.cpp
myproject/data/file.csv
myproject/Makefile
In eval.cpp I include mylib.h as follows:
#include "../../include/mylib.h"
and compile it through Makefile:
all:
g++ -I include ../include/mylib.h src/eval.cpp -o eval.out
Now in my eval.cpp I'm reading the file.csv from data directory and if I refer to it like this
../data/file.csv
it doesn't find it (gets empty lines all the time), but this
data/file.csv
works fine.
So, to include mylib.h it goes two directories up (from src folder) which seems right. But it doesn't make sense to me that to refer to another file from the same piece of code it assumes we are in project directory. I suppose it is connected with Makefile somehow, but I'm not sure.
Why is it so?
EDIT: After a few thing I tried it seems that the path which is used is not the path from binary location to the data location, but depends on where from I run the binary as well. I.e., if I have binary in bin directory and run it like:
./bin/eval.out
It works with data/file.csv.
This:
cd bin
./eval.out
works with ../data/file.csv.
Now it seems very confusing to me as depending on where I run the program from it will give different output. Can anyone please elaborate on the reasons for this behaviour and if it is normal or I'm making some mistake?
It is so because (as explained here ) the compiler will search for #included files with quotes (not with brackets) with the current working directory being the location of the source file.
Then, when you try to open your .csv file, it's now your program that looks for a file. But your program runs with the current working directory being myproject/ which explains why you must specify data/file.csv as your file path, and not ../data/file.csv. Your program does not run in your src folder, it will run in the directory the binary ends up being invoked from.
You could have noticed that in your Makefile, your -I options specify a different path for your header file than your .cpp file.
EDIT Answer: It's quite simple actually and completely normal. When you invoke your binary, the directory which you're in is the current working directory. That is, if you run it with the command ./myproject/bin/eval.out, the current working directory is . (e.g. /home/the_user/cpp_projects). My post was a bit misleading about that, I corrected it.
Note: You can use the command pwd in a command prompt to know which is the current working directory of this prompt (pwd stands for "print working directory").

Help with rake dependency mapping

I'm writing a Rakefile for a C++ project. I want it to identify #includes automatically, forcing the rebuilding of object files that depend on changed source files. I have a working solution, but I think it can be better. I'm looking for suggestions for:
Suggestions for improving my function
Libraries, gems, or tools that do the work for me
Links to cool C++ Rakefiles that I should check out that do similar things
Here's what I have so far. It's a function that returns the list of dependencies given a source file. I feed in the source file for a given object file, and I want a list of files that will force me to rebuild my object file.
def find_deps( file )
deps = Array.new
# Find all include statements
cmd = "grep -r -h -E \"#include\" #{file}"
includes = `#{cmd}`
includes.each do |line|
dep = line[ /\.\/(\w+\/)*\w+\.(cpp|h|hpp)/ ]
unless dep.nil?
deps << dep # Add the dependency to the list
deps += find_deps( dep )
end
end
return deps
end
I should note that all of my includes look like this right now:
#include "./Path/From/Top/Level/To/My/File.h" // For top-level files like main.cpp
#include "../../../Path/From/Top/To/My/File.h" // Otherwise
Note that I'm using double quotes for includes within my project and angle brackets for external library includes. I'm open to suggestions on alternative ways to do my include pathing that make my life easier.
Use the gcc command to generate a Make dependency list instead, and parse that:
g++ -M -MM -MF - inputfile.cpp
See man gcc or info gcc for details.
I'm sure there are different schools of thought with respect to what to put in #include directives. I advise against putting the whole path in your #includes. Instead, set up the proper include paths in your compile command (with -I). This makes it easier to relocate files in the future and more readable (in my opinion). It may sound minor, but the ability to reorganize as a project evolves is definitely valuable.
Using the preprocessor (see #greyfade) to generate the dependency list has the advantage that it will expand the header paths for you based on your include dirs.
Update: see also the Importing Dependencies section of the Rakefile doc for a library that reads the makefile dependency format.

Can I have one makefile to build a hierarchical project?

I have several hundred files in a non-flat directory structure. My Makefile lists each sourcefile, which, given the size of the project and the fact that there are multiple developers on the project, can create annoyances when we forget to put a new one in or take out the old ones. I'd like to generalize my Makefile so that make can simply build all .cpp and .h files without me having to specify all the filenames, given some generic rules for different types of files.
My question: given a large number of files in a directory with lots of subfolders, how do I tell make to build them all without having to specify each and every subfolder as part of the path? And how do I make it so that I can do this with only one Makefile in the root directory?
EDIT: this almost answers my question, but it requires that you specify all filenames :\
I'm sure a pure-gmake solution is possible, but using an external command to modify the makefile, or generate an external one (which you include in your makefile) is probably much simpler.
Something along the lines of:
all: myprog
find_sources:
zsh -c 'for x in **/*.cpp; echo "myprog: ${x/.cpp/.o}" >> deps.mk'
include deps.mk
and run
make find_sources && make
note: the exact zsh line probably needs some escaping to work in a make file, e.g. $$ instead of $. It can also be replaced with bash + find.
One way that would be platform independent (I mean independent from shell being in Windows or Linux) is this:
DIRS = relative/path1\
relative/path2
dd = absolute/path/to/subdirectories
all:
#$(foreach dir, $(DIRS), $(MAKE) -C $(dd)$(dir) build -f ../../Makefile ;)
build:
... build here
note that spaces and also the semicolon are important here, also it is important to specify the absolute paths, and also specify the path to the appropriate Makefile at the end (in this case I am using only one Makefile on grandparent folder)
But there is a better approach too which involves PHONY targets, it better shows the progress and errors and stops the build if one folder has problem instead of proceeding to other targets:
.PHONY: subdirs $(DIRS)
subdirs: $(DIRS)
$(DIRS):
$(MAKE) -C $# build -f ../../Makefile
all : prepare subdirs
...
build :
... build here
Again I am using only one Makefile here that is supposed to be applicable to all sub-projects. For each sub-project in the grandchild folder the target "build" is created usinf one Makefile in the root.
I would start by using a combination of the wildcard function:
http://www.gnu.org/software/make/manual/make.html#Wildcard-Function
VPATH/vpath
http://www.gnu.org/software/make/manual/make.html#Selective-Search
and the file functions
http://www.gnu.org/software/make/manual/make.html#File-Name-Functions
For exclusion (ie: backups, as Jonathan Leffler mentioned), use a seperate folder not in the vpath for backups, and use good implicit rules.
You will still need to define which folders to do to, but not each file in them.
I'm of two minds on this one. On one hand, if your Make system compiles and links everything it finds, you'll find out in a hurry if someone has left conflicting junk in the source directories. On the other hand, non-conflicting junk will proliferate and you'll have no easy way of distinguishing it from the live code...
I think it depends on a lot of things specific to your shop, such as source source control system and whether you plan to ever have another project with an overlapping code base. That said, if you really want to compile every source file below a given directory and then link them all, I'd suggest simple recursion: to make objects, compile all source files here, add the resultant objects (with full paths) to a list in the top source directory, recurse into all directories here. To link, use the list.