I have recently started using Yocto. I 'm looking for option to include/add altered package into final build image. Below I have described the scenario.
I'm working on RDK, which is yocto based system for STB(Set-top Box) Emulator. I have already build complete system once. Now I'm making some changes in some particular module, to see final effect of that in build/image, I rebuilt that particular module(at this point I came to know bitbake doesn't work like makefile utility, that you make changes and it will take care of rest and your package will be compiled as well as included into final image/binary), I used bitbake -c cleansstate <module_name>, then bitbake <module_name> to rebuild the package.
Next thing was to get it inside the final image, but the same thing I had to go through the pain again, bitbake -c cleansstate <image_name>, then bitbake <image_name> to rebuild the image.
Basically, only once package is changed and to include that into final image I have create complete image again.Which is very time-consuming process!!!
I'm wondering is there any way that I can reduce this build time and include altered package into final image?
NOTE: Not looking for optimization option, I know about local.conf BB_NUMBER_THREADS and PARALLEL_MAKE options. It is just about, can we add package into final image without generating all dependency for final image as described in scenario.
Assuming by "making changes" you mean modifying the underlying code, I would suggest using devtool modify - this will set up a local source tree for the recipe where you can make your changes, and each time you make a change and then run bitbake on the recipe or something that depends upon it (such as your image) it will rebuild it including your changes. Basic steps:
devtool modify <recipe>
Make your changes within the source tree that is set up
bitbake <recipe> or bitbake <image>
Test the result; loop back to step 2 if you need to make further changes
devtool finish <recipe> to write your changes back as patches against the recipe
I happened to me that after adding a recipe on meta/recipes-extended/myrecipe_0.0.1.bb
I was able to build my new recipe with the command
bitbake myrecipe
but the binaries never got included on the rootfs image when running
bitbake core-image-minimal
To add the output of my recipe to the output images, I've added the following to my ${BUILDDIR}/conf/local.conf file:
IMAGE_INSTALL_append = " myrecipe"
in my local.conf file.
Related
How I can build local sources and dependancies with flatpak-builder?
I can build local sources
flatpak build ../dictionary ./configure --prefix=/app
I can extract and build application with dependancies with a .json
flatpak-builder --repo=repo dictionary2 org.gnome.Dictionary.json
But no way to build dependancies and local sources? I don't find sources type
like dir or other, only archive, git (no hg?) ...
flatpak-builder is meant to automate the whole build process, with a single entry-point: the JSON manifest.
Everything else it obtains from Git, Bazaar or tarballs. Note that for these the "url" property may be a local URL starting with file://.
(There is indeed no support for Hg. If that's important for you, feel free to request it.)
In addition to that, there are a few more source types (see the flatpak-manifest(5) manpage), which can be used to modify the extracted sources:
file which point to a local file to copy somewhere in the extracted sources;
patch which point to a local patch file to apply to the extracted sources;
script which creates a script in the extracted sources, from an array of commands;
shell which modifies the extracted sources by running an array of commands;
Adding a dir source type might be useful.
However (and I only flatpaked a few apps, and contributed 2 or 3 patches to the code, so I might be completely wrong) care must be taken as this would easily make builds completely unreproducible, which is one thing flatpak-builder tries very hard to enable.
For example, when using a local file source, flatpak-builder will base64-econde the content of that file and use it as a data:text/plain;charset=utf8;base64,<content> URL for the file which it stores in the manifest included inside the final build.
Something similar might be needed for a dir source (tar the folder then base64-encode the content of the tar?), otherwise it would be impossible to reproduce the build. I've just been told (after submitting this answer) that this changed in Git master, in favour of a new flatpak-builder --bundle-sources option. This would probably make it easier to support reproducible builds with a dir source type.
In any case, feel free to start the conversation around a new dir source type in the upstream bug tracker. :)
There's a expermental cli tool if you want to use it https://gitlab.com/csoriano/flatpak-dev-cli
You can read the docs
http://docs.flatpak.org/en/latest/building-simple-apps.html
http://docs.flatpak.org/en/latest/flatpak-builder.html
In a nutshell this is what you need to use flatpak as develop workbench
https://github.com/albfan/gnome-builder/wiki/flatpak
I'm a beginner of Yocto project.
So, I'm really hope to know how to build *.bb files which I added.
I am added a .bb file (dlt-daemon) to meta-/meta-*/recipes-expends/dlt-daemon/dlt-daemon_v2.14.1.bb.
However, whenever I try to build it (bitbake core-image-weston) it isn't built.
I tried to build a *.bb file only ( bitbake -b ******/*.bb -c compile ) but there is no output file in the rootfs. ( I found the output files at build/tmp/work/arch****/dlt-daemon/2.14.1-r0/build/***** )
I'm not sure why it doesn't work?
Please, can I know how to build *.bb files which I added?
Preferably, you should add your own recipes in your own layer.
But nevertheless, just adding a recipe (ie .bb-file) won't add it to any rootfs. If you can run
bitbake your-recipe
without getting any errors, your recipe is working as it should (there could still be some issues if you're not installing any files etc). You can confirm that it's working by either looking at the logs for the different tasks (in ${WORKDIR}/<arch>/recipe-name/recipe-version/temp/).
Still being able to build your recipe isn't enough for what you want. For the application in question to appear in you rootfs, you need to add it to your image. Temporarily, you can add the following line to your conf/local.conf:
IMAGE_INSTALL_append = " <package-name>"
Note the leading space. To make it permanent, you should add the <package-name> to IMAGE_INSTALL directly in you image recipe.
Open your local.conf file and add below line e.g: hello.bb
IMAGE_INSTALL_append = " hello" # "space" before hello. this will add to your rfs image
Then compile your rfs using bitbake core-image-minimal
I would like to edit an existing software to add a new source file (Source.cpp).
But, I can't manage the compilation process (it seems to be automake and it looks very complicated).
The software (iperf 2: https://sourceforge.net/projects/iperf2/files/?source=navbar) is compiled using a classical ./configure make then make install.
If I just add the file to the corresponding source and include directory, I got this error message:
Settings.cpp:(.text+0x969) : undefined reference to ...
It looks like the makefile isn't able to produce the output file associated with my new source file (Source.cpp). So, I probably need to indicate it manually somewhere.
I searched a bit in the project files and it seemed that the file to edit was: "Makefile.am".
I added my source to the variable iperf_SOURCES in that file but it didn't workded.
Could you help me to find the file where I need to indicate my new source file (it seems a pretty standard compilation scheme but I never used automake softwares and this one seems very complicated).
Thank you in advance
This project is built with the autotools, as you already figured out.
The makefiles are built by automake. It takes its input in files that usually have a am file name extension.
The iperf program is built by the makefile generated from src/Makefile.am. This is indicated by:
bin_PROGRAMS = iperf
All (actually this is a simplification, but which holds in this case) source files of a to be built binary are in the corresponding name_SOURCES variable, thus in this case iperf_SOURCES. Just add your source file to the end of that list, like so (keeping their formatting):
iperf_SOURCES = \
Client.cpp \
# lines omitted
tcp_window_size.c \
my_new_file.c
Now, to reflect this change in any future generated src/Makefile you need to run automake. This will modify src/Makefile.in, which is a template that is used by config.sub at the end of configure to generate the actual makefile.
Running automake can happen in various ways:
If you already have makefiles that were generated after an configure these should take care of rebuilding themselves. This seems to fail sometimes though!
You could run automake (in the top level directory) by hand. I've never done this, as there is the better solution to...
Run autoreconf --install (possibly add --force to the arguments) in the top level directory. This will regenerate the entire build system, calling all needed programs such as autoheader, autoconf and of course automake. This is my favorite solution.
The later two options require calling configure again, IMO ideally doing an out of source built:
# in top level dir
mkdir build
cd build
../configure # arguments
make # should now also compile and link your new source file
I have a project which is using yocto for building libraries including gstreamer. I found out that I need to patch some gstreamer element thus creating new bitbake recipe with patch..
I usually have to run bitbake with image name as parameter which will rebuild whole yocto (which is quite long):
MACHINE=some_machine nice bitbake yocto-etc-etc
How do I rebuild just that part which I need and not whole yocto?
I heard about devtool, but I am not sure how to use that.
you can pass different command to bitbake based on what you need.
To remove temp:
bitbake -c clean gstreamer
To remove temp and sstate cache (I use this most):
bitbake -c cleansstate gstreamer
To remove download as well, and lets begin build starting from do_fetch and all
bitbake -c cleanall gstreamer
Once you are done with either of these clean, which ever suits you, you can simple give build command for the specified:
bitbake gstreamer
Certainly, this is easy to do. Just specify the recipe you want to build instead of the image name, for example if it was the main gstreamer recipe you had changed (which at least in current versions is called gstreamer1.0):
MACHINE=some-machine bitbake gstreamer1.0
Note that the name expected on the command line is always a recipe name or something from PROVIDES in a recipe, and not a runtime package name.
Regarding devtool, it can certainly put you into an environment where you can more easily make changes to the source for a recipe and generate patches from them, but the actual building part we are discussing here doesn't really change. You can find more information on how to use devtool in the Yocto Project Development Manual
You can also
clean: Removes all output files for a target
cleanall: Removes all output files, shared state cache, and downloaded source files for a target, depending on the changes
bitbake -c clean task
bitbake -c cleanall task
First you can create a patch on the gstreamer using quilt or diff etc...
Put the patch into your meta layer and include it into,SRC_URI += "file://xxxx.patch".
Make sure you have added FILESEXTRAPATHS_PREPEND variable in the bbappend file of recipe.
Then do a cleansstate of the package.
bitabake gstreamer** -c cleansstate
Then execute do_patch operation and check our patch has been applied properly.
bitabake gstreamer*** -c patch
Then do the full build of the component followed by built the final target.
You can also launch the taskes you are interested in, for example:
If you want to apply only the patch you can do something like:
# Apply the patch you have located and sourced in SRC_URI variable previously
MACHINE=some_machine nice bitbake -c patch gstreamer
# Compile the recipe
MACHINE=some_machine nice bitbake -c compile gstreamer
# In case there are more necessary tasks, launch them as previous
Now you can get the generated package, and pass it to your board (eg. via ssh/serial(zmodem) ), test it and repeat until you like the resul, then regenerate the image doing:
for i in clean cleanall cleansstate;do bitbake -c ${i} gstreamer;done
MACHINE=some_machine nice bitbake yocto-etc-etc
you can build any specific recipe by providing the recipe name along with the bitbake command
For instance, If you want to build gstreamer
poky/meta/recipes-multimedia/gstreamer1.0_1.16.3.bb
you can use the following command
MACHINE=<your-machine-name> bitbake gstreamer1.0
Note that the PROVIDES value will be parsed from .bb filename excluding the characters after underscore.
Additional suggestions
If you want to do some experimental changes in your source and wants to compile for each minimalistic change, you could do that by navigating to
work directory
cd build/tmp/work/armv5e-poky-linux-gnueabi/gstreamer1.0/1.16.3-r0/
here you can apply your changes in src directory and you can use ./temp/run.do_compile to compile, which will take very less time compared to the entire build time.
I have two question
I have done some changes in C-backend of LLVM-2.9 source code. Now what I wanna is how can I reflect these changes in build folder?
What does make update do ? When I run this command this is what happen in my terminal.
arpit#arpit-HP-dx2480-MT-KL969AV:~/llvm1/build$ sudo make update
svn update /home/arpit/llvm1/llvm
Skipped '/home/arpit/llvm1/llvm'
svn: warning: '/home/arpit/llvm1/llvm' is not a working copy
Skipped '.'
make update updates LLVM and Clang and other sub-projects, so it's not what you need. What you do need is just make in the build folder. If any of the source files changed, make is supposed to pick it up and re-build the relevant files.
Maybe the specific changes you made are not picked up by make, have you added a new file? You can always try to build everything anew (e.g. make -B) and see if the change is included.