How to create a qmake function that creates a custom make target? - c++

I know that we have QMAKE_EXTRA_TARGETS to create new makefile targets, that is used as follows (as seen in http://blog.qt.io/blog/2008/04/16/the-power-of-qmake/):
conv.target=convert
conv.input=file.in
conv.output=file.out
conv.commands=convert.sh file.in file.out
QMAKE_EXTRA_TARGETS+=conv
In my case, convert.sh is used for multiple files and targets. I would like to create a method with arguments (target_name, input_file, output_file), that creates the task for me, so that I don't have to repeat the above lines.
The documentation on qmake is quite lacking, or I haven't found the correct source, but to my understanding, there are two types of functions in qmake: replace and test (http://doc.qt.io/qt-5/qmake-language.html#replace-functions) and we can create custom ones using defineReplace and defineTest.
I have tried:
defineTest(createConvertTask) {
custom.target = $$1
custom.input = $$2
custom.output = $$3
custom.commands = convert.sh $$2 > $$3
QMAKE_EXTRA_TARGETS += custom
}
but that doesn't really work, as after calling createConvertTask multiple times, QMAKE_EXTRA_TARGETS will just contain multiple copies of the string custom.
However, this
defineTest(createConvertTask) {
$$1.target = $$1
$$1.input = $$2
$$1.output = $$3
$$1.commands = convert.sh $$2 > $$3
QMAKE_EXTRA_TARGETS += $$1
}
fails with error example.pro:2: error: Left hand side of assignment must expand to exactly one word.
Any ideas on how to approach this?

1.Option: custom compiler
Use a custom compiler like this:
convert.input = LIST_OF_IN_FILES # note: no $$
convert.output = $${SOME_DIR}/${QMAKE_FILE_BASE}.ext
convert.commands = convert.sh ${QMAKE_FILE_IN} > $${SOME_DIR}/${QMAKE_FILE_BASE}.ext
convert.CONFIG += no_link target_predeps
QMAKE_EXTRA_COMPILERS += convert
The variables ${QMAKE_FILE_IN} contains the current input file, same as ${QMAKE_FILE_BASE} but without extension. Here, the output filenames are generated out of the input files. The CONFIG options tell qmake not to add the output files to the objects list and to add them as prerequisites of the main target. Additionally a make target compiler_convert_make_all will be generated.
Just add files:
LIST_OF_IN_FILES += file1 file2
and make
make compiler_convert_make_all
This option also adds all output files to the clean target (will be deleted on make clean.
2.Option: use eval() and export()
To use variables as left hand expression you can use the eval() function, that 'Evaluates the contents of the string using qmake syntax rules'.
eval($${1}.target = $$1)
Since this is done inside a function you need to export() all variables to the global scope.
eval(export($${1}.target))
Afterwards add target and export QMAKE_EXTRA_TARGETS as well:
QMAKE_EXTRA_TARGETS += $${1}
export(QMAKE_EXTRA_TARGETS)
Complete with a replace function, the return value will be added to the dependencies of the custom convert target:
convert.target = convert
defineReplace(createConvertTask) {
eval($${2}_custom.target = $$2)
eval($${2}_custom.depends = $$1)
eval($${2}_custom.commands = convert.sh $$1 > $$2)
eval(export($${2}_custom.target))
eval(export($${2}_custom.depends))
eval(export($${2}_custom.commands))
QMAKE_EXTRA_TARGETS += $${2}_custom
export(QMAKE_EXTRA_TARGETS)
return($${2}_custom)
}
convert.depends += $$createConvertTask(in_file_1, out_file_1)
convert.depends += $$createConvertTask(in_file_2, out_file_2)
QMAKE_EXTRA_TARGETS += convert
The results in the generated Makefile:
out_file_1: in_file_1
convert.sh in_file_1 > out_file_1
out_file_2: in_file_2
convert.sh in_file_2 > out_file_2
convert: out_file_1 out_file_2
This approach is more flexible and can be extended to support the a variable target parameter (here constant convert).

Related

Bitbake does not include my recipe in build

I am trying to do something straightforward: add a new layer and a new recipe. I used the
bitbake-layers create-layer
command to create the layer, and added the layer directory path to BBLAYERS variable in BUILDDIR/conf/bblayers.conf.
layer.conf
BBPATH .= ":${LAYERDIR}"
BBFILES += "${LAYERDIR}/recipes-*/*/*.bb \
${LAYERDIR}/recipes-*/*/*.bbappend"
BBFILE_COLLECTIONS += "myname-mytest"
BBFILE_PATTERN_myname-mytest = "^${LAYERDIR}/"
BBFILE_PRIORITY_myname-mytest = "6"
LAYERVERSION_myname-mytest = "1"
LAYERSERIES_COMPAT_myname-mytest = "sumo"
Added this in local.conf:
local.conf
IMAGE_INSTALL_APPEND = " mytest-app"
bitbake-layers show-recipes
shows my layer and recipe.
mytest-app:
meta-myname-mytest 1.0
Errors in my recipe are caught in the bitbake build, but without any error, no output is produced under WORKDIR/image or logs are produced under WORKDIR/temp!
Have done this on other platforms, and can't for the life of me tell what I am doing wrong. Thanks for any help!!
It's IMAGE_INSTALL_append, not IMAGE_INSTALL_APPEND. c.f. https://docs.yoctoproject.org/bitbake/bitbake-user-manual/bitbake-user-manual-metadata.html#appending-and-prepending-override-style-syntax
For the new yocto version, you need to use
IMAGE_INSTALL:append
instead of
IMAGE_INSTALL_append
and you will be OK

accessing recources when compiling QT application with Bazel

I'm using Bazel to compile a Qt application (https://github.com/bbreslauer/qt-bazel-example) that is using shaders defined in a qrc file.
When I'm trying to access the resource file, it is not available (as I did not connect the qrc file to the compilation).
How can I define the qrc file content in the build?
UPDATE
following the response by #ypnos, I'm trying to add a macro to my qt.bzl file. I would like the macro to recieve a list of files as an argument, create the (temporary) qrc file, and run the rcc command.
I am currently struggling with:
running a python script in the bzl file is not as straightforward as I though. It cannot generate a file ("open" is undefined). Is it possible? if yes how (see example below)
even with a given qrc file, I cant get the command to work, I guess i'm doing somthing wrong with the command line arguments but I cant find refrence/manual for that
this is what I got so far(my qt.bzl file)
...
def qt_resource(name,file_list, **kwargs):
## following doesnt work inside the bzl file:
# fid = open('%s.qrc' % name, 'w')
# fid.write("<RCC>\n")
# fid.write("\t<qresource prefix=\"/%s\">\n" % name)
# for x in file_list:
# fid.write("\t\t<file>%s</file>\n" % x)
# fid.write("\t</qresource>\n")
# fid.write("</RCC>\n")
# fid.close()
native.genrule(
name = "%s_res" % name,
outs = ["rcc_%s.cpp" % name],
cmd = "rcc %s.qrc -o $#/rcc_%s.cpp"%(name,name) ,
)
srcs = [":rcc_%s.cpp" % name]
native.cc_library(
name = name,
srcs = srcs,
hdrs = [],
deps = [],
**kwargs
)
It seems the bazel example that you are using does not come with support for qrc (it only does moc and ui files).1
QRC files need to be transformed into C++ sources using rcc and then compiled.2 The concept is similar to the one of .ui files which are converted to headers.
Maybe you can patch qt.bzl to add that functionality.

How to integrate pretty-printing as part of build in bazel

Right now, I have a really dumb pretty-print script which does a little git-fu to find files to format (unconditionally) and then runs those through clang-format -i. This approach has several shortcomings:
There are certain files which are enormous and take forever to pretty print.
The pretty printing is always done, regardless of whether or not the underlying file actually changed or not.
In the past, I was able to do things with CMake that had several nice properties which I would like to reproduce in bazel:
Only ever build code after it has gone through linting / pretty printing / etc.
Only lint / pretty print / etc. stuff that has changed
Pretty print stuff regardless of whether or not it is under VC or not
In CMake-land, I used this strategy, inspired by SCons proxy-target trickery:
Introduce a dummy target (e.g. source -> source.formatted). The action associated with this target does two things: a) run clang-format -i source, b) output/touch a file called source.formatted (this guarantees that for reasonable file systems, if source.formatted is newer than source, source doesn't need to be reformatted)
Add a dummy target (target_name.aggregated_formatted) which aggregates all the .formatted files corresponding to a particular library / executable target's sources
Make library / executable targets depend on target_name.aggregated_formatted as a pre-build step
Any help would be greatly appreciated.
#abergmeier is right. Let's take it one step further by implementing the macro and its components.
We'll use the C++ stage 1 tutorial in bazelbuild/examples.
Let's first mess up hello-world.cc:
#include <ctime>
#include <string>
#include <iostream>
std::string get_greet(const std::string& who) {
return "Hello " + who;
}
void print_localtime() {
std::time_t result =
std::time(nullptr);
std::cout << std::asctime(std::localtime(&result));
}
int main(int argc, char** argv) {
std::string who = "world";
if (argc > 1) {who = argv[1];}
std::cout << get_greet(who) << std::endl;
print_localtime();
return 0;
}
This is the BUILD file:
cc_binary(
name = "hello-world",
srcs = ["hello-world.cc"],
)
Since cc_binary doesn't know anything about clang-format or linting in general, let's create a macro called clang_formatted_cc_binary and replace cc_binary with it. The BUILD file now looks like this:
load(":clang_format.bzl", "clang_formatted_cc_binary")
clang_formatted_cc_binary(
name = "hello-world",
srcs = ["hello-world.cc"],
)
Next, create a file called clang_format.bzl with a macro named clang_formatted_cc_binary that's just a wrapper around native.cc_binary:
# In clang_format.bzl
def clang_formatted_cc_binary(**kwargs):
native.cc_binary(**kwargs)
At this point, you can build the cc_binary target, but it's not running clang-format yet. We'll need to add an intermediary rule to do that in clang_formatted_cc_binary which we'll call clang_format_srcs:
def clang_formatted_cc_binary(name, srcs, **kwargs):
# Using a filegroup for code cleaniness
native.filegroup(
name = name + "_unformatted_srcs",
srcs = srcs,
)
clang_format_srcs(
name = name + "_formatted_srcs",
srcs = [name + "_unformatted_srcs"],
)
native.cc_binary(
name = name,
srcs = [name + "_formatted_srcs"],
**kwargs
)
Note that we have replaced the native.cc_binary's sources with the formatted files, but kept the name to allow for in-place replacements of cc_binary -> clang_formatted_cc_binary in BUILD files.
Finally, we'll write the implementation of the clang_format_srcs rule, in the same clang_format.bzl file:
def _clang_format_srcs_impl(ctx):
formatted_files = []
for unformatted_file in ctx.files.srcs:
formatted_file = ctx.actions.declare_file("formatted_" + unformatted_file.basename)
formatted_files += [formatted_file]
ctx.actions.run_shell(
inputs = [unformatted_file],
outputs = [formatted_file],
progress_message = "Running clang-format on %s" % unformatted_file.short_path,
command = "clang-format %s > %s" % (unformatted_file.path, formatted_file.path),
)
return struct(files = depset(formatted_files))
clang_format_srcs = rule(
attrs = {
"srcs": attr.label_list(allow_files = True),
},
implementation = _clang_format_srcs_impl,
)
This rule goes through every file in the target's srcs attribute, declaring a "dummy" output file with the formatted_ prefix, and running clang-format on the unformatted file to produce the dummy output.
Now if you run bazel build :hello-world, Bazel will run the actions in clang_format_srcs before running the cc_binary compilation actions on the formatted files. We can prove this by running bazel build with the --subcommands flag:
$ bazel build //main:hello-world --subcommands
..
SUBCOMMAND: # //main:hello-world_formatted_srcs [action 'Running clang-format on main/hello-world.cc']
..
SUBCOMMAND: # //main:hello-world [action 'Compiling main/formatted_hello-world.cc']
..
SUBCOMMAND: # //main:hello-world [action 'Linking main/hello-world']
..
Looking at the contents of formatted_hello-world.cc, looks like clang-format did its job:
#include <ctime>
#include <string>
#include <iostream>
std::string get_greet(const std::string& who) { return "Hello " + who; }
void print_localtime() {
std::time_t result = std::time(nullptr);
std::cout << std::asctime(std::localtime(&result));
}
int main(int argc, char** argv) {
std::string who = "world";
if (argc > 1) {
who = argv[1];
}
std::cout << get_greet(who) << std::endl;
print_localtime();
return 0;
}
If all you want are the formatted sources without compiling them, you can run build the target with the _formatted_srcs suffix from clang_format_srcs directly:
$ bazel build //main:hello-world_formatted_srcs
INFO: Analysed target //main:hello-world_formatted_srcs (0 packages loaded).
INFO: Found 1 target...
Target //main:hello-world_formatted_srcs up-to-date:
bazel-bin/main/formatted_hello-world.cc
INFO: Elapsed time: 0.247s, Critical Path: 0.00s
INFO: 0 processes.
INFO: Build completed successfully, 1 total action
You might be able to use aspects for that. Being not certain, a Bazel-dev will probably point that out if it indeed is possible.
If you are familiar with Rules and Actions and the like, the quick and dirty way (which is similar to the CMake hackery) is to write a Macro. For e.g. cc_library you would do:
def clean_cc_library(name, srcs, **kwargs):
lint_sources(
name = "%s_linted" % name,
srcs = srcs,
)
pretty_print_sources(
name = "%s_pretty" % name,
srcs = ["%s_linted"],
)
return native.cc_library(
name = name,
srcs = ["%s_pretty"],
**kwargs
)
Then you of course need to replace every cc_library with clean_cc_library. And lint_sources and pretty_print_sources are rules that you have to implement yourself and need to produce the list of cleaned up files.
#abergmeier mentions maybe being able to use Aspects. You can, and I've made a prototype of a general linting system that leverages Aspects functionality so that BUILD files do not need to be modified to use Macros like clang_formatted_cc_library in-place of the core rules.
The basic idea is to have a bazel build step that is a pure function f(linter, sources) -> linted_sources_diff and a subsequent bazel run step that takes those diffs and applies them back to your source code to fix lint errors.
The prototype implementation is available at https://github.com/thundergolfer/bazel-linting-system.

Makefile conditional statements

Scenario :
Consider a source directory which has multiple ".cpp" files that creates a static library
consider files: XYZ.cpp & ABC.cpp (used specificly based on condition described below ) as well as PQR.cpp, JKL.cpp etc.., output library name is out.a
Here p ( is an environment variable) whose value if matches to q then,
out.a should be created using XYZ.cpp else it should be created using ABC.cpp
Ex: i.e. something like this
ifeq($p, q)
SRC = XYZ.cpp
else
SRC = ABC.cpp
endif
SRC += PQR.cpp \
JKL.cpp \
MNO.cpp
How could I do the same optimizely in Makefile ?
Thanks in advance for any help...
That's almost exactly it. You just need a space after ifeq, (and some parentheses around p, in case you want to use a variable name longer than one letter):
ifeq ($(p), q)
SRC = XYZ.cpp
else
SRC = ABC.cpp
endif
SRC += PQR.cpp \
JKL.cpp \
MNO.cpp
Although Beta's answer is quite correct, you might alternatively consider constructed variable names. In my opinion, they lead to cleaner and more readable makefiles. For example:
# if p is not set, default to "default"
p ?= default
q_SRC = XYZ.cpp
default_SRC = ABC.cpp
SRC = $($(p)_SRC) PQR.cpp JKL.cpp MNO.cpp
etc. Especially if you have lots of alternatives this can be much more understandable (again, IMO).

How do I deal with output files when using argparse?

I have a program and I want to make it so that if the user specifies the exact name of the 2 output files given, then those files will be named at the user's preference.
For ex:
-o file1.txt file2.txt
If the output files aren't specified then the script will automatically generate the files with default names.
You can use the argparse module, which provides clean ways of handling input arguments. For your task, you can use something like
arguments = argparse.ArgumentParser()
arguments.add_argument('fileNames', nargs='*', help='Output file names', default = ['val1', 'val2'])
inputArgs = arguments.parse_args()
# User may decide to give just one file name
outFileName1 = inputArgs.fileNames[0]
outFileName2 = 'val2' if len(inputArgs.fileNames) == 1 else inputArgs.fileNames[1]