How to execute Rake file tasks in parallel - build

I'm using Rake to build a C language build system.
When compiling multiple files,
I want to compile multiple files in parallel, using multiple cores of the CPU.
Can you give me some advice on how to write a Rake file?
I would like to achieve the make -j in Rake.
As a restriction, I prefer not to install a new Gem.
For clarity, here is a simplified Rake file.
CC = "gcc"
task :default => "hello"
file "hello" => ["hello.o", "message.o"] do
sh "#{CC} -o hello hello.o message.o"
end
file "hello.o" => "hello.c" do (1)
sh "#{CC} -c hello.c"
end
file "message.o" => "message.c" do (2)
sh "#{CC} -c message.c"
end
For tasks, I can use multitask.
However, for file tasks, I don't know how to describe it.
I would like to run the file tasks (1) and (2) concurrently.
my environment:
ruby 2.6.4p104 (2019-08-28 revision 67798) [i386-cygwin]
I thank you in advance.

You can create threads and you can run your files in new thread.
For example
CC = "gcc"
task :default => "hello"
file "hello" => ["hello.o", "message.o"] do
Thread.new do
sh "#{CC} -o hello hello.o message.o"
end
end
file "hello.o" => "hello.c" do
Thread.new do
sh "#{CC} -c hello.c"
end
end
file "message.o" => "message.c" do
Thread.new do
sh "#{CC} -c message.c"
end
end

I found out that rake -m considers all tasks as multitasking. I defined individual tasks only for the tasks that I do not want to be executed in parallel. By specifying something like "rake -m -j 8", I confirmed that the build time was reduced.

Related

How do you run and compile C++ code on alpine linux?

I’m new to alpine linux, and I’ve been trying to use it to run and compile C++ code, but I can’t get it to run the code.
First, I put the code in the opt folder, than I did this:
cd / → cd opt → clang++ -S -emit-llvm myFile.cpp
That created myFile in the opt folder, then I tried to use
sh myFile
To run the code, but it said “no such file or directory found,” If I do sh myFile.cpp, I get a few errors.
What am I doing wrong here?
Some of the packages I have installed are build-base, and clang.
By default, the result of clang (clang++ as well) is saved to a.out file, so you may run ./a.out or sh a.out. If you want to change output, use -o <filename> in clang command

CentOS7: rpmbuild - Unable to recognise the format of the input file

I'm trying to build an extremely simple rpm over centos7.
I just copy some pre-compiled executables from the tar.gz to /usr/bin/my_rpms/rpm1.
Here is my install section:
%install
mkdir -p %{buildroot}/usr/bin/my_rpms/rpm1/
install -D prog prog.o -t %{buildroot}/usr/bin/my_rpms/rpm1/
it used to work find for the most part.
but today when after i made some changes to the prog and re-compiled it keeps gettings these errors:
+ mkdir -p /root/rpmbuild/BUILDROOT/rpm1.x86_64/usr/bin/my_rpms/rpm1/
+ install -D prog prog.o -t /root/rpmbuild/BUILDROOT/rpm1.x86_64/usr/bin/my_rpms/rpm1/
+ /usr/lib/rpm/check-buildroot
+ /usr/lib/rpm/redhat/brp-compress
+ /usr/lib/rpm/redhat/brp-strip /usr/bin/strip
/usr/bin/strip: Unable to recognise the format of the input file `/root/rpmbuild/BUILDROOT/rpm1.x86_64/usr/bin/drivertest_rpms/rpm1/prog.o'
As you can see in error log, problem is with binary file striping which is default behavior of install command. I think your build environment is maybe different then rpm environment. cross-compiling? as suggested by #aaron-d-marasco
So I recommend to build rpm from project source. i.e move your build commands into %build section of .spec file.
Or strip your files in the same place where you have build them, and then in rpm use cp command in %install section instead of install command to move your files to target directory.

ocamlbuild links libraries in wrong order

I'm somehow trying to use _CoqProject parser from coq's library in OCaml (I'd welcome better alternatives to grab the .v files of a coq project if that library is not meant for external use, is it?), but ocamlbuild seems to be linking libraries in the wrong order.
Consider this minimal example file
open CoqProject_file
let x = read_project_file
The coq.lib package (bundled with coq) somehow depends on threads, and following this answer suggests to use -tag thread for that, but I still get the following error that threads is not found when linking coq.lib:
$ ocamlbuild -pkg coq.lib -tag thread -cflag -rectypes a.native /tmp/p
+ /home/sam/.opam/4.06.0+coq-8.7/bin/ocamlopt.opt -I /home/sam/.opam/4.06.0+coq-8.7/lib/coq/config -I /home/sam/.opam/4.06.0+coq-8.7/lib/coq/lib -I /home/sam/.opam/4.06.0+coq-8.7/lib/ocaml /home/sam/.opam/4.06.0+coq-8.7/lib/ocaml/str.cmxa /home/sam/.opam/4.06.0+coq-8.7/lib/ocaml/unix.cmxa /home/sam/.opam/4.06.0+coq-8.7/lib/coq/lib/clib.cmxa -thread threads.cmxa a.cmx -o a.native
File "_none_", line 1:
Error: No implementations provided for the following modules:
Thread referenced from /home/sam/.opam/4.06.0+coq-8.7/lib/coq/lib/clib.cmxa(Exninfo)
Mutex referenced from /home/sam/.opam/4.06.0+coq-8.7/lib/coq/lib/clib.cmxa(Exninfo)
Command exited with code 2.
Yet that compiles if I take the ocamlopt invocation apart and put -thread threads.cmxa before clib.cmxa
$ cd _build/
$ /home/sam/.opam/4.06.0+coq-8.7/bin/ocamlopt.opt -I /home/sam/.opam/4.06.0+coq-8.7/lib/coq/config -I /home/sam/.opam/4.06.0+coq-8.7/lib/coq/lib -I /home/sam/.opam/4.06.0+coq-8.7/lib/ocaml /home/sam/.opam/4.06.0+coq-8.7/lib/ocaml/str.cmxa /home/sam/.opam/4.06.0+coq-8.7/lib/ocaml/unix.cmxa -thread threads.cmxa /home/sam/.opam/4.06.0+coq-8.7/lib/coq/lib/clib.cmxa a.cmx -o a.native
What is the right way to call ocamlbuild?
If you use ocamlfind packages, you should use the -use-ocamlfind flag.
There is no good solution as to why -tag thread is needed¹. There are two different implementations of the OCaml Threads interface (one with os threads and one with green threads), and coq.lib depends on the interface but won't decide for the user which one to use, so you have to specify it manually, for example by using -tag thread.
¹: one solution would be to remove this choice by deprecating vmthreads (the green threads), which is rarely used in practice.

go unit test runs from %APPDATA%

I am trying to run some of my Go unit tests using "go test" but the test executable is built and run from my machine's %APPDATA%/local/temp directory. My PC has IT enforcement which blocks any unrecognized executable from being run other than from a pre-sanctioned directory (i.e C:/dev/projects"). All my Go source code are in that directory, including my *_test.go files. Is there a way to tell the Go test module to build and run from the current directory?
Yes you can.
Setting temp directory before executing the go test. By default temp directory environment variable gets evaluated in the order of TMP, TEMP, USERPROFILE, Windows directory; refer to msdn doc.
Basically it complies the go test under given temp directory and execute it.
C:\> cd dev\projects\src\mygotest
C:\dev\projects\src\mygotest>echo %CD%
C:\dev\projects\src\mygotest
C:\dev\projects\src\mygotest>set TMP=%CD%
C:\dev\projects\src\mygotest>go test -x
WORK=C:\dev\projects\src\mygotest\go-build306298926
mkdir -p $WORK\mygotest\_test\
mkdir -p $WORK\mygotest\_test\_obj_test\
cd C:\dev\projects\src\mygotest
"C:\\Go\\pkg\\tool\\windows_amd64\\compile.exe" -o "C:\\dev\\projects\\src\\mygotest\\go-build306298926\\mygotest\\_test\\mygotest.a" -trimpath "C:\\dev\\projects\\src\\mygotest\\go-build306298926" -p main -complete -buildid 86cb7a423d355c7468ad98c4f8bffe77b68d2265 -D _/C_/dev/projects/src/mygotest -I "C:\\dev\\projects\\src\\mygotest\\go-build306298926" -pack "C:\\dev\\projects\\src\\mygotest\\sample.go" "C:\\dev\\projects\\src\\mygotest\\sample_test.go"
cd $WORK\mygotest\_test
"C:\\Go\\pkg\\tool\\windows_amd64\\compile.exe" -o "C:\\dev\\projects\\src\\mygotest\\go-build306298926\\mygotest\\_test\\main.a" -trimpath "C:\\dev\\projects\\src\\mygotest\\go-build306298926" -p main -complete -D "" -I "C:\\dev\\projects\\src\\mygotest\\go-build306298926\\mygotest\\_test" -I "C:\\dev\\projects\\src\\mygotest\\go-build306298926" -pack "C:\\dev\\projects\\src\\mygotest\\go-build306298926\\mygotest\\_test\\_testmain.go"
cd .
"C:\\Go\\pkg\\tool\\windows_amd64\\link.exe" -o "C:\\dev\\projects\\src\\mygotest\\go-build306298926\\mygotest\\_test\\mygotest.test.exe" -L "C:\\dev\\projects\\src\\mygotest\\go-build306298926\\mygotest\\_test" -L "C:\\dev\\projects\\src\\mygotest\\go-build306298926" -w -extld=gcc -buildmode=exe "C:\\dev\\projects\\src\\mygotest\\go-build306298926\\mygotest\\_test\\main.a"
$WORK\mygotest\_test\mygotest.test.exe
Hello, playground
PASS
ok mygotest 0.526s
C:\dev\projects\src\mygotest>
Note: TMP set to current terminal session only, it doesn't affect system environment variable.
Important thing to note from above test output is WORK=C:\dev\projects\src\mygotest\go-build306298926.
Happy testing!

Separate compilation of OCaml modules

I have read this question and others,
but my compile problem is unsolved.
I am testing separate compilation with these files:
testmoda.ml
module Testmoda = struct
let greeter () = print_endline "greetings from module a"
end
testmodb.ml
module Testmodb = struct
let dogreet () = print_endline "Modul B:"; Testmoda.greeter ()
end
testmod.ml
let main () =
print_endline "Calling modules now...";
Testmoda.greeter ();
Testmodb.dogreet ();
print_endline "End."
;;
let _ = main ()
Now I generate the .mli file
ocamlc -c -i testmoda.ml >testmoda.mli
and the testmoda.cmi is there.
Next I create the .cmo file without errors:
ocamlc -c testmoda.ml
Fine, so do the same with testmodb.ml:
strobel#s131-amd:~/Ocaml/ml/testmod> ocamlc -c -i testmodb.ml >testmodb.mli
File "testmodb.ml", line 3, characters 45-61:
Error: Unbound value Testmoda.greeter
Another try:
strobel#s131-amd:~/Ocaml/ml/testmod> ocamlc -c testmoda.cmo testmodb.ml
File "testmodb.ml", line 3, characters 45-61:
Error: Unbound value Testmoda.greeter
Other combinations failed as well.
How do I compile testmodb.ml and testmod.ml? This should be easy - without ocamlbuild / omake /
oasis, I think.
Syntax errors in the files are excluded,
if I cat them together to one file (with the required space between) it compiles
and executes perfectly.
OCaml gives you a module for free at the top level of each source file. So your first module is actually named Testmoda.Testmoda, the function is named Testmoda.Testmoda.greeter, and so on. Things will work better if your files just contain the function definitions.
As a side comment, if you're going to use the interface generated by ocamlc -i, you really don't need mli files. The interface in the absence of an mli file is the same as the one generated by ocamlc -i. If you don't want the default interface, using ocamlc -i gives a good starting point for your mli file. But for a simple example like this, it just makes things look a lot more complicated than they really are (IMHO).
If you modify your files as I describe (remove extra module declarations), you can compile and run from scratch as follows:
$ ls
testmod.ml testmoda.ml testmodb.ml
$ cat testmoda.ml
let greeter () = print_endline "greetings from module a"
$ cat testmodb.ml
let dogreet () = print_endline "Modul B:"; Testmoda.greeter ()
$ ocamlc -o testmod testmoda.ml testmodb.ml testmod.ml
$ ./testmod
Calling modules now...
greetings from module a
Modul B:
greetings from module a
End.
If you have already compiled a file (with ocamlc -c file.ml) you can replace .ml with .cmo in the above command. This works even if all the filenames are .cmo files; in that case ocamlc just links them together for you.