Any creative idea how to use enviroment variables to calculate full build time of a makefile? Can I store the time somewhere and compare it in the end?
I know the OP meant Windows OS, but for the record, on any Linux machine you can use time:
>time make -j
real 0m9.774s
user 0m31.562s
sys 0m1.092s
On Windows 7 you've got PowerShell. So we'll assume:
Your makefile Makefile is in directory C:\Dev\ProjDir
Your make tool is mingw32-make and is in your PATH
If PowerShell isn't your usual console then do:
C:\Dev\ProjDir>powershell
At the PowerShell prompt, run:
PS C:\Dev\ProjDir> Measure-Command { mingw32-make | Out-Default }
This will run your build and display the output, followed by
a timing report, e.g.
Days : 0
Hours : 0
Minutes : 0
Seconds : 0
Milliseconds : 244
Ticks : 2443841
TotalDays : 2.82851967592593E-06
TotalHours : 6.78844722222222E-05
TotalMinutes : 0.00407306833333333
TotalSeconds : 0.2443841
TotalMilliseconds : 244.3841
Continued for OP's follow-up:
I was wondering if it is possible to be implemented as part of the makefile,
so I can decide which part to measure
The idea of timing "part of a makefile" does not make clear sense. When you run make, it parses
the entire makefile before it makes anything and works out the entire sequence of
steps it must take to make the specified or default target(s). Then, it takes all those
steps.
Perhaps you want to measure the time taken to make a particular set of targets? You
can do that in the way already described. For example, suppose that your makefile
can make two libraries libfoo.lib and libbar.lib and you would like to time the
build of just libfoo.lib. To make libfoo.lib by itself, the make command you would
run is:
mingw32-make libfoo.lib
So to time this command, run:
PS C:\Dev\ProjDir> Measure-Command { mingw32-make libfoo.lib | Out-Default }
Or suppose that your makefile makes app.exe from source files foo.c and bar.c, and
you would like to time just the build of the object files foo.obj, bar.obj. The
make command you would run to build just those object files is:
mingw32-make foo.obj bar.obj
So you would time it with:
PS C:\Dev\ProjDir> Measure-Command { mingw32-make foo.obj bar.obj | Out-Default }
Perhaps you would like to be able to invoke powershell's Measure-Command inside your makefile
to time the building of particular targets?
For this, you need a command that invokes PowerShell to run some other command. That is:
powershell -c "some other command"
So in your makefile you can add a target for timing the build of any other targets:
.phony: time
time:
powershell -c "Measure-Command { $(MAKE) $(targets) | Out-Default }"
You would use the time target like so:
C:\Dev\ProjDir>mingw32-make time targets=app.exe
or:
C:\Dev\ProjDir>mingw32-make time targets="foo.obj bar.obj"
And of course, in your makefile, the commands to build a particular target can
include powershell -c "some other command" wherever you like.
Related
I have a C++ project setup with CMake, running on Mac. Recently I am looking into adding Fortify to do auto code analyzation. I am using Fortify version 22.1.
After set up the CMake and shell scripts, I found that if I compile with more than one CPU (using -j), the compiler (c++ or g++) will have issues generating the libs. Sometimes it will pass and successfully generate the Fortify output, but others it will just error out. Multi CPUs compile fine for this project without running Fortify.
I also see this error when I compile with Fortify (no matter it success or not):
[error]: Translator execution failed. Please consult the Troubleshooting section of the User Manual.
Translator returned status 1:
error: unable to handle compilation, expected exactly one compiler job in ''
This error always happens after a "Linking CXX xxxxx xxxx". I can't find any documentation about them.
Does anyone know how to solve this? Thank you.
Update more details about my setup:
I use shell files to wrap the sourceanalyzer like this:
#!/bin/bash
exec sourceanalyzer -b MyApp /Library/Developer/CommandLineTools/usr/bin/c++ "$#"
And my CMake setup like this:
if (${ENABLE_FORTIFY} EQUAL 1)
set(CMAKE_CC_COMPILER ${AVSxAppDALDefaultImplementation_SOURCE_DIR}/scripts/fortify-build-cc.sh)
set(CMAKE_CXX_COMPILER ${AVSxAppDALDefaultImplementation_SOURCE_DIR}/scripts/fortify-build-cxx.sh)
endif()
My shell script to run CMake and then to the scan:
cmake $PACKAGEPATH \
...
-DENABLE_FORTIFY="${ENABLE_FORTIFY}"
echo "---BUILDING---"
make release
if [[ $ENABLE_FORTIFY == 1 ]]; then
echo "---RUNNING FORTIFY SCAN---"
sourceanalyzer -b ${CURRENT_PROJECT_NAME} -scan -f fortify_scan_result_${CURRENT_PROJECT_NAME}.txt
fi
I have some of github that I am trying to run using AFL.
The code: https://github.com/karimmd/CScanner/tree/cfe7d08bf46b1eed0443f9e27bc089d68a830a45
I wanna run the project and find vulnerablities. I have put the github all files inside a folder code , so the file structure is CScanner-master/code/all the files here.
I am using this command on terminal :
hemlatamahaur#Hemlatas-MacBook-Pro desktop % afl-fuzz -i CScanner-master -o code ./input-testcode.c
afl-fuzz 2.56b by <lcamtuf#google.com>
[+] You have 4 CPU cores and 2 runnable tasks (utilization: 50%).
[+] Try parallel jobs - see /usr/local/Cellar/afl-fuzz/2.57b/share/doc/afl/parallel_fuzzing.txt.
[*] Setting up output directories...
[+] Output directory exists but deemed OK to reuse.
[*] Deleting old session data...
[+] Output dir cleanup successful.
[*] Scanning 'CScanner-master'...
[+] No auto-generated dictionary tokens to reuse.
[*] Creating hard links for all input files...
[*] Validating target binary...
[-] PROGRAM ABORT : Program './input-testcode.c' not found or not executable
Location : check_binary(), afl-fuzz.c:6873
It keep saying there is no file as input-testcode.c
I am new to AFL, so I might be doing it wrong. How do I run this code using AFL to find the vulnerabilities. Any help is very appreciated.
you have to build your code using afl-clang first
afl-clang
$ afl-clang input-testcode.c -o input-testcode .
Then:
$ afl-fuzz -i CScanner-master -o code ./input-testcode .
I hope it works
Afl-fuzz works on the executable
I've been recently attempting to run my scripts in parallel in a more convenient way than to open a several instances of terminal and executing in scripts separately.
I've been trying to learn how to use gnu_parallel for the past couple of days and I am still a bit clueless, and hoping if someone can provide a direct example.
Suppose I have a g++ compiled code called blah.exe and a bash script called blah.sh that will run alone perfectly fine, but I want to execute them in different directories.
I've been reading
https://www.gnu.org/software/parallel/man.html#EXAMPLE:-Working-as-xargs--n1.-Argument-appending
and
https://www.biostars.org/p/182136/
but I am not totally clear about the syntax
To run these in series, I would do:
for i in 1 2 3 4
mv ./blah.exe directory$i
cd directory$i
./blah.exe all
cd ..
end
similarly
for i in 1 2 3 4
mv ./blah.sh directory$i
cd directory$i
source ./blah.sh all
cd ..
end
I am trying to under stand how I would split this load to 4 logical-threads in one command using parallel.
Could someone provide an example for this?
Thank you for your time.
Something like:
parallel --dry-run 'cd directory{}; ../blah.exe all; source ../blah.sh all' ::: {1..4}
No need to copy/move the executable, just run the same one.
No need to cd .. afterwards, as it's a new process each time.
Note this is not multi-threading, it is multi-processing.
If you want to process discontiguous directory numbers, you can use:
parallel ... ::: {1..4} 6 7 {11..14}
If you want to process all directories, you can use:
printf "%s\0" */ | parallel -0 'cd {}; pwd'
If you want to process all directories starting with FRED, you can use:
printf "%s\0" FRED*/ | parallel -0 'cd {}; pwd'
Im back porting ffmpeg to an older version of debian.
everything is going well, but its so slow.
I am running dpkg-buildpackage -us -uc
with a debian rules file that looks like this:
#!/usr/bin/make -f
%:
dh $#
override_dh_auto_configure:
./configure
I notice, this is only processing on 1 core.
is there anything like make -j 4 that I could use to speed this up?
I've been using this guide, but i don't see anything for speeding up the build step
https://www.debian.org/doc/manuals/maint-guide/
Sure, you can use -j 4 as an argument to dpkg-buildpackage. It is documented in the man page. The relevant section is:
-jjobs Number of jobs allowed to be run simultaneously, equivalent to
the make(1) option of the same name. Will add itself to
the MAKEFLAGS environment variable, which should cause all
subsequent make invocations to inherit the option. Also adds
parallel=jobs to the DEB_BUILD_OPTIONS environment variable which
allows debian/rules files to use this information for their own
purposes. The parallel=jobs in DEB_BUILD_OPTIONS environment
variable will override the -j value if this option is given.
I have inherited an ANSI C++ program that: has no GUIs and is supposed to run in batch mode, generating lots of data (we are talking 100,000+ ASCII files). We are thinking that in long term we’ll run it under UNIX. For now, I have a MacBook Air running OS X 10.9.4 and I loaded Xcode 5.1.1. It compiles without errors or warnings.
I need to test a program as follows:
<prompt> myprogram datain dataout1 datout2
Where is the compiled program? In which directory? Can I copy my datain file in that directory?
For repeated execution under Windows (Command Prompt window) I normally would have a batch file of the type:
myprogram datain1 dataout11 datout12
myprogram datain2 dataout21 datout22
myprogram datain3 dataout31 datout32
........
myprogram datainn dataoutn1 datoutn2
Can I do something similar with OS X? Where can I find the applicable documentation?
You will want to look for your terminal emulation program. See http://en.wikipedia.org/wiki/Terminal_(OS_X) for how to use it, and it should be the bash shell which is one of the unix shells
You can also do a shell script see
http://tldp.org/HOWTO/Bash-Prog-Intro-HOWTO.html for some bash shell scripting info
For such a simple operation, you can write a shell script that will look almost exactly the same as the batch file you use on Windows. The key difference between Windows' cmd.exe and *nix shells here is that the current directory is not part of the search path for executables (the way it is on Windows), so if you put the shell script in the same folder as the compiled executable, you will need to prefix the program name with ./ (to mean "look in the current directory"). For example:
#!/bin/sh
./myprogram datain1 dataout11 datout12
./myprogram datain2 dataout21 datout22
./myprogram datain3 dataout31 datout32
........
./myprogram datainn dataoutn1 datoutn2
If the shell script and executable are not in the same folder, you can use either an absolute path or an appropriate relative path.
Also, to run the script you will either need to make it executable:
$ chmod +x myscript.sh
$ ./myscript.sh
or invoke the shell with the script as an argument:
$ sh myscript.sh