How to pass argument to Makefile from command line?
I understand I can do
$ make action VAR="value"
$ value
with Makefile
VAR = "default"
action:
#echo $(VAR)
How do I get the following behavior?
$ make action value
value
How about
$make action value1 value2
value1 value2
You probably shouldn't do this; you're breaking the basic pattern of how Make works. But here it is:
action:
#echo action $(filter-out $#,$(MAKECMDGOALS))
%: # thanks to chakrit
#: # thanks to William Pursell
EDIT:
To explain the first command,
$(MAKECMDGOALS) is the list of "targets" spelled out on the command line, e.g. "action value1 value2".
$# is an automatic variable for the name of the target of the rule, in this case "action".
filter-out is a function that removes some elements from a list. So $(filter-out bar, foo bar baz) returns foo baz (it can be more subtle, but we don't need subtlety here).
Put these together and $(filter-out $#,$(MAKECMDGOALS)) returns the list of targets specified on the command line other than "action", which might be "value1 value2".
Here is a generic working solution based on #Beta's
I'm using GNU Make 4.1 with SHELL=/bin/bash atop my Makefile, so YMMV!
This allows us to accept extra arguments (by doing nothing when we get a job that doesn't match, rather than throwing an error).
%:
#:
And this is a macro which gets the args for us:
args = `arg="$(filter-out $#,$(MAKECMDGOALS))" && echo $${arg:-${1}}`
Here is a job which might call this one:
test:
#echo $(call args,defaultstring)
The result would be:
$ make test
defaultstring
$ make test hi
hi
Note! You might be better off using a "Taskfile", which is a bash pattern that works similarly to make, only without the nuances of Maketools. See https://github.com/adriancooney/Taskfile
Much easier aproach. Consider a task:
provision:
ansible-playbook -vvvv \
-i .vagrant/provisioners/ansible/inventory/vagrant_ansible_inventory \
--private-key=.vagrant/machines/default/virtualbox/private_key \
--start-at-task="$(AT)" \
-u vagrant playbook.yml
Now when I want to call it I just run something like:
AT="build assets" make provision
or just:
make provision in this case AT is an empty string
Few years later, want to suggest just for this: https://github.com/casey/just
action v1 v2=default:
#echo 'take action on {{v1}} and {{v2}}...'
You will be better of defining variables and calling your make instead of using parameters:
Makefile
action: ## My action helper
#echo $$VAR_NAME
Terminal
> VAR_NAME="Hello World" make action
Hello World
don't try to do this
$ make action value1 value2
instead create script:
#! /bin/sh
# rebuild if necessary
make
# do action with arguments
action "$#"
and do this:
$ ./buildthenaction.sh value1 value2
for more explanation why do this and caveats of makefile hackery read my answer to another very similar but seemingly not duplicate question: Passing arguments to "make run"
Related
I have a nextflow script with a channel for paired file inputs. I am trying to extract a substring from the file inputs to use as part of the shell call. I am trying to use Groovy's regex matching to extract the substring, but since it is based on an input value, I am having trouble executing the matching. An alternative would be to perform the regex in bash as part of the process shell call, but I am interested in figuring out how to manipulate inputs within a process anyways, as I feel it would be useful for other things too. How can I perform intermediate Groovy code with the process inputs prior to the shell call?
process alignment {
input:
val files
output:
stdout
def matcher = "${files[1][0]}" =~ /.+\/bcl2fastq_out\/([^\/]+)\/.+/
# this is the culprit, as if I hardcode the first string it works
def project = matcher.findAll()[0][1]
"""
echo ${project}
"""
}
workflow {
files = Channel
.fromFilePairs("${params.out_dir}/**{_R1,_R2}_00?.fastq.gz", checkIfExists:true, size: 2)
alignment(files)
}
when I execute this, I get the error
No such variable: files
an example input string would look like extractions/test/bcl2fastq_out/project1/example_L001_R1_001.fastq.gz where I'm trying to extract the project1 substring
As you've already discovered, you can declare variables in the script block, before the command string. For example:
process alignment {
input:
tuple val(sample), path(fastq_files)
output:
tuple val(sample), path(output_file)
script:
output_file = "${sample}.bam"
"""
>&2 echo "Aligning ${sample}..."
touch "${output_file}"
"""
}
Note that these are global (within the process scope) unless you define them using the def keyword. If you don't need these elsewhere in your process definition, like in your example, a local variable (using def) is usually preferable. If, however, you need to access these in your output declaration, for example, then they will need to be global.
Note that the fromFilePairs factory method emits a tuple, where the first element is a group key and the second element is a list of files. The problem with just using val to declare the inputs is that the files in the second element will not be localized to the working directory when your script is run. To fix this, you can just change your input definition to something like:
input:
tuple val(sample), path(fastq_files)
The problem with this approach, is that we're unable to extract the parent directory name from the localized filenames. So you will need to pass this in somehow. Usually, you could just get the parent name from the first file in the tuple, using:
params.input_dir = './path/to/files'
params.pattern = '**_R{1,2}_00?.fastq.gz'
process alignment {
debug true
input:
tuple val(sample), val(project), path(fastq_files)
"""
echo "${sample}: ${project}: ${fastq_files}"
"""
}
workflow {
Channel
.fromFilePairs( "${params.input_dir}/${params.pattern}" )
.map { sample, reads ->
def project = reads[0].parent.name
tuple( sample, project, reads )
}
.set { reads }
alignment( reads )
}
But since the glob pattern has an additional wildcard, i.e. _00?, you may not necessarily get the results you expect. For example:
$ mkdir -p path/to/files/project{1,2,3}
$ touch path/to/files/project1/sample1_R{1,2}_00{1,2,3,4}.fastq.gz
$ touch path/to/files/project2/sample2_R{1,2}_00{1,2,3,4}.fastq.gz
$ touch path/to/files/project3/sample3_R{1,2}_00{1,2,3,4}.fastq.gz
$ nextflow run main.nf
N E X T F L O W ~ version 22.04.0
Launching `main.nf` [determined_roentgen] DSL2 - revision: f80ab33ac8
executor > local (12)
[a8/9235cc] process > alignment (12) [100%] 12 of 12 ✔
sample2: project2: sample2_R1_001.fastq.gz sample2_R1_004.fastq.gz
sample1: project1: sample1_R1_003.fastq.gz sample1_R2_001.fastq.gz
sample1: project1: sample1_R1_004.fastq.gz sample1_R2_003.fastq.gz
sample3: project3: sample3_R1_001.fastq.gz sample3_R2_001.fastq.gz
sample1: project1: sample1_R1_001.fastq.gz sample1_R2_004.fastq.gz
sample1: project1: sample1_R1_002.fastq.gz sample1_R2_002.fastq.gz
sample2: project2: sample2_R1_002.fastq.gz sample2_R2_002.fastq.gz
sample2: project2: sample2_R2_001.fastq.gz sample2_R2_004.fastq.gz
sample2: project2: sample2_R1_003.fastq.gz sample2_R2_003.fastq.gz
sample3: project3: sample3_R2_002.fastq.gz sample3_R2_004.fastq.gz
sample3: project3: sample3_R1_003.fastq.gz sample3_R1_004.fastq.gz
sample3: project3: sample3_R1_002.fastq.gz sample3_R2_003.fastq.gz
Fortunately, we can supply a custom file pair grouping strategy using a closure. This uses the readPrefix helper function:
workflow {
Channel
.fromFilePairs( "${params.input_dir}/${params.pattern}" ) { file ->
prefix = Channel.readPrefix(file, params.pattern)
suffix = file.simpleName.tokenize('_').last()
"${file.parent.name}/${prefix}_${suffix}"
}
.map { key, reads ->
def (project, sample) = key.tokenize('/')
tuple( sample, project, reads )
}
.set { reads }
alignment( reads )
}
Results:
$ nextflow run main.nf
N E X T F L O W ~ version 22.04.0
Launching `main.nf` [loving_cantor] DSL2 - revision: 5a76ac712f
executor > local (12)
[f4/74edbc] process > alignment (12) [100%] 12 of 12 ✔
sample1_003: project1: sample1_R1_003.fastq.gz sample1_R2_003.fastq.gz
sample2_002: project2: sample2_R1_002.fastq.gz sample2_R2_002.fastq.gz
sample1_002: project1: sample1_R1_002.fastq.gz sample1_R2_002.fastq.gz
sample2_003: project2: sample2_R1_003.fastq.gz sample2_R2_003.fastq.gz
sample2_004: project2: sample2_R1_004.fastq.gz sample2_R2_004.fastq.gz
sample2_001: project2: sample2_R1_001.fastq.gz sample2_R2_001.fastq.gz
sample1_001: project1: sample1_R1_001.fastq.gz sample1_R2_001.fastq.gz
sample1_004: project1: sample1_R1_004.fastq.gz sample1_R2_004.fastq.gz
sample3_001: project3: sample3_R1_001.fastq.gz sample3_R2_001.fastq.gz
sample3_004: project3: sample3_R1_004.fastq.gz sample3_R2_004.fastq.gz
sample3_002: project3: sample3_R1_002.fastq.gz sample3_R2_002.fastq.gz
sample3_003: project3: sample3_R1_003.fastq.gz sample3_R2_003.fastq.gz
I figured it out, if instead of just jumping into the shell script with the triple quotes, you can start specifying the process execution script with "script:" then run Groovy using the process inputs
process alignment {
input:
val files
output:
stdout
script:
test = (files[1][0] =~ '.+/test/([^/]+)/.+').findAll()[0][1]
"""
echo $test
"""
I have an env var INTEGRATION that is set to true or false.
I would like to set the values of variables based on whether INTEGRATION is set to true or false.
I have tried:
ifeq ($(INTEGRATION),false)
PRODUCTION_URL="production"
else
PRODUCTION_URL="integration"
endif
and also
if [ ${SNOOTY_INTEGRATION} == false ]; then \
echo "HIT THE TRUTH"; \
PRODUCTION_URL="https://docs.mongodb.com"; \
fi
after running
export INTEGRATION=true
make help
in terminal, I run make help:
help:
echo "prod url ${PRODUCTION_URL}";
in the first case, it print prod url integration and in the second case, it prints prod url -- when the desired behavior is for it to print, prod url production
Running env INTEGRATION=true is wrong.
That doesn't set the environment variable in the current shell: it only sets that environment variable in the env command.
You can either use:
$ export INTEGRATION=true
$ make help
(set the variable in the shell, so all subsequent commands can see it until you un-export it again), or you can use:
$ env INTEGRATION=true make help
Which is essentially the same thing as (the more common and shorter):
$ INTEGRATION=true make help
(set the variable only for this invocation of make).
But you can't use:
$ env INTEGRATION=true
$ make help
ETA
If that doesn't work there must be something else about your environment you haven't explained. Please provide a complete self-contained example that shows the problem. For example, I don't know what the shell if-statement you show in your question is for or how it relates to the rest of your question. This works for me:
$ cat Makefile
ifeq ($(INTEGRATION),false)
PRODUCTION_URL="production"
else
PRODUCTION_URL="integration"
endif
help:
echo "prod url ${PRODUCTION_URL}";
$ make help
echo "prod url "integration"";
prod url integration
$ export INTEGRATION=true
$ make help
echo "prod url "integration"";
prod url integration
$ export INTEGRATION=false
$ make help
echo "prod url "production"";
prod url production
UPDATE
At the end, I came up with a lightening beautiful build Makefile. If you want take a look at this gist - https://gist.github.com/jonataswalker/5961bfcc0f335f3b51ea.
To concatenate some files I can do:
# JS files
JS_TARGETS = wrapper.js \
file1.js \
file2.js
combine-js:
#cat $(JS_TARGETS) > file-combined.js
But that's not what I want/need. My wrapper.js is like:
(function(Foo, win, doc){
'use strict';
/*{CODE_TO_BE_REPLACED}*/
})(window.Foo = window.Foo || {}, window, document);
So I need to put my other files inside wrapper.js. After that I will run jshint and uglify but this is another issue.
Simplest way would be splitting wrapper.js into two parts - head and tail - and then concatenating in strict order, e.g.:
cat wrapper_head.js file1.js file2.js wrapper_tail.js > file-combined.js
This should do it - it uses perl though:
JS_WRAPPED_SOURCES = file1.js file2.js
JS_WRAPPER = wrapper.js
JS_SOURCES = $(JS_WRAPPER) $(JS_WRAPPED_SOURCES)
combined.js: $(JS_SOURCES)
cat $< | perl -e '$$c=join("\n",<>); $$c =~ s/{CODE_TO_BE_REPLACED}/`cat $(JS_WRAPPED_SOURCES)`/e; print $$c' \
> $#
Since you're running jshint and uglify I'm guessing you're using Grunt.
In that case, I'd recommend you use a Grunt task like grunt-replace.
I can't figure out why in the following the two different versions yield different results:
$(INCLUDE_DIR)/%: TRGT_PATH = \
$(INCLUDE_DIR)/$(patsubst $(word 1,$(subst /, ,$(dir $*)))/%,%,$*)
$(INCLUDE_DIR)/%: INCLUDEDS = $(TRGT_PATH)
$(INCLUDED_FILES) : $(INCLUDE_DIR)/%: %
ifeq ($(TRGT_PATH),$(findstring $(TRGT_PATH),$(INCLUDEDS)))
#echo [INC][WARN] File $(TRGT_PATH) already exists while copying from $*
else
#echo $(findstring $(TRGT_PATH),$(INCLUDEDS))
#echo [INC] $* $(TRGT_PATH)
#cp $* $(TRGT_PATH)
endif
Output:
[INC][WARN] File include/GeometricObject.h already exists while copying from engine/GeometricObject.h
[INC][WARN] File include/util.h already exists while copying from engine/util.h
[INC][WARN] File include/util.h already exists while copying from test/util.h
If I change the line ifeq ($(TRGT_PATH),$(findstring $(TRGT_PATH),$(INCLUDEDS))) to ifneq (,$(findstring $(TRGT_PATH),$(INCLUDEDS))) the output is:
include/GeometricObject.h
[INC] engine/GeometricObject.h include/GeometricObject.h
include/util.h
[INC] engine/util.h include/util.h
include/util.h
[INC] test/util.h include/util.h
As far as I know $(findstring t,l) returns the t if t is in l and else an empty string. But the output (if VAR is equals LIST) still is:
foo
bar
Can someone explain?
PS: I tested a more simple code and that worked fine...
If you'd provided a complete example, including the values of VAR and LIST, we'd be able to help. As it is, all I can say is "it works for me", so you must not be accurately reporting your environment in your question:
$ cat Makefile
VAR = v
LIST = v
all:
ifneq (,$(findstring $(VAR),$(LIST)))
#echo foo
else
#echo bar
endif
$ make
foo
ETA:
Aha. Well, your problem has absolutely nothing to do with findstring, so it's not surprising that your original question was not answerable.
The issue is that you're trying to use target-specific variables in a non-recipe context, and the documentation clearly states that they are not available outside of recipes.
The ifeq, etc. statements are like preprocessor statements: they are evaluated as the makefile is being parsed, not later when the recipes are being invoked. So the values of TRGT_PATH and INCLUDEDS when the ifeq/ifneq is invoked are the global values of those variables (which might be the empty string if they're not set otherwise) not the target-specific values.
I tried to get some aliases from a specific config file in a short bash script. The config looks like:
[other config]
[alias]
alias: command -o option
alias2: command -l 2
alias3: command -o option
[other config]
How would you get these aliases separated? I would prefer a output like this:
alias: command -o option
alias2: command -l 2
alias3: command -o option
I already did some bad stuff like getting line numbers and so on...
Any Ideas? Would be great!
You can do this using sed:
sed -n -e '/\[alias\]/,/\[.*\]/s/:/:/p'
This will print all lines between [alias] and the next line containing [ and ] that have a colon on them.
Perl has a Config::GitLike module that may come in handy for parsing that git config file.
Looking at the documentation, you want to do:
#!perl -w
use strict;
use 5.010;
use Config::GitLike;
my $c = Config::GitLike->new(confname => 'config');
$c->load;
my $alias = $c->get( key => 'alias.alias' );
my $alias2 = $c->get( key => 'alias.alias2' );
my $alias3 = $c->get( key => 'alias.alias3' );
say "alias: $alias";
say "alias2: $alias2";
say "alias3: $alias3";