How do I retrieve AWS Batch job parameters? - aws-batch

Question

How do I retrieve parameters from an AWS Batch job request? Suppose I have a job submitter app that sends a job request with the following code (in C#):
SubmitJobRequest submitJobRequest = new SubmitJobRequest()
{
JobName = "MyJobName",
JobQueue = "MyJobQueue",
JobDefinition = "MyJobDefinition:1",
Parameters = new Dictionary<string, string>() { {"Foo", "Bar" } },
};
SubmitJobResponse submitJobResponse = AWSBatchClient.SubmitJob(submitJobRequest);
What I want to be able to do now is retrieve what's in the Parameters field in submitJobRequest in my docker app that gets launched. How do I do that? It's not passed in as program args, as I've tested that (the only args I see are those were statically defined for 'Command' my job definition). I know that I can set environment variables via container overrides and then retrieve them via Environment.GetEnvironmentVariable (in C#). But I don't know how to get the parameters. Thanks.

Answer 1

Here is an example using yaml cloudformation.(or you can use json for same properties).You can declare the parameters using a Ref in the command section. I am using user_name but you can add more. We might have a limit of 30KB payload
ContainerProperties:
Command:
- "python"
- "Your command here"
- "--user_name"
- "Ref::user_name"
Now you can submit your job to the queue like this. I am using python and boto3 client:
# Submit the job
job1 = client.submit_job(
jobName=jobName,
jobQueue=jobQueue,
jobDefinition=jobDefinition,
parameters={
'user_name':user_name
}
)
To retrieve the parameters use this(I am using argparse):
parser = argparse.ArgumentParser(description='AWS Driver Batch Job Runner')
parser.add_argument('--user_name', dest='user_name', required=True)
args = parser.parse_args()
print(args.user_name)

Answer 2

Found the answer I was looking for. I just had to add a ref to the parameter for Command in the job definition. In the question example, I would've needed to specify Ref::Foo for Command in the job definition and then "Bar" would've gotten passed as program args to my container app.
To expand on my example, in my specific case, my program uses the CommandLineParser package for passing parameters. Suppose one of the CommandLine options is called Foo. If I were running the program from a command line, I'd set a value for Foo with something like "--Foo Bar". To effectively do the same for my batch job, in my job definition, for Command, I would specify "--Foo Ref::Foo" (without quotes). Then for the Parameters field in my SubmitJobRequest object, I would set Foo exactly as per my original example and then my batch program would see "Bar" for the Foo CommandLine option (just like as if it was run with "--Foo Bar"). Hope that helps.