Triggering DAG in a loop - airflow-scheduler

Question

I am trying to trigger DAG task for 3 times , how can this be done using python script.
Currently my flow job is
dag = DAG('dag1', default_args=default_args, concurrency=1,
max_active_runs=1,
schedule_interval=None, catchup=False)
task = BashOperator(
task_id='task1',
bash_command='ssh "runJob.sh"',
dag=dag)
Is there any better way to trigger job for a given specified number of times ?

Answer 1

I assume you use Airflow 2, you can use the following command to trigger a DAG run.
airflow dags trigger [-h] [-c CONF] [-e EXEC_DATE] [-r RUN_ID] [-S SUBDIR] dag_id
You can read more on this command here.
Based on that command, we can use use subprocess to run it.
import subprocess
command = [
"airflow",
"dags",
"trigger",
"-e",
"2021-02-05 00:00:00",
"-r",
"manual__2021-02-05T00:00:00+00:00",
"my_dag",
]
for i in range(3):
process = subprocess.Popen(command, stdout=subprocess.PIPE)
output, error = process.communicate()

Answer 2

In airflow Airflow 2.0 you can use the TriggerDagRunOperator.
You should be able to use that in a loop.
https://airflow.apache.org/docs/apache-airflow/stable/_api/airflow/operators/trigger_dagrun/index.html
trigger = TriggerDagRunOperator(
task_id="some-id",
trigger_dag_id="dag1",
conf={"some_param": 'some_value'},
reset_dag_run = True
)
Alternatively you can also put in the a task and call the execute method,
like TriggerDagRunOperator(...).execute() and pass in the current context to the execute method
which you can find using the get_current_context function from airflow.operators.python.