What to do when Airflow BashOperator fails with TemplateNotFound error
Airflow

What to do when Airflow BashOperator fails with TemplateNotFound error

How to fix TemplateNotFound error when using Airflow BashOperator
Bartosz Mikulski
Bartosz Mikulski – 1 min read

Airflow has an annoying error that causes a failure when we try to use the BashOperator like this:

operator = BashOperator(
    task_id='operator_id',
    bash_command='some_bash_script.sh',
    dag=dag)

What is wrong? For some reason, it tries to apply the template variables to the “bash_command” parameter even if we don’t use them. Consequently, the template loader fails with the TemplateNotFound error because it cannot find any template variables in the given string.

Of course, in this case, it is the correct behavior because we don’t use any template variables. Unfortunately, it is not the behavior we want.

To make Airflow ignore the missing template and just use the given string as the file name (as it should do anyway), we have to trick the template parsing code by adding a single space at the end of the parameter:

operator = BashOperator(
    task_id='operator_id',
    bash_command='some_bash_script.sh ',
    dag=dag)
Older post

Copy directories in S3 using s3-dist-cp

How to copy files in S3 and preserve the directory structure

Newer post

How to use xcom_pull to get a variable from another DAG

Get an XCOM variable from another DAG

Related Posts

Airflow Prophet

Anomaly detection in Airflow DAG using Prophet library

How to detect problems in Airflow pipeline using Prophet for time series anomaly detection
Published on: 12 Feb 2021

Airflow

How to restart a stuck Airflow DAG

What to do when an Airflow DAG gets stuck and does not want to run
Published on: 31 Dec 2020

Airflow

Why does the DayOfWeekSensor exist in Airflow?

How to make an Airflow DAG wait until a specified day of the week
Published on: 30 Dec 2020