How to deal with the jinja2 TemplateNotFound error in Airflow
Occasionally, when we are making a custom Airflow operator, we may end up in a situation when Airflow complains about not existing jinja template. It may happen when we have an Airflow operator, which calls another operator inside its execute function when both operators have a common template field.
In this situation, Airflow may raise an error saying that: jinja2.exception.TemplateNotFound: content_of_the_field
. It is strange because if we have not used a jinja template inside the field value, we may expect it to pass the string without using it as a template. Unfortunately, that does not happen.
Dealing with the issue is quite simple. The wrapping Airflow operator should include the field’s content as a jinja constant and pass it into the inner operator. For example, when our code looks like this:
1
2
3
4
5
6
7
8
9
10
11
12
13
class SomeOperator(BaseOperator):
template_fields = ('the_field')
def __init__(self, field, *args, **kwargs):
super().__init__(*args, **kwargs)
self.field = field
self.extra_args = args
self.extra_kwargs = kwargs
def execute(self, context):
self.another_operator = SomeOtherOperator(field=self.field, *self.extra_args, **self.extra_kwargs)
self.another_operator.execute(context)

We can fix the jinja template issue like this:
1
2
3
4
5
6
7
8
9
10
11
class SomeOperator(BaseOperator):
template_fields = ('the_field')
def __init__(self, field, *args, **kwargs):
super().__init__(*args, **kwargs)
self.field = "{{ '" + field + "' }}"
self.extra_args = args
self.extra_kwargs = kwargs
... rest of the code
You may also like
- What to do when Airflow BashOperator fails with TemplateNotFound error
- Why does the DayOfWeekSensor exist in Airflow?
- Use a custom function in Airflow templates
- Dependencies between DAGs: How to wait until another DAG finishes in Airflow?
- How to add a manual step to an Airflow DAG using the JiraOperator
Remember to share on social media! If you like this text, please share it on Facebook/Twitter/LinkedIn/Reddit or other social media.
If you want to contact me, send me a message on LinkedIn or Twitter.
Would you like to have a call and talk? Please schedule a meeting using this link.