Use a custom function in Airflow templates

This article is a part of my "100 data engineering tutorials in 100 days" challenge. (47/100)

Adding a custom function to Airflow is quite simple. First, we have do define a function in Python, for example, this one:

def do_something_with_execution_date(execution_date):
    # Imagine that there is some useful code ;)

When the function is ready, we use the user_defined_macros parameter of the DAG object to pass a dictionary of custom functions:

dag = DAG(
        'custom_function': do_something_with_execution_date,

Now, we can use the custom function in any place that supports Airflow templates. Of course, only in the DAGs that have access to the functions.

{{ custom_function(execution_date) }};

Note that, I can pass parameters to the function and rename it by using a different name as the dictionary key.

Remember to share on social media!
If you like this text, please share it on Facebook/Twitter/LinkedIn/Reddit or other social media.

If you want to contact me, send me a message on LinkedIn or Twitter.

Would you like to have a call and talk? Please schedule a meeting using this link.

Bartosz Mikulski
Bartosz Mikulski * data/machine learning engineer * conference speaker * co-founder of Software Craft Poznan & Poznan Scala User Group

This website DOES NOT use cookies
but you may still see the cookies set earlier if you have already visited it.