Use a custom function in Airflow templates
Adding a custom function to Airflow is quite simple. First, we have do define a function in Python, for example, this one:
1
2
3
def do_something_with_execution_date(execution_date):
# Imagine that there is some useful code ;)
...
When the function is ready, we use the user_defined_macros
parameter of the DAG object to pass a dictionary of custom functions:
1
2
3
4
5
6
dag = DAG(
...,
user_defined_macros={
'custom_function': do_something_with_execution_date,
}
)
Now, we can use the custom function in any place that supports Airflow templates. Of course, only in the DAGs that have access to the functions.
1
{{ custom_function(execution_date) }};
Note that, I can pass parameters to the function and rename it by using a different name as the dictionary key.
Did you enjoy reading this article?
Would you like to learn more about leveraging AI to drive growth and innovation, software craft in data engineering, and MLOps?
Subscribe to the newsletter or add this blog to your RSS reader (does anyone still use them?) to get a notification when I publish a new essay!
You may also like
- How to use AWSAthenaOperator in Airflow to verify that a DAG finished successfully
- How to use Airflow backfill to run DAGs for a specified date in the past?
- How to set Airflow variables while creating a dev environment
- How to set a different retry delay for every task in an Airflow DAG
- Doing data quality checks using the SQLCheckOperator

Bartosz Mikulski
- MLOps engineer by day
- AI and data engineering consultant by night
- Python and data engineering trainer
- Conference speaker
- Contributed a chapter to the book "97 Things Every Data Engineer Should Know"
- Twitter: @mikulskibartosz
- Mastodon: @mikulskibartosz@mathstodon.xyz