How to render an Airflow template for testing
Do you know what I do when I have to test a failing Airflow task? I copy the rendered template and run the code on my own, without Airflow. It is way easier to fiddle with the parameters, for example, to get more verbose output.
To get a rendered template, I used to open an instance of the task, copy the code, and change the parameters to match the execution date I wanted. That is the long and error-prone way to do it.
It turns out that the Airflow command-line interface has a command that generates a rendered template of a given task for the execution date we choose. No more copy-pasting and looking for parameters to change!
All we need is this command:
1
airflow render dag_id task_id execution_date
You may also like
- Use LatestOnlyOperator to skip some tasks while running a backfill in Airflow
- How to conditionally skip tasks in an Airflow DAG
- How to use Virtualenv to prepare a separate environment for Python function running in Airflow
- Why my Airflow tasks got stuck in "no_status" and how I fixed it
- Remove a directory from S3 using Airflow S3Hook
Remember to share on social media! If you like this text, please share it on Facebook/Twitter/LinkedIn/Reddit or other social media.
If you want to contact me, send me a message on LinkedIn or Twitter.
Would you like to have a call and talk? Please schedule a meeting using this link.
