Get an XCom value in the Airflow on_failure_callback function
When we want to get an XCom variable in the on_failure_callback, we will face a nasty bug. Using the ti
key to retrieve a value from the context gives us a None value.
1
2
# This does not work
context.get("ti").xcom_pull(key="test")
It happens because in the on_failure_callback the task instance is passed to the function using the task_instance
key. Therefore to get a value from XCom, we must execute this code:
1
2
task: TaskInstance = context.get('task_instance')
task.xcom_pull(key="test")
Alternatively, we can import XCom and access it directly:
1
2
from airflow.models import XCom
XCom.get_one(execution_date = context.get('execution_date'), key='test')
You may also like
- Why does the ExternalTaskSensor get stuck?
- How to retrieve the statuses of the recent DAG executions from Airflow database
- Why does the DayOfWeekSensor exist in Airflow?
- How to add a manual step to an Airflow DAG using the JiraOperator
- Why my Airflow tasks got stuck in "no_status" and how I fixed it
Remember to share on social media! If you like this text, please share it on Facebook/Twitter/LinkedIn/Reddit or other social media.
If you want to contact me, send me a message on LinkedIn or Twitter.
Would you like to have a call and talk? Please schedule a meeting using this link.