Get an XCom value in the Airflow on_failure_callback function
When we want to get an XCom variable in the on_failure_callback, we will face a nasty bug. Using the
ti key to retrieve a value from the context gives us a None value.
1 2 # This does not work context.get("ti").xcom_pull(key="test")
It happens because in the on_failure_callback the task instance is passed to the function using the
task_instance key. Therefore to get a value from XCom, we must execute this code:
1 2 task: TaskInstance = context.get('task_instance') task.xcom_pull(key="test")
Alternatively, we can import XCom and access it directly:
1 2 from airflow.models import XCom XCom.get_one(execution_date = context.get('execution_date'), key='test')
You may also like
- How to find the Hive partition closest to a given date
- How to use AWSAthenaOperator in Airflow to verify that a DAG finished successfully
- How to conditionally skip tasks in an Airflow DAG
- How to set Airflow variables while creating a dev environment
- How to run Airflow DAGs for a specified date in the past?