Send a Slack message from an Airflow DAG

This article is a part of my "100 data engineering tutorials in 100 days" challenge. (37/100)

What to do when we want to send a Slack notification telling us how many values have we processed in our DAG? I suggest doing it like this:

  • First, we have to store the value in the XCom. We can do it by returning the value from the PythonOperator. In this case, Airflow will automatically store it in XCom using the return_value as the key.

  • In the following step, we can define a SlackAPIPostOperator that uses a template to retrieve the value from XCom and puts it in the message sent to a Slack channel:

```python from airflow.operators.slack_operator import SlackAPIPostOperator

send_to_slack = SlackAPIPostOperator( task_id=’notify_slack’, channel=’#airflow’, username=’Airflow’, text=”DAG run {{ execution_date }} processed {{ task_instance.xcom_pull(‘other_task’, key=’return_value’) }} values.” slack_conn_id=’slack_connection_id’ )

Subscribe to the newsletter and join the free email course.

Remember to share on social media!
If you like this text, please share it on Facebook/Twitter/LinkedIn/Reddit or other social media.

If you want to contact me, send me a message on LinkedIn or Twitter.

Would you like to have a call and talk? Please schedule a meeting using this link.

Bartosz Mikulski
Bartosz Mikulski * data/machine learning engineer * conference speaker * co-founder of Software Craft Poznan & Poznan Scala User Group

Subscribe to the newsletter and get access to my free email course on building trustworthy data pipelines.

Do you want to work with me at riskmethods?

REMOTE position (available in Poland or Germany)