Send a Slack message from an Airflow DAG

This article is a part of my "100 data engineering tutorials in 100 days" challenge. (37/100)

What to do when we want to send a Slack notification telling us how many values have we processed in our DAG? I suggest doing it like this:

  • First, we have to store the value in the XCom. We can do it by returning the value from the PythonOperator. In this case, Airflow will automatically store it in XCom using the return_value as the key.

  • In the following step, we can define a SlackAPIPostOperator that uses a template to retrieve the value from XCom and puts it in the message sent to a Slack channel:

```python from airflow.operators.slack_operator import SlackAPIPostOperator

send_to_slack = SlackAPIPostOperator( task_id=’notify_slack’, channel=’#airflow’, username=’Airflow’, text=”DAG run {{ execution_date }} processed {{ task_instance.xcom_pull(‘other_task’, key=’return_value’) }} values.” slack_conn_id=’slack_connection_id’ )

Did you enjoy reading this article?
Would you like to learn more about leveraging AI to drive growth and innovation, software craft in data engineering, and MLOps?

Subscribe to the newsletter or add this blog to your RSS reader (does anyone still use them?) to get a notification when I publish a new essay!

Newsletter

Do you enjoy reading my articles?
Subscribe to the newsletter if you don't want to miss the new content, business offers, and free training materials.

Bartosz Mikulski

Bartosz Mikulski

  • MLOps engineer by day
  • AI and data engineering consultant by night
  • Python and data engineering trainer
  • Conference speaker
  • Contributed a chapter to the book "97 Things Every Data Engineer Should Know"
  • Twitter: @mikulskibartosz
  • Mastodon: @mikulskibartosz@mathstodon.xyz
Newsletter

Do you enjoy reading my articles?
Subscribe to the newsletter if you don't want to miss the new content, business offers, and free training materials.