Send SMS from an Airflow DAG using AWS SNS

This article is a part of my "100 data engineering tutorials in 100 days" challenge. (97/100)

In this article, I am going to show you how to use Terraform to create and configure an SNS topic that delivers SMS messages to a mobile phone number. In the second part, I will use the SnsPublishOperator to send messages from Airflow.

Define an SNS topic and subscription

First, we have to define a new aws_sns_topic resource in Terraform:

1
2
3
resource "aws_sns_topic" "send-sms" {
    name = "send-sms"
}

After that, we have to add each recipient as a new SNS topic subscription:

1
2
3
4
5
resource "aws_sns_topic_subscription" "send-sms-recipient" {
    topic_arn = aws_sns_topic.send-sms.arn
    protocol = "sms"
    endpoint = "phone number with country code"
}

In the end, we have to define an access policy that allows publishing messages:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
data "aws_iam_policy_document" "sns_topic_policy" {
  policy_id = "__default_policy_ID"

  statement {
    actions = [
      "SNS:Publish"
    ]

    effect = "Allow"

    principals {
      type        = "AWS"
      identifiers = ["aws_identifier_of_the_AWS_user_used_by_Airflow"]
    }

    resources = [
      aws_sns_topic.send-sms.arn,
    ]

    sid = "__default_statement_ID"
  }
}

Subscribe to the newsletter and join the free email course.

Sending SMS message from Airflow

To send a text message, we have to call the SnsPublishOperator inside an Airflow DAG. Of course, we can use templates to pass values to the message:

1
2
3
4
5
6
7
8
from airflow.contrib.operators.sns_publish_operator import SnsPublishOperator

send_sms = SnsPublishOperator(
    task_id='send_sms',
    target_arn='sns_topic_arn',
    message='Here is the message. You can use the template variables to get values from XCom or DAG parameters.',
    aws_conn_id='aws_connection_id'
)

Remember to share on social media!
If you like this text, please share it on Facebook/Twitter/LinkedIn/Reddit or other social media.

If you want to contact me, send me a message on LinkedIn or Twitter.

Would you like to have a call and talk? Please schedule a meeting using this link.


Bartosz Mikulski
Bartosz Mikulski * MLOps Engineer / data engineer * conference speaker * co-founder of Software Craft Poznan & Poznan Scala User Group

Subscribe to the newsletter and get access to my free email course on building trustworthy data pipelines.