Send SMS from an Airflow DAG using AWS SNS

This article is a part of my "100 data engineering tutorials in 100 days" challenge. (97/100)

In this article, I am going to show you how to use Terraform to create and configure an SNS topic that delivers SMS messages to a mobile phone number. In the second part, I will use the SnsPublishOperator to send messages from Airflow.

Define an SNS topic and subscription

First, we have to define a new aws_sns_topic resource in Terraform:

1
2
3
resource "aws_sns_topic" "send-sms" {
    name = "send-sms"
}

After that, we have to add each recipient as a new SNS topic subscription:

1
2
3
4
5
resource "aws_sns_topic_subscription" "send-sms-recipient" {
    topic_arn = aws_sns_topic.send-sms.arn
    protocol = "sms"
    endpoint = "phone number with country code"
}

In the end, we have to define an access policy that allows publishing messages:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
data "aws_iam_policy_document" "sns_topic_policy" {
  policy_id = "__default_policy_ID"

  statement {
    actions = [
      "SNS:Publish"
    ]

    effect = "Allow"

    principals {
      type        = "AWS"
      identifiers = ["aws_identifier_of_the_AWS_user_used_by_Airflow"]
    }

    resources = [
      aws_sns_topic.send-sms.arn,
    ]

    sid = "__default_statement_ID"
  }
}


Sending SMS message from Airflow

To send a text message, we have to call the SnsPublishOperator inside an Airflow DAG. Of course, we can use templates to pass values to the message:

1
2
3
4
5
6
7
8
from airflow.contrib.operators.sns_publish_operator import SnsPublishOperator

send_sms = SnsPublishOperator(
    task_id='send_sms',
    target_arn='sns_topic_arn',
    message='Here is the message. You can use the template variables to get values from XCom or DAG parameters.',
    aws_conn_id='aws_connection_id'
)
Newsletter

Do you enjoy reading my articles?
Subscribe to the newsletter if you don't want to miss the new content, business offers, and free training materials.

Bartosz Mikulski

Bartosz Mikulski

  • Data/MLOps engineer by day
  • DevRel/copywriter by night
  • Python and data engineering trainer
  • Conference speaker
  • Contributed a chapter to the book "97 Things Every Data Engineer Should Know"
  • Twitter: @mikulskibartosz
Newsletter

Do you enjoy reading my articles?
Subscribe to the newsletter if you don't want to miss the new content, business offers, and free training materials.