How to retry a Python function call

This article is a part of my "100 data engineering tutorials in 100 days" challenge. (38/100)

The worst thing about making calls to external service is that those calls sometimes fail because of network errors or because the service is temporarily unavailable. It makes sense to retry the function call a few times and hope that the problem disappears in those situations.

The retry package gives us an easy method to implement retries in Python. All we need is a decorator added to the function we want to retry.

First, we have to install the package. If we use pip, the installation looks like this: pip install retry.

After that, we must find the definition of the function we want to retry, for example, this one:

def a_function_that_fails_for_no_reason():

Now, we add the decorator to the function. To define a decorator, we need the names of the errors that should cause a retry. All other errors will be propagated to the caller without retrying the function.

Subscribe to the newsletter and join the free email course.

Additionally, we need the number of retries and the delay between the subsequent function calls.

from retry import retry

@retry((HTTPError, DNSError), tries = 3, delay = 2, backoff = 2)
def a_function_that_fails_for_no_reason():

Remember to share on social media!
If you like this text, please share it on Facebook/Twitter/LinkedIn/Reddit or other social media.

If you want to contact me, send me a message on LinkedIn or Twitter.

Bartosz Mikulski
Bartosz Mikulski * MLOps Engineer / data engineer * conference speaker * co-founder of Software Craft Poznan & Poznan Scala User Group

Subscribe to the newsletter and get access to my free email course on building trustworthy data pipelines.