How to add dependencies to AWS lambda
The process of adding dependencies to an AWS Lambda consists of two steps. First, we have to install the dependencies in the source code directory. Later, we have to package our lambda function into a zip file that also contains all of the dependency files.
As an example, I will install the Python Scrapy dependency because, in addition to pip packages, it also requires some Linux dependencies.
There is one little problem with installing Lambda dependencies. Lambdas run on a special Linux distribution made by AWS, and we need the dependencies compiled for that particular distribution.
Because of that, I am going to prepare a Docker image in which I will install all of the required packages and then copy them to the local directory that contains the lambda function.
First, I must create a Dockerfile.
FROM amazonlinux RUN yum update -y && \ yum install -y python3 pip3 libxml2 libxslt && \ pip3 install -t /deps_source lxml scrapy CMD cp -r /deps_source/* /deps
When I build the Docker image using that file, it will install all of the required packages and store them in a directory inside the image.
1 docker build - t lambdadeps
The only thing left to do when I run the image is copying the already installed dependencies to a directory that was mounted as a Docker volume.
1 docker run -v $(pwd)/deps:/deps lambdadeps:latest
That will copy the Python dependencies for Amazon Linux into my local directory.
Packaging the lambda function
Now, I can write a Make script that packages the lambda function and its dependencies. Note that, I have the dependencies in the
deps directory, and the lambda function is in the
build: docker run -v $(pwd)/deps:/deps lambdadeps:latest cd deps && zip -r9 function.zip . mv deps/function.zip function.zip zip -g function.zip test_lambda.py
You MUST use tabs for indentation in the Makefile. If your IDE replaces tabs with spaces, it will not work! If you copy-paste my code, you have to replace four spaces with a tab.
When the Makefile is ready, I can run the
make build command to package the lambda and all its dependencies into a zip file.
After that, I have to deploy the function on AWS. The process depends on the deployment of choice, and it is not in the scope of this article.
Remember to share on social media! If you like this text, please share it on Facebook/Twitter/LinkedIn/Reddit or other social media.