How to release Lambda functions to AWS using Azure DevOps Release Pipelines?

Hi Guys, I hope you guys are doing well. When I was assigned to migrate our Github repositories and Pipelines to Azure DevOps, I had this problem. Because our complete application and their resources reside in AWS and our client wanted to have pipelines and code inside the Azure DevOps. After some research and a couple of POCs, I have figured out how I will explain it in this article.

The project’s architecture has microservices that run on many Lambda functions that utilize the Serverless framework and API Gateways. So from the atomic view, it is deploying Cloud formation templates written with Serverless Framework. So it is easy to deploy from anywhere like this without even opening the AWS Console.

So I am also using the exact implementation for this article as well. I have a Lambda function that was created with the Serverless framework.

Also, you need an AWS account and an Azure DevOps account. This article will not explain how to set up the Azure DevOps account or Azure Repos.

Alright, first of all, you have to push the code to the Azure Repos. Also, you can use Github as the source as well. But I am using Azure Repos. Now, I have the repo inside the Azure Repos and am ready to deploy.

In the Azure Pipelines, there are two types of the Pipelines. They are Build Pipelines and Release Pipelines. If you have multiple environments like Dev, Test, Staging, and Production. You can Build and Test at one time on Build Pipeline and create a common artefact and push it to several environments using Release Pipeline. Also, if you need to push it directly to the environment, you need the Release Pipeline, and all the other things can be done in the Release Pipelines.

If you need to know more about the Build/Test Pipelines, go through the below article I have written.

Build and Test Pipelines with Azure

Before implementing our Pipeline, we have to do two things first.

1. Install AWS Toolkit for Azure DevOps
2. Create a Service connection with AWS

So first, you need to install AWS Toolkit for Azure DevOps on the Azure DevOps. This will help run AWS SDK commands inside the Azure DevOps without manually installing the AWS SDK. For installing it, Go to the below URL,

AWS Toolkit for Azure DevOps - Visual Studio Marketplace

And click on the Get it free button.

AWS Toolkit for Azure DevOps (marketplace.visualstudio.com)

Then Select your organization and click Install. You will need an Administrator’s permission from a DevOps organization to do this.

The second thing is Creating a Service connection to AWS from Azure DevOps. It’s using these credentials to Deploy services to AWS. For this, you first need to create an IAM user with Programmatic Access.

Adding new User for Service Connection (console.aws.amazon.com)

Then click on Attach existing policies directly button on Set Permissions. Then select AdministratorAccess (I do not need administrator privileges, but I am creating one for ease.) from the below table.

Attach existing policies directly (console.aws.amazon.com)

Then press the Press Next button a couple of times and the Press Create User button. Then this will create the User.

Created User (console.aws.amazon.com)

Copy and keep these two values somewhere. We need them in Azure DevOps.

In the Azure DevOps, Project Settings, Select the Service Connections tab from the left-hand pane.

Then click on the New Service connection button from the top of the page. Then from the list of Connection types, Select AWS and Press Next.

From this form, For the Access Key ID field, give the value we have copied from AWS IAM Access Key ID and for the Secret, provide the value copied as Secret Access Key from AWS IAM. Then Give a Service connection name and click on Save.

The initial setup is done and ready to deploy from Azure DevOps to AWS.

Now head over to Release Pipelines and press the New + button to create a New Release Pipeline. Then select the Empty Job because we are going to develop from scratch.

New Pipeline (dev.azure.com)

First, I will give a name for the Pipeline and a Name for the stage.

Then we need an Artifact to start the Pipeline. So click on Add an Artifact, and you will see a popup like this.

Add an Artifact (dev.azure.com)

From here, you have lots of options for Artifact sources. You can select a Build artefact from a build Pipeline or directly from the Repo. If you open the five more Artifact types collapsible, you will see more sources, even the docker hub.

Release Pipeline Sources (dev.azure.com)

Because my code is in Azure Repos, I will select that option from here.

So I have a sample project in Azure Repos with one Lambda function with a GET API endpoint that outputs the current time with the moment.js library.

Select Azure Repo (dev.azure.com)

I will select the Project, Repo, and Default Branch for the deployments from the subsequent inputs. Default version I have chosen as the Latest from the default branch. Then I will click on Add button.

Now in the Stages, Stage 1 is already created. Here we define our Deployment sequences. Click on the one job link here.

Add Stages (dev.azure.com)

So here, our first job is created as an agent job. So we need to add tasks or steps to deployment under this agent job. I will show the steps we need to deploy a Lambda function to AWS.

1. Use NodeJS version
2. Install Serverless
3. Install dependencies
4. Deploying the Serverless app

So, we have to create these steps inside the Agent Job.

First, click on the + mark on Agent Job and search the Node.js tool installer to tell the Agent to use Node Version. So I am using NodeJS 12 for this project. So I am changing Version Spec to 12.x

Node.js Tool Installer (dev.azure.com)

Now we need to install Serverless. I have used Serverless v2 to create the Lambda function in my project. So we need to install Serverless v2. Again click on + and add an npm task.

NPM Task (dev.azure.com)

Here, Change the command to custom and add command as install serverless@2 -g and Select the folder containing package.json from the Working folder that includes the package.json option.

Now again, add another NPM task and do the same thing above, and only change the command to install -no-cache

Install dependencies (dev.azure.com)

This will install project dependencies.

Now click on the + icon and Add the AWS Shell Script task in the last step.

Deploy to AWS (dev.azure.com)

Here, I have selected the Service Connection I created earlier for the AWS Credentials, and for the Region, you can choose anything. Select Inline Script and add this command in the Text Area for the Script Source.

serverless deploy --region us-east-1

You need to have the region code you have selected in the last option for the region.

Then, In the Advanced options, select the Working Directory as same as the directory the serverless.yml placed.

Now all the steps are done. Click on the Save button top of the page.

Now, you can see the Created Pipeline under the Release pipelines in Azure DevOps. To run this Pipeline, click on the Create Release Button. Then you can see a Release is Created.

Created Release (dev.azure.com)

Click on this and,

Click on the Stage to view the Progress log. Here you can monitor how it is running.

Pipeline Log (dev.azure.com)

If all the things went well, you could see that Stage 1 is successfully deployed with a Green Tick like below.

Successfully Deployed (dev.azure.com)

Then you go to the Lambda functions section on the AWS console; you can see our Lambda function successfully deployed to the AWS.

Deployed Lambda Function (console.aws.amazon.com)

That’s it. I hope you guys got some idea about this. Keep cool until the following article.

Good Day to you all.