Sure, you can manage AWS cloud components just with Serverless. But why would you, when Terraform was designed to do it better? By default, they don’t integrate. Follow this guide to learn how to bridge Serverless and Terraform in 3 ways to handle infrastructure work with greater efficiency.
Dear cloud providers, what are Terraform and Serverless good for?
Terraform is an “infrastructure as a code” tool used to manage the lifecycle of cloud components. You write code that defines the cloud infrastructure, and then Terraform maps it to actual resources like databases, virtual networks, event buses, and so on.
The Serverless Framework is a command-line tool that can handle cloud infrastructure (just like Terraform) but rather serves as a place where you create code. After the code’s deployment, serverless interacts with cloud components to execute business logic. It’s not without downsides, though.
Imagine having multiple applications where all of them need to interact with cloud components to do their job. What is more, each of them sometimes may need to change some part of the infrastructure.
Let’s say you have cloud component definitions linked to application A. Now, whenever application B, C, or D needs to change the cloud infrastructure, you’d have to make changes to application A.
It is really inconvenient and completely against the single responsibility principle.
The solution here is to create a separate entity that will keep only the cloud infrastructure definition. It is way better to have it written down in a tool for this very purpose — like Terraform.
Which tool is better for handling state changes?
Since Terraform works as “infrastructure as a code”, it can create an execution plan before we confirm changes. When you’re unsure what will happen after you deploy the infrastructure definition, you can ask Terraform to simulate changes.
Unfortunately, the Serverless Framework doesn’t support this mechanism out of the box. For you to have it, it would require a workaround. Let’s go for a clean solution — let me show you how to combine both tools.
Why fusing Terraform and Serverless is a problem
Let’s say you have an application written on the Serverless Framework where a single lambda has to be executed every time a record is added to DynamoDB. It could look something like this.
Configuration for the Serverless Framework application
I used CloudFormation’s
Fn::GetAtt function to tell the lambda which DynamoDB stream it should subscribe to. The function returns a value of an attribute
StreamArn from the resource
serverless deploy in a system console and go to
AWS console -> Lambda -> Functions -> blog-post-dev-hello. See that our lambda was created successfully and subscribes to the DynamoDB stream.
The Serverless Framework was able to resolve
Fn::GetAtt: [ExampleTable, StreamArn] without any problems because we wrote the cloud components and the function in the same application.
Now, let’s move our DynamoDB definition to the Terraform application. First, we will delete our current application to prevent later conflicts by executing
serverless remove. Then we create a new Terraform application with one file.
But what about our Serverless Framework application?
As you can see, we lost the logical name of the
ExampleTable and we cannot use
Fn::GetAtt at all. What can we do?
The dirty way: use hardcoded values
The simplest solution is to hardcode DynamoDB stream ARN. So! Deploy your infrastructure. In your Terraform application, execute
terraform init and
terraform apply. Now you can go to the
AWS console -> DynamoDB -> Tables -> example_table -> Exports and streams where you will find our DynamoDB stream ARN.
Let’s copy this value and paste it into
After that, we can deploy the Serverless Framework application with
serverless deploy to see the lambda once again subscribes to the DynamoDB stream.
But hold up! We had to execute a lot of additional steps to get the same solution that Serverless Framework offers. Is there a better way to do this?
The standard way: work with the Systems Manager Parameter Store
A simpler solution for auto-injecting DynamoDB stream ARN — just like the Serverless Framework does it — is to make use of AWS’ Systems Manager Parameter Store. This service is a simple container with a purpose to store configuration data of your applications. Let’s make use of it!
Add the following code to the Terraform application.
I defined there a new Parameter Store named
/blog-post/example-table-stream-arn and the value of the DynamoDB stream ARN as
aws_dynamodb_table.this.stream_arn. Let’s look at how I built that value.
aws_dynamodb_table is a type of resource,
this is a name of resource, and
stream_arn is one of many attributes that this resource exposes. The whole list of these attributes can be found in Terraform’s documentation.
Ok, now that you know what was added, run a deploy with
terraform apply. Check if the resource was created by going to the
AWS console -> Systems Manager -> Parameter Store -> /blog-post/example-table-stream-arn.
Now, you have to tell the Serverless Framework application how to fetch this data. Serverless provides a series of special variables that are resolved to various values after deployment. One of those variables references the System Manager Parameter Store:
Add the syntax to the Serverless Framework application.
Deploy the application with
serverless deploy and see that the lambda function successfully subscribed to DynamoDB stream once again.
Nice! You now know how to solve the problem using the System Manager Parameter Store. But, surprise, there’s another approach you can use! Let’s see a different scenario.
The exotic way: connecting through AWS CloudFormation Outputs
Imagine you have to declare the infrastructure using a native CloudFormation template. The template is exactly the same one used to declare the infrastructure in the Serverless Framework.
Because of the CloudFormation template’s limitations, you lose access to resource attributes. You now can’t use
resource.resource_name.resource_attribute to fetch the stream ARN. Shoo the problem away by using CloudFormation Outputs.
The Outputs is an optional section in CloudFormation templates where you can declare values to be used as cross-stack references. This means you can select which attributes of our infrastructure we want to be easily fetchable.
Adding DynamoDB stream ARN to the Outputs section looks like this.
Now, remove the current Terraform application to prevent a race condition between creating a new database and removing the old one. Run
terraform destroy. Then, you can safely deploy our beauty with
Now make the reference to the CloudFormation Outputs variable in the Serverless Framework application.
cf:STACK_NAME.OUTPUT_KEY is the syntax for this. In our case, this should look like this.
Deploy it with
serverless deploy, and voilà, the lambda function once again subscribes to the DynamoDB stream.
Remember the 3 ways
You might find other approaches to connecting the Serverless Framework with Terraform. During the development of a spaceship-big project, I relied on the three described in the tutorial:
- Hardcoding the DynamoDB stream ARN
- Fetching data from AWS’ Systems Manager Parameter Store
- Connecting through CloudFormation Outputs to the Serverless Framework
I hope they will serve you as good as they served me. Make good use of them and good luck : )