Terraform Loops

Power and Pitfalls of the Count Parameter

Terraform Loops

Introduction

When managing infrastructure as code with Terraform, efficiency and modularity are key aspects to consider. As your infrastructure grows in complexity, the potential for code duplication increases, leading to maintenance challenges and a higher likelihood of errors. To combat these issues, Terraform offers several powerful looping constructs that allow you to iterate over resources and modules effectively, promoting cleaner and more adaptable configurations.

Why Looping Constructs are Essential

Imagine you are tasked with deploying a fleet of virtual machines (VMs) in a cloud environment. Without looping constructs, you would need to define each VM instance individually, which could result in significant code duplication. For instance, if you needed to create three identical VMs, your Terraform code might look something like this:

resource "ibm_is_instance" "web_server_1" {
  name              = "web-server-1"
  profile           = "bx2-2x8"  # Instance profile
  zone              = "us-south-1"
  image             = "r0-12345678"  # Image reference
  primary_network_interface {
    subnet_id = "subnet-12345678"
  }
}

resource "ibm_is_instance" "web_server_2" {
  name              = "web-server-2"
  profile           = "bx2-2x8"
  zone              = "us-south-1"
  image             = "r0-12345678"
  primary_network_interface {
    subnet_id = "subnet-12345678"
  }
}

resource "ibm_is_instance" "web_server_3" {
  name              = "web-server-3"
  profile           = "bx2-2x8"
  zone              = "us-south-1"
  image             = "r0-12345678"
  primary_network_interface {
    subnet_id = "subnet-12345678"
  }
}

This approach results in a long configuration file where each VM is defined separately, making it tedious to manage and prone to inconsistencies. For example, if you want to update the instance profile or the image, you would have to do so in multiple places.

The Main Looping Constructs

With Terraform's looping constructs, you can streamline this process significantly, reducing repetition and potential errors.

  1. Count Parameter: A straightforward way to create multiple instances of a resource or module.

  2. For_each Parameter: Enables iteration over resources, inline blocks within a resource, and entire modules, offering greater flexibility than the count parameter.

  3. For Expressions: Useful for iterating over lists and maps, allowing for dynamic data manipulation and resource creation.

In this article, we will focus specifically on the count parameter in Terraform. We will delve into its syntax, practical use cases and limitations. In future articles in this series, we will cover for_each parameter and for expressions.

Count Parameter

The following resource block creates a single Cloud Object Storage instance named “cos-instance“.

resource "ibm_resource_instance" "cos_instance" {
  name              = "cos-instance"
  resource_group_id = "12345"
  service           = "cloud-object-storage"
  plan              = "standard"
  location          = "global"
}
Terraform will perform the following actions:

  # ibm_resource_instance.cos_instance will be created
  + resource "ibm_resource_instance" "cos_instance" {
      + account_id              = (known after apply)
      + id                      = (known after apply)
      + name                    = "cos-instance"
      + resource_group_id       = "12345"
      # Note: output has been removed to reduce the plan length
    }

Plan: 1 to add, 0 to change, 0 to destroy.

Now, to create multiple such instances, we require a loop. In a general-purpose programming language like Python, you would wrap the code with a “for” loop to create multiple instances of this instance.

for i in range(0,3):
    resource "ibm_resource_instance" "cos_instance" {
      name              = "cos-instance-${i}" # Appending iterator as suffix to make it unique instance 
      resource_group_id = "12345"
      service           = "cloud-object-storage"
      plan              = "standard"
      location          = "global"
    }

Unfortunately, this isn't directly possible in Terraform because its configuration language (HCL) doesn't execute imperative loops. Terraform works declaratively, meaning you describe the desired state, and Terraform figures out the "how."

To create multiple instances of a resource, we use the count parameter. It is added as a meta-parameter to repeat the desired resource or module block.

resource "ibm_resource_instance" "cos_instance" {
  count = 3
  name              = "cos-instance-${count.index+1}"
  resource_group_id = "12345"
  service           = "cloud-object-storage"
  plan              = "standard"
  location          = "global"
}

The above code creates three “cos_instance” with names “cos-instance-1", "cos-instance-2", and "cos-instance-3".

The count.index is a special variable that acts as an iterator, representing the current iteration of the count loop. It starts from 0 for the first instance and increments by 1 for each subsequent instance. This makes it extremely useful for generating dynamic and unique configurations for each resource instance. In the example provided, count.index is used to create unique names for the resources by appending a number to the resource name.

Explanation:

  • count = 3: This instructs Terraform to create three instances of the ibm_resource_instance resource.

  • count.index: For each iteration, this variable takes the values 0, 1, and 2, corresponding to the first, second, and third instances, respectively.

  • count.index + 1: Adding 1 to count.index results in 1, 2, and 3, which are used to generate the names cos-instance-1, cos-instance-2, and cos-instance-3.

Terraform will perform the following actions:

  # ibm_resource_instance.cos_instance[0] will be created
  + resource "ibm_resource_instance" "cos_instance" {
      + account_id              = (known after apply)
      + id                      = (known after apply)
      + name                    = "cos-instance-1"
      + resource_group_id       = "12345"
    }

  # ibm_resource_instance.cos_instance[1] will be created
  + resource "ibm_resource_instance" "cos_instance" {
      + account_id              = (known after apply)
      + id                      = (known after apply)
      + name                    = "cos-instance-2"
      + resource_group_id       = "12345"    
    }

  # ibm_resource_instance.cos_instance[2] will be created
  + resource "ibm_resource_instance" "cos_instance" {
      + account_id              = (known after apply)
      + id                      = (known after apply)
      + name                    = "cos-instance-3"
      + resource_group_id       = "12345"
    }

Plan: 3 to add, 0 to change, 0 to destroy.

In addition to directly specifying the value of count parameter, you can derive the number of instances dynamically by using the length() function on a variable. This is particularly useful when the desired number of instances is based on a list of names or configurations, and usually when the length of required input is unknown.

resource "ibm_resource_instance" "cos_instance" {
  count             = length(var.cos_instances)
  name              = var.cos_instances[count.index]
  resource_group_id = "12345"
  service           = "cloud-object-storage"
  plan              = "standard"
  location          = "global"
}

variable "cos_instances" {
  description = "This variable contains the names of COS instances"
  type = list(string)
  default     = ["cos-1", "cos-2", "cos-3"]
}

Explanation:

  1. Variable cos_instances: This variable contains the list of names for the COS instances.

  2. count = length(var.cos_instances): The length() function calculates the number of items in the cos_instances list. In this case, the count will be 3.

  3. name = var.cos_instances[count.index]: The count.index acts as an iterator to fetch the name from the cos_instances list which is similar to array lookup syntax in other languages like python ARRAY[<INDEX>]. For each iteration:

    • count.index = 0 → Name is cos-1

    • count.index = 1 → Name is cos-2

    • count.index = 2 → Name is cos-3

Terraform will perform the following actions:

  # ibm_resource_instance.cos_instance[0] will be created
  + resource "ibm_resource_instance" "cos_instance" {
      + name                    = "cos-1"
      + resource_group_id       = "12345"
      + service                 = "cloud-object-storage"
      + plan                    = "standard"
      + location                = "global"
    }

  # ibm_resource_instance.cos_instance[1] will be created
  + resource "ibm_resource_instance" "cos_instance" {
      + name                    = "cos-2"
      + resource_group_id       = "12345"
      + service                 = "cloud-object-storage"
      + plan                    = "standard"
      + location                = "global"
    }

  # ibm_resource_instance.cos_instance[2] will be created
  + resource "ibm_resource_instance" "cos_instance" {
      + name                    = "cos-3"
      + resource_group_id       = "12345"
      + service                 = "cloud-object-storage"
      + plan                    = "standard"
      + location                = "global"
    }

Plan: 3 to add, 0 to change, 0 to destroy.

Understanding the Plan Output:

When you use the count parameter on a resource, Terraform creates an array of resources rather than a single resource. Each instance of the resource has an index starting from 0, representing its position in the array.

The resource ibm_resource_instance.cos_instance has been expanded into an array with three elements:

  • ibm_resource_instance.cos_instance[0]: Corresponds to the first instance with the name cos-1.

  • ibm_resource_instance.cos_instance[1]: Corresponds to the second instance with the name cos-2.

  • ibm_resource_instance.cos_instance[2]: Corresponds to the third instance with the name cos-3.

Extracting Specific Attributes:

To fetch a specific attribute from one of the instances, use the syntax:

<PROVIDER>_<TYPE>.<NAME>[INDEX].ATTRIBUTE

  • If you want to retrieve the name attribute of the second COS instance (cos-instance-2), you would use: ibm_resource_instance.cos_instance[1].name

  • If you want to retrieve the name attribute of all COS instances in the array, you would use: ibm_resource_instance.cos_instance[*].name

You can define an output in your Terraform configuration to retrieve and display the attribute values of the created resources:

output "cos_instances_id" {
  description = "ID of all COS instances"
  value = ibm_resource_instance.cos_instance[*].id
}

output "cos_instance_1_id" {
  description = "ID of first COS instance"
  value = ibm_resource_instance.cos_instance[0].id
}

Using count with Modules

The count can be added to a module block. Let's say there is a module that creates COS(cloud object storage). You could use this module with a count parameter to create three cos instances as follows:

module "cos_instance" {
  count             = length(var.cos_instances)
  source            = "./modules/cos_instance"
  name              = var.cos_instances[count.index]
  resource_group_id = "12345"
  plan              = "standard"
  location          = "global"
}

variable "cos_instances" {
  description = "This variable contains names of COS instances"
  default     = ["cos-1", "cos-2", "cos-3"]
}

output "cos_instances_id" {
  description = "ID of all COS instances"
  value = module.cos_instance[*].id
}

Just as the count parameter transforms a resource into an array of resources, adding count to a module turns it into an array of modules. This is particularly useful when you need to reuse a module multiple times with slightly different configurations.

Limitations of Count

The count parameter in Terraform is a powerful tool for creating multiple instances of a resource, but it comes with certain limitations.

Terraform's count parameter applies to an entire resource or module, but it cannot be used to loop over inline blocks (nested blocks) inside a resource. This means you cannot dynamically repeat sections like filters in the following example:

resource "ibm_logs_alert" "logs_alert_instance" {
  instance_id = "56789"
  region      = "us-south"
  name        = "example-alert-description"
  is_active   = true
  severity    = "info_or_unspecified"

  # This is an inline block that cannot use count to repeat or loop through filters
  filters {
    text        = "text"
    filter_type = "text_or_unspecified"
  }
}

You might think of using the count parameter inside the filters block to create multiple filter conditions dynamically for your alerts. However, Terraform does not support applying count within inline blocks like filters. Inline blocks are tied to a single resource instance and cannot independently leverage count for looping or repetition.

The count parameter comes with another limitation that can lead to unexpected and potentially dangerous outcomes. Let’s explore this with an example we discussed earlier.

resource "ibm_resource_instance" "cos_instance" {
  count             = length(var.cos_instances)
  name              = var.cos_instances[count.index]
  resource_group_id = "12345"
  service           = "cloud-object-storage"
  plan              = "standard"
  location          = "global"
}

variable "cos_instances" {
  description = "This variable contains the names of COS instances"
  type = list(string)
  default     = ["cos-1", "cos-2", "cos-3"]
}

Now as discussed previously, the following code will produce 3 resources. Each resource will be identified by its index:

  • ibm_resource_instance.cos_instance[0]: Corresponds to "cos-1."

  • ibm_resource_instance.cos_instance[1]: Corresponds to "cos-2."

  • ibm_resource_instance.cos_instance[2]: Corresponds to "cos-3."

The Problem: Removing an Item from the List

Now, suppose we decide to remove the second COS instance, "cos-2," by updating the cos_instances list:

variable "cos_instances" {
  default = ["cos-1", "cos-3"]
}

When Terraform processes this change, the execution plan may not behave as expected:

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following
symbols:
  ~ update in-place
  - destroy

Terraform will perform the following actions:

  # ibm_resource_instance.cos_instance[1] will be updated in-place
  ~ resource "ibm_resource_instance" "cos_instance" {
      ~ name                    = "cos-2" -> "cos-3"
        tags                    = []
        # (37 unchanged attributes hidden)
    }

  # ibm_resource_instance.cos_instance[2] will be destroyed
  # (because index [2] is out of range for count)
  - resource "ibm_resource_instance" "cos_instance" {
      - name                    = "cos-3" -> null
      - resource_group_id       = "12345" -> null
      - service                 = "cloud-object-storage" -> null
      - plan                    = "standard" -> null
      - location                = "global" -> null
    }

Plan: 0 to add, 1 to change, 1 to destroy.

Instead of simply removing "cos-2," Terraform renames "cos-2" to "cos-3" and deletes the original "cos-3."

Why Does This Happen?

Terraform uses the count.index value as a resource’s identity. When an item is removed from the middle of the list, the indices shift. In this example:

Before:
ibm_resource_instance.cos_instance[0]: cos-1
ibm_resource_instance.cos_instance[1]: cos-2
ibm_resource_instance.cos_instance[2]: cos-3

After:
ibm_resource_instance.cos_instance[0]: cos-1
ibm_resource_instance.cos_instance[1]: cos-3

Terraform interprets this as a need to:

  1. Rename the resource at index 1 from "cos-2" to "cos-3."

  2. Destroy the resource at index 2 ("cos-3"), as it is now out of range.

The Consequences

  • Loss of Availability: Resources that should remain untouched may be destroyed and recreated.

  • Potential Data Loss: If the resource being destroyed is a database or storage instance which is in our example, valuable data could be lost.


Addressing the Limitation

To avoid these pitfalls, you can use the for_each expression instead of count. Unlike count, for_each uses unique keys to identify resources, preventing unintentional renaming or deletion. We will explore for_each in detail in the next article.


Conclusion

In this article, we explored the count parameter in Terraform, learning how it can be used to create multiple instances of resources and modules. We also discussed its limitations, particularly how changes to the resource list can lead to unintended renaming or deletion.

In the next article, we will delve into the for_each expression to understand how it overcomes the limitations of count, ensuring safer and more predictable resource management.