One place for hosting & domains

      Terraform

      How To Manage Infrastructure Data with Terraform Outputs


      [*]

      Introduction

      Terraform outputs are used to extract information about the infrastructure resources from the project state. Using other features of the Hashicorp Configuration Language (HCL), which Terraform uses, resource information can be queried and transformed into more complex data structures, such as lists and maps. Outputs are useful for providing information to external software, which can operate on the created infrastructure resources.

      In this tutorial, you’ll learn about Terraform output syntax and its parameters by creating a simple infrastructure that deploys Droplets. You’ll also parse the outputs programmatically by converting them to JSON.

      Prerequisites

      Note: This tutorial has specifically been tested with Terraform 0.13.

      Defining Outputs

      In this section, you’ll declare a Droplet, deploy it to the cloud, and learn about outputs by defining one that will show the Droplet’s IP address.

      Assuming you are in the terraform-outputs directory, create and open the droplets.tf file for editing:

      Add the following Droplet resource and output definition:

      terraform-outputs/droplets.tf

      resource "digitalocean_droplet" "web" {
        image  = "ubuntu-18-04-x64"
        name   = "test-droplet"
        region = "fra1"
        size   = "s-1vcpu-1gb"
      }
      
      output "droplet_ip_address" {
        value = digitalocean_droplet.web.ipv4_address
      }
      

      You first declare a Droplet resource, called web. Its actual name in the cloud will be test-droplet, in the region fra1, running Ubuntu 18.04.

      Then, you declare an output called droplet_ip_address. In Terraform, outputs are used to export and show internal and computed values and information about the resources. Here, you set the value parameter, which accepts the data to output, to the IP address of the declared Droplet. At declare time, it’s unknown, but it will become available once the Droplet is deployed. Outputs are shown and accessible after each deployment.

      Save and close the file, then deploy the project by running the following command:

      • terraform apply -var "do_token=${DO_PAT}"

      Enter yes to apply when prompted. The end of the output you’ll see will be similar to this:

      Output

      ... digitalocean_droplet.web: Creating... ... digitalocean_droplet.web: Creation complete after 32s [id=207631771] Apply complete! Resources: 1 added, 0 changed, 0 destroyed. Outputs: droplet_ip_address = ip_address

      The highlighted IP address belongs to your newly deployed Droplet. Applying the project deploys the resources to the cloud and shows the outputs at the end, when all resource attributes are available. Without the droplet_ip_address output, Terraform would show no further information about the Droplet, except that it’s deployed.

      Outputs can also be shown using the output command:

      The output will list all outputs in the project:

      Output

      droplet_ip_address = ip_address

      You can also query a specific output by name by specifying it as an argument:

      • terraform output output_name

      For droplet_ip_address, the output will consist of the IP address only:

      Output

      ip_address

      Except for specifying the mandatory value, outputs have a few optional parameters:

      • description: embeds short documentation detailing what the output shows.
      • sensitive: accepts a boolean value, which prevents the content of the output from being shown after deploying if set to true.
      • depends_on: a meta parameter available at each resource that allows you to explicitly specify resources the output depends on, that Terraform is not able to automatically deduce during planning.

      The sensitive parameter is useful when the logs of the Terraform deployment will be publicly available, but the output contents should be kept hidden. You’ll now add it to your Droplet resource definition.

      Open droplets.tf for editing and add the highlighted line:

      terraform-outputs/droplets.tf

      resource "digitalocean_droplet" "web" {
        image  = "ubuntu-18-04-x64"
        name   = "test-droplet"
        region = "fra1"
        size   = "s-1vcpu-1gb"
      }
      
      output "droplet_ip_address" {
        value      = digitalocean_droplet.web.ipv4_address
        sensitive = true
      }
      

      Save and close the file when you’re done. You can try deploying the project again by running:

      • terraform apply -var "do_token=${DO_PAT}"

      You’ll see that the output is redacted:

      Output

      digitalocean_droplet.web: Refreshing state... [id=207631771] Apply complete! Resources: 0 added, 0 changed, 0 destroyed. Outputs: droplet_ip_address = <sensitive>

      Even if it’s marked as sensitive, the output and its contents will still be available through other channels, such as viewing the Terraform state or querying the outputs directly.

      In the next step, you’ll create a different Droplet and output structure, so destroy the currently deployed ones by running:

      • terraform destroy -var "do_token=${DO_PAT}"

      The output at the very end will be:

      Output

      ... Destroy complete! Resources: 1 destroyed.

      You’ve declared and deployed a Droplet and created an output that shows its IP address. You’ll now learn about using outputs to show more complex structures, such as lists and maps.

      Outputting Complex Structures

      In this section, you’ll deploy multiple Droplets from the same definition using the count keyword, and output their IP addresses in various formats.

      Using the for loop

      You’ll need to modify the Droplet resource definition, so open it for editing:

      Modify it to look like this:

      terraform-outputs/droplets.tf

      resource "digitalocean_droplet" "web" {
        count  = 3
        image  = "ubuntu-18-04-x64"
        name   = "test-droplet-${count.index}"
        region = "fra1"
        size   = "s-1vcpu-1gb"
      }
      

      You’ve specified that three Droplets should be created using the count key and added the current index to the Droplet name, so that you’ll be able to later discern between them. When you’re done, save and close the file.

      Apply the code by running:

      • terraform apply -var "do_token=${DO_PAT}"

      Terraform will plan the creation of three numbered Droplets, called test-droplet-0, test-droplet-1, and test-droplet-2. Enter yes when prompted to finish the process. You’ll see the following output in the end:

      Output

      ... Apply complete! Resources: 3 added, 0 changed, 0 destroyed.

      This means that all three Droplets are successfully deployed and that all information about them is stored in the project state.

      The easiest way to access their resource attributes is to use outputs, though creating one for each of the Droplets is not scalable. The solution is to use the for loop to traverse through the list of Droplets and gather their attributes, or to alternatively use splat expressions. You’ll learn about them later in this step.

      You’ll first define an output that will output the IP addresses of the three Droplets, paired with their names. Open droplets.tf for editing:

      Add the following lines:

      terraform-outputs/droplets.tf

      resource "digitalocean_droplet" "web" {
        count  = 3
        image  = "ubuntu-18-04-x64"
        name   = "test-droplet-${count.index}"
        region = "fra1"
        size   = "s-1vcpu-1gb"
      }
      
      output "droplet_ip_addresses" {
        value = {
          for droplet in digitalocean_droplet.web:
          droplet.name => droplet.ipv4_address
        }
      }
      

      The output value of droplet_ip_addresses is constructed using a for loop. Because it’s surrounded by braces, the resulting type will be a map. The loop traverses the list of Droplets, and for each instance, pairs its name with its IP address and appends it to the resulting map.

      Save and close the file, then try applying the project again:

      • terraform apply -var "do_token=${DO_PAT}"

      Enter yes when prompted and you’ll receive the output contents at the end:

      Output

      Apply complete! Resources: 0 added, 0 changed, 0 destroyed. Outputs: droplet_ip_addresses = { "test-droplet-0" = "ip_address" "test-droplet-1" = "ip_address" "test-droplet-2" = "ip_address" }

      The droplet_ip_addresses output details the IP addresses of the three deployed droplets.

      Using the Terraform output command, you can get the contents of the output as JSON using its command argument:

      • terraform output -json droplet_ip_addresses

      The result will be similar to the following:

      Output

      {"test-droplet-0":"ip_address","test-droplet-1":"ip_address","test-droplet-2":"ip_address"}

      JSON parsing is widely used and supported in many programming languages. This way, you can programmatically parse the information about the deployed Droplet resources.

      Using Splat Expressions

      Splat expressions offer a compact way of iterating over all elements of a list, and collecting contents of an attribute from each of them, resulting in a list. A splat expression that would extract the IP addresses of the three deployed droplets would have the following syntax:

      digitalocean_droplet.web[*].ipv4_address
      

      The [*] symbol traverses the list on its left and for each of the elements, takes the contents of its attribute specified on the right. If the reference on the left is not a list by itself, it will be converted to one where it will be the sole element.

      You can open droplets.tf for editing and modify the following lines to implement this:

      terraform-outputs/droplets.tf

      resource "digitalocean_droplet" "web" {
        count  = 3
        image  = "ubuntu-18-04-x64"
        name   = "test-droplet-${count.index}"
        region = "fra1"
        size   = "s-1vcpu-1gb"
      }
      
      output "droplet_ip_addresses" {
        value = digitalocean_droplet.web[*].ipv4_address
      }
      

      After saving the file, apply the project by running the following command:

      • terraform apply -var "do_token=${DO_PAT}"

      You’ll receive output that is now a list, and contains only the IP addresses of the Droplets:

      Output

      Apply complete! Resources: 0 added, 0 changed, 0 destroyed. Outputs: droplet_ip_addresses = [ "ip_address", "ip_address", "ip_address", ]

      To receive the output as JSON, run the following command:

      • terraform output -json droplet_ip_addresses

      The output will be a single array:

      Output

      ["ip_address","ip_address","ip_address"]

      You’ve used outputs together with splat expressions and for loops to export IP addresses of the deployed Droplets. You’ve also received the output contents as JSON, and you’ll now use jq—a tool for dynamically filtering JSON according to given expressions—to parse them.

      Parsing Outputs Using jq

      In this step, you’ll install and learn the basics of jq, a tool for manipulating JSON documents. You’ll use it to parse the outputs of your Terraform project.

      If you’re on Ubuntu, run the following command to install jq:

      On macOS, you can use Homebrew to install it:

      jq applies the provided processing expression on given input, which can be piped in. The easiest task in jq is to pretty print the input:

      • terraform output -json droplet_ip_addresses | jq '.'

      Passing in the identity operator (.) means that the whole JSON document parsed from the input should be outputted without modifications:

      Output

      [ "first_ip_address", "second_ip_address", "third_ip_address" ]

      You can request just the second IP address using the array bracket notation, counting from zero:

      • terraform output -json droplet_ip_addresses | jq '.[1]'

      The output will be:

      Output

      "second_ip_address"

      To make the result of the processing an array, wrap the expression in brackets:

      • terraform output -json droplet_ip_addresses | jq '[.[1]]'

      You’ll get a pretty printed JSON array:

      Output

      [ "second_ip_address" ]

      You can retrieve parts of arrays instead of single elements by specifying a range of indexes inside the brackets:

      • terraform output -json droplet_ip_addresses | jq '.[0:2]'

      The output will be:

      Output

      [ "first_ip_address", "second_ip_address" ]

      The range 0:2 returns the first two elements—the upper part of the range (2) is not inclusive, so only elements at positions 0 and 1 are fetched.

      You can now destroy the deployed resources by running:

      • terraform destroy -var "do_token=${DO_PAT}"

      In this step, you have installed jq and used it to parse and manipulate the output of your Terraform project, which deploys three Droplets.

      Conclusion

      You have learned about Terraform outputs, how they are used to show details about the deployed resources, and how they can be used to export data structures for later external processing. You’ve also seen how to use outputs to show attributes of a single resource, as well as for showing constructed maps and lists containing resource attributes.

      For more detailed information about the features of jq, visit the official docs.

      To learn more about Terraform check out the series: How To Manage Infrastructure with Terraform.

      [*]
      [*]Source link

      How To Improve Flexibility Using Terraform Variables, Dependencies, and Conditionals


      Introduction

      Hashicorp Configuration Language (HCL), which Terraform uses, provides many useful structures and capabilities that are present in other programming languages. Using loops in your infrastructure code can greatly reduce code duplication and increase readability, allowing for easier future refactoring and greater flexibility. HCL also provides a few common data structures, such as lists and maps (also called arrays and dictionaries respectively in other languages), as well as conditionals for execution path branching.

      Unique to Terraform is the ability to manually specify the resources one depends on. While the execution graph it builds when running your code already contains the detected links (which are correct in most scenarios), you may find yourself in need of forcing a dependency relationship that Terraform was unable to detect.

      In this article, we’ll review the data structures HCL provides, its looping features for resources (the count key, for_each, and for), and writing conditionals to handle known and unknown values, as well as explicitly specifying dependency relationships between resources.

      Prerequisites

      • A DigitalOcean account. If you do not have one, sign up for a new account.

      • A DigitalOcean Personal Access Token, which you can create via the DigitalOcean control panel. Instructions to do that can be found in this link: How to Generate a Personal Access Token.

      • Terraform installed on your local machine and a project set up with the DigitalOcean provider. Complete Step 1 and Step 2 of the How To Use Terraform with DigitalOcean tutorial, and be sure to name the project folder terraform-flexibility, instead of loadbalance. During Step 2, you do not need to include the pvt_key variable and the SSH key resource.

      • A fully registered domain name added to your DigitalOcean account. For instructions on how to do that, visit the official docs.

      Note: This tutorial has specifically been tested with Terraform 0.13.

      Data Types in HCL

      In this section, before you learn more about loops and other features of HCL that make your code more flexible, we’ll first go over the available data types and their uses.

      The Hashicorp Configuration Language supports primitive and complex data types. Primitive data types are strings, numbers, and boolean values, which are the basic types that can not be derived from others. Complex types, on the other hand, group multiple values into a single one. The two types of complex values are structural and collection types.

      Structural types allow values of different types to be grouped together. The main example is the resource definitions you use to specify what your infrastructure will look like. Compared to the structural types, collection types also group values, but only ones of the same type. The three collection types available in HCL that we are interested in are lists, maps, and sets.

      Lists

      Lists are similar to arrays in other programming languages. They contain a known number of elements of the same type, which can be accessed using the array notation ([]) by their whole-number index, starting from 0. Here is an example of a list variable declaration holding names of Droplets you’ll deploy in the next steps:

      variable "droplet_names" {
        type    = list(string)
        default = ["first", "second", "third"]
      }
      

      For the type, you explicitly specify that it’s a list whose element type is string, and then provide its default value. Values enumerated in brackets signify a list in HCL.

      Maps

      Maps are collections of key-value pairs, where each value is accessed using its key of type string. There are two ways of specifying maps inside curly brackets: by using colons (:) or equal signs (=) for specifying values. In both situations, the value must be enclosed with quotes. When using colons, the key must too be enclosed.

      The following map definition containing Droplet names for different environments is written using the equal sign:

      variable "droplet_env_names" {
        type = map(string)
      
        default = {
          development = "dev-droplet"
          staging = "staging-droplet"
          production = "prod-droplet"
        }
      }
      

      If the key starts with a number, you must use the colon syntax:

      variable "droplet_env_names" {
        type = map(string)
      
        default = {
          "1-development": "dev-droplet"
          "2-staging": "staging-droplet"
          "3-production": "prod-droplet"
        }
      }
      

      Sets

      Sets do not support element ordering, meaning that traversing sets is not guaranteed to yield the same order each time and that their elements can not be accessed in a targeted way. They contain unique elements repeated exactly once, and specifying the same element multiple times will result in them being coalesced with only one instance being present in the set.

      Declaring a set is similar to declaring a list, the only difference being the type of the variable:

      variable "droplet_names" {
        type    = set(string)
        default = ["first", "second", "third", "fourth"]
      }
      

      Now that you’ve learned about the types of data structures HCL offers and reviewed the syntax of lists, maps, and sets, which we’ll use throughout this tutorial, you’ll move on to trying some flexible ways of deploying multiple instances of the same resource in Terraform.

      Setting the Number of Resources Using the count Key

      In this section, you’ll create multiple instances of the same resource using the count key. The count key is a parameter available on all resources that specifies how many instances of it to create.

      You’ll see how it works by writing a Droplet resource, which you’ll store in a file named droplets.tf, in the project directory you created as part of the prerequisites. Create and open it for editing by running:

      Add the following lines:

      terraform-flexibility/droplets.tf

      resource "digitalocean_droplet" "test_droplet" {
        count  = 3
        image  = "ubuntu-18-04-x64"
        name   = "web"
        region = "fra1"
        size   = "s-1vcpu-1gb"
      }
      

      This code defines a Droplet resource called test_droplet, running Ubuntu 18.04 with 1GB RAM.

      Note that the value of count is set to 3, which means that Terraform will attempt to create three instances of the same resource. When you are done, save and close the file.

      You can plan the project to see what actions Terraform would take by running:

      • terraform plan -var "do_token=${DO_PAT}"

      The output will be similar to this:

      Output

      ... An execution plan has been generated and is shown below. Resource actions are indicated with the following symbols: + create Terraform will perform the following actions: # digitalocean_droplet.test_droplet[0] will be created + resource "digitalocean_droplet" "test_droplet" { ... name = "web" ... } # digitalocean_droplet.test_droplet[1] will be created + resource "digitalocean_droplet" "test_droplet" { ... name = "web" ... } # digitalocean_droplet.test_droplet[2] will be created + resource "digitalocean_droplet" "test_droplet" { ... name = "web" ... } Plan: 3 to add, 0 to change, 0 to destroy. ...

      The output details that Terraform would create three instances of test_droplet, all with the same name web. While possible, it is not preferred, so let’s modify the Droplet definition to make the name of each instance different. Open droplets.tf for editing:

      Modify the highlighted line:

      terraform-flexibility/droplets.tf

      resource "digitalocean_droplet" "test_droplet" {
        count  = 3
        image  = "ubuntu-18-04-x64"
        name   = "web.${count.index}"
        region = "fra1"
        size   = "s-1vcpu-1gb"
      }
      

      Save and close the file.

      The count object provides the index parameter, which contains the index of the current iteration, starting from 0. The current index is substituted into the name of the Droplet using string interpolation, which allows you to dynamically build a string by substituting variables. You can plan the project again to see the changes:

      • terraform plan -var "do_token=${DO_PAT}"

      The output will be similar to this:

      Output

      ... An execution plan has been generated and is shown below. Resource actions are indicated with the following symbols: + create Terraform will perform the following actions: # digitalocean_droplet.test_droplet[0] will be created + resource "digitalocean_droplet" "test_droplet" { ... name = "web.0" ... } # digitalocean_droplet.test_droplet[1] will be created + resource "digitalocean_droplet" "test_droplet" { ... name = "web.1" ... } # digitalocean_droplet.test_droplet[2] will be created + resource "digitalocean_droplet" "test_droplet" { ... name = "web.2" ... } Plan: 3 to add, 0 to change, 0 to destroy. ...

      This time, the three instances of test_droplet will have their index in their names, making them easier to track.

      You now know how to create multiple instances of a resource using the count key, as well as fetch and use the index of an instance during provisioning. Next, you’ll learn how to fetch the Droplet’s name from a list.

      Getting Droplet Names From a List

      In situations when multiple instances of the same resource need to have custom names, you can dynamically retrieve them from a list variable you define. During the rest of the tutorial, you’ll see several ways of automating Droplet deployment from a list of names, promoting flexibility and ease of use.

      You’ll first need to define a list containing the Droplet names. Create a file called variables.tf and open it for editing:

      Add the following lines:

      terraform-flexibility/variables.tf

      variable "droplet_names" {
        type    = list(string)
        default = ["first", "second", "third", "fourth"]
      }
      

      Save and close the file. This code defines a list called droplet_names, containing the strings first, second, third, and fourth.

      Open droplets.tf for editing:

      Modify the highlighted lines:

      terraform-flexibility/droplets.tf

      resource "digitalocean_droplet" "test_droplet" {
        count  = length(var.droplet_names)
        image  = "ubuntu-18-04-x64"
        name   =  var.droplet_names[count.index]
        region = "fra1"
        size   = "s-1vcpu-1gb"
      }
      

      To improve flexibility, instead of manually specifying a constant number of elements, you pass in the length of the droplet_names list to the count parameter, which will always return the number of elements in the list. For the name, you fetch the element of the list positioned at count.index, using the array bracket notation. Save and close the file when you’re done.

      Try planning the project again. You’ll receive output similar to this:

      Output

      ... An execution plan has been generated and is shown below. Resource actions are indicated with the following symbols: + create Terraform will perform the following actions: # digitalocean_droplet.test_droplet[0] will be created + resource "digitalocean_droplet" "test_droplet" { ... + name = "first" ... } # digitalocean_droplet.test_droplet[1] will be created + resource "digitalocean_droplet" "test_droplet" { ... + name = "second" ... } # digitalocean_droplet.test_droplet[2] will be created + resource "digitalocean_droplet" "test_droplet" { ... + name = "third" ... } # digitalocean_droplet.test_droplet[3] will be created + resource "digitalocean_droplet" "test_droplet" { ... + name = "fourth" ... Plan: 4 to add, 0 to change, 0 to destroy. ...

      As a result of modifications, four Droplets would be deployed, successively named after the elements of the droplet_names list.

      You’ve learned about count, its features and syntax, and using it together with a list to modify the resource instances. You’ll now see its disadvantages, and how to overcome them.

      Understanding the Disadvantages of count

      Now that you know how count is used, you’ll see its disadvantages when modifying the list it’s used with.

      Let’s try deploying the Droplets to the cloud:

      • terraform apply -var "do_token=${DO_PAT}"

      Enter yes when prompted. The end of your output will be similar to this:

      Output

      Apply complete! Resources: 4 added, 0 changed, 0 destroyed.

      Now let’s create one more Droplet instance by enlarging the droplet_names list. Open variables.tf for editing:

      Add a new element to the beginning of the list:

      terraform-flexibility/variables.tf

      variable "droplet_names" {
        type    = list(string)
        default = ["zero", "first", "second", "third", "fourth"]
      }
      

      When you’re done, save and close the file.

      Plan the project:

      • terraform plan -var "do_token=${DO_PAT}"

      You’ll receive output like this:

      Output

      ... An execution plan has been generated and is shown below. Resource actions are indicated with the following symbols: + create ~ update in-place Terraform will perform the following actions: # digitalocean_droplet.test_droplet[0] will be updated in-place ~ resource "digitalocean_droplet" "test_droplet" { ... ~ name = "first" -> "zero" ... } # digitalocean_droplet.test_droplet[1] will be updated in-place ~ resource "digitalocean_droplet" "test_droplet" { ... ~ name = "second" -> "first" ... } # digitalocean_droplet.test_droplet[2] will be updated in-place ~ resource "digitalocean_droplet" "test_droplet" { ... ~ name = "third" -> "second" ... } # digitalocean_droplet.test_droplet[3] will be updated in-place ~ resource "digitalocean_droplet" "test_droplet" { ... ~ name = "fourth" -> "third" ... } # digitalocean_droplet.test_droplet[4] will be created + resource "digitalocean_droplet" "test_droplet" { ... + name = "fourth" ... } Plan: 1 to add, 4 to change, 0 to destroy. ...

      The output shows that Terraform would rename the first four Droplets and create a fifth one called fourth, because it considers the instances as an ordered list and identifies the elements (Droplets) by their index number in the list. This is how Terraform initially considers the four Droplets:

      Index Number 0 1 2 3
      Droplet Name first second third fourth

      When the a new Droplet zero is added to the beginning, its internal list representation looks like this:

      Index Number 0 1 2 3 4
      Droplet Name zero first second third fourth

      The four initial Droplets are now shifted one place to the right. Terraform then compares the two states represented in tables: at position 0, the Droplet was called first, and because it’s different in the second table, it plans an update action. This continues until position 4, which does not have a comparable element in the first table, and instead a Droplet provisioning action is planned.

      This means that adding a new element to the list anywhere but to the very end would result in resources being modified when they do not need to be. Similar update actions would be planned if an element of the droplet_names list was removed.

      Incomplete resource tracking is the main downfall of using count for deploying a dynamic number of differing instances of the same resource. For a constant number of constant instances, count is a simple solution that works well. In situations like this, though, when some attributes are being pulled in from a variable, the for_each loop, which you’ll learn about later in this tutorial, is a much better choice.

      Referencing the Current Resource (self)

      Another downside of count is that referencing an arbitrary instance of a resource by its index is not possible in some cases.

      The main example is destroy-time provisioners, which run when the resource is planned to be destroyed. The reason is that the requested instance may not exist (it’s already destroyed) or would create a mutual dependency cycle. In such situations, instead of referring to the object through the list of instances, you can access only the current resource through the self keyword.

      To demonstrate its usage, you’ll now add a destroy-time local provisioner to the test_droplet definition, which will show a message when run. Open droplets.tf for editing:

      Add the following highlighted lines:

      terraform-flexibility/droplets.tf

      resource "digitalocean_droplet" "test_droplet" {
        count  = length(var.droplet_names)
        image  = "ubuntu-18-04-x64"
        name   =  var.droplet_names[count.index]
        region = "fra1"
        size   = "s-1vcpu-1gb"
      
        provisioner "local-exec" {
          when    = destroy
          command = "echo 'Droplet ${self.name} is being destroyed!'"
        }
      }
      

      Save and close the file.

      The local-exec provisioner runs a command on the local machine Terraform is running on. Because the when parameter is set to destroy, it will run only when the resource is going to be destroyed. The command it runs echoes a string to stdout, which substitutes the name of the current resource using self.name.

      Because you’ll be creating the Droplets in a different way in the next section, destroy the currently deployed ones by running the following command:

      • terraform destroy -var "do_token=${DO_PAT}"

      Enter yes when prompted. You’ll receive the local-exec provisioner being run four times:

      Output

      ... digitalocean_droplet.test_droplet["first"] (local-exec): Executing: ["/bin/sh" "-c" "echo 'Droplet first is being destroyed!'"] digitalocean_droplet.test_droplet["second"] (local-exec): Executing: ["/bin/sh" "-c" "echo 'Droplet second is being destroyed!'"] digitalocean_droplet.test_droplet["second"] (local-exec): Droplet second is being destroyed! digitalocean_droplet.test_droplet["third"] (local-exec): Executing: ["/bin/sh" "-c" "echo 'Droplet third is being destroyed!'"] digitalocean_droplet.test_droplet["third"] (local-exec): Droplet third is being destroyed! digitalocean_droplet.test_droplet["fourth"] (local-exec): Executing: ["/bin/sh" "-c" "echo 'Droplet fourth is being destroyed!'"] digitalocean_droplet.test_droplet["fourth"] (local-exec): Droplet fourth is being destroyed! digitalocean_droplet.test_droplet["first"] (local-exec): Droplet first is being destroyed! ...

      In this step, you learned the disadvantages of count. You’ll now learn about the for_each loop construct, which overcomes them and works on a wider array of variable types.

      Looping Using for_each

      In this section, you’ll consider the for_each loop, its syntax, and how it helps flexibility when defining resources with multiple instances.

      for_each is a parameter available on each resource, but unlike count, which requires a number of instances to create, for_each accepts a map or a set. Each element of the provided collection is traversed once and an instance is created for it. for_each makes the key and value available under the each keyword as attributes (the pair’s key and value as each.key and each.value, respectively). When a set is provided, the key and value will be the same.

      Because it provides the current element in the each object, you won’t have to manually access the desired element as you did with lists. In case of sets, that’s not even possible, as it has no observable ordering internally. Lists can also be passed in, but they must first be converted into a set using the toset function.

      The main advantage of using for_each, aside from being able to enumerate all three collection data types, is that only the actually affected elements will be modified, created, or deleted. If you change the order of the elements in the input, no actions will be planned, and if you add, remove, or modify an element from the input, appropriate actions will be planned only for that element.

      Let’s convert the Droplet resource from count to for_each and see how it works in practice. Open droplets.tf for editing by running:

      Modify the highlighted lines:

      terraform-flexibility/droplets.tf

      resource "digitalocean_droplet" "test_droplet" {
        for_each = toset(var.droplet_names)
        image    = "ubuntu-18-04-x64"
        name     = each.value
        region   = "fra1"
        size     = "s-1vcpu-1gb"
      }
      

      You can remove the local-exec provisioner. When you’re done, save and close the file.

      The first line replaces count and invokes for_each, passing in the droplet_names list in the form of a set using the toset function, which automatically converts the given input. For the Droplet name, you specify each.value, which holds the value of the current element from the set of Droplet names.

      Plan the project by running:

      • terraform plan -var "do_token=${DO_PAT}"

      The output will detail steps Terraform would take:

      Output

      ... An execution plan has been generated and is shown below. Resource actions are indicated with the following symbols: + create Terraform will perform the following actions: # digitalocean_droplet.test_droplet["first"] will be created + resource "digitalocean_droplet" "test_droplet" { ... + name = "first" ... } # digitalocean_droplet.test_droplet["fourth"] will be created + resource "digitalocean_droplet" "test_droplet" { ... + name = "fourth" ... } # digitalocean_droplet.test_droplet["second"] will be created + resource "digitalocean_droplet" "test_droplet" { ... + name = "second" ... } # digitalocean_droplet.test_droplet["third"] will be created + resource "digitalocean_droplet" "test_droplet" { ... + name = "third" ... } # digitalocean_droplet.test_droplet["zero"] will be created + resource "digitalocean_droplet" "test_droplet" { ... + name = "zero" ... } Plan: 5 to add, 0 to change, 0 to destroy. ...

      Unlike when using count, Terraform now considers each instance individually, and not as elements of an ordered list. Every instance is linked to an element of the given set, as signified by the shown string element in the brackets next to each resource that will be created.

      Apply the plan to the cloud by running:

      • terraform apply -var "do_token=${DO_PAT}"

      Enter yes when prompted. When it finishes, you’ll remove one element from the droplet_names list to demonstrate that other instances won’t be affected. Open variables.tf for editing:

      Modify the list to look like this:

      terraform-flexibility/variables.tf

      variable "droplet_names" {
        type    = list(string)
        default = ["first", "second", "third", "fourth"]
      }
      

      Save and close the file.

      Plan the project again, and you’ll receive the following output:

      Output

      ... An execution plan has been generated and is shown below. Resource actions are indicated with the following symbols: - destroy Terraform will perform the following actions: # digitalocean_droplet.test_droplet["zero"] will be destroyed - resource "digitalocean_droplet" "test_droplet" { ... - name = "zero" -> null ... } Plan: 0 to add, 0 to change, 1 to destroy. ...

      This time, Terraform would destroy only the removed instance (zero), and would not touch any of the other instances, which is the correct behavior.

      In this step, you’ve learned about for_each, how to use it, and its advantages over count. Next, you’ll learn about the for loop, its syntax and usage, and when it can be used to automate certain tasks.

      Looping Using for

      The for loop works on collections, and creates a new collection by applying a transformation to each element of the input. The exact type of the output will depend on whether the loop is surrounded by brackets ([]) or braces ({}), which give a list or a map, respectively. As such, it is suitable for querying resources and forming structured outputs for later processing.

      The general syntax of the for loop is:

      for element in collection:
      transform(element)
      if condition
      

      Similarly to other programming languages, you first name the traversal variable (element) and specify the collection to enumerate. The body of the loop is the transformational step, and the optional if clause can be used for filtering the input collection.

      You’ll now work through a few examples using outputs. You’ll store them in a file named outputs.tf. Create it for editing by running the following command:

      Add the following lines to output pairs of deployed Droplet names and their IP addresses:

      terraform-flexibility/outputs.tf

      output "ip_addresses" {
        value = {
          for instance in digitalocean_droplet.test_droplet:
          instance.name => instance.ipv4_address
        }
      }
      

      This code specifies an output called ip_addresses, and specifies a for loop that iterates over the instances of the test_droplet resource you’ve been customizing in the previous steps. Because the loop is surrounded by curly brackets, its output will be a map. The transformational step for maps is similar to lambda functions in other programming languages, and here it creates a key-value pair by combining the instance name as the key with its private IP as its value.

      Save and close the file, then refresh Terraform state to account for the new output by running:

      • terraform refresh -var "do_token=${DO_PAT}"

      The Terraform refresh command updates the local state with the actual infrastructure state in the cloud.

      Then, check the contents of the outputs:

      Output

      ip_addresses = { "first" = "ip_address" "fourth" = "ip_address" "second" = "ip_address" "third" = "ip_address" }

      Terraform has shown the contents of the ip_addresses output, which is a map constructed by the for loop. (The order of the entries may be different for you.) The loop will work seamlessly for every number of entries—meaning that you can add a new element to the droplet_names list and the new Droplet, which would be created without any further manual input, would also show up in this output automatically.

      By surrounding the for loop in square brackets, you can make the output a list. For example, you could output only Droplet IP addresses, which is useful for external software that may be parsing the data. The code would look like this:

      terraform-flexibility/outputs.tf

      output "ip_addresses" {
        value = [
          for instance in digitalocean_droplet.test_droplet:
          instance.ipv4_address
        ]
      }
      

      Here, the transformational step simply selects the IP address attribute. It would give the following output:

      Output

      ip_addresses = [ "ip_address", "ip_address", "ip_address", "ip_address", ]

      As was noted before, you can also filter the input collection using the if clause. Here is how you would write the loop if you’d filter it by the fra1 region:

      terraform-flexibility/outputs.tf

      output "ip_addresses" {
        value = [
          for instance in digitalocean_droplet.test_droplet:
          instance.ipv4_address
          if instance.region == "fra1"
        ]
      }
      

      In HCL, the == operator checks the equality of the values of the two sides—here it checks if instance.region is equal to fra1. If it is, the check passes and the instance is transformed and added to the output, otherwise it is skipped. The output of this code would be the same as the prior example, because all Droplet instances are in the fra1 region, according to the test_droplet resource definition. The if conditional is also useful when you want to filter the input collection for other values in your project, like the Droplet size or distribution.

      Because you’ll be creating resources differently in the next section, destroy the currently deployed ones by running the following command:

      • terraform destroy -var "do_token=${DO_PAT}"

      Enter yes when prompted to finish the process.

      We’ve gone over the for loop, its syntax, and examples of usage in outputs. You’ll now learn about conditionals and how they can be used together with count.

      Directives and Conditionals

      In one of the previous sections, you’ve seen the count key and how it works. You’ll now learn about ternary conditional operators, which you can use elsewhere in your Terraform code, and how they can be used with count.

      The syntax of the ternary operator is:

      condition ? value_if_true : value_if_false
      

      condition is an expression that computes to a boolean (true or false). If the condition is true, then the expression evaluates to value_if_true. On the other hand, if the condition is false, the result will be value_if_false.

      The main use of ternary operators is to enable or disable single resource creation according to the contents of a variable. This can be achieved by passing in the result of the comparison (either 1 or 0) to the count key on the desired resource.

      Let’s add a variable called create_droplet, which will control if a Droplet will be created. First, open variables.tf for editing:

      Add the highlighted lines:

      terraform-flexibility/variables.tf

      variable "droplet_names" {
        type    = list(string)
        default = ["first", "second", "third", "fourth"]
      }
      
      variable "create_droplet" {
        type = bool
        default = true
      }
      

      This code defines the create_droplet variable of type bool. Save and close the file.

      Then, to modify the Droplet declaration, open droplets.tf for editing by running:

      Modify your file like the following:

      terraform-flexibility/droplets.tf

      resource "digitalocean_droplet" "test_droplet" {
        count  = var.create_droplet ? 1 : 0
        image  = "ubuntu-18-04-x64"
        name   =  "test_droplet"
        region = "fra1"
        size   = "s-1vcpu-1gb"
      }
      

      For count, you use a ternary operator to return either 1 if the create_droplet variable is true, and 0 if false, which will result in no Droplets being provisioned. Save and close the file when you’re done.

      Plan the project execution plan with the variable set to false by running:

      • terraform plan -var "do_token=${DO_PAT}" -var "create_droplet=false"

      You’ll receive the following output:

      Output

      Refreshing Terraform state in-memory prior to plan... The refreshed state will be used to calculate this plan, but will not be persisted to local or remote state storage. ------------------------------------------------------------------------ No changes. Infrastructure is up-to-date. This means that Terraform did not detect any differences between your configuration and real physical resources that exist. As a result, no actions need to be performed.

      Because create_droplet was passed in the value of false, the count of instances is 0, and no Droplets will be created.

      You’ve reviewed how to use the ternary conditional operator together with the count key to enable a higher level of flexibility in choosing whether to deploy desired resources. Next you’ll learn about explicitly setting resource dependencies for your resources.

      Explicitly Setting Resource Dependencies

      While creating the execution plan for your project, Terraform detects dependency chains between resources and implicitly orders them so that they will be built in the appropriate order. In the majority of cases, it is able to detect relationships by scanning all expressions in resources and building a graph.

      However, when one resource requires access control settings to already be deployed at the cloud provider, in order to be provisioned, there is no clear sign to Terraform that they are related. In turn, Terraform will not know they are dependent on each other behaviorally. In such cases, the dependency must be manually specified using the depends_on argument.

      The depends_on key is available on each resource and used to specify to which resources one has hidden dependency links. Hidden dependency relationships form when a resource depends on another one’s behavior, without using any of its data in its declaration, which would prompt Terraform to connect them one way.

      Here is an example of how depends_on is specified in code:

      resource "digitalocean_droplet" "droplet" {
        image  = "ubuntu-18-04-x64"
        name   = "web"
        region = "fra1"
        size   = "s-1vcpu-1gb"
      
        depends_on = [
          # Resources...
        ]
      }
      

      It accepts a list of references to other resources, and it does not accept arbitrary expressions.

      depends_on should be used sparingly, and only when all other options are exhausted. Its use signifies that what you are trying to declare is stepping outside the boundaries of Terraform’s automated dependency detection system; it may signify that the resource is explicitly depending on more resources than it needs to.

      You’ve now learned about explicitly setting additional dependencies for a resource using the depends_on key, and when it should be used.

      Conclusion

      In this article, we’ve gone over the features of HCL that improve flexibility and scalability of your code, such as count for specifying the number of resource instances to deploy and for_each as an advanced way of looping over collection data types and customizing instances. When used correctly, they greatly reduce code duplication and operational overhead of managing the deployed infrastructure.

      You’ve also learned about conditionals and ternary operators, and how they can be utilized to control if a resource will get deployed. While Terraform’s automated dependency analysis system is quite capable, there may be cases where you need to manually specify resource dependencies using the depends_on key.

      To learn more about Terraform, check out our How To Manage Infrastructure with Terraform series.



      Source link

      How To Structure a Terraform Project


      Introduction

      Structuring Terraform projects appropriately according to their use cases and perceived complexity is essential to ensure their maintainability and extensibility in day-to-day operations. A systematic approach to properly organizing code files is necessary to ensure that the project remains scalable during deployment and usable to you and your team.

      In this tutorial, you’ll learn about structuring Terraform projects according to their general purpose and complexity. Then, you’ll create a project with a simple structure using the more common features of Terraform: variables, locals, data sources, and provisioners. In the end, your project will deploy an Ubuntu 18.04 server (Droplet) on DigitalOcean, install an Apache web server, and point your domain to the web server.

      Prerequisites

      Note: This tutorial has specifically been tested with Terraform 0.13.

      Understanding a Terraform Project’s Structure

      In this section, you’ll learn what Terraform considers a project, how you can structure the infrastructure code, and when to choose which approach. You’ll also learn about Terraform workspaces, what they do, and how Terraform is storing state.

      A resource is an entity of a cloud service (such as a DigitalOcean Droplet) declared in Terraform code that is created according to specified and inferred properties. Multiple resources form infrastructure with their mutual connections.

      Terraform uses a specialized programming language for defining infrastructure, called Hashicorp Configuration Language (HCL). HCL code is typically stored in files ending with the extension tf. A Terraform project is any directory that contains tf files and has been initialized using the init command, which sets up Terraform caches and default local state.

      Terraform state is the mechanism via which it keeps track of resources that are actually deployed in the cloud. State is stored in backends—locally, on disk, or remotely, on a file storage cloud service or specialized state management software, for optimal redundancy and reliability. You can read more about different backends in the Terraform documentation.

      Project workspaces allow you to have multiple states in the same backend, tied to the same configuration. This allows you to deploy multiple distinct instances of the same infrastructure. Each project starts with a workspace named default—this will be used if you do not explicitly create or switch to another one.

      Modules in Terraform (akin to libraries in other programming languages) are parametrized code containers enclosing multiple resource declarations. They allow you to abstract away a common part of your infrastructure and reuse it later with different inputs.

      A Terraform project can also include external code files for use with dynamic data inputs, which can parse the JSON output of a CLI command and offer it for use in resource declarations. In this tutorial, you’ll do this with a Python script.

      Now that you know what a Terraform project consists of, let’s review two general approaches of Terraform project structuring.

      Simple Structure

      Suitable for small and testing projects, with a few resources of varying types and variables. It has a few configuration files, usually one per resource type (or more helper ones together with a main), and no custom modules, because most of the resources are unique and there aren’t enough to be generalized and reused. Following this, most of the code is stored in the same directory, next to each other. These projects often have a few variables (such as an API key for accessing the cloud) and may use dynamic data inputs and other Terraform and HCL features, though not prominently.

      As an example of the file structure of this approach, this is what the project we’ll build in this tutorial will look like in the end:

      .
      └── tf/
          ├── versions.tf
          ├── variables.tf
          ├── provider.tf
          ├── droplets.tf
          ├── dns.tf
          ├── data-sources.tf
          └── external/
              └── name-generator.py
      

      As this project will deploy an Apache web server Droplet and set up DNS records, the definitions of project variables, the DigitalOcean Terraform provider, the Droplet, and DNS records will be stored in their respective files. The minimum required Terraform and DigitalOcean provider versions will be specified in versions.tf, while the Python script that will generate a name for the Droplet (and be used as a dynamic data source in data-sources.tf) will be stored in the external folder, to separate it from HCL code.

      Complex Structure

      Contrary to the simple structure, this approach is suitable for large projects, with clearly defined subdirectory structures containing multiple modules of varying levels of complexity, aside from the usual code. These modules can depend on each other. Coupled with version control systems, these projects can make extensive use of workspaces. This approach is suitable for larger projects managing multiple apps, while reusing code as much as possible.

      Development, staging, quality assurance, and production infrastructure instances can also be housed under the same project in different directories by relying on common modules, thus eliminating duplicate code and making the project the central source of truth. Here is the file structure of an example project with a more complex structure, containing multiple deployment apps, Terraform modules, and target cloud environments:

      .
      └── tf/
          ├── modules/
          │   ├── network/
          │   │   ├── main.tf
          │   │   ├── dns.tf
          │   │   ├── outputs.tf
          │   │   └── variables.tf
          │   └── spaces/
          │       ├── main.tf
          │       ├── outputs.tf
          │       └── variables.tf
          └── applications/
              ├── backend-app/
              │   ├── env/
              │   │   ├── dev.tfvars
              │   │   ├── staging.tfvars
              │   │   ├── qa.tfvars
              │   │   └── production.tfvars
              │   └── main.tf
              └── frontend-app/
                  ├── env/
                  │   ├── dev.tfvars
                  │   ├── staging.tfvars
                  │   ├── qa.tfvars
                  │   └── production.tfvars
                  └── main.tf
      

      This approach will further be explored later in this series.

      You now know what a Terraform project is, how to best structure it according to perceived complexity, and what role Terraform workspaces serve. In the next steps, you’ll create a project with a simple structure that will provision a Droplet with an Apache web server installed and DNS records set up for your domain. You’ll first initialize your project with the DigitalOcean provider and variables, and then proceed to define the Droplet, a dynamic data source to provide its name, and a DNS record for deployment.

      Step 1 — Setting Up Your Initial Project

      In this section, you’ll add the DigitalOcean Terraform provider to your project, define the project variables, and declare a DigitalOcean provider instance, so that Terraform will be able to connect to your account.

      Start off by creating a directory for your Terraform project with the following command:

      • mkdir ~/apache-droplet-terraform

      Navigate to it:

      • cd ~/apache-droplet-terraform

      Since this project will follow the simple structuring approach, you’ll store the provider, variables, Droplet, and DNS record code in separate files, per the file structure from the previous section. First, you’ll need to add the DigitalOcean Terraform provider to your project as a required provider.

      Create a file named versions.tf and open it for editing by running:

      Add the following lines:

      ~/apache-droplet-terraform/versions.tf

      terraform {
        required_providers {
          digitalocean = {
            source = "digitalocean/digitalocean"
            version = "1.22.2"
          }
        }
        required_version = ">= 0.13"
      }
      

      In this terraform block, you list the required providers (DigitalOcean, version 1.22.2) and set the minimal required version of Terraform to be higher or equal to 0.13. When you are done, save and close the file.

      Then, define the variables your project will expose in the variables.tf file, following the approach of storing different resource types in separate code files:

      Add the following variables:

      ~/apache-droplet-terraform/variables.tf

      variable "do_token" {}
      variable "domain_name" {}
      

      Save and close the file.

      The do_token variable will hold your DigitalOcean Personal Access Token and domain_name will specify your desired domain name. The deployed Droplet will have the SSH key, identified by the SSH fingerprint, automatically installed.

      Next, let’s define the DigitalOcean provider instance for this project. You’ll store it in a file named provider.tf. Create and open it for editing by running:

      Add the provider:

      ~/apache-droplet-terraform/provider.tf

      provider "digitalocean" {
        token = var.do_token
      }
      

      Save and exit when you’re done. You’ve defined the digitalocean provider, which corresponds to the required provider you specified earlier in provider.tf, and set its token to the value of the variable, which will be supplied during runtime.

      In this step, you have created a directory for your project, requested the DigitalOcean provider to be available, declared project variables, and set up the connection to a DigitalOcean provider instance to use an auth token that will be provided later. You’ll now write a script that will generate dynamic data for your project definitions.

      Step 2 — Creating a Python Script for Dynamic Data

      Before continuing on to defining the Droplet, you’ll create a Python script that will generate the Droplet’s name dynamically and declare a data source resource to parse it. The name will be generated by concatenating a constant string (web) with the current time of the local machine, expressed in the UNIX epoch format. A naming script can be useful when multiple Droplets are generated according to a naming scheme, to easily differentiate between them.

      You’ll store the script in a file named name-generator.py, in a directory named external. First, create the directory by running:

      The external directory resides in the root of your project and will store non-HCL code files, like the Python script you’ll write.

      Create name-generator.py under external and open it for editing:

      • nano external/name-generator.py

      Add the following code:

      external/name-generator.py

      import json, time
      
      fixed_name = "web"
      result = {
        "name": f"{fixed_name}-{int(time.time())}",
      }
      
      print(json.dumps(result))
      

      This Python script imports the json and time modules, declares a dictionary named result, and sets the value of the name key to an interpolated string, which combines the fixed_name with the current UNIX time of the machine it runs on. Then, the result is converted into JSON and outputted on stdout. The output will be different each time the script is run:

      Output

      {"name": "web-1597747959"}

      When you’re done, save and close the file.

      Note: Large and complex structured projects require more thought put into how external data sources are created and used, especially in terms of portability and error handling. Terraform expects the executed program to write a human-readable error message to stderr and gracefully exit with a non-zero status, which is something not shown in this step because of the simplicity of the task. Additionally, it expects the program to have no side effects, so that it can be re-run as many times as needed.

      For more info on what Terraform expects, visit the official docs on data sources.

      Now that the script is ready, you can define the data source, which will pull the data from the script. You’ll store the data source in a file named data-sources.tf in the root of your project as per the simple structuring approach.

      Create it for editing by running:

      Add the following definition:

      ~/apache-droplet-terraform/data-sources.tf

      data "external" "droplet_name" {
        program = ["python3", "${path.module}/external/name-generator.py"]
      }
      

      Save and close the file.

      This data source is called droplet_name and executes the name-generator.py script using Python 3, which resides in the external directory you just created. It automatically parses its output and provides the deserialized data under its result attribute for use within other resource definitions.

      With the data source now declared, you can define the Droplet that Apache will run on.

      Step 3 — Defining the Droplet

      In this step, you’ll write the definition of the Droplet resource and store it in a code file dedicated to Droplets, as per the simple structuring approach. Its name will come from the dynamic data source you have just created, and will be different each time it’s deployed.

      Create and open the droplets.tf file for editing:

      Add the following Droplet resource definition:

      ~/apache-droplet-terraform/droplets.tf

      data "digitalocean_ssh_key" "ssh_key" {
        name = "your_ssh_key_name"
      }
      
      resource "digitalocean_droplet" "web" {
        image  = "ubuntu-18-04-x64"
        name   = data.external.droplet_name.result.name
        region = "fra1"
        size   = "s-1vcpu-1gb"
        ssh_keys = [
          data.digitalocean_ssh_key.ssh_key.id
        ]
      }
      

      You first declare a DigitalOcean SSH key resource called ssh_key, which will fetch a key from your account by its name. Make sure to replace the highlighted code with your SSH key name.

      Then, you declare a Droplet resource, called web. Its actual name in the cloud will be different, because it’s being requested from the droplet_name external data source. To bootstrap the Droplet resource with a SSH key each time it’s deployed, the ID of the ssh_key is passed into the ssh_keys parameter, so that DigitalOcean will know which key to apply.

      For now, this is all you need to configure related to droplet.tf, so save and close the file when you’re done.

      You’ll now write the configuration for the DNS record that will point your domain to the just declared Droplet.

      Step 4 — Defining DNS Records

      The last step in the process is to configure the DNS record pointing to the Droplet from your domain.

      You’ll store the DNS config in a file named dns.tf, because it’s a separate resource type from the others you have created in the previous steps. Create and open it for editing:

      Add the following lines:

      ~/apache-droplet-terraform/dns.tf

      resource "digitalocean_record" "www" {
        domain = var.domain_name
        type   = "A"
        name   = "@"
        value  = digitalocean_droplet.web.ipv4_address
      }
      

      This code declares a DigitalOcean DNS record at your domain name (passed in using the variable), of type A. The record has a name of @, which is a placeholder routing to the domain itself and with the Droplet IP address as its value. You can replace the name value with something else, which will result in a subdomain being created.

      When you’re done, save and close the file.

      Now that you’ve configured the Droplet, the name generator data source, and a DNS record, you’ll move on to deploying the project in the cloud.

      Step 5 — Planning and Applying the Configuration

      In this section, you’ll initialize your Terraform project, deploy it to the cloud, and check that everything was provisioned correctly.

      Now that the project infrastructure is defined completely, all that is left to do before deploying it is to initialize the Terraform project. Do so by running the following command:

      You’ll receive the following output:

      Output

      Initializing the backend... Initializing provider plugins... - Finding digitalocean/digitalocean versions matching "1.22.2"... - Finding latest version of hashicorp/external... - Installing hashicorp/external v1.2.0... - Installed hashicorp/external v1.2.0 (signed by HashiCorp) - Installing digitalocean/digitalocean v1.22.2... - Installed digitalocean/digitalocean v1.22.2 (signed by a HashiCorp partner, key ID F82037E524B9C0E8) Partner and community providers are signed by their developers. If you'd like to know more about provider signing, you can read about it here: https://www.terraform.io/docs/plugins/signing.html The following providers do not have any version constraints in configuration, so the latest version was installed. To prevent automatic upgrades to new major versions that may contain breaking changes, we recommend adding version constraints in a required_providers block in your configuration, with the constraint strings suggested below. * hashicorp/external: version = "~> 1.2.0" Terraform has been successfully initialized! You may now begin working with Terraform. Try running "terraform plan" to see any changes that are required for your infrastructure. All Terraform commands should now work. If you ever set or change modules or backend configuration for Terraform, rerun this command to reinitialize your working directory. If you forget, other commands will detect it and remind you to do so if necessary.

      You’ll now be able to deploy your Droplet with a dynamically generated name and an accompanying domain to your DigitalOcean account.

      Start by defining the domain name, SSH key fingerprint, and your personal access token as environment variables, so you won’t have to copy the values each time you run Terraform. Run the following commands, replacing the highlighted values:

      • export DO_PAT="your_do_api_token"
      • export DO_DOMAIN_NAME="your_domain"

      You can find your API token in your DigitalOcean Control Panel.

      Run the plan command with the variable values passed in to see what steps Terraform would take to deploy your project:

      • terraform plan -var "do_token=${DO_PAT}" -var "domain_name=${DO_DOMAIN_NAME}"

      The output will be similar to the following:

      Output

      Refreshing Terraform state in-memory prior to plan... The refreshed state will be used to calculate this plan, but will not be persisted to local or remote state storage. data.digitalocean_ssh_key.ssh_key: Refreshing state... data.external.droplet_name: Refreshing state... ------------------------------------------------------------------------ An execution plan has been generated and is shown below. Resource actions are indicated with the following symbols: + create Terraform will perform the following actions: # digitalocean_droplet.web will be created + resource "digitalocean_droplet" "web" { + backups = false + created_at = (known after apply) + disk = (known after apply) + id = (known after apply) + image = "ubuntu-18-04-x64" + ipv4_address = (known after apply) + ipv4_address_private = (known after apply) + ipv6 = false + ipv6_address = (known after apply) + ipv6_address_private = (known after apply) + locked = (known after apply) + memory = (known after apply) + monitoring = false + name = "web-1597780013" + price_hourly = (known after apply) + price_monthly = (known after apply) + private_networking = (known after apply) + region = "fra1" + resize_disk = true + size = "s-1vcpu-1gb" + ssh_keys = [ + "...", ] + status = (known after apply) + urn = (known after apply) + vcpus = (known after apply) + volume_ids = (known after apply) + vpc_uuid = (known after apply) } # digitalocean_record.www will be created + resource "digitalocean_record" "www" { + domain = "your_domain" + fqdn = (known after apply) + id = (known after apply) + name = "@" + ttl = (known after apply) + type = "A" + value = (known after apply) } Plan: 2 to add, 0 to change, 0 to destroy. ------------------------------------------------------------------------ Note: You didn't specify an "-out" parameter to save this plan, so Terraform can't guarantee that exactly these actions will be performed if "terraform apply" is subsequently run.

      The lines starting with a green + signify that Terraform will create each of the resources that follow after—which is exactly what should happen, so you can apply the configuration:

      • terraform apply -var "do_token=${DO_PAT}" -var "domain_name=${DO_DOMAIN_NAME}"

      The output will be the same as before, except that this time you’ll be asked to confirm:

      Output

      Plan: 2 to add, 0 to change, 0 to destroy. Do you want to perform these actions? Terraform will perform the actions described above. Only 'yes' will be accepted to approve. Enter a value: `yes`

      Enter yes, and Terraform will provision your Droplet and the DNS record:

      Output

      digitalocean_droplet.web: Creating... ... digitalocean_droplet.web: Creation complete after 33s [id=204432105] digitalocean_record.www: Creating... digitalocean_record.www: Creation complete after 1s [id=110657456] Apply complete! Resources: 2 added, 0 changed, 0 destroyed.

      Terraform has now recorded the deployed resources in its state. To confirm that the DNS records and the Droplet were connected successfully, you can extract the IP address of the Droplet from the local state and check if it matches to public DNS records for your domain. Run the following command to get the IP address:

      • terraform show | grep "ipv4"

      You’ll receive your Droplet’s IP address:

      Output

      ipv4_address = "your_Droplet_IP"

      You can check the public A records by running:

      • nslookup -type=a your_domain | grep "Address" | tail -1

      The output will show the IP address to which the A record points:

      Output

      Address: your_Droplet_IP

      They are the same, as they should be, meaning that the Droplet and DNS record were provisioned successfully.

      For the changes in the next step to take place, destroy the deployed resources by running:

      • terraform destroy -var "do_token=${DO_PAT}" -var "domain_name=${DO_DOMAIN_NAME}"

      When prompted, enter yes to continue.

      In this step, you have created your infrastructure and applied it to your DigitalOcean account. You’ll now modify it to automatically install the Apache web server on the provisioned Droplet using Terraform provisioners.

      Step 6 — Running Code Using Provisioners

      Now you’ll set up the installation of the Apache web server on your deployed Droplet by using the remote-exec provisioner to execute custom commands.

      Terraform provisioners can be used to execute specific actions on created remote resources (the remote-exec provisioner) or the local machine the code is executing on (using the local-exec provisioner). If a provisioner fails, the node will be marked as tainted in current state, which means that it will be deleted and recreated during the next run.

      To connect to a provisioned Droplet, Terraform needs the private SSH key of the one set up on the Droplet. The best way to pass in the location of the private key is by using variables, so open variables.tf for editing:

      Add the highlighted line:

      ~/apache-droplet-terraform/variables.tf

      variable "do_token" {}
      variable "domain_name" {}
      variable "private_key" {}
      

      You have now added a new variable, called private_key, to your project. Save and close the file.

      Next, you’ll add the connection data and remote provisioner declarations to your Droplet configuration. Open droplets.tf for editing by running:

      Extend the existing code with the highlighted lines:

      ~/apache-droplet-terraform/droplets.tf

      data "digitalocean_ssh_key" "ssh_key" {
        name = "your_ssh_key_name"
      }
      
      resource "digitalocean_droplet" "web" {
        image  = "ubuntu-18-04-x64"
        name   = data.external.droplet_name.result.name
        region = "fra1"
        size   = "s-1vcpu-1gb"
        ssh_keys = [
          data.digitalocean_ssh_key.ssh_key.id
        ]
      
        connection {
          host        = self.ipv4_address
          user        = "root"
          type        = "ssh"
          private_key = file(var.private_key)
          timeout     = "2m"
        }
      
        provisioner "remote-exec" {
          inline = [
            "export PATH=$PATH:/usr/bin",
            # Install Apache
            "apt update",
            "apt -y install apache2"
          ]
        }
      }
      

      The connection block specifies how Terraform should connect to the target Droplet. The provisioner block contains the array of commands, within the inline parameter, that it will execute after provisioning. That is, updating the package manager cache and installing Apache. Save and exit when you’re done.

      You can create a temporary environment variable for the private key path as well:

      • export DO_PRIVATE_KEY="private_key_location"

      Note: The private key, and any other file that you wish to load from within Terraform, must be placed within the project. You can see the How to Set Up SSH Keys on Ubuntu 18.04 tutorial for more info regarding SSH key set up on Ubuntu 18.04 or other distributions.

      Try applying the configuration again:

      • terraform apply -var "do_token=${DO_PAT}" -var "domain_name=${DO_DOMAIN_NAME}" -var "private_key=${DO_PRIVATE_KEY}"

      Enter yes when prompted. You’ll receive output similar to before, but followed with long output from the remote-exec provisioner:

      Output

      digitalocean_droplet.web: Creating... digitalocean_droplet.web: Still creating... [10s elapsed] digitalocean_droplet.web: Still creating... [20s elapsed] digitalocean_droplet.web: Still creating... [30s elapsed] digitalocean_droplet.web: Provisioning with 'remote-exec'... digitalocean_droplet.web (remote-exec): Connecting to remote host via SSH... digitalocean_droplet.web (remote-exec): Host: ... digitalocean_droplet.web (remote-exec): User: root digitalocean_droplet.web (remote-exec): Password: false digitalocean_droplet.web (remote-exec): Private key: true digitalocean_droplet.web (remote-exec): Certificate: false digitalocean_droplet.web (remote-exec): SSH Agent: false digitalocean_droplet.web (remote-exec): Checking Host Key: false digitalocean_droplet.web (remote-exec): Connected! ... digitalocean_droplet.web: Creation complete after 1m5s [id=204442200] digitalocean_record.www: Creating... digitalocean_record.www: Creation complete after 1s [id=110666268] Apply complete! Resources: 2 added, 0 changed, 0 destroyed.

      You can now navigate to your domain in a web browser. You will see the default Apache welcome page.

      Apache Web Server - Default Page

      This means that Apache was installed successfully, and that Terraform provisioned everything correctly.

      To destroy the deployed resources, run the following command and enter yes when prompted:

      • terraform destroy -var "do_token=${DO_PAT}" -var "domain_name=${DO_DOMAIN_NAME}" -var "private_key=${DO_PRIVATE_KEY}"

      You have now completed a small Terraform project with a simple structure, that deploys the Apache web server on a Droplet and sets up DNS records for the desired domain.

      Conclusion

      You have learned about two general approaches of structuring your Terraform projects, according to their complexity. You’ve then deployed a Droplet running Apache with DNS records for your domain, following the simple structuring approach, and using the remote-exec provisioner to execute commands.

      For reference, here is the file structure of the project you created in this tutorial:

      .
      └── tf/
          ├── versions.tf
          ├── variables.tf
          ├── provider.tf
          ├── droplets.tf
          ├── dns.tf
          ├── data-sources.tf
          └── external/
              └── name-generator.py
      

      The resources you defined (the Droplet, the DNS record and dynamic data source, the DigitalOcean provider and variables) are stored each in its own separate file, according to the simple project structure outlined in the first section of this tutorial.

      For more information about Terraform provisioners and their parameters, visit the official documentation.



      Source link