One place for hosting & domains

      Terraform

      How To Troubleshoot Terraform


      The author selected the Free and Open Source Fund to receive a donation as part of the Write for DOnations program.

      Introduction

      Things do not always go according to plan: deployments can fail, existing resources may break unexpectedly, and you and your team could be forced to fix the issue as soon as possible. Understanding the methods to approach debugging your Terraform project is crucial when you need to make a swift response.

      Similarly to developing with other programming languages and frameworks, setting log levels in Terraform to gain insight into its internal workflows with the necessary verbosity is a feature that can help you when troubleshooting. By logging the internal actions, you can uncover implicitly hidden errors, such as variables defaulting to an unsuitable data type. Also common with frameworks is the ability to import and use third-party modules (or libraries) to reduce code duplication between projects.

      In this tutorial, you’ll verify that variables always have sensible values and you’ll specify exactly which versions of providers and modules you need to prevent conflicts. You’ll also enable various levels of debug mode verbosity, which can help you diagnose an underlying issue in Terraform itself.

      Prerequisites

      • A DigitalOcean Personal Access Token, which you can create via the DigitalOcean Control Panel. You can find instructions to do this in How to Generate a Personal Access Token.
      • Terraform installed on your local machine and a project set up with the DigitalOcean provider. Complete Step 1 and Step 2 of the How To Use Terraform with DigitalOcean tutorial, and be sure to name the project folder terraform-troubleshooting, instead of loadbalance. During Step 2, do not include the pvt_key variable and the SSH key resource.

      Note: We have specifically tested this tutorial using Terraform 0.13.

      Setting Version Constraints

      Although the ability to make use of third-party modules and providers can minimize code duplication and effort when writing your Terraform code, it is also possible for developers of third-party modules to release new versions that can potentially bring breaking changes to your specific code. To prevent this, Terraform allows you to specify version boundaries to ensure that only the versions you want are installed and used. Specifying versions means that you will require the versions you’ve tested in your code, but it also leaves the possibility for a future update.

      Version constraints are specified as strings and passed in to the version parameter when you define module or provider requirements. As part of the Prerequisites, you’ve already requested the digitalocean provider in provider.tf. Run the following command to review what version requirements it specifies:

      You’ll find the provider code as follows:

      terraform-troubleshooting/provider.tf

      terraform {
        required_providers {
          digitalocean = {
            source = "digitalocean/digitalocean"
            version = "1.22.2"
          }
        }
      }
      ...
      

      In this case, you have requested the digitalocean provider, and explicitly set the version to 1.22.2. When your project only requires a specific version, this is the most effortless way to accomplish that.

      Version constraints can be more complex than just specifying one version. They can contain one or more groups of conditions, separated by a comma (,). The groups each define an acceptable range of versions and may include operators, such as:

      • >, <, >=, <=: for comparisons, such as >=1.0, which would require the version to be equal to or greater than 1.0.
      • !=: for excluding a specific version—!= 1.0 would deny version 1.0 from being used and requested.
      • ~>: for matching the specified version up to the right-most version part, which is allowed to increment (~>1.5.10 will match 1.5.10 and 1.5.11, but won’t match 1.5.9).

      Here are two examples of version constraints with multiple groups:

      • >=1.0, <2.0: allows all versions from the 1.0 series onward, up to 2.0.
      • >1.0, != 1.5: allows versions greater than, but not equal to 1.0, with the exception of 1.5, which it also excludes.

      For a potential available version to be selected, it must pass every specified constraint and remain compatible with other modules and providers, as well as the version of Terraform that you’re using. If Terraform deems no combination acceptable, it won’t be able to perform any tasks because the dependencies remain unresolved. When Terraform identifies acceptable versions satisfying the constraints, it uses the latest one available.

      In this section, you’ve learned about locking the range of module and resource versions that you can install in your project by specifying version constraints. This is useful when you want stability by using only tested and approved versions of third-party code. In the next section, you’ll configure Terraform to show more verbose logs, which are necessary for bug reports and further debugging in case of crashes.

      Enabling Debug Mode

      There could be a bug or malformed input within your workflow, which may result in your resources not provisioning as intended. In such rare cases, it’s important to know how to access detailed logs describing what Terraform is doing. They may aid in pinpointing the cause of the error, tell you if it’s user-made, or prompt you to report the issue to Terraform developers if it’s an internal bug.

      Terraform exposes the TF_LOG environment variable for setting the level of logging verbosity, of which there are five:

      • TRACE: the most elaborate verbosity, shows every step taken by Terraform and produces enormous outputs with internal logs.
      • DEBUG: describes what happens internally in a more concise way compared to TRACE.
      • ERROR: shows errors that prevent Terraform from continuing.
      • WARN: logs warnings, which may indicate misconfiguration or mistakes, but are not critical to execution.
      • INFO: shows general, high-level messages about the execution process.

      To specify a desired log level, you’ll have to set the environment variable to the appropriate value:

      If TF_LOG is defined, but the value is not one of the five listed verbosity levels, Terraform will default to TRACE.

      You’ll now define a Droplet resource and try deploying it with different log levels. You’ll store the Droplet definition in a file named droplets.tf, so create and open it for editing:

      Add the following lines:

      terraform-troubleshooting/droplets.tf

      resource "digitalocean_droplet" "test-droplet" {
        image  = "ubuntu-18-04-x64"
        name   = "test-droplet"
        region = "fra1"
        size   = "s-1vcpu-1gb"
      }
      

      This Droplet will run Ubuntu 18.04 with one CPU core and 1GB RAM in the fra1 region; you’ll call it test-droplet. That is all you need to define, so save and close the file.

      Before deploying the Droplet, set the log level to DEBUG by running:

      Then, plan the Droplet provisioning:

      • terraform plan -var "do_token=${DO_PAT}"

      The output will be very long, and you can inspect it more closely to find that each line starts with the level of verbosity (importance) in double brackets. You’ll see that most of the lines start with [DEBUG].

      [WARN] and [INFO] are also present; that’s because TF_LOG sets the lowest log level. This means that you’d have to set TF_LOG to TRACE to show TRACE and all other log levels at the same time.

      If an internal error occurred, Terraform will show the stack trace and exit, stopping execution. From there, you’ll be able to locate where in the source code the error occurred, and if it’s a bug, report it to Terraform developers. Otherwise, if it’s an error in your code, Terraform will point it out to you, so you can fix it in your project.

      Here is how the log output would be when the DigitalOcean backend can’t verify your API token. It throws a user error because of incorrect input:

      Output

      ... digitalocean_droplet.test-droplet: Creating... 2021/01/20 06:54:35 [ERROR] eval: *terraform.EvalApplyPost, err: Error creating droplet: POST https://api.digitalocean.com/v2/droplets: 401 Unable to authenticate you 2021/01/20 06:54:35 [ERROR] eval: *terraform.EvalSequence, err: Error creating droplet: POST https://api.digitalocean.com/v2/droplets: 401 Unable to authenticate you Error: Error creating droplet: POST https://api.digitalocean.com/v2/droplets: 401 Unable to authenticate you on droplets.tf line 1, in resource "digitalocean_droplet" "test-droplet": 1: resource "digitalocean_droplet" "test-droplet" { ...

      You’ve now learned to enable more verbose logging modes. They are very useful for diagnosing crashes and unexpected Terraform behavior. In the next section, you’ll review verifying variables and preventing edge cases.

      Validating Variables

      In this section, you’ll ensure that variables always have sensible and appropriate values according to their type and validation parameters.

      In HCL (HashiCorp Configuration Language), when defining a variable you do not necessarily need to specify anything except its name. You would declare an example variable called test_ip like this:

      variable "test_ip" { }
      

      You can then use this value through the code; passing its value in when you run Terraform.

      While that will work, this definition has two shortcomings: first, you can not pass a value in at runtime; and second, it can be of any type (bool, string and so on), which may not be suitable for its purpose. To remedy this, you should always specify its default value and type:

      variable "test_ip" {
        type    = string
        default = "8.8.8.8"
      }
      

      By setting a default value, you ensure that the code referencing the variable remains operational in the event that a more specific value was not provided. When you specify a type, Terraform can validate the new value the variable should be set to, and show an error if it’s non-conforming to the type. An instance of this behavior would be trying to fit a string into a number.

      A new feature of Terraform 0.13 is that you can provide a validation routine for variables that can give an error message if the validation fails. Examples of validation would be checking the length of the new value if it’s a string, or looking for at least one match with a RegEx expression in case of structured data.

      To add input validation to your variable, define a validation block:

      variable "test_ip" {
        type    = string
        default = "8.8.8.8"
      
        validation {
          condition     = can(regex("d{1,3}.d{1,3}.d{1,3}.d{1,3}", var.test_ip))
          error_message = "The provided value is not a valid IP address."
        }
      }
      

      Under validation, you can specify two parameters within the curly braces:

      • A condition that accepts a bool it will calculate, which will signify if the validation passes.
      • An error_message that specifies the error message in case the validation does not pass.

      In this example, you compute the condition by searching for a regex match in the variable value. You pass that to the can function. The can function returns true if the function that’s passed in as a parameter ran without errors, so it’s useful for checking when a function completed successfully or returned results.

      The regex function we’re using here accepts a Regular Expression (RegEx), applies it to a given string, and returns the matched substrings. The RegEx matches four pairs of three digit numbers, separated with dots in-between. You can learn more about RegEx by visiting the Introduction to Regular Expressions tutorial.

      You now know how to specify a default value for a variable, how to set its type, and how to enable input validation using RegEx expressions.

      Conclusion

      In this tutorial, you’ve troubleshooted Terraform by enabling debug mode and setting the log verbosity to appropriate levels. You’ve also learned about some of the advanced features of variables, such as declaring validation procedures and setting good defaults. Leaving out default values is a common pitfall that may cause strange issues further along in your project’s development.

      For the stability of your project, locking third-party module and provider versions is recommended, since it leaves you with a stable system and an upgrade path when it becomes necessary.

      Verification of input values for variables is not confined to matching with regex. For more built-in functions that you can make use of, visit the official docs.

      To learn more about Terraform, check out our How To Manage Infrastructure with Terraform series.



      Source link

      How To Deploy Multiple Environments in Your Terraform Project Without Duplicating Code


      The author selected the Free and Open Source Fund to receive a donation as part of the Write for DOnations program.

      Introduction

      Terraform offers advanced features that become increasingly useful as your project grows in size and complexity. It’s possible to alleviate the cost of maintaining complex infrastructure definitions for multiple environments by structuring your code to minimize repetitions and introducing tool-assisted workflows for easier testing and deployment.

      Terraform associates a state with a backend, which determines where and how state is stored and retrieved. Every state has only one backend and is tied to an infrastructure configuration. Certain backends, such as local or s3, may contain multiple states. In that case, the pairing of state and infrastructure to the backend is describing a workspace. Workspaces allow you to deploy multiple distinct instances of the same infrastructure configuration without storing them in separate backends.

      In this tutorial, you’ll first deploy multiple infrastructure instances using different workspaces. You’ll then deploy a stateful resource, which, in this tutorial, will be a DigitalOcean Volume. Finally, you’ll reference pre-made modules from the Terraform Registry, which you can use to supplement your own.

      Prerequisites

      • A DigitalOcean Personal Access Token, which you can create via the DigitalOcean Control Panel. You can find instructions for this in the How to Generate a Personal Access Token tutorial.
      • Terraform installed on your local machine and a project set up with the DO provider. Complete Step 1 and Step 2 of the How To Use Terraform with DigitalOcean tutorial, and be sure to name the project folder terraform-advanced, instead of loadbalance. During Step 2, do not include the pvt_key variable and the SSH key resource.

      Note: We have specifically tested this tutorial using Terraform 0.13.

      Deploying Multiple Infrastructure Instances Using Workspaces

      Multiple workspaces are useful when you want to deploy or test a modified version of your main infrastructure without creating a separate project and setting up authentication keys again. Once you have developed and tested a feature using the separate state, you can incorporate the new code into the main workspace and possibly delete the additional state. When you init a Terraform project, regardless of backend, Terraform creates a workspace called default. It is always present and you can never delete it.

      However, multiple workspaces are not a suitable solution for creating multiple environments, such as for staging and production. Therefore workspaces, which only track the state, do not store the code or its modifications.

      Since workspaces do not track the actual code, you should manage the code separation between multiple workspaces at the version control (VCS) level by matching them to their infrastructure variants. How you can achieve this is dependent on the VCS tool itself; for example, in Git branches would be a fitting abstraction. To make it easier to manage the code for multiple environments, you can break them up into reusable modules, so that you avoid repeating similar code for each environment.

      Deploying Resources in Workspaces

      You’ll now create a project that deploys a Droplet, which you’ll apply from multiple workspaces.

      You’ll store the Droplet definition in a file called droplets.tf.

      Assuming you’re in the terraform-advanced directory, create and open it for editing by running:

      Add the following lines:

      droplets.tf

      resource "digitalocean_droplet" "web" {
        image  = "ubuntu-18-04-x64"
        name   = "web-${terraform.workspace}"
        region = "fra1"
        size   = "s-1vcpu-1gb"
      }
      

      This definition will create a Droplet running Ubuntu 18.04 with one CPU core and 1 GB RAM in the fra1 region. Its name will contain the name of the current workspace it is deployed from. When you’re done, save and close the file.

      Apply the project for Terraform to run its actions with:

      • terraform apply -var "do_token=${DO_PAT}"

      Your output will be similar to the following:

      Output

      An execution plan has been generated and is shown below. Resource actions are indicated with the following symbols: + create Terraform will perform the following actions: # digitalocean_droplet.web will be created + resource "digitalocean_droplet" "web" { + backups = false + created_at = (known after apply) + disk = (known after apply) + id = (known after apply) + image = "ubuntu-18-04-x64" + ipv4_address = (known after apply) + ipv4_address_private = (known after apply) + ipv6 = false + ipv6_address = (known after apply) + ipv6_address_private = (known after apply) + locked = (known after apply) + memory = (known after apply) + monitoring = false + name = "web-default" + price_hourly = (known after apply) + price_monthly = (known after apply) + private_networking = (known after apply) + region = "fra1" + resize_disk = true + size = "s-1vcpu-1gb" + status = (known after apply) + urn = (known after apply) + vcpus = (known after apply) + volume_ids = (known after apply) + vpc_uuid = (known after apply) } Plan: 1 to add, 0 to change, 0 to destroy. ...

      Enter yes when prompted to deploy the Droplet in the default workspace.

      The name of the Droplet will be web-default, because the workspace you start with is called default. You can list the workspaces to confirm that it’s the only one available:

      You’ll receive the following output:

      Output

      * default

      The asterisk (*) means that you currently have that workspace selected.

      Create and switch to a new workspace called testing, which you’ll use to deploy a different Droplet, by running workspace new:

      • terraform workspace new testing

      You’ll have output similar to:

      Output

      Created and switched to workspace "testing"! You're now on a new, empty workspace. Workspaces isolate their state, so if you run "terraform plan" Terraform will not see any existing state for this configuration.

      You plan the deployment of the Droplet again by running:

      • terraform plan -var "do_token=${DO_PAT}"

      The output will be similar to the previous run:

      Output

      An execution plan has been generated and is shown below. Resource actions are indicated with the following symbols: + create Terraform will perform the following actions: # digitalocean_droplet.web will be created + resource "digitalocean_droplet" "web" { + backups = false + created_at = (known after apply) + disk = (known after apply) + id = (known after apply) + image = "ubuntu-18-04-x64" + ipv4_address = (known after apply) + ipv4_address_private = (known after apply) + ipv6 = false + ipv6_address = (known after apply) + ipv6_address_private = (known after apply) + locked = (known after apply) + memory = (known after apply) + monitoring = false + name = "web-testing" + price_hourly = (known after apply) + price_monthly = (known after apply) + private_networking = (known after apply) + region = "fra1" + resize_disk = true + size = "s-1vcpu-1gb" + status = (known after apply) + urn = (known after apply) + vcpus = (known after apply) + volume_ids = (known after apply) + vpc_uuid = (known after apply) } Plan: 1 to add, 0 to change, 0 to destroy. ...

      Notice that Terraform plans to deploy a Droplet called web-testing, which it has named differently from web-default. This is because the default and testing workspaces have separate states and have no knowledge of each other’s resources—even though they stem from the same code.

      To confirm that you’re in the testing workspace, output the current one you’re in with workspace show:

      The output will be the name of the current workspace:

      Output

      testing

      To delete a workspace, you first need to destroy all its deployed resources. Then, if it’s active, you need to switch to another one using workspace select. Since the testing workspace here is empty, you can switch to default right away:

      • terraform workspace select default

      You’ll receive output of Terraform confirming the switch:

      Output

      Switched to workspace "default".

      You can then delete it by running workspace delete:

      • terraform workspace delete testing

      Terraform will then perform the deletion:

      Output

      Deleted workspace "testing"!

      You can destroy the Droplet you’ve deployed in the default workspace by running:

      • terraform destroy -var "do_token=${DO_PAT}"

      Enter yes when prompted to finish the process.

      In this section, you’ve worked in multiple Terraform workspaces. In the next section, you’ll deploy a stateful resource.

      Deploying Stateful Resources

      Stateless resources do not store data, so you can create and replace them quickly, because they are not unique. Stateful resources, on the other hand, contain data that is unique or not simply re-creatable; therefore, they require persistent data storage.

      Since you may end up destroying such resources, or multiple resources require their data, it’s best to store it in a separate entity, such as DigitalOcean Volumes.

      Volumes are objects that you can attach to Droplets (servers), but are separate from them, and provide additional storage space. In this step, you’ll define the Volume and connect it to a Droplet in droplets.tf.

      Open it for editing:

      Add the following lines:

      droplets.tf

      resource "digitalocean_droplet" "web" {
        image  = "ubuntu-18-04-x64"
        name   = "web-${terraform.workspace}"
        region = "fra1"
        size   = "s-1vcpu-1gb"
      }
      
      resource "digitalocean_volume" "volume" {
        region                  = "fra1"
        name                    = "new-volume"
        size                    = 10
        initial_filesystem_type = "ext4"
        description             = "New Volume for Droplet"
      }
      
      resource "digitalocean_volume_attachment" "volume_attachment" {
        droplet_id = digitalocean_droplet.web.id
        volume_id  = digitalocean_volume.volume.id
      }
      

      Here you define two new resources, the Volume itself and a Volume attachment. The Volume will be 10GB, formatted as ext4, called new-volume, and located in the same region as the Droplet. To connect the Volume to the Droplet, since they are separate entities, you define a Volume attachment object. volume_attachment takes the Droplet and Volume IDs and instructs the DigitalOcean cloud to make the Volume available to the Droplet as a disk device.

      When you’re done, save and close the file.

      Plan this configuration by running:

      • terraform plan -var "do_token=${DO_PAT}"

      The actions that Terraform will plan will be the following:

      Output

      An execution plan has been generated and is shown below. Resource actions are indicated with the following symbols: + create Terraform will perform the following actions: # digitalocean_droplet.web will be created + resource "digitalocean_droplet" "web" { + backups = false + created_at = (known after apply) + disk = (known after apply) + id = (known after apply) + image = "ubuntu-18-04-x64" + ipv4_address = (known after apply) + ipv4_address_private = (known after apply) + ipv6 = false + ipv6_address = (known after apply) + ipv6_address_private = (known after apply) + locked = (known after apply) + memory = (known after apply) + monitoring = false + name = "web-default" + price_hourly = (known after apply) + price_monthly = (known after apply) + private_networking = (known after apply) + region = "fra1" + resize_disk = true + size = "s-1vcpu-1gb" + status = (known after apply) + urn = (known after apply) + vcpus = (known after apply) + volume_ids = (known after apply) + vpc_uuid = (known after apply) } # digitalocean_volume.volume will be created + resource "digitalocean_volume" "volume" { + description = "New Volume for Droplet" + droplet_ids = (known after apply) + filesystem_label = (known after apply) + filesystem_type = (known after apply) + id = (known after apply) + initial_filesystem_type = "ext4" + name = "new-volume" + region = "fra1" + size = 10 + urn = (known after apply) } # digitalocean_volume_attachment.volume_attachment will be created + resource "digitalocean_volume_attachment" "volume_attachment" { + droplet_id = (known after apply) + id = (known after apply) + volume_id = (known after apply) } Plan: 3 to add, 0 to change, 0 to destroy. ...

      The output details that Terraform would create a Droplet, a Volume, and a Volume attachment, which connects the Volume to the Droplet.

      You’ve now defined and connected a Volume (a stateful resource) to a Droplet. In the next section, you’ll review public, pre-made Terraform modules that you can incorporate in your project.

      Referencing Pre-made Modules

      Aside from creating your own custom modules for your projects, you can also use pre-made modules and providers from other developers, which are publicly available at Terraform Registry.

      In the modules section you can search the database of available modules and sort by provider in order to find the module with the functionality you need. Once you’ve found one, you can read its description, which lists the inputs and outputs the module provides, as well as its external module and provider dependencies.

      Terraform Registry - SSH key Module

      You’ll now add the DigitalOcean SSH key module to your project. You’ll store the code separate from existing definitions in a file called ssh-key.tf. Create and open it for editing by running:

      Add the following lines:

      ssh-key.tf

      module "ssh-key" {
        source         = "clouddrove/ssh-key/digitalocean"
        key_path       = "~/.ssh/id_rsa.pub"
        key_name       = "new-ssh-key"
        enable_ssh_key = true
      }
      

      This code defines an instance of the clouddrove/droplet/digitalocean module from the registry and sets some of the parameters it offers. It should add a public SSH key to your account by reading it from the ~/.ssh/id_rsa.pub.

      When you’re done, save and close the file.

      Before you plan this code, you must download the referenced module by running:

      You’ll receive output similar to the following:

      Output

      Initializing modules... Downloading clouddrove/ssh-key/digitalocean 0.13.0 for ssh-key... - ssh-key in .terraform/modules/ssh-key Initializing the backend... Initializing provider plugins... - Using previously-installed digitalocean/digitalocean v1.22.2 Terraform has been successfully initialized! ...

      You can now plan the code for the changes:

      • terraform plan -var "do_token=${DO_PAT}"

      You’ll receive output similar to this:

      Output

      Refreshing Terraform state in-memory prior to plan... The refreshed state will be used to calculate this plan, but will not be persisted to local or remote state storage. ------------------------------------------------------------------------ An execution plan has been generated and is shown below. Resource actions are indicated with the following symbols: + create Terraform will perform the following actions: ... # module.ssh-key.digitalocean_ssh_key.default[0] will be created + resource "digitalocean_ssh_key" "default" { + fingerprint = (known after apply) + id = (known after apply) + name = "devops" + public_key = "ssh-rsa ... demo@clouddrove" } Plan: 4 to add, 0 to change, 0 to destroy. ...

      The output shows that you would create the SSH key resource, which means that you downloaded and invoked the module from your code.

      Conclusion

      Bigger projects can make use of some advanced features Terraform offers to help reduce complexity and make maintenance easier. Workspaces allow you to test new additions to your code without touching the stable main deployments. You can also couple workspaces with a version control system to track code changes. Using pre-made modules can also shorten development time, but may incur additional expenses or time in the future if the module becomes obsolete.

      For further resources on using Terraform, check out our How To Manage Infrastructure With Terraform series.



      Source link

      How To Use Ansible with Terraform for Configuration Management


      The author selected the Free and Open Source Fund to receive a donation as part of the Write for DOnations program.

      Introduction

      Ansible is a configuration management tool that executes playbooks, which are lists of customizable actions written in YAML on specified target servers. It can perform all bootstrapping operations, like installing and updating software, creating and removing users, and configuring system services. As such, it is suitable for bringing up servers you deploy using Terraform, which are created blank by default.

      Ansible and Terraform are not competing solutions, because they resolve different phases of infrastructure and software deployment. Terraform allows you to define and create the infrastructure of your system, encompassing the hardware that your applications will run on. Conversely, Ansible configures and deploys software by executing its playbooks on the provided server instances. Running Ansible on the resources Terraform provisioned directly after their creation allows you to make the resources usable for your use case much faster. It also enables easier maintenance and troubleshooting, because all deployed servers will have the same actions applied to them.

      In this tutorial, you’ll deploy Droplets using Terraform, and then immediately after their creation, you’ll bootstrap the Droplets using Ansible. You’ll invoke Ansible directly from Terraform when a resource deploys. You’ll also avoid introducing race conditions using Terraform’s remote-exec and local-exec provisioners in your configuration, which will ensure that the Droplet deployment is fully complete before further setup commences.

      Prerequisites

      Note: This tutorial has specifically been tested with Terraform 0.13.

      Step 1 — Defining Droplets

      In this step, you’ll define the Droplets on which you’ll later run an Ansible playbook, which will set up the Apache web server.

      Assuming you are in the terraform-ansible directory, which you created as part of the prerequisites, you’ll define a Droplet resource, create three copies of it by specifying count, and output their IP addresses. You’ll store the definitions in a file named droplets.tf. Create and open it for editing by running:

      Add the following lines:

      ~/terraform-ansible/droplets.tf

      resource "digitalocean_droplet" "web" {
        count  = 3
        image  = "ubuntu-18-04-x64"
        name   = "web-${count.index}"
        region = "fra1"
        size   = "s-1vcpu-1gb"
      
        ssh_keys = [
            data.digitalocean_ssh_key.terraform.id
        ]
      }
      
      output "droplet_ip_addresses" {
        value = {
          for droplet in digitalocean_droplet.web:
          droplet.name => droplet.ipv4_address
        }
      }
      

      Here you define a Droplet resource running Ubuntu 18.04 with 1GB RAM on a CPU core in the region fra1. Terraform will pull the SSH key you defined in the prerequisites from your account and add it to the provisioned Droplet with the specified unique ID list element passed into ssh_keys. Terraform will deploy the Droplet three times because the count parameter is set. The output block following it will show the IP addresses of the three Droplets. The loop traverses the list of Droplets, and for each instance, pairs its name with its IP address and appends it to the resulting map.

      Save and close the file when you’re done.

      You have now defined the Droplets that Terraform will deploy. In the next step, you’ll write an Ansible playbook that will execute on each of the three deployed Droplets and will deploy the Apache web server. You’ll later go back to the Terraform code and add in the integration with Ansible.

      Step 2 — Writing an Ansible Playbook

      You’ll now create an Ansible playbook that performs the initial server setup tasks, such as creating a new user and upgrading the installed packages. You’ll instruct Ansible on what to do by writing tasks, which are units of action that are executed on target hosts. Tasks can use built-in functions, or specify custom commands to be run. Besides the tasks for the initial setup, you’ll also install the Apache web server and enable its mod_rewrite module.

      Before writing the playbook, ensure that your public and private SSH keys, which correspond to the one in your DigitalOcean account, are available and accessible on the machine from which you’re running Terraform and Ansible. A typical location for storing them on Linux would be ~/.ssh (though you can store them in other places).

      Note: On Linux, you’ll need to ensure that the private key file has the appropriate permissions. You can set them by running:

      • chmod 600 your_private_key_location

      You already have a variable for the private key defined, so you’ll only need to add in one for the public key location.

      Open provider.tf for editing by running:

      Add the following line:

      ~/terraform-ansible/provider.tf

      terraform {
        required_providers {
          digitalocean = {
            source = "digitalocean/digitalocean"
            version = "1.22.2"
          }
        }
      }
      
      variable "do_token" {}
      variable "pvt_key" {}
      variable "pub_key" {}
      
      provider "digitalocean" {
        token = var.do_token
      }
      
      data "digitalocean_ssh_key" "terraform" {
        name = "terraform"
      }
      

      When you’re done, save and close the file.

      With the pub_key variable now defined, you’ll start writing the Ansible playbook. You’ll store it in a file called apache-install.yml. Create and open it for editing:

      You’ll be building the playbook gradually. First, you’ll need to define on which hosts the playbook will run, its name, and if the tasks should be run as root. Add the following lines:

      ~/terraform-ansible/apache-install.yml

      - become: yes
        hosts: all
        name: apache-install
      

      By setting become to yes, you instruct Ansible to run commands as the superuser, and by specifying all for hosts, you allow Ansible to run the tasks on any given server—even the ones passed in through the command line, as Terraform does.

      The first task that you’ll add will create a new, non-root user. Append the following task definition to your playbook:

      ~/terraform-ansible/apache-install.yml

      . . .
        tasks:
          - name: Add the user 'sammy' and add it to 'sudo'
            user:
              name: sammy
              group: sudo
      

      You first define a list of tasks and then add a task to it. It will create a user named sammy and grant them superuser access using sudo by adding them to the appropriate group.

      The next task will add your public SSH key to the user, so you’ll be able to connect to it later on:

      ~/terraform-ansible/apache-install.yml

      . . .
          - name: Add SSH key to 'sammy'
            authorized_key:
              user: sammy
              state: present
              key: "{{ lookup('file', pub_key) }}"
      

      This task will ensure that the public SSH key, which is looked up from a local file, is present on the target. You’ll supply the value for the pub_key variable from Terraform in the next step.

      Once you have set up the user task, the next is to update the software on the Droplet using apt:

      ~/terraform-ansible/apache-install.yml

      . . .
          - name: Update all packages
            apt:
              upgrade: dist
              update_cache: yes
              cache_valid_time: 3600
      

      The target Droplet has the newest versions of available packages and a non-root user available so far. You can now order the installation of Apache and the mod_rewrite module by appending the following tasks:

      ~/terraform-ansible/apache-install.yml

      . . .
          - name: Install apache2
            apt: name=apache2 update_cache=yes state=latest
      
          - name: Enable mod_rewrite
            apache2_module: name=rewrite state=present
            notify:
              - Restart apache2
      
        handlers:
          - name: Restart apache2
            service: name=apache2 state=restarted
      

      The first task will run the apt package manager to install Apache. The second one will ensure that the mod_rewrite module is present. After it’s enabled, you need to ensure that you restart Apache, which you can’t configure from the task itself. To resolve that, you call a handler to issue the restart.

      At this point, your playbook will be as follows:

      ~/terraform-ansible/apache-install.yml

      - become: yes
        hosts: all
        name: apache-install
        tasks:
          - name: Add the user 'sammy' and add it to 'sudo'
            user:
              name: sammy
              group: sudo
          - name: Add SSH key to 'sammy'
            authorized_key:
              user: sammy
              state: present
              key: "{{ lookup('file', pub_key) }}"
          - name: Update all packages
            apt:
              upgrade: dist
              update_cache: yes
              cache_valid_time: 3600
          - name: Install apache2
            apt: name=apache2 update_cache=yes state=latest
          - name: Enable mod_rewrite
            apache2_module: name=rewrite state=present
            notify:
              - Restart apache2
      
        handlers:
          - name: Restart apache2
            service: name=apache2 state=restarted
      

      This is all you need to define on the Ansible side, so save and close the playbook. You’ll now modify the Droplet deployment code to execute this playbook when the Droplets have finished provisioning.

      Step 3 — Running Ansible on Deployed Droplets

      Now that you have defined the actions Ansible will take on the target servers, you’ll modify the Terraform configuration to run it upon Droplet creation.

      Terraform offers two provisioners that execute commands: local-exec and remote-exec, which run commands locally or remotely (on the target), respectively. remote-exec requires connection data, such as type and access keys, while local-exec does everything on the machine Terraform is executing on, and so does not require connection information. It’s important to note that local-exec runs immediately after the resource you have defined it for has finished provisioning; therefore, it does not wait for the resource to actually boot up. It runs after the cloud platform acknowledges its presence in the system.

      You’ll now add provisioner definitions to your Droplet to run Ansible after deployment. Open droplets.tf for editing:

      Add the highlighted lines:

      ~/terraform-ansible/droplets.tf

      resource "digitalocean_droplet" "web" {
        count  = 3
        image  = "ubuntu-18-04-x64"
        name   = "web-${count.index}"
        region = "fra1"
        size   = "s-1vcpu-1gb"
      
        ssh_keys = [
            data.digitalocean_ssh_key.terraform.id
        ]
      
        provisioner "remote-exec" {
          inline = ["sudo apt update", "sudo apt install python3 -y", "echo Done!"]
      
          connection {
            host        = self.ipv4_address
            type        = "ssh"
            user        = "root"
            private_key = file(var.pvt_key)
          }
        }
      
        provisioner "local-exec" {
          command = "ANSIBLE_HOST_KEY_CHECKING=False ansible-playbook -u root -i '${self.ipv4_address},' --private-key ${var.pvt_key} -e 'pub_key=${var.pub_key}' apache-install.yml"
        }
      }
      
      output "droplet_ip_addresses" {
        value = {
          for droplet in digitalocean_droplet.web:
          droplet.name => droplet.ipv4_address
        }
      }
      

      Like Terraform, Ansible is run locally and connects to the target servers via SSH. To run it, you define a local-exec provisioner in the Droplet definition that runs the ansible-playbook command. This passes in the username (root), the IP of the current Droplet (retrieved with ${self.ipv4_address}), the SSH public and private keys, and specifies the playbook file to run (apache-install.yml). By setting the ANSIBLE_HOST_KEY_CHECKING environment variable to False, you skip checking if the server was connected to beforehand.

      As was noted, the local-exec provisioner runs without waiting for the Droplet to become available, so the execution of the playbook may precede the actual availability of the Droplet. To remedy this, you define the remote-exec provisioner to contain commands to execute on the target server. For remote-exec to execute the target server must be available. Since remote-exec runs before local-exec the server will be fully initialized by the time Ansible is invoked. python3 comes preinstalled on Ubuntu 18.04, so you can comment out or remove the command as necessary.

      When you’re done making changes, save and close the file.

      Then, deploy the Droplets by running the following command. Remember to replace private_key_location and public_key_location with the locations of your private and public keys respectively:

      • terraform apply -var "do_token=${DO_PAT}" -var "pvt_key=private_key_location" -var "pub_key=public_key_location"

      The output will be long. Your Droplets will provision and then a connection will establish with each. Next the remote-exec provisioner will execute and install python3:

      Output

      ... digitalocean_droplet.web[1] (remote-exec): Connecting to remote host via SSH... digitalocean_droplet.web[1] (remote-exec): Host: ... digitalocean_droplet.web[1] (remote-exec): User: root digitalocean_droplet.web[1] (remote-exec): Password: false digitalocean_droplet.web[1] (remote-exec): Private key: true digitalocean_droplet.web[1] (remote-exec): Certificate: false digitalocean_droplet.web[1] (remote-exec): SSH Agent: false digitalocean_droplet.web[1] (remote-exec): Checking Host Key: false digitalocean_droplet.web[1] (remote-exec): Connected! ...

      After that, Terraform will run the local-exec provisioner for each of the Droplets, which executes Ansible. The following output shows this for one of the Droplets:

      Output

      ... digitalocean_droplet.web[2] (local-exec): Executing: ["/bin/sh" "-c" "ANSIBLE_HOST_KEY_CHECKING=False ansible-playbook -u root -i 'ip_address,' --private-key private_key_location -e 'pub_key=public_key_location' apache-install.yml"] digitalocean_droplet.web[2] (local-exec): PLAY [apache-install] ********************************************************** digitalocean_droplet.web[2] (local-exec): TASK [Gathering Facts] ********************************************************* digitalocean_droplet.web[2] (local-exec): ok: [ip_address] digitalocean_droplet.web[2] (local-exec): TASK [Add the user 'sammy' and add it to 'sudo'] ******************************* digitalocean_droplet.web[2] (local-exec): changed: [ip_address] digitalocean_droplet.web[2] (local-exec): TASK [Add SSH key to 'sammy''] ******************************* digitalocean_droplet.web[2] (local-exec): changed: [ip_address] digitalocean_droplet.web[2] (local-exec): TASK [Update all packages] ***************************************************** digitalocean_droplet.web[2] (local-exec): changed: [ip_address] digitalocean_droplet.web[2] (local-exec): TASK [Install apache2] ********************************************************* digitalocean_droplet.web[2] (local-exec): changed: [ip_address] digitalocean_droplet.web[2] (local-exec): TASK [Enable mod_rewrite] ****************************************************** digitalocean_droplet.web[2] (local-exec): changed: [ip_address] digitalocean_droplet.web[2] (local-exec): RUNNING HANDLER [Restart apache2] ********************************************** digitalocean_droplet.web[2] (local-exec): changed: [ip_address] digitalocean_droplet.web[2] (local-exec): PLAY RECAP ********************************************************************* digitalocean_droplet.web[2] (local-exec): [ip_address] : ok=7 changed=6 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0 ...

      At the end of the output, you’ll receive a list of the three Droplets and their IP addresses:

      Output

      droplet_ip_addresses = { "web-0" = "..." "web-1" = "..." "web-2" = "..." }

      You can now navigate to one of the IP addresses in your browser. You will reach the default Apache welcome page, signifying the successful installation of the web server.

      Apache Welcome Page

      This means that Terraform provisioned your servers and your Ansible playbook executed on it successfully.

      To check that the SSH key was correctly added to sammy on the provisioned Droplets, connect to one of them with the following command:

      • ssh -i private_key_location sammy@droplet_ip_address

      Remember to put in the private key location and the IP address of one of the provisioned Droplets, which you can find in your Terraform output.

      The output will be similar to the following:

      Output

      Welcome to Ubuntu 18.04.5 LTS (GNU/Linux 4.15.0-121-generic x86_64) * Documentation: https://help.ubuntu.com * Management: https://landscape.canonical.com * Support: https://ubuntu.com/advantage System information as of ... System load: 0.0 Processes: 88 Usage of /: 6.4% of 24.06GB Users logged in: 0 Memory usage: 20% IP address for eth0: ip_address Swap usage: 0% IP address for eth1: ip_address 0 packages can be updated. 0 updates are security updates. New release '20.04.1 LTS' available. Run 'do-release-upgrade' to upgrade to it. *** System restart required *** Last login: ... ...

      You’ve successfully connected to the target and obtained shell access for the sammy user, which confirms that the SSH key was correctly configured for that user.

      You can destroy the deployed Droplets by running the following command, entering yes when prompted:

      • terraform destroy -var "do_token=${DO_PAT}" -var "pvt_key=private_key_location" -var "pub_key=public_key_location"

      In this step, you have added in Ansible playbook execution as a local-exec provisioner to your Droplet definition. To ensure that the server is available for connections, you’ve included the remote-exec provisioner, which can serve to install the python3 prerequisite, after which Ansible will run.

      Conclusion

      Terraform and Ansible together form a flexible workflow for spinning up servers with the needed software and hardware configurations. Running Ansible directly as part of the Terraform deployment process allows you to have the servers up and bootstrapped with dependencies for your development work and applications much faster.

      For more on using Terraform, check out our How To Manage Infrastructure with Terraform series. You can also find further Ansible content resources on our Ansible topic page.



      Source link