One place for hosting & domains

      Infrastructure

      Infrastructure as Code Explained


      Introduction

      Cloud computing provides on-demand computing resources, which are decoupled from physical hardware and necessary underlying configuration. Autonomous software systems provision these computing resources in the cloud to achieve the automation that cloud computing offers. Because of such automation, it’s possible to control and manipulate the available resources programmatically by interfacing with the cloud providers. This way, infrastructure changes (such as resource scaling) can be implemented more quickly and reliably and operated mostly without manual interaction, but still with the ability to oversee the whole process and revert changes if something does not go according to plan.

      Infrastructure as Code (IaC) is the approach of automating infrastructure deployment and changes by defining the desired resource states and their mutual relationships in code. The code is written in specialized, human-readable languages of IaC tools. The actual resources in the cloud are created (or modified) when you execute the code. This then prompts the tool to interface with the cloud provider or deployment system on your behalf to apply the necessary changes, without using a cloud provider’s web interface. The code can be modified whenever needed—upon code execution the IaC tool will take care of finding the differences between the desired infrastructure in code and the actual infrastructure in the cloud, taking steps to make the actual state equal to the desired one.

      For IaC to work in practice, created resources must not be manually modified afterward (an immutable infrastructure), as this creates discord between the expected infrastructure in code and the actual state in the cloud. In addition, the manually modified resources could get recreated or deleted during future code executions, and all such customization would be lost. The solution to this is to incorporate the modifications into the infrastructure code.

      In this conceptual article, we’ll explore the IaC approach, its benefits, and examples of real-world implementations. We’ll also introduce Terraform, an open source IaC provisioning tool. We’ll review Terraform’s role in this approach and how it compares to other IaC tools.

      Benefits of IaC

      With IaC, you can quickly create as many instances of your entire infrastructure as you need, in multiple provider regions, from a single source of truth: your declarative code. This has many advantages that ensures you’re creating resources consistently without error while reducing management and manual setup time.

      The main benefits of IaC are:

      • Deployment: removing the manual provisioning interaction with cloud providers means a quicker deployment speed.
      • Recovery: identifying issues in the configuration can mean quick recovery from failures.
      • Consistency: deploying resources are the same each time, resolving infrastructure fragility.
      • Modification: modifying resources can have a quick turnaround time.
      • Reusability: reusing parts of the infrastructure architecture in future projects.
      • Version Control: storing the infrastructure code in version control systems.
      • Visibility: writing configuration as code acts as documentation for the infrastructure.

      Within an IaC workflow you can repeatedly spin up infrastructure in a standardized fashion, which means software development and testing is a quicker process because development, staging, quality-assurance testing, and production environments are separated. You can repeat the process of writing code and testing it live by deploying the infrastructure as many times as needed. Once your written infrastructure fulfills all requirements, you can deploy it in desired cloud environments. When new requirements arise, you can reiterate the process.

      General IaC Workflow

      IaC, being based on code, should always be coupled with version control software (VCS), such as Git. Storing your infrastructure declarations in VCS makes it easily retrievable, with changes visible to everyone on your team, and provides snapshots at historical points, so you can always roll back to an earlier version if new modifications create errors. Advanced VCS can be configured to automatically trigger the IaC tool to update the infrastructure in the cloud when an approved change is added.

      You now know what the IaC approach is, and what benefits it brings. You’ll now learn about states, the resource tracking mechanism employed in IaC. Then, you’ll follow up with the role of Terraform and other tools using IaC.

      What is ‘state’?

      In an IaC environment, the term ‘state’ refers to the state of affairs of desired infrastructure resources in a deployment. There are at least three states at a given moment: the actual one in the cloud, the ideal state presented in code, and the cached state that the IaC tool maintains. The cached state describes the state in the cloud as it was when the code was last executed. Terraform allows you to deploy the same code multiple times, forming multiple states for each deployment.

      How State is Changed

      The actual state in the cloud (of the managed resources) should always be the same as the cached state of the tool. When executing the code, the tool will compare the ideal state with the cached one and apply the detected differences to the cloud. If the cached and actual states do not match, it’s highly likely that the execution will fail or that resources will be incorrectly provisioned.

      Role of Terraform in IaC

      Terraform is an open source IaC resource provisioning tool, written in Go and developed by Hashicorp. It supports multiple cloud providers, including DigitalOcean. The infrastructure definitions are written in the Hashicorp Configuration Language (HCL), and source code files written in it have the file extension tf.

      Terraform works by reading the code describing your infrastructure and generating a graph containing all resources with their mutual relationships. It then compares it to the cached state of the resources in the cloud, and prepares an execution plan that details what will be applied to the cloud, and in what order, to reach the desired state.

      The two main types of underlying components in Terraform are providers and provisioners. Providers are responsible for interacting with a given cloud provider, creating, managing, and deleting resources, while provisioners are used to execute specific actions on created remote resources or the local machine the code is executing on.

      Terraform supports managing basic cloud provider components, such as compute instances, load balancers, storage, and DNS records, though more providers and provisioners can be added, owing to its extensible nature.

      In IaC Terraform’s role is to ensure that the state of resources in the cloud is equal to the state expressed in code. It does not monitor the deployed resources, and its main focus is not on further bootstrapping of the provisioned compute instances with software and tasks. In the next section, you’ll learn how it compares to other tools and how they extend each other in a typical workflow.

      The IaC approach is widespread in modern deployment, configuration management, virtualization, and orchestration software. Docker and Kubernetes, the leading tools used for container creation and orchestration, both use YAML as their language for declaring the desired end result. On the other hand, Hashicorp Packer, a tool for creating snapshots of deployments, uses JSON to declare the template and variables from which a snapshot of the system will be built.

      Ansible, Chef, and Puppet, the three most popular configuration management tools, all use the IaC approach to define the desired state of the servers they manage.

      Ansible bootstraps provided servers according to a given playbook. A playbook is a textual file written in appropriate YAML, instructing Ansible what operations to perform on the existing target resources. Examples of such operations include running and starting services, installing packages using the system-provided package manager, or executing custom bash commands. To learn more about writing Ansible playbooks, read Configuration Managment 101: Writing Ansible Playbooks.

      Chef and Puppet both require central servers with agents installed on each of the managed ones. Unlike Ansible, Chef uses Ruby, and Puppet uses its own declarative language for describing the resources.

      Terraform is not mutually exclusive with other IaC tools and DevOps systems. Its strength is in provisioning hardware resources, rather than further software installation and initial server setup.

      Unlike configuration management tools such as Ansible and Chef, Terraform is not suitable for installing software on the target resources and setting up tasks. Instead Terraform offers providers for interacting with their supported resources.

      Terraform can work from a single machine and does not require central servers with client agents installed on the provisioned resources, unlike some other tools. It does not check their actual state and reapplies the configuration automatically, because its main focus is on provisioning them. A typical workflow is to provision the infrastructure resources using Terraform and then bootstrap them using a configuration management tool, if needed.

      For Chef, Terraform has a built-in provisioner that sets up its client agent on provisioned remote resources. With it you can automatically have all your provisioned servers added to the main server, from where you can additionally configure them using cookbooks, Chef’s infrastructure declarations. You can learn more about writing them in Configuration Management 101: Writing Chef Recipes.

      Conclusion

      This article covered the paradigms of the IaC approach, its advantages over traditional manual system administration, the basics of Terraform as an IaC resource provisioning tool, and how it compares to other popular infrastructure automation tools.

      If you’re looking to incorporate Infrastructure as Code into your workflow, check out our Terraform series to learn the fundamentals of using this tool in your development and deployment process.

      One way to start with Terraform is to read How To Structure Your Terraform Project to understand how to ensure your infrastructure stays scalable and extensible.



      Source link

      Top 10 Security Practices for Protecting Yourself and Your Infrastructure


      How to Join

      This Tech Talk is free and open to everyone. Register on Eventbrite here to receive a link to join on Thursday, October 29, 2020, 11:00 a.m.–12:00 p.m. ET.

      About the Talk

      From firewalls, to user privileges, to access controls — this talk will not only explain the how, but also dive into the why of common security practices.

      What You’ll Learn

      • 10 different tips and tricks for securing your infrastructure

      This Talk is Designed For

      • Everyone! Security is only effective if everyone is aware of the how and why things are done.

      About the Presenter

      Mason Egger (@masonegger) is currently a Developer Advocate at DigitalOcean, specializing in cloud infrastructure, distributed systems, and Python. Prior to his work at DigitalOcean, he was an SRE (Site Reliability Engineer), helping build and maintain a highly available hybrid multi-cloud PaaS. He is an avid programmer, speaker, educator, and writer/blogger. He is a maintainer of the DigitalOcean Terraform provider and contributes to random open source projects here and there. In his spare time, he enjoys reading, camping, kayaking, and exploring new places.

      To join the live Tech Talk, register here.



      Source link

      How To Manage Infrastructure Data with Terraform Outputs


      [*]

      Introduction

      Terraform outputs are used to extract information about the infrastructure resources from the project state. Using other features of the Hashicorp Configuration Language (HCL), which Terraform uses, resource information can be queried and transformed into more complex data structures, such as lists and maps. Outputs are useful for providing information to external software, which can operate on the created infrastructure resources.

      In this tutorial, you’ll learn about Terraform output syntax and its parameters by creating a simple infrastructure that deploys Droplets. You’ll also parse the outputs programmatically by converting them to JSON.

      Prerequisites

      Note: This tutorial has specifically been tested with Terraform 0.13.

      Defining Outputs

      In this section, you’ll declare a Droplet, deploy it to the cloud, and learn about outputs by defining one that will show the Droplet’s IP address.

      Assuming you are in the terraform-outputs directory, create and open the droplets.tf file for editing:

      Add the following Droplet resource and output definition:

      terraform-outputs/droplets.tf

      resource "digitalocean_droplet" "web" {
        image  = "ubuntu-18-04-x64"
        name   = "test-droplet"
        region = "fra1"
        size   = "s-1vcpu-1gb"
      }
      
      output "droplet_ip_address" {
        value = digitalocean_droplet.web.ipv4_address
      }
      

      You first declare a Droplet resource, called web. Its actual name in the cloud will be test-droplet, in the region fra1, running Ubuntu 18.04.

      Then, you declare an output called droplet_ip_address. In Terraform, outputs are used to export and show internal and computed values and information about the resources. Here, you set the value parameter, which accepts the data to output, to the IP address of the declared Droplet. At declare time, it’s unknown, but it will become available once the Droplet is deployed. Outputs are shown and accessible after each deployment.

      Save and close the file, then deploy the project by running the following command:

      • terraform apply -var "do_token=${DO_PAT}"

      Enter yes to apply when prompted. The end of the output you’ll see will be similar to this:

      Output

      ... digitalocean_droplet.web: Creating... ... digitalocean_droplet.web: Creation complete after 32s [id=207631771] Apply complete! Resources: 1 added, 0 changed, 0 destroyed. Outputs: droplet_ip_address = ip_address

      The highlighted IP address belongs to your newly deployed Droplet. Applying the project deploys the resources to the cloud and shows the outputs at the end, when all resource attributes are available. Without the droplet_ip_address output, Terraform would show no further information about the Droplet, except that it’s deployed.

      Outputs can also be shown using the output command:

      The output will list all outputs in the project:

      Output

      droplet_ip_address = ip_address

      You can also query a specific output by name by specifying it as an argument:

      • terraform output output_name

      For droplet_ip_address, the output will consist of the IP address only:

      Output

      ip_address

      Except for specifying the mandatory value, outputs have a few optional parameters:

      • description: embeds short documentation detailing what the output shows.
      • sensitive: accepts a boolean value, which prevents the content of the output from being shown after deploying if set to true.
      • depends_on: a meta parameter available at each resource that allows you to explicitly specify resources the output depends on, that Terraform is not able to automatically deduce during planning.

      The sensitive parameter is useful when the logs of the Terraform deployment will be publicly available, but the output contents should be kept hidden. You’ll now add it to your Droplet resource definition.

      Open droplets.tf for editing and add the highlighted line:

      terraform-outputs/droplets.tf

      resource "digitalocean_droplet" "web" {
        image  = "ubuntu-18-04-x64"
        name   = "test-droplet"
        region = "fra1"
        size   = "s-1vcpu-1gb"
      }
      
      output "droplet_ip_address" {
        value      = digitalocean_droplet.web.ipv4_address
        sensitive = true
      }
      

      Save and close the file when you’re done. You can try deploying the project again by running:

      • terraform apply -var "do_token=${DO_PAT}"

      You’ll see that the output is redacted:

      Output

      digitalocean_droplet.web: Refreshing state... [id=207631771] Apply complete! Resources: 0 added, 0 changed, 0 destroyed. Outputs: droplet_ip_address = <sensitive>

      Even if it’s marked as sensitive, the output and its contents will still be available through other channels, such as viewing the Terraform state or querying the outputs directly.

      In the next step, you’ll create a different Droplet and output structure, so destroy the currently deployed ones by running:

      • terraform destroy -var "do_token=${DO_PAT}"

      The output at the very end will be:

      Output

      ... Destroy complete! Resources: 1 destroyed.

      You’ve declared and deployed a Droplet and created an output that shows its IP address. You’ll now learn about using outputs to show more complex structures, such as lists and maps.

      Outputting Complex Structures

      In this section, you’ll deploy multiple Droplets from the same definition using the count keyword, and output their IP addresses in various formats.

      Using the for loop

      You’ll need to modify the Droplet resource definition, so open it for editing:

      Modify it to look like this:

      terraform-outputs/droplets.tf

      resource "digitalocean_droplet" "web" {
        count  = 3
        image  = "ubuntu-18-04-x64"
        name   = "test-droplet-${count.index}"
        region = "fra1"
        size   = "s-1vcpu-1gb"
      }
      

      You’ve specified that three Droplets should be created using the count key and added the current index to the Droplet name, so that you’ll be able to later discern between them. When you’re done, save and close the file.

      Apply the code by running:

      • terraform apply -var "do_token=${DO_PAT}"

      Terraform will plan the creation of three numbered Droplets, called test-droplet-0, test-droplet-1, and test-droplet-2. Enter yes when prompted to finish the process. You’ll see the following output in the end:

      Output

      ... Apply complete! Resources: 3 added, 0 changed, 0 destroyed.

      This means that all three Droplets are successfully deployed and that all information about them is stored in the project state.

      The easiest way to access their resource attributes is to use outputs, though creating one for each of the Droplets is not scalable. The solution is to use the for loop to traverse through the list of Droplets and gather their attributes, or to alternatively use splat expressions. You’ll learn about them later in this step.

      You’ll first define an output that will output the IP addresses of the three Droplets, paired with their names. Open droplets.tf for editing:

      Add the following lines:

      terraform-outputs/droplets.tf

      resource "digitalocean_droplet" "web" {
        count  = 3
        image  = "ubuntu-18-04-x64"
        name   = "test-droplet-${count.index}"
        region = "fra1"
        size   = "s-1vcpu-1gb"
      }
      
      output "droplet_ip_addresses" {
        value = {
          for droplet in digitalocean_droplet.web:
          droplet.name => droplet.ipv4_address
        }
      }
      

      The output value of droplet_ip_addresses is constructed using a for loop. Because it’s surrounded by braces, the resulting type will be a map. The loop traverses the list of Droplets, and for each instance, pairs its name with its IP address and appends it to the resulting map.

      Save and close the file, then try applying the project again:

      • terraform apply -var "do_token=${DO_PAT}"

      Enter yes when prompted and you’ll receive the output contents at the end:

      Output

      Apply complete! Resources: 0 added, 0 changed, 0 destroyed. Outputs: droplet_ip_addresses = { "test-droplet-0" = "ip_address" "test-droplet-1" = "ip_address" "test-droplet-2" = "ip_address" }

      The droplet_ip_addresses output details the IP addresses of the three deployed droplets.

      Using the Terraform output command, you can get the contents of the output as JSON using its command argument:

      • terraform output -json droplet_ip_addresses

      The result will be similar to the following:

      Output

      {"test-droplet-0":"ip_address","test-droplet-1":"ip_address","test-droplet-2":"ip_address"}

      JSON parsing is widely used and supported in many programming languages. This way, you can programmatically parse the information about the deployed Droplet resources.

      Using Splat Expressions

      Splat expressions offer a compact way of iterating over all elements of a list, and collecting contents of an attribute from each of them, resulting in a list. A splat expression that would extract the IP addresses of the three deployed droplets would have the following syntax:

      digitalocean_droplet.web[*].ipv4_address
      

      The [*] symbol traverses the list on its left and for each of the elements, takes the contents of its attribute specified on the right. If the reference on the left is not a list by itself, it will be converted to one where it will be the sole element.

      You can open droplets.tf for editing and modify the following lines to implement this:

      terraform-outputs/droplets.tf

      resource "digitalocean_droplet" "web" {
        count  = 3
        image  = "ubuntu-18-04-x64"
        name   = "test-droplet-${count.index}"
        region = "fra1"
        size   = "s-1vcpu-1gb"
      }
      
      output "droplet_ip_addresses" {
        value = digitalocean_droplet.web[*].ipv4_address
      }
      

      After saving the file, apply the project by running the following command:

      • terraform apply -var "do_token=${DO_PAT}"

      You’ll receive output that is now a list, and contains only the IP addresses of the Droplets:

      Output

      Apply complete! Resources: 0 added, 0 changed, 0 destroyed. Outputs: droplet_ip_addresses = [ "ip_address", "ip_address", "ip_address", ]

      To receive the output as JSON, run the following command:

      • terraform output -json droplet_ip_addresses

      The output will be a single array:

      Output

      ["ip_address","ip_address","ip_address"]

      You’ve used outputs together with splat expressions and for loops to export IP addresses of the deployed Droplets. You’ve also received the output contents as JSON, and you’ll now use jq—a tool for dynamically filtering JSON according to given expressions—to parse them.

      Parsing Outputs Using jq

      In this step, you’ll install and learn the basics of jq, a tool for manipulating JSON documents. You’ll use it to parse the outputs of your Terraform project.

      If you’re on Ubuntu, run the following command to install jq:

      On macOS, you can use Homebrew to install it:

      jq applies the provided processing expression on given input, which can be piped in. The easiest task in jq is to pretty print the input:

      • terraform output -json droplet_ip_addresses | jq '.'

      Passing in the identity operator (.) means that the whole JSON document parsed from the input should be outputted without modifications:

      Output

      [ "first_ip_address", "second_ip_address", "third_ip_address" ]

      You can request just the second IP address using the array bracket notation, counting from zero:

      • terraform output -json droplet_ip_addresses | jq '.[1]'

      The output will be:

      Output

      "second_ip_address"

      To make the result of the processing an array, wrap the expression in brackets:

      • terraform output -json droplet_ip_addresses | jq '[.[1]]'

      You’ll get a pretty printed JSON array:

      Output

      [ "second_ip_address" ]

      You can retrieve parts of arrays instead of single elements by specifying a range of indexes inside the brackets:

      • terraform output -json droplet_ip_addresses | jq '.[0:2]'

      The output will be:

      Output

      [ "first_ip_address", "second_ip_address" ]

      The range 0:2 returns the first two elements—the upper part of the range (2) is not inclusive, so only elements at positions 0 and 1 are fetched.

      You can now destroy the deployed resources by running:

      • terraform destroy -var "do_token=${DO_PAT}"

      In this step, you have installed jq and used it to parse and manipulate the output of your Terraform project, which deploys three Droplets.

      Conclusion

      You have learned about Terraform outputs, how they are used to show details about the deployed resources, and how they can be used to export data structures for later external processing. You’ve also seen how to use outputs to show attributes of a single resource, as well as for showing constructed maps and lists containing resource attributes.

      For more detailed information about the features of jq, visit the official docs.

      To learn more about Terraform check out the series: How To Manage Infrastructure with Terraform.

      [*]
      [*]Source link