One place for hosting & domains

      December 2018

      Upgrade Debian to the Newest Release


      Updated by Linode

      Written by Linode

      Debian repositories can be tracked either by codename (Wheezy, Jessie, etc.), or by status name (stable, testing, etc.). For example, Debian 9 Stretch is the stable release at the time of this writing; the status of Debian 8 (Jessie) is oldstable. Debian stable releases are eventually managed by the Debian Long Term Support (LTS) team for a total lifespan of about 5 years.

      Linode offers Debian’s stable and oldstable releases. When exclusively tracking the stable releases with APT, your system will upgrade whenever the stable release reaches its end of life. For example, if you’re tracking the stable release of Debian 8 and it reaches its end of life, your system will make available a number of new packages which will upgrade you to Debian 9.

      On the other hand, if you’re currently tracking repositories by codename, as Debian does by default, you will never upgrade beyond that codename release. This is the safest option and you can still manually upgrade to a newer Debian codename or release status name at any time.

      Caution

      While upstream maintainers try to ensure cross-compatibility and problem-free upgrades, there is risk involved in upgrading operating system versions.

      Before You Begin

      • You will need root access to your Linode, or a user account with sudo privileges.

      • Back up any important data stored on your Linode! If you subscribe to the Linode Backups service, we recommend taking a manual snapshot before upgrading your system. If you use a different backup service or application, you should do a manual backup now.

        Note

        You may also want to back up your configuration files (usually located in /etc/) in case they have changed in later versions of the software you are using. See our backup guides for more information.

      Prepare to Upgrade

      1. Verify that you are booting with Debian’s kernel using the GRUB 2 boot setting in the Linode Cloud Manager. We recommend you use the distribution-supplied kernel unless you have a specific reason not to.

      2. Exit the SSH session if you’re currently logged in to one and instead open a Lish session to your Linode. Lish will give you continuous access to your Linode whereas SSH could disconnect during the upgrade. Read more about Lish here.

      3. Install all available updates for your current Debian system:

        sudo apt update && sudo apt upgrade
        
      4. If you’ve set APT to pin any packages to a specific Debian version other than stable, You’ll need to disable APT pinning for those packages if you want them upgraded to those offered in the newest release.

      5. You may want to stop services which are non-essential to the system but important to your setup, such as a database service. This would be to ensure a graceful shutdown of the service to prevent data loss or system locks from causing problems. To stop a service, enter the following command, replacing mariadb with the name of the service you want to stop:

        sudo systemctl stop mariadb
        

        You can view all enabled services with:

        sudo systemctl list-unit-files --state=enabled
        

        View all currently running services:

        sudo systemctl list-units --state=running
        

      Upgrade Debian

      1. Edit your sources.list file to change all instances of the current codename to the new release codename. The example of upgrading from Debian 8 (Jessie) to Debian 9 (Stretch) is used below, so jessie is changed to stretch (or, alternatively from jessie to stable).

        /etc/apt/sources.list
        1
        2
        3
        4
        5
        6
        7
        8
        9
        
        deb http://mirrors.linode.com/debian stretch main
        deb-src http://mirrors.linode.com/debian stretch main
        
        deb http://mirrors.linode.com/debian-security/ stretch/updates main
        deb-src http://mirrors.linode.com/debian-security/ stretch/updates main
        
        # stretch-updates, previously known as 'volatile'
        deb http://mirrors.linode.com/debian stretch-updates main
        deb-src http://mirrors.linode.com/debian stretch-updates main

        Note

        Ensure any third party repositories are also tracking stretch. You will need to check with the maintainers of each package to ensure that their own repositories have been updated.

      2. Update your package lists and remove any old packages which were previously downloaded for installation:

        sudo apt update
        sudo apt-get clean
        
      3. The Debian 8 release notes recommend a two-part upgrade path to avoid removing packages you may want to keep. Perform the minimal upgrade.

        sudo apt upgrade
        
        • During the upgrade process, you will prompted whether you want to replace or keep the current GRUB 2 file. This is because Linode must edit /etc/default/grub from upstream to work properly with our infrastructure.

          Keep current Grub 2 configuration

          Choose Keep the local version currently installed. Further prompts about installing GRUB should be answered with installing to /dev/sda, then Continue without installing GRUB. GRUB is not needed in your disk MBR because your Linode boots from a GRUB installation provided by Linode’s host servers.

        • During the upgrade process, you’ll be prompted to review configuration files which you’ve modified to decide whether to keep or replace them with the upstream default file. An example:

            
          Configuration file '/etc/mysql/my.cnf'
          ==> Modified (by you or by a script) since installation.
          ==> Package distributor has shipped an updated version.
          What would you like to do about it ?  Your options are:
          Y or I  : install the package maintainer's version
          N or O  : keep your currently-installed version
          D     : show the differences between the versions
          
          
        • If your system is running Fail2ban, the upgrade will end with the error shown below. This is a known issue. See the troubleshooting section of this page to fix before proceeding further.

            
          Errors were encountered while processing:
           fail2ban
          E: Sub-process /usr/bin/dpkg returned an error code (1)
          
          
      4. Once the minimal upgrade above is completed, start the main upgrade:

        sudo apt dist-upgrade
        
      5. Reboot your system when the upgrade completes. You should still be in Lish. Monitor the Linode’s console output for errors as the system shuts down and reboots. Your Linode is now running the newest version of Debian Stable.

      6. Remove old and unused packages:

        sudo apt-get autoremove
        

      Troubleshooting

      Below are some known issues you may encounter when upgrading Debian. These are mainly just issues reported by our customers so you’ll want to monitor the debian-announce mailing list for more comprehensive information, and the lists of any third party packages you will install.

      Fail2ban

      When upgrading from Debian 8 to 9, you may experience problems because of a duplicate configuration option in /etc/fail2ban/jail.local if you copied it directly from /etc/fail2ban/jail.conf. This is currently a known issue. To fix this:

      1. Comment out port = anyport in /etc/fail2ban/jail.local (around line 155). The block should look similar to below:

        /etc/fail2ban/jail.local
         1
         2
         3
         4
         5
         6
         7
         8
         9
        10
        11
        
        [pam-generic]
        
        enabled  = false
        # pam-generic filter can be customized to monitor specific subset of 'tty's
        filter   = pam-generic
        # port actually must be irrelevant but lets leave it all for some possible uses
        port     = all
        banaction = iptables-allports
        #port     = anyport
        logpath  = /var/log/auth.log
        maxretry = 6
      2. Tell dpkg to reconfigure anything necessary:

        sudo dpkg --configure -a
        sudo apt-get install --fix-broken
        

      Upgrading Apache 2.2 to 2.4

      Upgrading from Debian 7 to 8 moves Apache from version 2.2 to 2.4. This version change can break existing websites if you’re already running Apache and requires adjusting configuration files. See our Upgrading Apache guide for more information.

      Find answers, ask questions, and help others.

      This guide is published under a CC BY-ND 4.0 license.



      Source link

      Secrets Management with Terraform


      Updated by Linode Contributed by Linode

      Terraform is an Infrastructure as Code (IaC) tool that allows you to write declarative code to manage your infrastructure. In order to implement IaC with Terraform it is necessary to supply secrets, such as server passwords and API tokens, within your code. This guide will discuss methods for securing those secrets within Terraform.

      Keeping Secrets Out of .tf Files

      In Terraform, .tf files contain the declarative code used to create, manage, and destroy infrastructure. This code is often committed to a version control system like Git, using a platform like GitHub, and shared within a team. Because it is easy for this information to become public-facing, it is important that you make sure your committed code is free of secrets.

      Input Variables

      Terraform configurations in .tf files can accept values from input variables. These variables are included in your configuration using Terraform’s interpolation syntax.

      For example, you might have a linode-infrastructure.tf file with a provider block that requires an API access token. The token variable definition is declared inside your .tf file and is then interpolated inside the provider declaration with the "${var.token}" syntax:

      linode-infrastructure.tf
      1
      2
      3
      4
      5
      6
      7
      
      variable "token" {
        description = "Your API access token"
      }
      
      provider "linode" {
          token = "${var.token}"
      }

      Variable definitions are written in .tf files. In this example, it’s the same file as your provider configuration, but the definition could have been in a separate .tf file too.

      Note

      Your variable definitions can have default values assigned to them. Here’s an example that encodes Linode’s Newark data center as the default value for a region variable:

      variables.tf
      1
      2
      3
      4
      
      variable "region" {
        description = "The region to deploy Linode instances in"
        default = "us-east"
      }

      You could later use this variable when declaring your Linode instances.

      Assigning Variable Values in a File

      The values assigned to your variables (aside from default values) are not included in the variable definitions in your .tf files. Instead, the values are stored in separate files with the .tfvars extension. When Terraform runs a command like plan or apply, it automatically looks through the working directory for a file named terraform.tfvars, or for files with the .auto.tfvars extension.

      Here’s an example terraform.tfvars which supplies a value for the token variable from the previous example:

      terraform.tfvars
      1
      
      token = 'your-token-value'

      You would then add the terraform.tfvars file to your .gitignore file and keep it out of version control. This strategy allows you to safely commit the linode-infrastructure.tf file.

      For ease of use with large terraform.tfvars files, it might be beneficial to include an example terraform.tfvars.example in your Git repository with all of the variable names recorded (but none of the values entered). Team members could then copy this example into their local repository’s terraform.tfvars and enter the appropriate values.

      Note

      Variable value files with names that don’t match terraform.tfvars or *.auto.tfvars can be specified with the -var-file option:

      terraform apply -var-file=myvars.tfvars
      

      Supplying multiple .tfvars files is another way to further separate secret variables and non-secret variables; e.g.:

      terraform apply 
      -var-file=non-secret-variables.tfvars 
      -var-file=secret-variables.tfvars
      

      Assigning Values in Environment Variables

      Terraform allows you to keep input variable values in environment variables. These variables have the prefix TF_VAR_ and are supplied at the command line. Using the above example of an API access token, you could export the variable and use it like so:

      export TF_VAR_token=your-token-value
      terraform apply
      

      You could also include the variable on the same line when running terraform plan or terraform apply:

      TF_VAR_token=your-token-value terraform apply
      

      Caution

      This method commits the environment variable to your shell’s history, so take care when using this method.

      Assigning Values in Command-Line Flags

      Variable values can be set with the -var option:

      terraform apply -var 'token=your-token-value'
      

      Caution

      This method commits the command-line variable to your shell’s history, so take care when using this method.

      Supply Variables at Prompt

      If Terraform does not find a default value for a defined variable; or a value from a .tfvars file, environment variable, or CLI flag; it will prompt you for a value before running an action:

        
      $ terraform plan
      var.token
        Your API access token
      
        Enter a value:
      
      

      This method is a bit easier to use than supplying environment variables, and has the added benefit of displaying the description you set up when defining your variable.

      How to Manage Your State File

      While it is relatively easy to keep secrets out of .tf files using any of the above methods, there is another file you need to be aware of when managing secrets, and that is the terraform.tfstate file.

      This state file contains a JSON object that holds your managed infrastructure’s current state. This state is a snapshot of the various attributes of your infrastructure at the time it was last modified. It is generated on terraform apply and is a necessary part of the Terraform process, as it maps the declarative code of your .tf files to your real world infrastructure.

      As of the writing of this guide, sensitive information used to generate your Terraform state can be stored as plain text in the terraform.tfstate file. For example, if you are working with the Linode provider and have supplied a root password for your Linode instance, that root password will be stored as plain text in the state file. Avoid checking your terraform.tfstate file into your version control repository. Instead, the following are some strategies for storing and sharing your state files.

      Remote Backends

      Terraform backends allow the user to securely store their state in a remote location, such as a key/value store like Consul, or an S3 compatible bucket storage like Minio. This allows the Terraform state to be read from the remote store, and because the state only ever exists locally in memory, there is no worry about storing secrets in plain text.

      Some backends, like Consul, also allow for state locking. If one user is applying a state, another user will be unable to make any changes.

      Using a Terraform backend is the preferred way to share a Terraform state file.

      Encrypting Secrets

      Third-party tools exist that allow you to encrypt your secrets. If you encrypt the secrets in your terraform.tfstate (or your .tfvars files), you can check them into version control securely:

      • git-crypt allows you to encrypt files when they are committed to a Git repository. git-crypt also decrypts files when they are checked out.

        Note

        You must initialize git-crypt in a repository before committing your state file or variable value files, or they will not be eligible for encryption.

      • Terrahelp allows you to encrypt and decrypt a whole state file, or just the variables you have include in your terraform.tfvars file.

      Use a Dummy Password

      It is possible to supply a dummy password to Terraform and later change that password manually to a more secure one. For instance, if you were to create a Linode instance with a dummy root password, you could later change that password from the command line or in the Linode Manager.

      Note

      Any attempt to change the password in a .tf file will result in the creation of new resources on terraform apply.

      Privatize Version Control

      If you are unwilling or unable to use the above options to help manage your state file, and if you are using a platform like GitHub or GitLab to share your state files, then at minimum the repository should be private.

      More Information

      You may wish to consult the following resources for additional information on this topic. While these are provided in the hope that they will be useful, please note that we cannot vouch for the accuracy or timeliness of externally hosted materials.

      Find answers, ask questions, and help others.

      This guide is published under a CC BY-ND 4.0 license.



      Source link

      Introduction to HashiCorp Configuration Language (HCL)


      Updated by Linode Written by Linode

      HCL is a configuration language authored by HashiCorp. HCL is used with HashiCorp’s cloud infrastructure automation tools, like Terraform. The language was created with the goal of being both human and machine friendly. It is JSON compatible, which means it is interoperable with other systems outside of the Terraform product line.

      This guide provides an introduction to HCL syntax and some commonly used HCL terminology.

      HCL Syntax Overview

      HashiCorp’s configuration syntax is easy to read and write. It was created to have a more clearly visible and defined structure when compared with other well known configuration languages, like YAML.

      ~/terraform/main.tf
       1
       2
       3
       4
       5
       6
       7
       8
       9
      10
      11
      12
      13
      14
      15
      16
      17
      18
      19
      20
      
      # Linode provider block. Installs Linode plugin.
      provider "linode" {
          token = "${var.token}"
      }
      
      variable "region" {
        description = "This is the location where the Linode instance is deployed."
      }
      
      /* A multi
         line comment. */
      resource "linode_instance" "example_linode" {
          image = "linode/ubuntu18.04"
          label = "example-linode"
          region = "${var.region}"
          type = "g6-standard-1"
          authorized_keys = [ "my-key" ]
          root_pass = "example-password"
      }
          

      Note

      Key Elements of HCL

      • HCL syntax is composed of stanzas or blocks that define a variety of configurations available to Terraform. Provider plugins expand on the available base Terraform configurations.

      • Stanzas or blocks are comprised of key = value pairs. Terraform accepts values of type string, number, boolean, map, and list.

      • Single line comments start with #, while multi-line comments use an opening /* and a closing */.

      • Interpolation syntax can be used to reference values stored outside of a configuration block, like in an input variable, or from a Terraform module’s output.

        An interpolated variable reference is constructed with the "${var.region}" syntax. This example references a variable named region, which is prefixed by var.. The opening ${ and closing } indicate the start of interpolation syntax.

      • You can include multi-line strings by using an opening <<EOF, followed by a closing EOF on its own line.

      • Strings are wrapped in double quotes.

      • Lists of primitive types (string, number, and boolean) are wrapped in square brackets: ["Andy", "Leslie", "Nate", "Angel", "Chris"].

      • Maps use curly braces {} and colons :, as follows: { "password" : "my_password", "db_name" : "wordpress" }.

      See Terraform’s Configuration Syntax documentation for more details.

      Providers

      In Terraform, a provider is used to interact with an Infrastructure as a Service (IaaS) or Platform as a Service (PaaS) API, like the Linode APIv4. The provider determines which resources are exposed and available to create, read, update, and delete. A credentials set or token is usually required to interface with your service account. For example, the Linode Terraform provider requires your Linode API access token. A list of all official Terraform providers is available from HashiCorp.

      Configuring a Linode as your provider requires that you include a block which specifies Linode as the provider and sets your Linode API token in one of your .tf files:

      ~/terraform/terraform.tf
      1
      2
      3
      
      provider "linode" {
          token = "my-token"
      }

      Once your provider is declared, you can begin configuring resources available from the provider.

      Note

      Providers are packaged as plugins for Terraform. Whenever declaring a new provider in your Terraform configuration files, the terraform init command should be run. This command will complete several initialization steps that are necessary before you can apply your Terraform configuration, including downloading the plugins for any providers you’ve specified.

      Resources

      A Terraform resource is any component of your infrastructure that can be managed by your provider. Resources available with the Linode provider range from a Linode instance, to a block storage volume, to a DNS record. Terraform’s Linode Provider documentation contains a full listing of all supported resources.

      Resources are declared with a resource block in a .tf configuration file. This example block deploys a 2GB Linode instance located in the US East data center from an Ubuntu 18.04 image. Values are also provided for the Linode’s label, public SSH key, and root password:

      ~/terraform/main.tf
      1
      2
      3
      4
      5
      6
      7
      8
      
      resource "linode_instance" "WordPress" {
          image = "linode/ubuntu18.04"
          label = "WPServer"
          region = "us-east"
          type = "g6-standard-1"
          authorized_keys = [ "example-key" ]
          root_pass = "example-root-pass"
      }

      HCL-specific meta-parameters are available to all resources and are independent of the provider you use. Meta-parameters allow you to do things like customize the lifecycle behavior of the resource, define the number of resources to create, or protect certain resources from being destroyed. See Terraform’s Resource Configuration documentation for more information on meta-parameters.

      Modules

      A module is an encapsulated set of Terraform configurations used to organize the creation of resources in reusable configurations.

      The Terraform Module Registry is a repository of community modules that can help you get started creating resources for various providers. You can also create your own modules to better organize your Terraform configurations and make them available for reuse. Once you have created your modules, you can distribute them via a remote version control repository, like GitHub.

      Using Modules

      A module block instructs Terraform to create an instance of a module. This block instantiates any resources defined within that module.

      The only universally required configuration for all module blocks is the source parameter which indicates the location of the module’s source code. All other required configurations will vary from module to module. If you are using a local module you can use a relative path as the source value. The source path for a Terraform Module Registry module will be available on the module’s registry page.

      This example creates an instance of a module named linode-module-example and provides a relative path as the location of the module’s source code:

      ~/terraform/main.tf
      1
      2
      3
      
      module "linode-module-example" {
          source = "/modules/linode-module-example"
      }

      Authoring modules involves defining resource requirements and parameterizing configurations using input variables, variable files, and outputs. To learn how to write your own Terraform modules, see Create a Terraform Module.

      Input Variables

      You can define input variables to serve as Terraform configuration parameters. By convention, input variables are normally defined within a file named variables.tf. Terraform will load all files ending in .tf, so you can also define variables in files with other names.

      • Terraform accepts variables of type string, number, boolean, map, and list. If a variable type is not explicitly defined, Terraform will default to type = "string".

      • It is good practice to provide a meaningful description for all your input variables.

      • If a variable does not contain a default value, or if you would like to override a variable’s default value, you must provide a value as an environment variable or within a variable values file.

      Variable Declaration Example

      ~/terraform/variables.tf
      1
      2
      3
      4
      5
      6
      7
      8
      
      variable "token" {
        description = "This is your Linode APIv4 Token."
      }
      
      variable "region" {
          description: "This is the location where the Linode instance is deployed."
          default = "us-east"
      }

      Two input variables named token and region are defined, respectively. The region variable defines a default value. Both variables will default to type = "string", since a type is not explicitly declared.

      Supplying Variable Values

      Variable values can be specified in .tfvars files. These files use the same syntax as Terraform configuration files:

      ~/terraform/terraform.tfvars
      1
      2
      
      token = "my-token"
      region = "us-west"

      Terraform will automatically load values from filenames which match terraform.tfvars or *.auto.tfvars. If you store values in a file with another name, you need to specify that file with the -var-file option when running terraform apply. The -var-file option can be invoked multiple times:

      terraform apply 
      -var-file="variable-values-1.tfvars" 
      -var-file="variable-values-2.tfvars"
      

      Values can also be specified in environment variables when running terraform apply. The name of the variable should be prefixed with TF_VAR_:

      TF_VAR_token=my-token-value TF_VAR_region=us-west terraform apply
      

      Note

      Environment variables can only assign values to variables of type = "string"

      Referencing Variables

      You can call existing input variables within your configuration file using Terraform’s interpolation syntax. Observe the value of the region parameter:

      ~/terraform/main.tf
      1
      2
      3
      4
      5
      6
      7
      8
      
      resource "linode_instance" "WordPress" {
          image = "linode/ubuntu18.04"
          label = "WPServer"
          region = "${var.region}"
          type = "g6-standard-1"
          authorized_keys = [ "example-key" ]
          root_pass = "example-root-pass"
      }

      Note

      If a variable value is not provided in any of the ways discussed above, and the variable is called in a resource configuration, Terraform will prompt you for the value when you run terraform apply.

      For more information on variables, see Terraform’s Input Variables documentation.

      Interpolation

      HCL supports the interpolation of values. Interpolations are wrapped in an opening ${ and a closing }. Input variable names are prefixed with var.:

      ~/terraform/terraform.tf
      1
      2
      3
      
      provider "linode" {
          token = "${var.token}"
      }

      Interpolation syntax is powerful and includes the ability to reference attributes of other resources, call built-in functions, and use conditionals and templates.

      This resource’s configuration uses a conditional to provide a value for the tags parameter:

      ~/terraform/terraform.tf
      1
      2
      3
      
      resource "linode_instance" "web" {
          tags = ["${var.env == "production" ? var.prod_subnet : var.dev_subnet}"]
      }

      If the env variable has the value production, then the prod_subnet variable is used. If not, then the variable dev_subent is used.

      Functions

      Terraform has built-in computational functions that perform a variety of operations, including reading files, concatenating lists, encrypting or creating a checksum of an object, and searching and replacing.

      ~/terraform/terraform.tf
      1
      2
      3
      4
      
      resource "linode_sshkey" "main_key" {
          label = "foo"
          ssh_key = "${chomp(file("~/.ssh/id_rsa.pub"))}"
      }

      In this example, ssh_key = "${chomp(file("~/.ssh/id_rsa.pub"))}" uses Terraform’s built-in function file() to provide a local file path to the public SSH key’s location. The chomp() function removes trailing new lines from the SSH key. Observe that the nested functions are wrapped in opening ${ and closing } to indicate that the value should be interpolated.

      Note

      Running terraform console creates an environment where you can test interpolation functions. For example:

      terraform console
      
        
      > list("newark", "atlanta", "dallas")
      [
        "newark",
        "atlanta",
        "dallas",
      ]
      >
      
      

      Terraform’s official documentation includes a complete list of supported built-in functions.

      Templates

      Templates can be used to store large strings of data. The template provider exposes the data sources for other Terraform resources or outputs to consume. The data source can be a file or an inline template.

      The data source can use Terraform’s standard interpolation syntax for variables. The template is then rendered with variable values that you supply in the data block.

      This example template resource substitutes in the value from ${linode_instance.web.ip_address} anywhere ${web_ip} appears inside the template file ips.json:

      1
      2
      3
      4
      5
      6
      7
      
      data "template_file" "web" {
          template = "${file("${path.module}/ips.json")}"
      
          vars {
              web_ip = "${linode_instance.web.ip_address}"
          }
      }

      You could then define an output variable to view the rendered template when you later run terraform apply:

      1
      2
      3
      
      output "ip" {
        value = "${data.template_file.web.rendered}"
      }

      Terraform’s official documentation has a list of all available components of interpolation syntax.

      Next Steps

      Now that you are familiar with HCL, you can begin creating your own Linode instance with Terraform by following the Use Terraform to Provision Linode Environments guide.

      More Information

      You may wish to consult the following resources for additional information on this topic. While these are provided in the hope that they will be useful, please note that we cannot vouch for the accuracy or timeliness of externally hosted materials.

      Find answers, ask questions, and help others.

      This guide is published under a CC BY-ND 4.0 license.



      Source link