One place for hosting & domains

      Manage

      How To Deploy and Manage Your DNS using OctoDNS on Ubuntu 18.04


      The author selected the Electronic Frontier Foundation to receive a donation as part of the Write for DOnations program.

      Introduction

      OctoDNS is an infrastructure-as-code tool that allows you to deploy and manage your DNS zones using standard software development principles, including version control, testing, and automated deployment. OctoDNS was created by GitHub and is written in Python.

      Using OctoDNS eliminates many of the pitfalls of manual DNS management, as zone files are stored in a structured format (YAML). This allows you to deploy zones to multiple DNS providers simultaneously, identify syntax errors, and push out your DNS configuration automatically, reducing the risk of human error. Another common usage of OctoDNS is to synchronize your DNS configuration between different providers, such as a testing and production system, or between live and failover environments.

      OctoDNS is similar to DNSControl, which is an equivalent tool created by Stack Exchange and written in Go. Unlike OctoDNS, DNSControl uses a JavaScript-based configuration language for defining DNS zones, which allows you to use advanced programmatic features such as loops to specify multiple similar records within the same zone. The article How to Deploy and Manage Your DNS Using DNSControl on Ubuntu 18.04 covers the basic setup and configuration of DNSControl.

      In this tutorial, you’ll install and configure OctoDNS, create a basic DNS configuration, and begin deploying DNS records to a live provider. As part of this tutorial, we will use DigitalOcean as the example DNS provider. If you wish to use a different provider, the setup is very similar. When you’re finished, you’ll be able to manage and test your DNS configuration in a safe, offline environment, and then automatically deploy it to production.

      Prerequisites

      Before you begin this guide you’ll need the following:

      • One Ubuntu 18.04 server set up by following the Initial Server Setup with Ubuntu 18.04, including a sudo non-root user and enabled firewall to block non-essential ports. your-server-ipv4-address and your-server-ipv6-address refer to the IP addresses of the server where you’re hosting your website or domain.
      • A fully registered domain name with DNS hosted by a supported provider. This tutorial will use your-domain throughout and DigitalOcean as the service provider.
      • A DigitalOcean API key (Personal Access Token) with read and write permissions. To create one, visit How to Create a Personal Access Token.

      Once you have these ready, log in to your server as your non-root user to begin.

      Step 1 — Installing OctoDNS

      OctoDNS is distributed as a Python pip package, and runs in a Python Virtual Environment (virtualenv), so you’ll start this step by installing the packages required for this. A virtualenv is an isolated Python environment that can have its own libraries and configuration, separate from the main system-wide Python installation. Python and virtualenv are available within Ubuntu’s default software repositories, making it possible to install using conventional package management tools.

      Begin by updating the local package index to reflect any new upstream changes:

      Then, install the python and virtualenv packages:

      • sudo apt install python virtualenv

      After confirming the installation, apt will download and install Python, virtualenv, and all of their required dependencies.

      Next, you'll create the required directories for OctoDNS, where your DNS and program configuration will be stored. Start by creating the ~/octodns and ~/octodns/config directories:

      • mkdir ~/octodns ~/octodns/config

      Now move into ~/octodns:

      Next, you need to create the Python Virtual Environment—an isolated Python environment with its own libraries and configuration to run OctoDNS in:

      Activate your environment with the following command:

      This will output something similar to the following:

      Output

      Running virtualenv with interpreter /usr/bin/python2 New python executable in /home/user/octodns/env/bin/python2 Also creating executable in /home/user/octodns/env/bin/python Installing setuptools, pkg_resources, pip, wheel...done.

      Your Bash shell prompt will now also be prefixed with the name of the virtual environment. This shows that you are currently operating within the virtualenv:

      (env) user@digitalocean:~/octodns$
      

      If you wish to exit the virtualenv, you can use the deactivate command at any time. However, you should stay in your virtualenv to continue with this tutorial.

      Now that you've installed and configured Python and virtualenv, you can install OctoDNS. OctoDNS is distributed as a Python pip package, which is the standard package-management tool for Python packages and libraries.

      You can install the OctoDNS pip package using the following command within your virtualenv:

      Once this is complete, you can check the installed version to make sure that everything is working:

      Your output will look similar to the following:

      Output

      octoDNS 0.9.6

      If you see a octodns-sync: command not found error, double-check that you're still inside your virtualenv.

      Now that you've installed OctoDNS, you can create the required configuration files to connect OctoDNS to your DNS provider to allow it to make changes to your DNS records.

      Step 2 — Configuring OctoDNS

      In this step, you'll create the required configuration files for OctoDNS, and connect it to your DNS provider so that it can begin to make live changes to your DNS records.

      Note: This tutorial will focus on the initial setup of OctoDNS; however for production use it is recommended to store your OctoDNS configuration in a version control system (VCS) such as Git. The advantages of this include full version control, integration with CI/CD for testing, seamlessly rolling-back deployments, and so on.

      Firstly, you need to configure the config.yaml file, which defines the DNS zones for OctoDNS to manage, and allows it to authenticate to your DNS provider and make changes.

      The format of config.yaml differs slightly depending on the DNS provider that you are using. Please see the Supported Providers list in the official OctoDNS documentation to find the configuration for your own provider. When viewing this hyperlink, the configuration details are presented as a code comment in the actual Python code for your provider, which is linked in the 'Provider' column of the table. Once you have found the Python code for your provider, such as cloudflare.py or route53.py, the relevant code comment can be found directly under the class ProviderNameProvider. For example:

      Excerpt of octodns/provider/route53.py

      class Route53Provider(BaseProvider):
          '''
          AWS Route53 Provider
          route53:
              class: octodns.provider.route53.Route53Provider
              # The AWS access key id
              access_key_id:
              # The AWS secret access key
              secret_access_key:
              # The AWS session token (optional)
              # Only needed if using temporary security credentials
              session_token:
      

      Move into the ~/octodns/config directory:

      Then create and open config.yaml for editing:

      Add the sample config.yaml configuration for your DNS provider to the file. If you're using DigitalOcean as your DNS provider, you can use the following:

      ~/octodns/config/config.yaml

      ---
      providers:
        config:
          class: octodns.provider.yaml.YamlProvider
          directory: ./config
          default_ttl: 300
          enforce_order: True
        digitalocean:
          class: octodns.provider.digitalocean.DigitalOceanProvider
          token: your-digitalocean-oauth-token
      
      zones:
        your-domain.:
          sources:
            - config
          targets:
            - digitalocean
      

      This file tells OctoDNS which DNS providers you want it to connect to, and which DNS zones it should manage for those providers.

      You'll need to provide some form of authentication for your DNS provider. This is usually an API key or OAuth token.

      If you do not wish to store your access token in plain text in the configuration file, you can instead pass it as an environment variable when the program runs. To do this, you should use the following token: line instead in config.yaml:

      ~/octodns/config/config.yaml

      token: env/DIGITALOCEAN_OAUTH_TOKEN
      

      Then, before running OctoDNS, set the relevant environment variable to your access token, and OctoDNS will read it from there when run:

      • export DIGITALOCEAN_OAUTH_TOKEN=your-digitalocean-oauth-token

      Warning: This token will grant access to your DNS provider account, so you should protect it as you would a password. Also, ensure that if you're using a version control system, either the file containing the token is excluded (e.g. using .gitignore), or is securely encrypted in some way.

      If you're using DigitalOcean as your DNS provider, you can use the required OAuth token in your DigitalOcean account settings that you generated as part of the prerequisites.

      If you have multiple different DNS providers—for example, for multiple domain names, or delegated DNS zones—you can define these all in the same config.yaml file.

      You've set up the initial OctoDNS configuration file to allow the program to authenticate to your DNS provider and make changes. Next you'll create the configuration for your DNS zones.

      Step 3 — Creating a DNS Configuration File

      In this step, you'll create an initial DNS configuration file, which will contain the DNS records for your domain name or delegated DNS zone.

      Each DNS zone that you want to manage using OctoDNS has its own file, for example your-domain.yaml. In this file, the DNS records for the zone are defined using YAML.

      To begin, move into the ~/octodns/config directory:

      Then create and open your-domain.yaml for editing:

      Add the following sample configuration to the file:

      ~/octodns/config/your-domain.yaml

      ---
      '':
        - type: A
          value: your-server-ipv4-address
      
      www:
        - type: A
          value: your-server-ipv4-address
      

      This sample file defines a DNS zone for your-domain with two A records, pointing to the IPv4 address that you're hosting your domain or website on. One A record is for the root domain (e.g. your-domain), and the other is for the www subdomain (e.g. www.your-domain).

      Once complete, save and close the file.

      You've set up a basic DNS zone configuration file for OctoDNS, with two basic A records pointing to the IPv4 address of your domain or website. Next, you'll expand the file with some useful DNS records.

      Step 4 — Populating Your DNS Configuration File

      Next, you can populate the DNS configuration file with a practical set of DNS records for your website or service, using the YAML structured configuration language.

      Unlike traditional BIND zone files, where DNS records are written in a raw, line-by-line format, DNS records within OctoDNS are defined as YAML keys and subkeys with a number of associated values, as shown briefly in Step 3.

      The top-level key is usually the 'name', which is essentially the record identifier. www, subdomain1, and mail are all examples of DNS 'name'. In OctoDNS, there are two special-use names, which are '', for the root record (usually referred to as @), and '*', for wildcard records. A required value of each key (DNS record) is type. This defines which type of DNS record you are defining within that YAML top-level key. A type exists for each of the standard DNS record types, including A, AAAA, MX, TXT, NS, CNAME, and so on. A full list of available record types is available in the Records section of the OctoDNS documentation.

      The values for your DNS records are defined either directly as values to the top-level keys (if you only have one value), or as a list (if you have multiple values, e.g. multiple IP addresses or MX addresses).

      For example, to define a single value, you could use the following configuration:

      ~/octodns/config/your-domain.yaml

      'www':
        type: A
        value: 203.0.113.1
      

      Alternatively, to define multiple values for a single record:

      ~/octodns/config/your-domain.yaml

      'www':
        type: A
        values:
        - 203.0.113.1
        - 203.0.113.2
      

      The syntax for setting DNS records varies slightly for each record type. Following are some examples for the most common record types:

      A records:

      Purpose: To point to an IPv4 address.

      Syntax:

      'name':
        type: A
        value: ipv4-address
      

      Example:

      'www':
        type: A
        value: your-server-ipv4-address
      

      AAAA records:

      Purpose: To point to an IPv6 address.

      Syntax:

      'name':
        type: AAAA
        value: ipv6-address
      

      Example:

      'www':
        type: AAAA
        value: your-server-ipv6-address
      

      CNAME records:

      Purpose: To make your domain/subdomain an alias of another.

      Syntax:

      'name':
        type: CNAME
        value: fully-qualified-domain-name
      

      Example:

      'www':
        type: CNAME
        value: www.example.org
      

      MX records:

      Purpose: To direct email to specific servers/addresses.

      Syntax:

      'name':
        type: MX
        value:
          exchange: mail-server
          preference: priority-value
      

      Note that a trailing . must be included if there are any dots in the MX value.

      Example:

      '':
        type: MX
        value:
          exchange: mail.your-domain.
          preference: 10
      

      TXT records:

      Purpose: To add arbitrary plain text, often used for configurations without their own dedicated record type.

      Syntax:

      'name':
        type: TXT
        value: content
      

      Example:

      '':
        type: TXT
        value: This is a TXT record.
      

      In order to begin adding DNS records for your domain or delegated DNS zone, edit your DNS configuration file:

      • cd ~/octodns/config
      • nano your-domain.yaml

      Next, you can begin populating your DNS zone using the syntax described in the previous list, as well as the Records section of the official OctoDNS documentation.

      For reference, the code block here contains a full sample configuration for an initial DNS setup:

      ~/octodns/config/your-domain.yaml

      ---
      '':
        - type: A
          value: your-server-ipv4-address
      
        - type: AAAA
          value: your-server-ipv6-address
      
        - type: MX
          value:
            exchange: mail.your-domain.
            preference: 10
      
        - type: TXT
          value: v=spf1 -all
      
      _dmarc:
        type: TXT
        value: v=DMARC1; p=reject; rua=mailto:abuse@your-domain; aspf=s; adkim=s;
      
      mail:
        - type: A
          value: your-server-ipv4-address
      
        - type: AAAA
          value: your-server-ipv6-address
      
      www:
        - type: A
          value: your-server-ipv4-address
      
        - type: AAAA
          value: your-server-ipv6-address
      

      Once you have completed your initial DNS configuration, save and close the file.

      In this step, you set up the initial DNS configuration file, containing your DNS records. Next, you will test the configuration and deploy it.

      Step 5 — Testing and Deploying Your DNS Configuration

      In this step, you will run a local syntax check on your DNS configuration, and then deploy the changes to the live DNS server/provider.

      Firstly, move into your octodns directory:

      Double check that you're still operating within your Python virtualenv by looking for the name of it before your Bash prompt:

      (env) user@digitalocean:~/octodns$
      

      Next, use the octodns-validate command to check the syntax of your configuration file(s). You'll need to specify the path to your configuration file:

      • octodns-validate --config=./config/config.yaml

      If the YAML syntax of your DNS configuration file is correct, OctoDNS will return with no output.

      If you see an error or warning in your output, OctoDNS will provide details on what and where the error is located within your YAML file.

      Next, you can perform a dry-run push of the DNS configuration, which will output which changes will be made, without actually making them:

      • octodns-sync --config=./config/config.yaml

      This should produce an output similar to the following:

      Output

      ******************************************************************************** * your-domain. ******************************************************************************** * digitalocean (DigitalOceanProvider) * Create <ARecord A 300, mail.your-domain., ['your-server-ipv4-address']> (config) * Create <AaaaRecord AAAA 300, mail.your-domain., ['your-server-ipv6-address']> (config) * Create <TxtRecord TXT 300, your-domain., ['v=spf1 -all']> (config) * Create <AaaaRecord AAAA 300, your-domain., ['your-server-ipv6-address']> (config) * Create <ARecord A 300, your-domain., ['your-server-ipv4-address']> (config) * Create <ARecord A 300, www.your-domain., ['your-server-ipv4-address']> (config) * Create <MxRecord MX 300, your-domain., [''10 mail.your-domain.'']> (config) * Create <TxtRecord TXT 300, _dmarc.your-domain., ['v=DMARC1; p=reject; rua=mailto:abuse@your-domain; aspf=s; adkim=s;']> (config) * Create <AaaaRecord AAAA 300, www.your-domain., ['your-server-ipv6-address']> (config) * Summary: Creates=9, Updates=0, Deletes=0, Existing Records=2 ********************************************************************************

      Warning: The next command will make live changes to your DNS records and possibly other settings. Please ensure that you are prepared for this, including taking a backup of your existing DNS configuration, as well as ensuring that you have the means to roll back if needed.

      Finally, you can push out the changes to your live DNS provider:

      • octodns-sync --config=./config/config.yaml --doit

      Note: In some cases, OctoDNS will refuse to push changes if it is making a significant number of adjustments. This is an automatic protection feature to prevent accidental misconfigurations. If you encounter this refusal, you can re-run octodns-sync using the --force option, but please ensure you are ready to do so.

      You'll see an output like the dry-run earlier in this step, but with the addition of something similar to the following:

      Output

      2019-07-07T23:17:27 INFO DigitalOceanProvider[digitalocean] apply: making changes 2019-07-07T23:17:30 INFO Manager sync: 9 total changes

      Now, if you check the DNS settings for your domain in the DigitalOcean control panel, you'll see the changes.

      A screenshot of the DigitalOcean control panel, showing some of the DNS changes that OctoDNS has made.

      You can also check the record creation by running a DNS query for your domain/delegated zone. You'll see that the records have been updated accordingly:

      You'll see output showing the IP address and relevant DNS record from your zone that was deployed using OctoDNS. DNS records can take some time to propagate, so you may need to wait and run this command again.

      In this final step, you ran a local syntax check of the DNS configuration file, then deployed it to your live DNS provider, and tested that the changes were made successfully.

      Conclusion

      In this article you set up OctoDNS and deployed a DNS configuration to a live provider. Now you can manage and test your DNS configuration changes in a safe, offline environment before deploying them to production.

      If you wish to explore this subject further, OctoDNS is designed to be integrated into your CI/CD pipeline, allowing you to run in-depth tests and have more control over your deployment to production. You could also look into integrating OctoDNS into your infrastructure build/deployment processes, allowing you to deploy servers and add them to DNS completely automatically.

      If you wish to go further with OctoDNS, the following DigitalOcean articles provide some interesting next steps to help integrate OctoDNS into your change management and infrastructure deployment workflows:



      Source link

      How To Deploy and Manage Your DNS using DNSControl on Ubuntu 18.04


      The author selected the Electronic Frontier Foundation Inc to receive a donation as part of the Write for DOnations program.

      Introduction

      DNSControl is an infrastructure-as-code tool that allows you to deploy and manage your DNS zones using standard software development principles, including version control, testing, and automated deployment. DNSControl was created by Stack Exchange and is written in Go.

      Using DNSControl eliminates many of the pitfalls of manual DNS management, as zone files are stored in a programmable format. This allows you to deploy zones to multiple DNS providers simultaneously, identify syntax errors, and push out your DNS configuration automatically, reducing the risk of human error. Another common usage of DNSControl is to quickly migrate your DNS to a different provider; for example, in the event of a DDoS attack or system outage.

      In this tutorial, you’ll install and configure DNSControl, create a basic DNS configuration, and begin deploying DNS records to a live provider. As part of this tutorial, we will use DigitalOcean as the example DNS provider. If you wish to use a different provider, the setup is very similar. When you’re finished, you’ll be able to manage and test your DNS configuration in a safe, offline environment, and then automatically deploy it to production.

      Prerequisites

      Before you begin this guide you’ll need the following:

      • One Ubuntu 18.04 server set up by following the Initial Server Setup with Ubuntu 18.04, including a sudo non-root user and enabled firewall to block non-essential ports. your-server-ip refers to the IP address of the server where you’re hosting your website or domain.
      • A fully registered domain name with DNS hosted by a supported provider. This tutorial will use example.com throughout and DigitalOcean as the service provider.
      • A DigitalOcean API key (Personal Access Token) with read and write permissions. To create one, visit How to Create a Personal Access Token.

      Once you have these ready, log in to your server as your non-root user to begin.

      Step 1 — Installing DNSControl

      DNSControl is written in Go, so you’ll start this step by installing Go to your server and setting your GOPATH.

      Go is available within Ubuntu’s default software repositories, making it possible to install using conventional package management tools.

      Begin by updating the local package index to reflect any new upstream changes:

      Then, install the golang-go package:

      • sudo apt install golang-go

      After confirming the installation, apt will download and install Go and all of its required dependencies.

      Next, you'll configure the required path environment variables for Go. If you would like to know more about this, you can read this tutorial on Understanding the GOPATH. Start by editing the ~/.profile file:

      Add the following lines to the very end of your file:

      ~/.profile

      ...
      export GOPATH="$HOME/go"
      export PATH="$PATH:$GOPATH/bin"
      

      Once you have added these lines to the bottom of the file, save and close it. Then reload your profile by either logging out and back in, or sourcing the file again:

      Now you've installed and configured Go, you can install DNSControl.

      The go get command can be used to fetch a copy of the code, automatically compile it and install it into your Go directory:

      • go get github.com/StackExchange/dnscontrol

      Once this is complete, you can check the installed version to make sure that everything is working:

      Your output will look similar to the following:

      Output

      dnscontrol 0.2.8-dev

      If you see a dnscontrol: command not found error, double-check your Go path setup.

      Now that you've installed DNSControl, you can create a configuration directory and connect DNSControl to your DNS provider in order to allow it to make changes to your DNS records.

      Step 2 — Configuring DNSControl

      In this step, you'll create the required configuration directories for DNSControl, and connect it to your DNS provider so that it can begin to make live changes to your DNS records.

      Firstly, create a new directory in which you can store your DNSControl configuration, and then move into it:

      • mkdir ~/dnscontrol
      • cd ~/dnscontrol

      Note: This tutorial will focus on the initial set up of DNSControl; however for production use it is recommended to store your DNSControl configuration in a version control system (VCS) such as Git. The advantages of this include full version control, integration with CI/CD for testing, seamlessly rolling-back deployments, and so on.

      If you plan to use DNSControl to write BIND zone files, you should also create the zones directory:

      BIND zone files are a raw, standardized method for storing DNS zones/records in plain text format. They were originally used for the BIND DNS server software, but are now widely adopted as the standard method for storing DNS zones. BIND zone files produced by DNSControl are useful if you want to import them to a custom or self-hosted DNS server, or for auditing purposes.

      However, if you just want to use DNSControl to push DNS changes to a managed provider, the zones directory will not be needed.

      Next, you need to configure the creds.json file, which is what will allow DNSControl to authenticate to your DNS provider and make changes. The format of creds.json differs slightly depending on the DNS provider that you are using. Please see the Service Providers list in the official DNSControl documentation to find the configuration for your own provider.

      Create the file creds.json in the ~/dnscontrol directory:

      • cd ~/dnscontrol
      • nano creds.json

      Add the sample creds.json configuration for your DNS provider to the file. If you're using DigitalOcean as your DNS provider, you can use the following:

      ~/dnscontrol/creds.json

      {
        "digitalocean": {
          "token": "your-digitalocean-oauth-token"
        }
      }
      

      This file tells DNSControl to which DNS providers you want it to connect.

      You'll need to provide some form of authentication for your DNS provider. This is usually an API key or OAuth token, but some providers require extra information, as documented in the Service Providers list in the official DNSControl documentation.

      Warning: This token will grant access to your DNS provider account, so you should protect it as you would a password. Also, ensure that if you're using a version control system, either the file containing the token is excluded (e.g. using .gitignore), or is securely encrypted in some way.

      If you're using DigitalOcean as your DNS provider, you can use the required OAuth token in your DigitalOcean account settings that you generated as part of the prerequisites.

      If you have multiple different DNS providers—for example, for multiple domain names, or delegated DNS zones—you can define these all in the same creds.json file.

      You've set up the initial DNSControl configuration directories, and configured creds.json to allow DNSControl to authenticate to your DNS provider and make changes. Next you'll create the configuration for your DNS zones.

      Step 3 — Creating a DNS Configuration File

      In this step, you'll create an initial DNS configuration file, which will contain the DNS records for your domain name or delegated DNS zone.

      dnsconfig.js is the main DNS configuration file for DNSControl. In this file, DNS zones and their corresponding records are defined using JavaScript syntax. This is known as a DSL, or Domain Specific Language. The JavaScript DSL page in the official DNSControl documentation provides further details.

      To begin, create the DNS configuration file in the ~/dnscontrol directory:

      • cd ~/dnscontrol
      • nano dnsconfig.js

      Then, add the following sample configuration to the file:

      ~/dnscontrol/dnsconfig.js

      // Providers:
      
      var REG_NONE = NewRegistrar('none', 'NONE');
      var DNS_DIGITALOCEAN = NewDnsProvider('digitalocean', 'DIGITALOCEAN');
      
      // Domains:
      
      D('example.com', REG_NONE, DnsProvider(DNS_DIGITALOCEAN),
          A('@', 'your-server-ip')
      );
      

      This sample file defines a domain name or DNS zone at a particular provider, which in this case is example.com hosted by DigitalOcean. An example A record is also defined for the zone root (@), pointing to the IP of the server that you're hosting your domain/website on.

      There are three main functions that make up a basic DNSControl configuration file:

      • NewRegistrar(name, type, metadata): defines the domain registrar for your domain name. DNSControl can use this to make required changes, such as modifying the authoritative nameservers. If you only want to use DNSControl to manage your DNS zones, this can generally be left as NONE.

      • NewDnsProvider(name, type, metadata): defines a DNS service provider for your domain name or delegated zone. This is where DNSControl will push the DNS changes that you make.

      • D(name, registrar, modifiers): defines a domain name or delegated DNS zone for DNSControl to manage, as well as the DNS records present in the zone.

      You should configure NewRegistrar(), NewDnsProvider(), and D() accordingly using the Service Providers list in the official DNSControl documentation.

      If you're using DigitalOcean as your DNS provider, and only need to be able to make DNS changes (rather than authoritative nameservers as well), the sample in the preceding code block is already correct.

      Once complete, save and close the file.

      In this step, you set up a DNS configuration file for DNSControl, with the relevant providers defined. Next, you'll populate the file with some useful DNS records.

      Step 4 — Populating Your DNS Configuration File

      Next, you can populate the DNS configuration file with useful DNS records for your website or service, using the DNSControl syntax.

      Unlike traditional BIND zone files, where DNS records are written in a raw, line-by-line format, DNS records within DNSControl are defined as a function parameter (domain modifier) to the D() function, as shown briefly in Step 3.

      A domain modifier exists for each of the standard DNS record types, including A, AAAA, MX, TXT, NS, CAA, and so on. A full list of available record types is available in the Domain Modifiers section of the DNSControl documentation.

      Modifiers for individual records are also available (record modifiers). Currently these are primarily used for setting the TTL (time to live) of individual records. A full list of available record modifiers is available in the Record Modifiers section of the DNSControl documentation. Record modifiers are optional, and in most basic use cases can be left out.

      The syntax for setting DNS records varies slightly for each record type. Following are some examples for the most common record types:

      • A records:

        • Purpose: To point to an IPv4 address.
        • Syntax: A('name', 'address', optional record modifiers)
        • Example: A('@', 'your-server-ip', TTL(30))
      • AAAA records:

        • Purpose: To point to an IPv6 address.
        • Syntax: AAAA('name', 'address', optional record modifiers)
        • Example: AAAA('@', '2001:db8::1') (record modifier left out, so default TTL will be used)
      • CNAME records:

        • Purpose: To make your domain/subdomain an alias of another.
        • Syntax: CNAME('name', 'target', optional record modifiers)
        • Example: CNAME('subdomain1', 'example.org.') (note that a trailing . must be included if there are any dots in the value)
      • MX records:

        • Purpose: To direct email to specific servers/addresses.
        • Syntax: MX('name', 'priority', 'target', optional record modifiers)
        • Example: MX('@', 10, 'mail.example.net') (note that a trailing . must be included if there are any dots in the value)
      • TXT records:

        • Purpose: To add arbitrary plain text, often used for configurations without their own dedicated record type.
        • Syntax: TXT('name', 'content', optional record modifiers)
        • Example: TXT('@', 'This is a TXT record.')
      • CAA records:

        • Purpose: To restrict and report on Certificate Authorities (CAs) who can issue TLS certificates for your domain/subdomains.
        • Syntax: CAA('name', 'tag', 'value', optional record modifiers)
        • Example: CAA('@', 'issue', 'letsencrypt.org')

      In order to begin adding DNS records for your domain or delegated DNS zone, edit your DNS configuration file:

      • cd ~/dnscontrol
      • nano dnsconfig.js

      Next, you can begin populating the parameters for the existing D() function using the syntax described in the previous list, as well as the Domain Modifiers section of the official DNSControl documentation. A comma (,) must be used in-between each record.

      For reference, the code block here contains a full sample configuration for a basic, initial DNS setup:

      ~/dnscontrol/dnsconfig.js

      ...
      
      D('example.com', REG_NONE, DnsProvider(DNS_DIGITALOCEAN),
          A('@', 'your-server-ip'),
          A('www', 'your-server-ip'),
          A('mail', 'your-server-ip'),
          AAAA('@', '2001:db8::1'),
          AAAA('www', '2001:db8::1'),
          AAAA('mail', '2001:db8::1'),
          MX('@', 10, 'mail.example.com.'),
          TXT('@', 'v=spf1 -all'),
          TXT('_dmarc', 'v=DMARC1; p=reject; rua=mailto:abuse@example.com; aspf=s; adkim=s;')
      );
      

      Once you have completed your initial DNS configuration, save and close the file.

      In this step, you set up the initial DNS configuration file, containing your DNS records. Next, you will test the configuration and deploy it.

      Step 5 — Testing and Deploying Your DNS Configuration

      In this step, you will run a local syntax check on your DNS configuration, and then deploy the changes to the live DNS server/provider.

      Firstly, move into your dnscontrol directory:

      Next, use the preview function of DNSControl to check the syntax of your file, and output what changes it will make (without actually making them):

      If the syntax of your DNS configuration file is correct, DNSControl will output an overview of the changes that it will make. This should look similar to the following:

      Output

      ******************** Domain: example.com ----- Getting nameservers from: digitalocean ----- DNS Provider: digitalocean...8 corrections #1: CREATE A example.com your-server-ip ttl=300 #2: CREATE A www.example.com your-server-ip ttl=300 #3: CREATE A mail.example.com your-server-ip ttl=300 #4: CREATE AAAA example.com 2001:db8::1 ttl=300 #5: CREATE TXT _dmarc.example.com "v=DMARC1; p=reject; rua=mailto:abuse@example.com; aspf=s; adkim=s;" ttl=300 #6: CREATE AAAA www.example.com 2001:db8::1 ttl=300 #7: CREATE AAAA mail.example.com 2001:db8::1 ttl=300 #8: CREATE MX example.com 10 mail.example.com. ttl=300 ----- Registrar: none...0 corrections Done. 8 corrections.

      If you see an error warning in your output, DNSControl will provide details on what and where the error is located within your file.

      Warning: The next command will make live changes to your DNS records and possibly other settings. Please ensure that you are prepared for this, including taking a backup of your existing DNS configuration, as well as ensuring that you have the means to roll back if needed.

      Finally, you can push out the changes to your live DNS provider:

      You'll see an output similar to the following:

      Output

      ******************** Domain: example.com ----- Getting nameservers from: digitalocean ----- DNS Provider: digitalocean...8 corrections #1: CREATE TXT _dmarc.example.com "v=DMARC1; p=reject; rua=mailto:abuse@example.com; aspf=s; adkim=s;" ttl=300 SUCCESS! #2: CREATE A example.com your-server-ip ttl=300 SUCCESS! #3: CREATE AAAA example.com 2001:db8::1 ttl=300 SUCCESS! #4: CREATE AAAA www.example.com 2001:db8::1 ttl=300 SUCCESS! #5: CREATE AAAA mail.example.com 2001:db8::1 ttl=300 SUCCESS! #6: CREATE A www.example.com your-server-ip ttl=300 SUCCESS! #7: CREATE A mail.example.com your-server-ip ttl=300 SUCCESS! #8: CREATE MX example.com 10 mail.example.com. ttl=300 SUCCESS! ----- Registrar: none...0 corrections Done. 8 corrections.

      Now, if you check the DNS settings for your domain in the DigitalOcean control panel, you'll see the changes.

      A screenshot of the DigitalOcean control panel, showing some of the DNS changes that DNSControl has made.

      You can also check the record creation by running a DNS query for your domain/delegated zone. You'll see that the records have been updated accordingly:

      You'll see output showing the IP address and relevant DNS record from your zone that was deployed using DNSControl. DNS records can take some time to propagate, so you may need to wait and run this command again.

      In this final step, you ran a local syntax check of the DNS configuration file, then deployed it to your live DNS provider, and tested that the changes were made successfully.

      Conclusion

      In this article you set up DNSControl and deployed a DNS configuration to a live provider. Now you can manage and test your DNS configuration changes in a safe, offline environment before deploying them to production.

      If you wish to explore this subject further, DNSControl is designed to be integrated into your CI/CD pipeline, allowing you to run in-depth tests and have more control over your deployment to production. You could also look into integrating DNSControl into your infrastructure build/deployment processes, allowing you to deploy servers and add them to DNS completely automatically.

      If you wish to go further with DNSControl, the following DigitalOcean articles provide some interesting next steps to help integrate DNSControl into your change management and infrastructure deployment workflows:



      Source link

      Use HashiCorp Vault to Manage Secrets


      Updated by Linode Contributed by Linode

      HashiCorp Vault is a secrets management tool that helps to provide secure, automated access to sensitive data. Vault meets these use cases by coupling authentication methods (such as application tokens) to secret engines (such as simple key/value pairs) using policies to control how access is granted. In this guide, you will install, configure, and access Vault in an example deployment to illustrate Vault’s features and API.

      This guide will use the latest version of Vault, which is 1.1.0 at the time of this writing.

      Why Use Vault?

      A service such as Vault requires operational effort to run securely and effectively. Given the added complexity of using Vault as part of an application, in what way does it add value?

      Consider a simple application that must use an API token or other secret value. How should this sensitive credential be given to the application at runtime?

      • Committing the secret alongside the rest of the application code in a version control system such as git is a poor security practice for a number of reasons, including that the sensitive value is recorded in plaintext and not protected in any way.
      • Recording a secret in a file that is passed to an application requires that the file be securely populated in the first place and strictly access-controlled.
      • Static credentials are challenging to rotate or restrict access to if an application is compromised.

      Vault solves these and other problems in a number of ways, including:

      • Services and applications that run without operator interaction can authenticate to Vault using values that can be rotated, revoked, and permission-controlled.
      • Some secrets engines can generate temporary, dynamically-generated secrets to ensure that credentials expire after a period of time.
      • Policies for users and machine accounts can be strictly controlled for specific types of access to particular paths.

      Concepts

      Before continuing, you should familiarize yourself with important Vault terms and concepts that will be used later in this guide.

      • A token is the the underlying mechanism that underpins access to Vault resources. Whether a user authenticates to Vault using a GitHub token or an application-driven service authenticates using an AppRole RoleID and SecretID, all forms of authentication are eventually normalized to a token. Tokens are typically short-lived (that is, expire after a period or time-to-live, or ttl) and have one or more policies attached to them.
      • A Vault policy dictates certain actions that may be performed upon a Vault path. Capabilities such as the ability to read a secret, write secrets, and delete them are all examples of actions that are defined in a policy for a particular path.
      • A path in Vault is similar in form to a Unix filesystem path (like /etc) or a URL (such as /blog/title). Users and machine accounts interact with Vault over particular paths in order to retrieve secrets, change settings, or otherwise interact with a running Vault service. All Vault access is performed over a REST interface, so these paths eventually take the form of an HTTP URL. While some paths interact with the Vault service itself to manage resources such as policies or settings, many paths serve as an endpoint to either authenticate to Vault or interact with a secret engine.
      • A secret engine is a backend used in Vault to provide secrets to Vault users. The simplest example of a secret engine is the key/value backend, which simply returns plain text values that may be stored at particular paths (these secrets remain encrypted on the backend). Other examples of secret backends include the PKI backend, which can generate and manage TLS certificates, and the TOTP backend, which can generate temporary one-time passwords for web sites that require multi-factor authentication (including the Linode Manager).

      Installation

      This guide will setup Vault in a simple, local filesystem-only configuration. The steps listed here apply equally to any distribution.

      These installation steps will:

      • Procure a TLS certificate to ensure that all communications between Vault and clients are encrypted.
      • Configure Vault for local filesystem storage.
      • Install the vault binary and set up the operating system to operate Vault as a service.

      Note

      The configuration outlined in this guide is suitable for small deployments. In situations that call for highly-available or fault-tolerant services, consider running more than one Vault instance with a highly-available storage backend such as Consul.

      Before you Begin

      1. Familiarize yourself with Linode’s Getting Started guide and complete the steps for deploying and setting up a Linode running a recent Linux distribution (such as Ubuntu 18.04 or CentOS 7), including setting the hostname and timezone.

        Note

        Setting the full hostname correctly in /etc/hosts is important in this guide in order to terminate TLS on Vault correctly. Your Linode’s fully qualified domain name and short hostname should be present in the /etc/hosts file before continuing.

      2. This guide uses sudo wherever possible. Complete the sections of our Securing Your Server guide to create a standard user account, harden SSH access, and remove unnecessary network services.

      3. Follow our UFW Guide in order to install and configure a firewall on your Ubuntu or Debian-based system, or our FirewallD Guide for rpm or CentOS-based systems. Consider reviewing Vault’s Production Hardening recommendations if this will be used in a production environment.

        Note

        When configuring a firewall, keep in mind that Vault listens on port 8200 by default and Let’s Encrypt utilizes ports 80 (HTTP) and 443 (HTTPS).

      4. Ensure your system is up to date. On Debian-based systems, use:

        sudo apt update && sudo apt upgrade
        

        While on rpm-based systems, such as CentOS, use:

        sudo yum update
        

      Acquire a TLS Certificate

      1. Follow the steps in our Secure HTTP Traffic with Certbot guide to acquire a TLS certificate.

      2. Add a system group in order to grant limited read access to the TLS files created by Certbot.

        sudo groupadd tls
        
      3. Change the group ownership of certificate files in the Let’s Encrypt directory to tls.

        sudo chgrp -R tls /etc/letsencrypt/{archive,live}
        
      4. Grant members of the tls group read access to the necessary directories and files.

        sudo chmod g+rx /etc/letsencrypt/{archive,live}
        sudo find /etc/letsencrypt/archive -name 'privkey*' -exec chmod g+r {} ';'
        

      Download Vault files

      1. Download the release binary for Vault.

        wget https://releases.hashicorp.com/vault/1.1.0/vault_1.1.0_linux_amd64.zip
        

        Note

        If you receive an error that indicates wget is missing from your system, install the wget package and try again.

      2. Download the checksum file, which will verify that the zip file is not corrupt.

        wget https://releases.hashicorp.com/vault/1.1.0/vault_1.1.0_SHA256SUMS
        
      3. Download the checksum signature file, which verifies that the checksum file has not been tampered with.

        wget https://releases.hashicorp.com/vault/1.1.0/vault_1.1.0_SHA256SUMS.sig
        

      Verify the Downloads

      1. Import the HashiCorp Security GPG key (listed on the HashiCorp Security page under Secure Communications):

        gpg --recv-keys 51852D87348FFC4C
        

        The output should show that the key was imported:

          
        gpg: /home/user/.gnupg/trustdb.gpg: trustdb created
        gpg: key 51852D87348FFC4C: public key "HashiCorp Security " imported
        gpg: no ultimately trusted keys found
        gpg: Total number processed: 1
        gpg:               imported: 1
        
        

        Note

        If an error occurs with the error message keyserver receive failed: Syntax error in URI, simply try rerunning the gpg command again.

        Note

        If you receive errors that indicate the dirmngr software is missing or inaccessible, install dirmngr using your package manager and run the GPG command again.

      2. Verify the checksum file’s GPG signature:

        gpg --verify vault*.sig vault*SHA256SUMS
        

        The output should contain the Good signature from "HashiCorp Security <security@hashicorp.com>" confirmation message:

          
        gpg: Signature made Mon 18 Mar 2019 01:44:51 PM MDT
        gpg:                using RSA key 91A6E7F85D05C65630BEF18951852D87348FFC4C
        gpg: Good signature from "HashiCorp Security <security@hashicorp.com>" [unknown]
        gpg: WARNING: This key is not certified with a trusted signature!
        gpg:          There is no indication that the signature belongs to the owner.
        Primary key fingerprint: 91A6 E7F8 5D05 C656 30BE  F189 5185 2D87 348F FC4C
        
        
      3. Verify that the fingerprint output matches the fingerprint listed in the Secure Communications section of the HashiCorp Security page.

      4. Verify the .zip archive’s checksum:

        sha256sum -c vault*SHA256SUMS 2>&1 | grep OK
        

        The output should show the file’s name as given in the vault*SHA256SUMS file:

          
        vault_1.1.0_linux_amd64.zip: OK
        
        

      Install the Vault Executable

      1. Extract the Vault executable to the local directory.

        unzip vault_*_linux_amd64.zip
        

        Note

        If you receive an error that indicates unzip is missing from your system, install the unzip package and try again.

      2. Move the vault executable into a system-wide location.

        sudo mv vault /usr/local/bin
        
      3. Reset the ownership and permissions on the executable.

        sudo chown root:root /usr/local/bin/vault
        sudo chmod 755 /usr/local/bin/vault
        
      4. Set executable capabilities on the vault binary. This will grant Vault privileges to lock memory, which is a best practice for running Vault securely (see the Vault documentation for additional information).

        sudo setcap cap_ipc_lock=+ep /usr/local/bin/vault
        
      5. Verify that vault is now available in the local shell.

        vault --version
        

        The output of this command should return the following.

          
        Vault v1.1.0 ('36aa8c8dd1936e10ebd7a4c1d412ae0e6f7900bd')
        
        

      System Vault Configuration

      1. Create a system user that vault will run as when the service is started.

        sudo useradd --system -d /etc/vault.d -s /bin/nologin vault
        
      2. Add the vault user to the previously created tls group, which will grant the user the ability to read Let’s Encrypt certificates.

        sudo gpasswd -a vault tls
        
      3. Create the data directory and configuration directory for vault with limited permissions.

        sudo install -o vault -g vault -m 750 -d /var/lib/vault
        sudo install -o vault -g vault -m 750 -d /etc/vault.d
        
      4. Create a systemd service file that will control how to run vault persistently as a system daemon.

        /etc/systemd/system/vault.service
         1
         2
         3
         4
         5
         6
         7
         8
         9
        10
        11
        12
        13
        14
        15
        16
        17
        18
        19
        20
        21
        22
        23
        24
        25
        26
        27
        28
        29
        30
        31
        32
        
        [Unit]
        Description="a tool for managing secrets"
        Documentation=https://www.vaultproject.io/docs/
        Requires=network-online.target
        After=network-online.target
        ConditionFileNotEmpty=/etc/vault.d/vault.hcl
        
        [Service]
        User=vault
        Group=vault
        ProtectSystem=full
        ProtectHome=read-only
        PrivateTmp=yes
        PrivateDevices=yes
        SecureBits=keep-caps
        AmbientCapabilities=CAP_IPC_LOCK
        Capabilities=CAP_IPC_LOCK+ep
        CapabilityBoundingSet=CAP_SYSLOG CAP_IPC_LOCK
        NoNewPrivileges=yes
        ExecStart=/usr/local/bin/vault server -config=/etc/vault.d/vault.hcl
        ExecReload=/bin/kill --signal HUP $MAINPID
        KillMode=process
        KillSignal=SIGINT
        Restart=on-failure
        RestartSec=5
        TimeoutStopSec=30
        StartLimitIntervalSec=60
        StartLimitBurst=3
        LimitNOFILE=65536
        
        [Install]
        WantedBy=multi-user.target

        These systemd service options define a number of important settings to ensure that Vault runs securely and reliably. Review the Vault documentation for a complete explanation of what these options achieve.

      Configuration

      Configure Vault

      1. Create a configuration file for Vault with the following contents, replacing example.com with the domain used in your Let’s Encrypt certificates.

        /etc/vault.d/vault.hcl
        1
        2
        3
        4
        5
        6
        7
        8
        9
        
        listener "tcp" {
          address = "0.0.0.0:8200"
          tls_cert_file = "/etc/letsencrypt/live/example.com/fullchain.pem"
          tls_key_file = "/etc/letsencrypt/live/example.com/privkey.pem"
        }
        
        storage "file" {
          path = "/var/lib/vault"
        }

        This configuration will use the Let’s Encrypt certificates created in the previous steps to terminate TLS for the Vault service. This ensures that secrets will never be transmitted in plaintext. The actual storage for Vault will be on the local filesystem at /var/lib/vault.

      Run The Vault Service

      1. Vault is now ready to run. Start the service using systemctl.

        sudo systemctl start vault
        
      2. If desired, enable the service as well so that Vault starts at system boot time.

        sudo systemctl enable vault
        
      3. Confirm that Vault is operational by using the vault executable to check for the service’s status. Set the VAULT_ADDR environment variable to https://example.com:8200, replacing example.com with your own domain:

        export VAULT_ADDR=https://example.com:8200
        
      4. vault commands should now be sent to your local Vault instance. To confirm this, run the vault status command:

        vault status
        

        The command should return output similar to the following:

          
        Key                Value
        ---                -----
        Seal Type          shamir
        Initialized        false
        Sealed             true
        Total Shares       0
        Threshold          0
        Unseal Progress    0/0
        Unseal Nonce       n/a
        Version            n/a
        HA Enabled         false
        
        

      The remainder of this tutorial assumes that the environment variable VAULT_ADDR is set to this value to ensure that requests are sent to the correct Vault host.

      Initializing Vault

      At this stage, Vault is installed and running, but not yet initialized. The following steps will initialize the Vault backend, which sets unseal keys and returns the initial root token. Initialization occurs only one time for a Vault deployment.

      There are two configurable options to choose when performing the initialization step. The first value is the number of key shares, which controls the total number of unseal keys that Vault will generate. The second value is the key threshold, which controls how many of these unseal key shares are required before Vault will successfully unseal itself. Unsealing is required whenever Vault is restarted or otherwise brought online after being in a previously offline state.

      To illustrate this concept, consider a secure server in a data center. Because the Vault database is only decrypted in-memory, stealing or bringing the server offline for any reason will leave the only copy of Vault’s database on the filesystem in encrypted form, or “sealed”.

      When starting the server again, a key share of 3 and key threshold of 2 means that 3 keys exist, but at least 2 must be provided at startup for Vault to derive its decryption key and load its database into memory for access once again.

      The key share count ensure that multiple keys can exist at different locations for a degree of fault tolerance and backup purposes. The key threshold count ensures that compromising one unseal key alone is not sufficient to decrypt Vault data.

      1. Choose a value for the number of key shares and key threshold. Your situation may vary, but as an example, consider a team of three people in charge of operating Vault. A key share of 3 ensures that each member holds one unseal key. A key threshold of 2 means that no single operator can lose their key and compromise the system or steal the Vault database without coordinating with another operator.

      2. Using these chosen values, execute the initialization command. Be prepared to save the output that is returned from the following command, as it is only viewable once.

        vault operator init -key-shares=3 -key-threshold=2
        

        This command will return output similar to the following:

          
        Unseal Key 1: BaR6GUWRY8hIeNyuzAn7FTa82DiIldgvEZhOKhVsl0X5
        Unseal Key 2: jzh7lji1NX9TsNVGycUudSIy/X4lczJgsCpRfm3m8Q03
        Unseal Key 3: JfdH8LqEyc4B+xLMBX6/LT9o8G/6isC2ZFfz+iNMIW/0
        
        Initial Root Token: s.YijNa8lqSDeho1tJBtY02983
        
        Vault initialized with 3 key shares and a key threshold of 2. Please securely
        distribute the key shares printed above. When the Vault is re-sealed,
        restarted, or stopped, you must supply at least 2 of these keys to unseal it
        before it can start servicing requests.
        
        Vault does not store the generated master key. Without at least 2 key to
        reconstruct the master key, Vault will remain permanently sealed!
        
        It is possible to generate new unseal keys, provided you have a quorum of
        existing unseal keys shares. See "vault operator rekey" for more information.
        
        
      3. In a production scenario, these unseal keys should be stored in separate locations. For example, store one in a password manager such as LastPass, encrypted one with gpg, and store another offline on a USB key. Doing so ensures that compromising one storage location is not sufficient to recover the number of unseal keys required to decrypt the Vault database.

      4. The Initial Root Token is equivalent to the “root” or superuser account for the Vault API. Record and protect this token in a similar fashion. Like the root account on a Unix system, this token should be used to create less-privileged accounts to use for day-to-day interactions with Vault and the root token should be used infrequently due to its widespread privileges.

      Unseal Vault

      After initialization, Vault will be sealed. The following unseal steps must be performed any time the vault service is brought down and then brought up again, such as when performing systemctl restart vault or restarting the host machine.

      1. With VAULT_ADDR set appropriately, execute the unseal command.

        vault operator unseal
        

        A prompt will appear:

          
        Unseal Key (will be hidden):
        
        
      2. Paste or enter one unseal key and press Enter. The command will finish with output similar to the following:

          
        Unseal Key (will be hidden):
        Key                Value
        ---                -----
        Seal Type          shamir
        Initialized        true
        Sealed             true
        Total Shares       3
        Threshold          2
        Unseal Progress    1/2
        Unseal Nonce       0124ce2a-6229-fac1-0e3f-da3e97e00583
        Version            1.1.0
        HA Enabled         false
        
        

        Notice that the output indicates that the one out of two required unseal keys have been provided.

      3. Perform the unseal command again.

        vault operator unseal
        
      4. Enter a different unseal key when the prompt appears.

          
        Unseal Key (will be hidden):
        
        
      5. The resulting output should indicate that Vault is now unsealed (notice the Sealed false line).

          
        Unseal Key (will be hidden):
        Key             Value
        ---             -----
        Seal Type       shamir
        Initialized     true
        Sealed          false
        Total Shares    3
        Threshold       2
        Version         1.1.0
        Cluster Name    vault-cluster-a397153e
        Cluster ID      a065557e-3ee8-9d26-4d90-b90c8d69fa5d
        HA Enabled      false
        
        

      Vault is now operational.

      Using Vault

      Token Authentication

      When interacting with Vault over its REST API, Vault identifies and authenticates most requests by the presence of a token. While the initial root token can be used for now, the Policies section of this guide explains how to provision additional tokens.

      1. Set the VAULT_TOKEN environment variable to the value of the previously-obtained root token. This token is the authentication mechanism that the vault command will rely on for future interaction with Vault. The actual root token will be different in your environment.

        export VAULT_TOKEN=s.YijNa8lqSDeho1tJBtY02983
        
      2. Use the token lookup subcommand to confirm that the token is valid and has the expected permissions.

        vault token lookup
        
      3. The output of this command should include the following:

          
        policies            [root]
        
        

      The KV Secret Backend

      Vault backends are the core mechanism Vault uses to permit users to read and write secret values. The simplest backend to illustrate this functionality is the KV backend. This backend lets clients write key/value pairs (such as mysecret=apikey) that can be read later.

      1. Enable the secret backend by using the enable Vault subcommand.

        vault secrets enable -version=2 kv
        
      2. Write an example value to the KV backend using the kv put Vault subcommand.

        vault kv put kv/myservice api_token=secretvalue
        

        This command should return output similar to the following:

          
        Key              Value
        ---              -----
        created_time     2019-03-31T04:35:38.631167678Z
        deletion_time    n/a
        destroyed        false
        version          1
        
        
      3. Read this value from the kv/myservice path.

        vault kv get kv/myservice
        

        This command should return output similar to the following:

          
        ====== Metadata ======
        Key              Value
        ---              -----
        created_time     2019-03-31T04:35:38.631167678Z
        deletion_time    n/a
        destroyed        false
        version          1
        
        ====== Data ======
        Key          Value
        ---          -----
        api_token    secretvalue
        
        
      4. Many utilities and script are better suited to process json output. Use the -format=json flag to do a read once more, with the results return in JSON form.

        vault kv get -format=json kv/myservice
        
        {
          "request_id": "2734ea81-6f39-c017-4c73-2719b2018b65",
          "lease_id": "",
          "lease_duration": 0,
          "renewable": false,
          "data": {
            "data": {
              "api_token": "secretvalue"
            },
            "metadata": {
              "created_time": "2019-03-31T04:35:38.631167678Z",
              "deletion_time": "",
              "destroyed": false,
              "version": 1
            }
          },
          "warnings": null
        }

      Policies

      Up until this point, we have performed API calls to Vault with the root token. Production best practices dictate that this token should rarely be used and most operations should be performed with lesser-privileged tokens associated with controlled policies.

      Policies are defined by specifying a particular path and the set of capabilities that are permitted by a user upon the path. In our previous commands, the path has been kv/myservice, so we can create a policy to only read this secret and perform no other operations, including reading or listing secrets. When no policy exists for a particular path, Vault denies operations by default.

      In the case of the KV backend, Vault distinguishes operations upon the stored data, which are the actual stored values, and metadata, which includes information such as version history. In this example, we will create a policy to control access to the key/value data alone.

      1. Create the following Vault policy file.

        policy.hcl
        1
        2
        3
        
        path "kv/data/myservice" {
          capabilities = ["read"]
        }

        This simple policy will permit any token associated with it to read the secret stored at the KV secret backend path kv/myservice.

      2. Load this policy into Vault using the policy write subcommand. The following command names the aforementioned policy read-myservice.

        vault policy write read-myservice policy.hcl
        
      3. To illustrate the use of this policy, create a new token with this new policy associated with it.

        vault token create -policy=read-myservice
        

        This command should return output similar to the following.

          
        Key                  Value
        ---                  -----
        token                s.YdpJWRRaEIgdOW4y72sSVygy
        token_accessor       07akQfzg0TDjj3YoZSGMPkHA
        token_duration       768h
        token_renewable      true
        token_policies       ["default" "read-myservice"]
        identity_policies    []
        policies             ["default" "read-myservice"]
        
        
      4. Open another terminal window or tab and login to the same host that Vault is running on. Set the VAULT_ADDR to ensure that new vault commands point at the local instance of Vault, replacing example.com with your domain.

        export VAULT_ADDR=https://example.com:8200
        
      5. Set the VAULT_TOKEN environment variable to the new token just created by the token create command. Remember that your actual token will be different than the one in this example.

        export VAULT_TOKEN=s.YdpJWRRaEIgdOW4y72sSVygy
        
      6. Now attempt to read our secret in Vault at the kv/myservice path.

        vault kv get kv/myservice
        

        Vault should return the key/value data.

          
        ====== Metadata ======
        Key              Value
        ---              -----
        created_time     2019-03-31T04:35:38.631167678Z
        deletion_time    n/a
        destroyed        false
        version          1
        
        ====== Data ======
        Key          Value
        ---          -----
        api_token    secretvalue
        
        
      7. To illustrate forbidden operations, attempt to list all secrets in the KV backend.

        vault kv list kv/
        

        Vault should deny this request.

          
        Error listing kv/metadata: Error making API request.
        
        URL: GET https://example.com:8200/v1/kv/metadata?list=true
        Code: 403. Errors:
        
        * 1 error occurred:
                * permission denied
        
        
      8. In contrast, attempt to perform the same operation in the previous terminal window that has been configured with the root token.

        vault kv list kv/
        
          
        Keys
        ----
        myservice
        
        

        The root token should have sufficient rights to return a list of all secret keys under the kv/ path.

      Authentication Methods

      In practice, when services that require secret values are deployed, a token should not be distributed as part of the deployment or configuration management. Rather, services should authenticate themselves to Vault in order to acquire a token that has a limited lifetime. This ensures that credentials eventually expire and cannot be reused if they are ever leaked or disclosed.

      Vault supports many types of authentication methods. For example, the Kubernetes authentication method can retrieve a token for individual pods. As a simple illustrative example, the following steps will demonstrate how to use the AppRole method.

      The AppRole authentication method works by requiring that clients provide two pieces of information: the AppRole RoleID and SecretID. The recommendation approach to using this method is to store these two pieces of information in separate locations, as one alone is not sufficient to authenticate against Vault, but together, they permit a client to retrieve a valid Vault token. For example, in a production service, a RoleID might be present in a service’s configuration file, while the SecretID could be provided as an environment variable.

      1. Enable the AppRole authentication method using the auth subcommand. Remember to perform these steps in the terminal window with the root token stored in the VAULT_TOKEN environment variable, otherwise Vault commands will fail.

        vault auth enable approle
        
      2. Create a named role. This will define a role that can be used to “log in” to Vault and retrieve a token with a policy associated with it. The following command creates a named role named my-application which creates tokens valid for 10 minutes which will have the read-myservice policy associated with them.

        vault write auth/approle/role/my-application 
            token_ttl=10m 
            policies=read-myservice
        
      3. Retrieve the RoleID of the named role, which uniquely identifies the AppRole. Note this value for later use.

        vault read auth/approle/role/my-application/role-id
        
          
        Key        Value
        ---        -----
        role_id    147cd412-d1c2-4d2c-c57e-d660da0b1fa8
        
        

        In this example case, RoleID is 147cd412-d1c2-4d2c-c57e-d660da0b1fa8. Note that your value will be different.

      4. Finally, read the secret-id of the named role, and save this value for later use as well.

        vault write -f auth/approle/role/my-application/secret-id
        
          
        Key                   Value
        ---                   -----
        secret_id             2225c0c3-9b9f-9a9c-a0a5-10bf06df7b25
        secret_id_accessor    30cbef6a-8834-94fe-6cf3-cf2e4598dd6a
        
        

        In this example output, the SecretID is 2225c0c3-9b9f-9a9c-a0a5-10bf06df7b25.

      5. Use these values to generate a limited-use token by performing a write operation against the AppRole API. Replace the RoleID and SecretID values here with your own.

        vault write auth/approle/login 
            role_id=147cd412-d1c2-4d2c-c57e-d660da0b1fa8 
            secret_id=2225c0c3-9b9f-9a9c-a0a5-10bf06df7b25
        

        The resulting output should include a new token, which in this example case is s.coRl4UR6YL1sqw1jXhJbuZfq

          
        Key                     Value
        ---                     -----
        token                   s.3uu4vwFO8D1mG5S76IG04mck
        token_accessor          fi3aW4W9kZNB3FAC20HRXeoT
        token_duration          10m
        token_renewable         true
        token_policies          ["default" "read-myservice"]
        identity_policies       []
        policies                ["default" "read-myservice"]
        token_meta_role_name    my-application
        
        
      6. Open one more terminal tab or window and log in to your remote host running Vault.

      7. Once again, set the VAULT_ADDR environment variable to the correct value to communicate with your local Vault instance.

        export VAULT_ADDR=https://example.com:8200
        
      8. Set the VAULT_TOKEN environment variable to this newly created token. From the previous example output, this would be the following (note that your token will be different).

        export VAULT_TOKEN=s.3uu4vwFO8D1mG5S76IG04mck
        
      9. Read the KV path that this token should be able to access.

        vault kv get kv/myservice
        

        The example should should be read and accessible.

      10. If you read this value using this Vault token after more than 10 minutes have elapsed, the token will have expired and any read operations using the token should be denied. Performing another vault write auth/approle/login operation (detailed in step 5) can generate new tokens to use.

      More Information

      You may wish to consult the following resources for additional information on this topic. While these are provided in the hope that they will be useful, please note that we cannot vouch for the accuracy or timeliness of externally hosted materials.

      Find answers, ask questions, and help others.

      This guide is published under a CC BY-ND 4.0 license.



      Source link