One place for hosting & domains

      Private

      How To Set Up a Private Docker Registry on Ubuntu 18.04


      The author selected the Apache Software Foundation to receive a donation as part of the Write for DOnations program.

      Introduction

      Docker Registry is an application that manages storing and delivering Docker container images. Registries centralize container images and reduce build times for developers. Docker images guarantee the same runtime environment through virtualization, but building an image can involve a significant time investment. For example, rather than installing dependencies and packages separately to use Docker, developers can download a compressed image from a registry that contains all of the necessary components. Furthermore, developers can automate pushing images to a registry using continuous integration tools, such as TravisCI, to seamlessly update images during production and development.

      Docker also has a free public registry, Docker Hub, that can host your custom Docker images, but there are situations where you will not want your image to be publicly available. Images typically contain all the code necessary to run an application, so using a private registry is preferable when using proprietary software.

      In this tutorial, you will set up and secure your own private Docker Registry. You will use Docker Compose to define configurations to run your Docker applications and Nginx to forward server traffic from HTTPS to the running Docker container. Once you’ve completed this tutorial, you will be able to push a custom Docker image to your private registry and pull the image securely from a remote server.

      Prerequisites

      Before you begin this guide, you’ll need the following:

      • Two Ubuntu 18.04 servers set up by following the Ubuntu 18.04 initial server setup guide, including a sudo non-root user and a firewall. One server will host your private Docker Registry and the other will be your client server.
      • Docker and Docker-Compose installed on both servers by following the How to Install Docker-Compose on Ubuntu 18.04 tutorial. You only need to complete the first step of this tutorial to install Docker Compose. This tutorial explains how to install Docker as part of its prerequisites.
      • Nginx installed on your private Docker Registry server by following the How to Install Nginx on Ubuntu 18.04.
      • Nginx secured with Let’s Encrypt on your server for the private Docker Registry, by following How to Secure Nginx With Let’s Encrypt. Make sure to redirect all traffic from HTTP to HTTPS in Step 4.
      • A domain name that resolves to the server you’re using for the private Docker Registry. You will set this up as part of the Let’s Encrypt prerequisite.

      Step 1 — Installing and Configuring the Docker Registry

      The Docker command line tool is useful for starting and managing one or two Docker containers, but, for full deployment most applications running inside Docker containers require other components to be running in parallel. For example, a lot of web applications consist of a web server, like Nginx, that serves up the application’s code, an interpreted scripting language such as PHP, and a database server like MySQL.

      With Docker Compose, you can write one .yml file to set up each container’s configuration and the information the containers need to communicate with each other. You can then use the docker-compose command line tool to issue commands to all the components that make up your application.

      Docker Registry is itself an application with multiple components, so you will use Docker Compose to manage your configuration. To start an instance of the registry, you’ll set up a docker-compose.yml file to define the location where your registry will be storing its data.

      On the server you have created to host your private Docker Registry, you can create a docker-registry directory, move into it, and then create a data subfolder with the following commands:

      • mkdir ~/docker-registry && cd $_
      • mkdir data

      Use your text editor to create the docker-compose.yml configuration file:

      Add the following content to the file, which describes the basic configuration for a Docker Registry:

      docker-compose.yml

      version: '3'
      
      services:
        registry:
          image: registry:2
          ports:
          - "5000:5000"
          environment:
            REGISTRY_STORAGE_FILESYSTEM_ROOTDIRECTORY: /data
          volumes:
            - ./data:/data
      

      The environment section sets an environment variable in the Docker Registry container with the path /data. The Docker Registry application checks this environment variable when it starts up, and as a result begins to save its data to the /data folder.

      However, as you have included the volumes: - ./data:/data line, Docker will start to map the /data directory in that container to /data on your registry server. The end result is that the Docker Registry's data all gets stored in ~/docker-registry/data on the registry server.

      The ports section, with configuration 5000:5000, tells Docker to map port 5000 on the server to port 5000 in the running container. This allows you to send a request to port 5000 on the server, and have the request forwarded to the registry application.

      You can now start Docker Compose to check the setup:

      You will see download bars in your output that show Docker downloading the Docker Registry image from Docker's own registry. Within a minute or two, you'll see output that looks similar to the following (versions might vary):

      Output of docker-compose up

      Starting docker-registry_registry_1 ... done Attaching to docker-registry_registry_1 registry_1 | time="2018-11-06T18:43:09Z" level=warning msg="No HTTP secret provided - generated random secret. This may cause problems with uploads if multiple registries are behind a load-balancer. To provide a shared secret, fill in http.secret in the configuration file or set the REGISTRY_HTTP_SECRET environment variable." go.version=go1.7.6 instance.id=c63483ee-7ad5-4205-9e28-3e809c843d42 version=v2.6.2 registry_1 | time="2018-11-06T18:43:09Z" level=info msg="redis not configured" go.version=go1.7.6 instance.id=c63483ee-7ad5-4205-9e28-3e809c843d42 version=v2.6.2 registry_1 | time="2018-11-06T18:43:09Z" level=info msg="Starting upload purge in 20m0s" go.version=go1.7.6 instance.id=c63483ee-7ad5-4205-9e28-3e809c843d42 version=v2.6.2 registry_1 | time="2018-11-06T18:43:09Z" level=info msg="using inmemory blob descriptor cache" go.version=go1.7.6 instance.id=c63483ee-7ad5-4205-9e28-3e809c843d42 version=v2.6.2 registry_1 | time="2018-11-06T18:43:09Z" level=info msg="listening on [::]:5000" go.version=go1.7.6 instance.id=c63483ee-7ad5-4205-9e28-3e809c843d42 version=v2.6.2

      You'll address the No HTTP secret provided warning message later in this tutorial. The output shows that the container is starting. The last line of the output shows it has successfully started listening on port 5000.

      By default, Docker Compose will remain waiting for your input, so hit CTRL+C to shut down your Docker Registry container.

      You have set up a full Docker Registry listening on port 5000. At this point the registry won't start unless you bring it up manually. Also, Docker Registry doesn't come with any built-in authentication mechanism, so it is currently insecure and completely open to the public. In the following steps, you will address these security concerns.

      Step 2 — Setting Up Nginx Port Forwarding

      You already have HTTPS set up on your Docker Registry server with Nginx, which means you can now set up port forwarding from Nginx to port 5000. Once you complete this step, you can access your registry directly at example.com.

      As part of the How to Secure Nginx With Let's Encrypt prerequisite, you have already set up the /etc/nginx/sites-available/example.com file containing your server configuration.

      Open this file with your text editor:

      • sudo nano /etc/nginx/sites-available/example.com

      Find the existing location line. It will look like this:

      /etc/nginx/sites-available/example.com

      ...
      location / {
        ...
      }
      ...
      

      You need to forward traffic to port 5000, where your registry will be running. You also want to append headers to the request to the registry, which provide additional information from the server with each request and response. Delete the contents of the location section, and add the following content into that section:

      /etc/nginx/sites-available/example.com

      ...
      location / {
          # Do not allow connections from docker 1.5 and earlier
          # docker pre-1.6.0 did not properly set the user agent on ping, catch "Go *" user agents
          if ($http_user_agent ~ "^(docker/1.(3|4|5(?!.[0-9]-dev))|Go ).*$" ) {
            return 404;
          }
      
          proxy_pass                          http://localhost:5000;
          proxy_set_header  Host              $http_host;   # required for docker client's sake
          proxy_set_header  X-Real-IP         $remote_addr; # pass on real client's IP
          proxy_set_header  X-Forwarded-For   $proxy_add_x_forwarded_for;
          proxy_set_header  X-Forwarded-Proto $scheme;
          proxy_read_timeout                  900;
      }
      ...
      

      The $http_user_agent section verifies that the Docker version of the client is above 1.5, and ensures that the UserAgent is not a Go application. Since you are using version 2.0 of the registry, older clients are not supported. For more information, you can find the nginx header configuration in Docker's Registry Nginx guide.

      Save and exit the file. Apply the changes by restarting Nginx:

      • sudo service nginx restart

      You can confirm that Nginx is forwarding traffic to port 5000 by running the registry:

      • cd ~/docker-registry
      • docker-compose up

      In a browser window, open up the following url:

      https://example.com/v2
      

      You will see an empty JSON object, or:

      {}
      

      In your terminal, you'll see output similar to the following:

      Output of docker-compose up

      registry_1 | time="2018-11-07T17:57:42Z" level=info msg="response completed" go.version=go1.7.6 http.request.host=cornellappdev.com http.request.id=a8f5984e-15e3-4946-9c40-d71f8557652f http.request.method=GET http.request.remoteaddr=128.84.125.58 http.request.uri="/v2/" http.request.useragent="Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_2) AppleWebKit/604.4.7 (KHTML, like Gecko) Version/11.0.2 Safari/604.4.7" http.response.contenttype="application/json; charset=utf-8" http.response.duration=2.125995ms http.response.status=200 http.response.written=2 instance.id=3093e5ab-5715-42bc-808e-73f310848860 version=v2.6.2 registry_1 | 172.18.0.1 - - [07/Nov/2018:17:57:42 +0000] "GET /v2/ HTTP/1.0" 200 2 "" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_2) AppleWebKit/604.4.7 (KHTML, like Gecko) Version/11.0.2 Safari/604.4.7"

      You can see from the last line that a GET request was made to /v2/, which is the endpoint you sent a request to from your browser. The container received the request you made, from the port forwarding, and returned a response of {}. The code 200 in the last line of the output means that the container handled the request successfully.

      Now that you have set up port forwarding, you can move on to improving the security of your registry.

      Step 3 — Setting Up Authentication

      With Nginx proxying requests properly, you can now secure your registry with HTTP authentication to manage who has access to your Docker Registry. To achieve this, you'll create an authentication file with htpasswd and add users to it. HTTP authentication is quick to set up and secure over a HTTPS connection, which is what the registry will use.

      You can install the htpasswd package by running the following:

      • sudo apt install apache2-utils

      Now you'll create the directory where you'll store our authentication credentials, and change into that directory. $_ expands to the last argument of the previous command, in this case ~/docker-registry/auth:

      • mkdir ~/docker-registry/auth && cd $_

      Next, you will create the first user as follows, replacing username with the username you want to use. The -B flag specifies bcrypt encryption, which is more secure than the default encryption. Enter the password when prompted:

      • htpasswd -Bc registry.password username

      Note: To add more users, re-run the previous command without the -c option, (the c is for create):

      • htpasswd registry.password username

      Next, you'll edit the docker-compose.yml file to tell Docker to use the file you created to authenticate users.

      • cd ~/docker-registry
      • nano docker-compose.yml

      You can add environment variables and a volume for the auth/ directory that you created, by editing the docker-compose.yml file to tell Docker how you want to authenticate users. Add the following highlighted content to the file:

      docker-compose.yml

      version: '3'
      
      services:
        registry:
          image: registry:2
          ports:
          - "5000:5000"
          environment:
            REGISTRY_AUTH: htpasswd
            REGISTRY_AUTH_HTPASSWD_REALM: Registry
            REGISTRY_AUTH_HTPASSWD_PATH: /auth/registry.password
            REGISTRY_STORAGE_FILESYSTEM_ROOTDIRECTORY: /data
          volumes:
            - ./auth:/auth
            - ./data:/data
      

      For REGISTRY_AUTH, you have specified htpasswd, which is the authentication scheme you are using, and set REGISTRY_AUTH_HTPASSWD_PATH to the path of the authentication file. Finally, REGISTRY_AUTH_HTPASSWD_REALM signifies the name of htpasswd realm.

      You can now verify that your authentication works correctly, by running the registry and checking that it prompts users for a username and password.

      In a browser window, open https://example.com/v2.

      After entering username and the corresponding password, you will see {} once again. You've confirmed the basic authentication setup: the registry only returned the result after you entered the correct username and password. You have now secured your registry, and can continue to using the registry.

      Step 4 — Starting Docker Registry as a Service

      You want to ensure that your registry will start whenever the system boots up. If there are any unforeseen system crashes, you want to make sure the registry restarts when the server reboots. Open up docker-compose.yml:

      Add the following line of content under registry::

      docker-compose.yml

      ...
        registry:
          restart: always
      ...
      

      You can start your registry as a background process, which will allow you to exit the ssh session and persist the process:

      With your registry running in the background, you can now prepare Nginx for file uploads.

      Step 5 — Increasing File Upload Size for Nginx

      Before you can push an image to the registry, you need to ensure that your registry will be able to handle large file uploads. Although Docker splits large image uploads into separate layers, they can sometimes be over 1GB. By default, Nginx has a limit of 1MB on file uploads, so you need to edit the configuration file for nginx and set the max file upload size to 2GB.

      • sudo nano /etc/nginx/nginx.conf

      Find the http section, and add the following line:

      /etc/nginx/nginx.conf

      ...
      http {
              client_max_body_size 2000M;
              ...
      }
      ...
      

      Finally, restart Nginx to apply the configuration changes:

      • sudo service nginx restart

      You can now upload large images to your Docker Registry without Nginx errors.

      Step 6 — Publishing to Your Private Docker Registry

      You are now ready to publish an image to your private Docker Registry, but first you have to create an image. For this tutorial, you will create a simple image based on the ubuntu image from Docker Hub. Docker Hub is a publicly hosted registry, with many pre-configured images that can be leveraged to quickly Dockerize applications. Using the ubuntu image, you will test pushing and pulling to your registry.

      From your client server, create a small, empty image to push to your new registry, the -i and -t flags give you interactive shell access into the container:

      • docker run -t -i ubuntu /bin/bash

      After it finishes downloading you'll be inside a Docker prompt, note that your container ID following root@ will vary. Make a quick change to the filesystem by creating a file called SUCCESS. In the next step, you'll be able to use this file to determine whether the publishing process is successful:

      Exit out of the Docker container:

      The following command creates a new image called test-image based on the image already running plus any changes you have made. In our case, the addition of the /SUCCESS file is included in the new image.

      Commit the change:

      • docker commit $(docker ps -lq) test-image

      At this point, the image only exists locally. Now you can push it to the new registry you have created. Log in to your Docker Registry:

      • docker login https://example.com

      Enter the username and corresponding password from earlier. Next, you will tag the image with the private registry's location in order to push to it:

      • docker tag test-image example.com/test-image

      Push the newly tagged image to the registry:

      • docker push example.com/test-image

      Your output will look similar to the following:

      Output

      The push refers to a repository [example.com/test-image] e3fbbfb44187: Pushed 5f70bf18a086: Pushed a3b5c80a4eba: Pushed 7f18b442972b: Pushed 3ce512daaf78: Pushed 7aae4540b42d: Pushed ...

      You've verified that your registry handles user authentication, and allows authenticated users to push images to the registry. Next, you will confirm that you are able to pull images from the registry as well.

      Step 7 — Pulling From Your Private Docker Registry

      Return to your registry server so that you can test pulling the image from your client server. It is also possible to test this from a third server.

      Log in with the username and password you set up previously:

      • docker login https://example.com

      You're now ready to pull the image. Use your domain name and image name, which you tagged in the previous step:

      • docker login example.com/test-image

      Docker will download the image and return you to the prompt. If you run the image on the registry server you'll see the SUCCESS file you created earlier is there:

      • docker run -it example.com/test-image /bin/bash

      List your files inside the bash shell:

      You will see the SUCCESS file you created for this image:

      SUCCESS  bin  boot  dev  etc  home  lib  lib64  media   mnt  opt  proc  root  run  sbin  srv  sys  tmp  usr  var
      

      You've finished setting up a secure registry to which users can push and pull custom images.

      Conclusion

      In this tutorial you set up your own private Docker Registry, and published a Docker image. As mentioned in the introduction, you can also use TravisCI or a similar CI tool to automate pushing to a private registry directly. By leveraging Docker and registries into your workflow, you can ensure that the image containing the code will result in the same behavior on any machine, whether in production or in development. For more information on writing Docker files, you can read this Docker tutorial explaining the process.



      Source link

      How To Configure BIND as a Private Network DNS Server on Debian 9


      Introduction

      An important part of managing server configuration and infrastructure includes maintaining an easy way to look up network interfaces and IP addresses by name, by setting up a proper Domain Name System (DNS). Using fully qualified domain names (FQDNs), instead of IP addresses, to specify network addresses eases the configuration of services and applications, and increases the maintainability of configuration files. Setting up your own DNS for your private network is a great way to improve the management of your servers.

      In this tutorial, we will go over how to set up an internal DNS server, using the BIND name server software (BIND9) on Debian 9, that can be used by your servers to resolve private hostnames and private IP addresses. This provides a central way to manage your internal hostnames and private IP addresses, which is indispensable when your environment expands to more than a few hosts.

      Prerequisites

      To complete this tutorial, you will need the following infrastructure. Create each server in the same datacenter with private networking enabled:

      • A fresh Debian 9 server to serve as the Primary DNS server, ns1
      • (Recommended) A second Debian 9 server to serve as a Secondary DNS server, ns2
      • Additional servers in the same datacenter that will be using your DNS servers

      On each of these servers, configure administrative access via a sudo user and a firewall by following our Debian 9 initial server setup guide.

      If you are unfamiliar with DNS concepts, it is recommended that you read at least the first three parts of our Introduction to Managing DNS.

      Example Infrastructure and Goals

      For the purposes of this article, we will assume the following:

      • We have two servers which will be designated as our DNS name servers. We will refer to these as ns1 and ns2 in this guide.
      • We have two additional client servers that will be using the DNS infrastructure we create. We will call these host1 and host2 in this guide. You can add as many as you’d like for your infrastructure.
      • All of these servers exist in the same datacenter. We will assume that this is the nyc3 datacenter.
      • All of these servers have private networking enabled (and are on the 10.128.0.0/16 subnet. You will likely have to adjust this for your servers).
      • All servers are connected to a project that runs on “example.com”. Since our DNS system will be entirely internal and private, you do not have to purchase a domain name. However, using a domain you own may help avoid conflicts with publicly routable domains.

      With these assumptions, we decide that it makes sense to use a naming scheme that uses “nyc3.example.com” to refer to our private subnet or zone. Therefore, host1‘s private Fully-Qualified Domain Name (FQDN) will be host1.nyc3.example.com. Refer to the following table the relevant details:

      Host Role Private FQDN Private IP Address
      ns1 Primary DNS Server ns1.nyc3.example.com 10.128.10.11
      ns2 Secondary DNS Server ns2.nyc3.example.com 10.128.20.12
      host1 Generic Host 1 host1.nyc3.example.com 10.128.100.101
      host2 Generic Host 2 host2.nyc3.example.com 10.128.200.102

      Note


      Your existing setup will be different, but the example names and IP addresses will be used to demonstrate how to configure a DNS server to provide a functioning internal DNS. You should be able to easily adapt this setup to your own environment by replacing the host names and private IP addresses with your own. It is not necessary to use the region name of the datacenter in your naming scheme, but we use it here to denote that these hosts belong to a particular datacenter’s private network. If you utilize multiple datacenters, you can set up an internal DNS within each respective datacenter.

      By the end of this tutorial, we will have a primary DNS server, ns1, and optionally a secondary DNS server, ns2, which will serve as a backup.

      Let’s get started by installing our Primary DNS server, ns1.

      Installing BIND on DNS Servers

      Note


      Text that is highlighted in red is important! It will often be used to denote something that needs to be replaced with your own settings or that it should be modified or added to a configuration file. For example, if you see something like host1.nyc3.example.com, replace it with the FQDN of your own server. Likewise, if you see host1_private_IP, replace it with the private IP address of your own server.

      On both DNS servers, ns1 and ns2, update the apt package cache by typing:

      Now install BIND:

      • sudo apt install bind9 bind9utils bind9-doc

      Setting Bind to IPv4 Mode

      Before continuing, let's set BIND to IPv4 mode since our private networking uses IPv4 exclusively. On both servers, edit the bind9 default settings file by typing:

      • sudo nano /etc/default/bind9

      Add "-4" to the end of the OPTIONS parameter. It should look like the following:

      /etc/default/bind9

      . . .
      OPTIONS="-u bind -4"
      

      Save and close the file when you are finished.

      Restart BIND to implement the changes:

      • sudo systemctl restart bind9

      Now that BIND is installed, let's configure the primary DNS server.

      Configuring the Primary DNS Server

      BIND's configuration consists of multiple files, which are included from the main configuration file, named.conf. These filenames begin with named because that is the name of the process that BIND runs (short for "domain name daemon"). We will start with configuring the options file.

      Configuring the Options File

      On ns1, open the named.conf.options file for editing:

      • sudo nano /etc/bind/named.conf.options

      Above the existing options block, create a new ACL (access control list) block called "trusted". This is where we will define a list of clients that we will allow recursive DNS queries from (i.e. your servers that are in the same datacenter as ns1). Using our example private IP addresses, we will add ns1, ns2, host1, and host2 to our list of trusted clients:

      /etc/bind/named.conf.options — 1 of 3

      acl "trusted" {
              10.128.10.11;    # ns1 - can be set to localhost
              10.128.20.12;    # ns2
              10.128.100.101;  # host1
              10.128.200.102;  # host2
      };
      
      options {
      
              . . .
      

      Now that we have our list of trusted DNS clients, we will want to edit the options block. Currently, the start of the block looks like the following:

      /etc/bind/named.conf.options — 2 of 3

              . . .
      };
      
      options {
              directory "/var/cache/bind";
              . . .
      }
      

      Below the directory directive, add the highlighted configuration lines (and substitute in the proper ns1 IP address) so it looks something like this:

      /etc/bind/named.conf.options — 3 of 3

              . . .
      
      };
      
      options {
              directory "/var/cache/bind";
      
              recursion yes;                 # enables resursive queries
              allow-recursion { trusted; };  # allows recursive queries from "trusted" clients
              listen-on { 10.128.10.11; };   # ns1 private IP address - listen on private network only
              allow-transfer { none; };      # disable zone transfers by default
      
              forwarders {
                      8.8.8.8;
                      8.8.4.4;
              };
      
              . . .
      };
      

      When you are finished, save and close the named.conf.options file. The above configuration specifies that only your own servers (the "trusted" ones) will be able to query your DNS server for outside domains.

      Next, we will configure the local file, to specify our DNS zones.

      Configuring the Local File

      On ns1, open the named.conf.local file for editing:

      • sudo nano /etc/bind/named.conf.local

      Aside from a few comments, the file should be empty. Here, we will specify our forward and reverse zones. DNS zones designate a specific scope for managing and defining DNS records. Since our domains will all be within the "nyc3.example.com" subdomain, we will use that as our forward zone. Because our servers' private IP addresses are each in the 10.128.0.0/16 IP space, we will set up a reverse zone so that we can define reverse lookups within that range.

      Add the forward zone with the following lines, substituting the zone name with your own and the secondary DNS server's private IP address in the allow-transfer directive:

      /etc/bind/named.conf.local — 1 of 2

      zone "nyc3.example.com" {
          type master;
          file "/etc/bind/zones/db.nyc3.example.com"; # zone file path
          allow-transfer { 10.128.20.12; };           # ns2 private IP address - secondary
      };
      

      Assuming that our private subnet is 10.128.0.0/16, add the reverse zone by with the following lines (note that our reverse zone name starts with "128.10" which is the octet reversal of "10.128"):

      /etc/bind/named.conf.local — 2 of 2

          . . .
      };
      
      zone "128.10.in-addr.arpa" {
          type master;
          file "/etc/bind/zones/db.10.128";  # 10.128.0.0/16 subnet
          allow-transfer { 10.128.20.12; };  # ns2 private IP address - secondary
      };
      

      If your servers span multiple private subnets but are in the same datacenter, be sure to specify an additional zone and zone file for each distinct subnet. When you are finished adding all of your desired zones, save and exit the named.conf.local file.

      Now that our zones are specified in BIND, we need to create the corresponding forward and reverse zone files.

      Creating the Forward Zone File

      The forward zone file is where we define DNS records for forward DNS lookups. That is, when the DNS receives a name query, "host1.nyc3.example.com" for example, it will look in the forward zone file to resolve host1's corresponding private IP address.

      Let's create the directory where our zone files will reside. According to our named.conf.local configuration, that location should be /etc/bind/zones:

      • sudo mkdir /etc/bind/zones

      We will base our forward zone file on the sample db.local zone file. Copy it to the proper location with the following commands:

      • sudo cp /etc/bind/db.local /etc/bind/zones/db.nyc3.example.com

      Now let's edit our forward zone file:

      • sudo nano /etc/bind/zones/db.nyc3.example.com

      Initially, it will look something like the following:

      /etc/bind/zones/db.nyc3.example.com — original

      $TTL    604800
      @       IN      SOA     localhost. root.localhost. (
                                    2         ; Serial
                               604800         ; Refresh
                                86400         ; Retry
                              2419200         ; Expire
                               604800 )       ; Negative Cache TTL
      ;
      @       IN      NS      localhost.      ; delete this line
      @       IN      A       127.0.0.1       ; delete this line
      @       IN      AAAA    ::1             ; delete this line
      

      First, you will want to edit the SOA record. Replace the first "localhost" with ns1's FQDN, then replace "root.localhost" with "admin.nyc3.example.com". Every time you edit a zone file, you need to increment the serial value before you restart the named process. We will increment it to "3". It should now look something like this:

      /etc/bind/zones/db.nyc3.example.com — updated 1 of 3

      @       IN      SOA     ns1.nyc3.example.com. admin.nyc3.example.com. (
                                    3         ; Serial
      
                                    . . .
      

      Next, delete the three records at the end of the file (after the SOA record). If you're not sure which lines to delete, they are marked with a "delete this line" comment above.

      At the end of the file, add your name server records with the following lines (replace the names with your own). Note that the second column specifies that these are "NS" records:

      /etc/bind/zones/db.nyc3.example.com — updated 2 of 3

      . . .
      
      ; name servers - NS records
          IN      NS      ns1.nyc3.example.com.
          IN      NS      ns2.nyc3.example.com.
      

      Now, add the A records for your hosts that belong in this zone. This includes any server whose name we want to end with ".nyc3.example.com" (substitute the names and private IP addresses). Using our example names and private IP addresses, we will add A records for ns1, ns2, host1, and host2 like so:

      /etc/bind/zones/db.nyc3.example.com — updated 3 of 3

      . . .
      
      ; name servers - A records
      ns1.nyc3.example.com.          IN      A       10.128.10.11
      ns2.nyc3.example.com.          IN      A       10.128.20.12
      
      ; 10.128.0.0/16 - A records
      host1.nyc3.example.com.        IN      A      10.128.100.101
      host2.nyc3.example.com.        IN      A      10.128.200.102
      

      Save and close the db.nyc3.example.com file.

      Our final example forward zone file looks like the following:

      /etc/bind/zones/db.nyc3.example.com — updated

      $TTL    604800
      @       IN      SOA     ns1.nyc3.example.com. admin.nyc3.example.com. (
                        3     ; Serial
                   604800     ; Refresh
                    86400     ; Retry
                  2419200     ; Expire
                   604800 )   ; Negative Cache TTL
      ;
      ; name servers - NS records
           IN      NS      ns1.nyc3.example.com.
           IN      NS      ns2.nyc3.example.com.
      
      ; name servers - A records
      ns1.nyc3.example.com.          IN      A       10.128.10.11
      ns2.nyc3.example.com.          IN      A       10.128.20.12
      
      ; 10.128.0.0/16 - A records
      host1.nyc3.example.com.        IN      A      10.128.100.101
      host2.nyc3.example.com.        IN      A      10.128.200.102
      

      Now let's move onto the reverse zone file(s).

      Creating the Reverse Zone File(s)

      Reverse zone files are where we define DNS PTR records for reverse DNS lookups. That is, when the DNS receives a query by IP address, "10.128.100.101" for example, it will look in the reverse zone file(s) to resolve the corresponding FQDN, "host1.nyc3.example.com" in this case.

      On ns1, for each reverse zone specified in the named.conf.local file, create a reverse zone file. We will base our reverse zone file(s) on the sample db.127 zone file. Copy it to the proper location with the following commands (substituting the destination filename so it matches your reverse zone definition):

      • sudo cp /etc/bind/db.127 /etc/bind/zones/db.10.128

      Edit the reverse zone file that corresponds to the reverse zone(s) defined in named.conf.local:

      • sudo nano /etc/bind/zones/db.10.128

      Initially, it will look something like the following:

      /etc/bind/zones/db.10.128 — original

      $TTL    604800
      @       IN      SOA     localhost. root.localhost. (
                                    1         ; Serial
                               604800         ; Refresh
                                86400         ; Retry
                              2419200         ; Expire
                               604800 )       ; Negative Cache TTL
      ;
      @       IN      NS      localhost.      ; delete this line
      1.0.0   IN      PTR     localhost.      ; delete this line
      

      In the same manner as the forward zone file, you will want to edit the SOA record and increment the serial value. It should look something like this:

      /etc/bind/zones/db.10.128 — updated 1 of 3

      @       IN      SOA     ns1.nyc3.example.com. admin.nyc3.example.com. (
                                    3         ; Serial
      
                                    . . .
      

      Now delete the two records at the end of the file (after the SOA record). If you're not sure which lines to delete, they are marked with a "delete this line" comment above.

      At the end of the file, add your name server records with the following lines (replace the names with your own). Note that the second column specifies that these are "NS" records:

      /etc/bind/zones/db.10.128 — updated 2 of 3

      . . .
      
      ; name servers - NS records
            IN      NS      ns1.nyc3.example.com.
            IN      NS      ns2.nyc3.example.com.
      

      Then add PTR records for all of your servers whose IP addresses are on the subnet of the zone file that you are editing. In our example, this includes all of our hosts because they are all on the 10.128.0.0/16 subnet. Note that the first column consists of the last two octets of your servers' private IP addresses in reversed order. Be sure to substitute names and private IP addresses to match your servers:

      /etc/bind/zones/db.10.128 — updated 3 of 3

      . . .
      
      ; PTR Records
      11.10   IN      PTR     ns1.nyc3.example.com.    ; 10.128.10.11
      12.20   IN      PTR     ns2.nyc3.example.com.    ; 10.128.20.12
      101.100 IN      PTR     host1.nyc3.example.com.  ; 10.128.100.101
      102.200 IN      PTR     host2.nyc3.example.com.  ; 10.128.200.102
      

      Save and close the reverse zone file (repeat this section if you need to add more reverse zone files).

      Our final example reverse zone file looks like the following:

      /etc/bind/zones/db.10.128 — updated

      $TTL    604800
      @       IN      SOA     nyc3.example.com. admin.nyc3.example.com. (
                                    3         ; Serial
                               604800         ; Refresh
                                86400         ; Retry
                              2419200         ; Expire
                               604800 )       ; Negative Cache TTL
      ; name servers
            IN      NS      ns1.nyc3.example.com.
            IN      NS      ns2.nyc3.example.com.
      
      ; PTR Records
      11.10   IN      PTR     ns1.nyc3.example.com.    ; 10.128.10.11
      12.20   IN      PTR     ns2.nyc3.example.com.    ; 10.128.20.12
      101.100 IN      PTR     host1.nyc3.example.com.  ; 10.128.100.101
      102.200 IN      PTR     host2.nyc3.example.com.  ; 10.128.200.102
      

      We're done editing our files, so next we can check our files for errors.

      Checking the BIND Configuration Syntax

      Run the following command to check the syntax of the named.conf* files:

      If your named configuration files have no syntax errors, you will return to your shell prompt and see no error messages. If there are problems with your configuration files, review the error message and the "Configure Primary DNS Server" section, then try named-checkconf again.

      The named-checkzone command can be used to check the correctness of your zone files. Its first argument specifies a zone name, and the second argument specifies the corresponding zone file, which are both defined in named.conf.local.

      For example, to check the "nyc3.example.com" forward zone configuration, run the following command (change the names to match your forward zone and file):

      • sudo named-checkzone nyc3.example.com /etc/bind/zones/db.nyc3.example.com

      And to check the "128.10.in-addr.arpa" reverse zone configuration, run the following command (change the numbers to match your reverse zone and file):

      • sudo named-checkzone 128.10.in-addr.arpa /etc/bind/zones/db.10.128

      When all of your configuration and zone files have no errors in them, you should be ready to restart the BIND service.

      Restarting BIND

      Restart BIND:

      • sudo systemctl restart bind9

      If you have the UFW firewall configured, open up access to BIND by typing:

      Your primary DNS server is now setup and ready to respond to DNS queries. Let's move on to creating the secondary DNS server.

      Configuring the Secondary DNS Server

      In most environments, it is a good idea to set up a secondary DNS server that will respond to requests if the primary becomes unavailable. Luckily, the secondary DNS server is much easier to configure.

      On ns2, edit the named.conf.options file:

      • sudo nano /etc/bind/named.conf.options

      At the top of the file, add the ACL with the private IP addresses of all of your trusted servers:

      /etc/bind/named.conf.options — updated 1 of 2 (secondary)

      acl "trusted" {
              10.128.10.11;   # ns1
              10.128.20.12;   # ns2 - can be set to localhost
              10.128.100.101;  # host1
              10.128.200.102;  # host2
      };
      
      options {
      
              . . .
      

      Below the directory directive, add the following lines:

      /etc/bind/named.conf.options — updated 2 of 2 (secondary)

              recursion yes;
              allow-recursion { trusted; };
              listen-on { 10.128.20.12; };      # ns2 private IP address
              allow-transfer { none; };          # disable zone transfers by default
      
              forwarders {
                      8.8.8.8;
                      8.8.4.4;
              };
      

      Save and close the named.conf.options file. This file should look exactly like ns1's named.conf.options file except it should be configured to listen on ns2's private IP address.

      Now edit the named.conf.local file:

      • sudo nano /etc/bind/named.conf.local

      Define slave zones that correspond to the master zones on the primary DNS server. Note that the type is "slave", the file does not contain a path, and there is a masters directive which should be set to the primary DNS server's private IP address. If you defined multiple reverse zones in the primary DNS server, make sure to add them all here:

      /etc/bind/named.conf.local — updated (secondary)

      zone "nyc3.example.com" {
          type slave;
          file "db.nyc3.example.com";
          masters { 10.128.10.11; };  # ns1 private IP
      };
      
      zone "128.10.in-addr.arpa" {
          type slave;
          file "db.10.128";
          masters { 10.128.10.11; };  # ns1 private IP
      };
      

      Now save and close the named.conf.local file.

      Run the following command to check the validity of your configuration files:

      Once that checks out, restart BIND:

      • sudo systemctl restart bind9

      Allow DNS connections to the server by altering the UFW firewall rules:

      Now you have primary and secondary DNS servers for private network name and IP address resolution. Now you must configure your client servers to use your private DNS servers.

      Configuring DNS Clients

      Before all of your servers in the "trusted" ACL can query your DNS servers, you must configure each of them to use ns1 and ns2 as name servers. This process varies depending on OS, but for most Linux distributions it involves adding your name servers to the /etc/resolv.conf file.

      Ubuntu 18.04 Clients

      On Ubuntu 18.04, networking is configured with Netplan, an abstraction that allows you to write standardized network configuration and apply it to incompatible backend networking software. To configure DNS, we need to write a Netplan configuration file.

      First, find the device associated with your private network by querying the private subnet with the ip address command:

      • ip address show to 10.128.0.0/16

      Output

      3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000 inet 10.128.100.101/16 brd 10.128.255.255 scope global eth1 valid_lft forever preferred_lft forever

      In this example, the private interface is eth1.

      Next, create a new file in /etc/netplan called 00-private-nameservers.yaml:

      • sudo nano /etc/netplan/00-private-nameservers.yaml

      Inside, paste the following contents. You will need to modify the interface of the private network, the addresses of your ns1 and ns2 DNS servers, and the DNS zone:

      Note: Netplan uses the YAML data serialization format for its configuration files. Because YAML uses indentation and whitespace to define its data structure, make sure that your definition uses consistent indentation to avoid errors.

      /etc/netplan 00-private-nameservers.yaml

      network:
          version: 2
          ethernets:
              eth1:                                 # Private network interface
                  nameservers:
                      addresses:
                      - 10.128.10.11                # Private IP for ns1
                      - 10.132.20.12                # Private IP for ns2
                      search: [ nyc3.example.com ]  # DNS zone
      
      

      Save and close the file when you are finished.

      Next, tell Netplan to attempt to use the new configuration file by using netplan try. If there are problems that cause a loss of networking, Netplan will automatically roll back the changes after a timeout:

      Output

      Warning: Stopping systemd-networkd.service, but it can still be activated by: systemd-networkd.socket Do you want to keep these settings? Press ENTER before the timeout to accept the new configuration Changes will revert in 120 seconds

      If the countdown is updating correctly at the bottom, the new configuration is at least functional enough to not break your SSH connection. Press ENTER to accept the new configuration.

      Now, check that the system's DNS resolver to determine if your DNS configuration has been applied:

      • sudo systemd-resolve --status

      Scroll down until you see the section for your private network interface. You should see the private IP addresses for your DNS servers listed first, followed by some fallback values. Your domain should should be in the "DNS Domain":

      Output

      . . . Link 3 (eth1) Current Scopes: DNS LLMNR setting: yes MulticastDNS setting: no DNSSEC setting: no DNSSEC supported: no DNS Servers: 10.128.10.11 10.128.20.12 67.207.67.2 67.207.67.3 DNS Domain: nyc3.example.com . . .

      Your client should now be configured to use your internal DNS servers.

      Ubuntu 16.04 and Debian Clients

      On Ubuntu 16.04 and Debian Linux servers, you can edit the /etc/network/interfaces file:

      • sudo nano /etc/network/interfaces

      Inside, find the dns-nameservers line. If it is attached to the lo interface, move it to your networking interface (eth0 or eth1 for example). Next, prepend your own name servers in front of the list that is currently there. Below that line, add a dns-search option pointed to the base domain of your infrastructure. In our case, this would be "nyc3.example.com":

      /etc/network/interfaces

          . . .
      
          dns-nameservers 10.128.10.11 10.128.20.12 8.8.8.8
          dns-search nyc3.example.com
      
          . . .
      

      Save and close the file when you are finished.

      Make sure that the resolvconf package is installed on your system:

      • sudo apt update
      • sudo apt install resolvconf

      Now, restart your networking services, applying the new changes with the following commands. Make sure you replace eth0 with the name of your networking interface:

      • sudo ifdown --force eth0 && sudo ip addr flush dev eth0 && sudo ifup --force eth0

      This should restart your network without dropping your current connection. If it worked correctly, you should see something like this:

      Output

      RTNETLINK answers: No such process Waiting for DAD... Done

      Double check that your settings were applied by typing:

      You should see your name servers in the /etc/resolv.conf file, as well as your search domain:

      Output

      # Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8) # DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN nameserver 10.128.10.11 nameserver 10.128.20.12 nameserver 8.8.8.8 search nyc3.example.com

      Your client is now configured to use your DNS servers.

      CentOS Clients

      On CentOS, RedHat, and Fedora Linux, edit the /etc/sysconfig/network-scripts/ifcfg-eth0 file. You may have to substitute eth0 with the name of your primary network interface:

      • sudo nano /etc/sysconfig/network-scripts/ifcfg-eth0

      Search for the DNS1 and DNS2 options and set them to the private IP addresses of your primary and secondary name servers. Add a DOMAIN parameter that with your infrastructure's base domain. In this guide, that would be "nyc3.example.com":

      /etc/sysconfig/network-scripts/ifcfg-eth0

      . . .
      DNS1=10.128.10.11
      DNS2=10.128.20.12
      DOMAIN='nyc3.example.com'
      . . .
      

      Save and close the file when you are finished.

      Now, restart the networking service by typing:

      • sudo systemctl restart network

      The command may hang for a few seconds, but should return you to the prompt shortly.

      Check that your changes were applied by typing:

      You should see your name servers and search domain in the list:

      /etc/resolv.conf

      nameserver 10.128.10.11
      nameserver 10.128.20.12
      search nyc3.example.com
      

      Your client should now be able to connect to and use your DNS servers.

      Testing Clients

      Use nslookup to test if your clients can query your name servers. You should be able to do this on all of the clients that you have configured and are in the "trusted" ACL.

      For CentOS clients, you may need to install the utility with:

      • sudo yum install bind-utils

      For Debian clients, you can install with:

      • sudo apt install dnsutils

      We can start by performing a forward lookup.

      Forward Lookup

      For example, we can perform a forward lookup to retrieve the IP address of host1.nyc3.example.com by running the following command:

      Querying "host1" expands to "host1.nyc3.example.com because of the search option is set to your private subdomain, and DNS queries will attempt to look on that subdomain before looking for the host elsewhere. The output of the command above would look like the following:

      Output

      Server: 127.0.0.53 Address: 127.0.0.53#53 Non-authoritative answer: Name: host1.nyc3.example.com Address: 10.128.100.101

      Next, we can check reverse lookups.

      Reverse Lookup

      To test the reverse lookup, query the DNS server with host1's private IP address:

      You should see output that looks like the following:

      Output

      11.10.128.10.in-addr.arpa name = host1.nyc3.example.com. Authoritative answers can be found from:

      If all of the names and IP addresses resolve to the correct values, that means that your zone files are configured properly. If you receive unexpected values, be sure to review the zone files on your primary DNS server (e.g. db.nyc3.example.com and db.10.128).

      Congratulations! Your internal DNS servers are now set up properly! Now we will cover maintaining your zone records.

      Maintaining DNS Records

      Now that you have a working internal DNS, you need to maintain your DNS records so they accurately reflect your server environment.

      Adding a Host to DNS

      Whenever you add a host to your environment (in the same datacenter), you will want to add it to DNS. Here is a list of steps that you need to take:

      Primary Name Server

      • Forward zone file: Add an "A" record for the new host, increment the value of "Serial"
      • Reverse zone file: Add a "PTR" record for the new host, increment the value of "Serial"
      • Add your new host's private IP address to the "trusted" ACL (named.conf.options)

      Test your configuration files:

      • sudo named-checkconf
      • sudo named-checkzone nyc3.example.com db.nyc3.example.com
      • sudo named-checkzone 128.10.in-addr.arpa /etc/bind/zones/db.10.128

      Then reload BIND:

      • sudo systemctl reload bind9

      Your primary server should be configured for the new host now.

      Secondary Name Server

      • Add your new host's private IP address to the "trusted" ACL (named.conf.options)

      Check the configuration syntax:

      Then reload BIND:

      • sudo systemctl reload bind9

      Your secondary server will now accept connections from the new host.

      Configure New Host to Use Your DNS

      • Configure /etc/resolv.conf to use your DNS servers
      • Test using nslookup

      Removing Host from DNS

      If you remove a host from your environment or want to just take it out of DNS, just remove all the things that were added when you added the server to DNS (i.e. the reverse of the steps above).

      Conclusion

      Now you may refer to your servers' private network interfaces by name, rather than by IP address. This makes configuration of services and applications easier because you no longer have to remember the private IP addresses, and the files will be easier to read and understand. Also, now you can change your configurations to point to a new servers in a single place, your primary DNS server, instead of having to edit a variety of distributed configuration files, which eases maintenance.

      Once you have your internal DNS set up, and your configuration files are using private FQDNs to specify network connections, it is critical that your DNS servers are properly maintained. If they both become unavailable, your services and applications that rely on them will cease to function properly. This is why it is recommended to set up your DNS with at least one secondary server, and to maintain working backups of all of them.



      Source link