One place for hosting & domains

      Set Up a Reverse Proxy in an LXD Container to Host Multiple Websites


      Updated by Linode Contributed by Simos Xenitellis

      Introduction

      LXD (pronounced “Lex-Dee”) is a system container manager build on top of Linux Containers (LXC) supported by Canonical. The goal of LXD is to provide an experience similar to a virtual machine but through containerization rather than hardware virtualization. Compared to Docker for delivering applications, LXD offers nearly full operating-system functionality with additional features such as snapshots, live migrations, and storage management.

      A reverse proxy is a server that sits between internal applications and external clients, forwarding client requests to the appropriate server. While many common applications, such as Node.js, are able to function as servers on their own, they may lack a number of advanced load balancing, security, and acceleration features.

      This guide explains the creation of a reverse proxy in an LXD container in order to host multiple websites, each in their own additional containers. You will utilize NGINX and Apache web servers, while also relying on NGINX as a reverse proxy.

      Please refer to the following diagram to understand the reverse proxy created in this guide.

      Diagram of LXD reverse proxy and web servers

      In this guide you will:

      Note

      For simplicity, the term container is used throughout this guide to describe the LXD system containers.

      Before You Begin

      1. Complete A Beginner’s Guide to LXD: Setting Up an Apache Web Server In a Container. The guide instructs you to create a container called web with the Apache web server for testing purposes. Remove this container by running the following commands.

        lxc stop web
        lxc delete web
        

        Note

      2. This guide will use the hostnames apache1.example.com and nginx1.example.com for the two example websites. Replace these names with hostnames you own and setup their DNS entries to point them to the IP address of the server you created. For help with DNS see our DNS Manager Guide.

      Creating the Containers

      1. Create two containers called apache1 and nginx1, one with the Apache web server and another with the NGINX web server, respectively. For any additional websites, you may create new containers with your chosen web server software.

        lxc launch ubuntu:18.04 apache1
        lxc launch ubuntu:18.04 nginx1
        
      2. Create the proxy container for the reverse proxy.

        lxc launch ubuntu:18.04 proxy
        
      3. List the containers with the list command.

        lxc list
        
      4. The output will look similar to the following.

          
        +---------+---------+---------------------+-----------------------------------------------+------------+-----------+
        |  NAME   |  STATE  |        IPV4         |                     IPV6                      |    TYPE    | SNAPSHOTS |
        +---------+---------+---------------------+-----------------------------------------------+------------+-----------+
        | apache1 | RUNNING | 10.10.10.204 (eth0) | fd42:67a4:b462:6ae2:216:3eff:fe01:1a4e (eth0) | PERSISTENT |           |
        +---------+---------+---------------------+-----------------------------------------------+------------+-----------+
        | nginx1  | RUNNING | 10.10.10.251 (eth0) | fd42:67a4:b462:6ae2:216:3eff:febd:67e3 (eth0) | PERSISTENT |           |
        +---------+---------+---------------------+-----------------------------------------------+------------+-----------+
        | proxy   | RUNNING | 10.10.10.28 (eth0)  | fd42:67a4:b462:6ae2:216:3eff:fe00:252e (eth0) | PERSISTENT |           |
        +---------+---------+---------------------+-----------------------------------------------+------------+-----------+
        
        

        There are three containers, all in the RUNNING state – each with their own private IP address. Take note of the IP addresses (both IPv4 and IPv6) for the container proxy. You will need them to configure the proxy container in a later section.

        Now that the containers have been created, the following steps will detail how to set up the web server software in the apache1 and nginx1 containers, and the proxy container so that the web servers are accessible from the internet.

      Configuring the Apache Web Server Container

      When using a reverse proxy in front of a web server, the web server does not know the IP addresses of visitors. The web server only sees the IP address of the reverse proxy. However, each web server has a way to identify the real remote IP address of a visitor. For Apache, this is performed with the Remote IP Apache module. For the module to work, the reverse proxy must be configured to pass the remote IP address’ information.

      1. Start a shell in the apache1 container.

        lxc exec apache1 -- sudo --user ubuntu --login
        
      2. Update the package list in the apache1 container.

        sudo apt update
        
      3. Install the package apache2 in the container.

        sudo apt install -y apache2
        
      4. Create the file /etc/apache2/conf-available/remoteip.conf.

        remoteip.conf
        1
        2
        
        RemoteIPHeader X-Real-IP
        RemoteIPTrustedProxy 10.10.10.28 fd42:67a4:b462:6ae2:216:3eff:fe00:252e

        You can use the nano text editor by running the command sudo nano /etc/apache2/conf-available/remoteip.conf. Note, these are the IP addresses of the proxy container shown earlier, for both IPv4 and IPv6. Replace these with the IPs from your lxc list output.

        Note

        Instead of specifying the IP addresses, you can also use the hostname proxy.lxd. However, the RemoteIP Apache module is peculiar when using the hostname and will use only one of the two IP addresses (either IPv4 or IPv6), which means the Apache web server will not know the real source IP address for some connections. By listing explicitly both IPv4 and IPv6 addresses, you can be certain that RemoteIP will successfully accept the source IP information from all connections of the reverse proxy.

      5. Enable the new remoteip.conf configuration.

        sudo a2enconf remoteip
        
          
        Enabling conf remoteip.
        To activate the new configuration, you need to run:
        systemctl reload apache2
        
        
      6. Enable the remoteip Apache module.

        sudo a2enmod remoteip
        
          
        Enabling module remoteip.
        To activate the new configuration, you need to run:
        systemctl restart apache2
        
        
      7. Edit the default web page for Apache to make a reference that it runs inside a LXD container.

        sudo nano /var/www/html/index.html
        

        Change the line “It works!” (line number 224) to “It works inside a LXD container!” Save and exit.

      8. Restart the Apache web server.

        sudo systemctl reload apache2
        
      9. Exit back to the host.

        exit
        

      You have created and configured the Apache web server, but the server is not yet accessible from the Internet. It will become accessible after you configure the proxy container in a later section.

      Creating the NGINX Web Server Container

      Like Apache, NGINX does not know the IP addresses of visitors when using a reverse proxy in front of a web server. It only sees the IP address of the reverse proxy instead. Each NGINX web server software can identify the real remote IP address of a visitor with the Real IP module. For the module to work, the reverse proxy must be configured accordingly to pass the information regarding the remote IP addresses.

      1. Start a shell in the nginx1 container.

        lxc exec nginx1 -- sudo --user ubuntu --login
        
      2. Update the package list in the nginx1 container.

        sudo apt update
        
      3. Install NGINX in the container.

        sudo apt install -y nginx
        
      4. Create the file /etc/nginx/conf.d/real-ip.conf.

        real-ip.conf
        1
        2
        
        real_ip_header    X-Real-IP;
        set_real_ip_from  proxy.lxd;

        You can use the nano text editor by running the command sudo nano /etc/nginx/conf.d/real-ip.conf.

        Note

        You have specified the hostname of the reverse proxy, proxy.lxd. Each LXD container gets automatically a hostname, which is the name of the container plus the suffix .lxd. By specifying the set_real_ip_from field with proxy.lxd, you are instructing the NGINX web server to accept the real IP address information for each connection, as long as that connection originates from proxy.lxd. The real IP address information will be found in the HTTP header X-Real-IP in each connection.

      5. Edit the default web page for NGINX to make a reference that it runs inside a LXD container.

        sudo nano /var/www/html/index.nginx-debian.html
        

        Change the line “Welcome to nginx!” (line number 14) to “Welcome to nginx running in a LXD system container!”. Save and exit.

      6. Restart the NGINX web server.

        sudo systemctl reload nginx
        
      7. Exit back to the host.

        exit
        

      You have created and configured the NGINX web server, but the server is not accessible yet from the Internet. It will become accessible after you configure the proxy container in the next section.

      Setting up the Reverse Proxy

      In this section you will configure the container proxy. You will install NGINX and set it up as a reverse proxy, then add the appropriate LXD proxy device in order to expose both ports 80 and 443 to the internet.

      1. Add LXD proxy devices to redirect connections from the internet to ports 80 (HTTP) and 443 (HTTPS) on the server to the respective ports at the proxy container.

        lxc config device add proxy myport80 proxy listen=tcp:0.0.0.0:80 connect=tcp:127.0.0.1:80 proxy_protocol=true
        lxc config device add proxy myport443 proxy listen=tcp:0.0.0.0:443 connect=tcp:127.0.0.1:443 proxy_protocol=true
        
          
        Device myport80 added to proxy
        Device myport443 added to proxy
        
        

        The lxc config device add command takes as arguments:

        Argument Explanation
        proxy The name of the container.
        myport80 A name for this proxy device.
        proxy The type of the LXD device (LXD proxy device).
        listen=tcp:0.0.0.0:80 The proxy device will listen on the host (default) on port 80, protocol TCP, on all interfaces.
        connect=tcp:127.0.0.1:80 The proxy device will connect to the container on port 80, protocol TCP, on the loopback interface. In previous versions of LXD you could have specified localhost here. However, in LXD 3.13 or newer, you can only specify IP addresses.
        proxy_protocol=true Request to enable the PROXY protocol so that the reverse proxy will get the originating IP address from the proxy device.

        Note

        If you want to remove a proxy device, use lxc config device remove. If you want to remove the above device myport80, run the following command:

        lxc config device remove proxy myport80
        

        Where proxy is the name of the container, and myport80 is the name of the device.

      2. Start a shell in the proxy container.

        lxc exec proxy -- sudo --user ubuntu --login
        
      3. Update the package list.

        sudo apt update
        
      4. Install NGINX in the container.

        sudo apt install -y nginx
        
      5. Logout from the container.

        logout
        

      Direct Traffic to the Apache Web Server From the Reverse Proxy

      The reverse proxy container is running and the NGINX package has been installed. To work as a reverse proxy, add the appropriate website configuration so that NGINX can identify (with server_name below) the appropriate hostname, and then pass (with proxy_pass below) the connection to the appropriate LXD container.

      1. Start a shell in the proxy container.

        lxc exec proxy -- sudo --user ubuntu --login
        
      2. Create the file apache1.example.com in /etc/nginx/sites-available/ for the configuration of your first website.

        apache1.example.com
         1
         2
         3
         4
         5
         6
         7
         8
         9
        10
        11
        12
        13
        14
        15
        
        server {
                listen 80 proxy_protocol;
                listen [::]:80 proxy_protocol;
        
                server_name apache1.example.com;
        
                location / {
                        proxy_set_header Host $host;
                        proxy_set_header X-Real-IP $remote_addr;
                        proxy_pass http://apache1.lxd;
                }
        
                real_ip_header proxy_protocol;
                set_real_ip_from 127.0.0.1;
        }

        You can run sudo nano /etc/nginx/sites-available/apache1.example.com to open up a text editor and add the configuration. Note, in this case you only need to edit the server_name to be the hostname of the website.

      3. Enable the website.

        sudo ln -s /etc/nginx/sites-available/apache1.example.com /etc/nginx/sites-enabled/
        
      4. Restart the NGINX reverse proxy. By restarting the service, NGINX will read and apply the new site instructions just added to /etc/nginx/sites-enabled.

        sudo systemctl reload nginx
        
      5. Exit the proxy container and return back to the host.

        logout
        
      6. From your local computer, visit the URL of your website with your web browser. You should see the default Apache page:

        Web page of Apache server running in a container

        Note

        If you look at the Apache access.log file (default file /var/log/apache2/access.log), it will still show the private IP address of the proxy container instead of the real IP address. This issue is specific to the Apache web server and has to do with how the server prints the logs. Other software on the web server will be able to use the real IP. To fix this through the Apache logs, see the section Troubleshooting.

      Direct Traffic to the NGINX Web Server From the Reverse Proxy

      The reverse proxy container is running and the NGINX package has been installed. To work as a reverse proxy, you will add the appropriate website configuration so NGINX can identify (with server_name below) the appropriate hostname, and then pass (with proxy_pass below) the connection to the appropriate LXD container with the actual web server software.

      1. Start a shell in the proxy container.

        lxc exec proxy -- sudo --user ubuntu --login
        
      2. Create the file nginx1.example.com in /etc/nginx/sites-available/ for the configuration of your second website.

        nginx1.example.com
         1
         2
         3
         4
         5
         6
         7
         8
         9
        10
        11
        12
        13
        14
        15
        
        server {
                listen 80 proxy_protocol;
                listen [::]:80 proxy_protocol;
        
                server_name nginx1.example.com;
        
                location / {
                        proxy_set_header Host $host;
                        proxy_set_header X-Real-IP $remote_addr;
                        proxy_pass http://nginx1.lxd;
                }
        
                real_ip_header proxy_protocol;
                set_real_ip_from 127.0.0.1;
        }

        You can run sudo nano /etc/nginx/sites-available/nginx1.example.com to create the configuration. Note, you only need to edit the fields server_name to be the hostname of the website.

      3. Enable the website.

        sudo ln -s /etc/nginx/sites-available/nginx1.example.com /etc/nginx/sites-enabled/
        
      4. Restart the NGINX reverse proxy service.

        sudo systemctl reload nginx
        
      5. Exit the proxy container and return back to the host.

        logout
        
      6. From your local computer, visit the URL of your website with your web browser. You should see the following default NGINX page.

        Web page of the nginx server running in a container

      Adding Support for HTTPS with Let’s Encrypt

      1. Start a shell in the proxy container.

        lxc exec proxy -- sudo --user ubuntu --login
        
      2. Add the repository ppa:certbot/certbot by running the following command.

        sudo add-apt-repository ppa:certbot/certbot
        
      3. Output will look similar to the following.

          
              This is the PPA for packages prepared by Debian Let's Encrypt Team and backported for Ubuntu(s).
              More info: https://launchpad.net/~certbot/+archive/ubuntu/certbot
             Press [ENTER] to continue or Ctrl-c to cancel adding it.
        
             Get:1 http://security.ubuntu.com/ubuntu bionic-security InRelease [88.7 kB]
             ...
             Fetched 3360 kB in 2s (2018 kB/s)
             Reading package lists... Done
        
        
      4. Install the following two packages to a) support the creation of Let’s Encrypt certificates; and b) auto-configure the NGINX reverse proxy to use Let’s Encrypt certificates. The packages are pulled from the newly-created repository.

        sudo apt-get install certbot python-certbot-nginx
        

        Note

        This configures the reverse proxy to also act as a TLS Termination Proxy. Any HTTPS configuration is only found in the proxy container. By doing so, it is not necessary to perform any tasks inside the web server containers relating to certificates and Let’s Encrypt.

      5. Run certbot as root with the --nginx parameter in order to perform the auto-configuration of Let’s Encrypt for the first website. You will be asked to supply a valid email address for urgent renewal and security notices. You will then be asked to accept the Terms of Service and whether you would like to be contacted by the Electronic Frontier Foundation in the future. Next, you will provide the website for which you are activating HTTPS. Finally, you can choose to set up a facility that automatically redirects HTTP connections to HTTPS connections.

        sudo certbot --nginx
        
          
        Saving debug log to /var/log/letsencrypt/letsencrypt.log
        Plugins selected: Authenticator nginx, Installer nginx
        Enter email address (used for urgent renewal and security notices) (Enter 'c' to
        cancel): myemail@example.com
        
        - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
        Please read the Terms of Service at
        https://letsencrypt.org/documents/LE-SA-v1.2-November-15-2017.pdf. You must
        agree in order to register with the ACME server at
        https://acme-v02.api.letsencrypt.org/directory
        - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
        (A)gree/(C)ancel: A
        
        - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
        Would you be willing to share your email address with the Electronic Frontier
        Foundation, a founding partner of the Let's Encrypt project and the non-profit
        organization that develops Certbot? We'd like to send you email about our work
        encrypting the web, EFF news, campaigns, and ways to support digital freedom.
        - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
        (Y)es/(N)o: N
        
        Which names would you like to activate HTTPS for?
        - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
        1: apache1.example.com
        2: nginx1.example.com
        - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
        Select the appropriate numbers separated by commas and/or spaces, or leave input
        blank to select all options shown (Enter 'c' to cancel): 1
        Obtaining a new certificate
        Performing the following challenges:
        http-01 challenge for apache1.example.com
        Waiting for verification...
        Cleaning up challenges
        Deploying Certificate to VirtualHost /etc/nginx/sites-enabled/apache1.example.com
        
        Please choose whether or not to redirect HTTP traffic to HTTPS, removing HTTP access.
        - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
        1: No redirect - Make no further changes to the webserver configuration.
        2: Redirect - Make all requests redirect to secure HTTPS access. Choose this for
        new sites, or if you're confident your site works on HTTPS. You can undo this
        change by editing your web server's configuration.
        - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
        Select the appropriate number [1-2] then [enter] (press 'c' to cancel): 2
        Redirecting all traffic on port 80 to ssl in /etc/nginx/sites-enabled/apache1.example.com
        
        - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
        Congratulations! You have successfully enabled https://apache1.example.com
        
        You should test your configuration at:
        https://www.ssllabs.com/ssltest/analyze.html?d=apache1.example.com
        - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
        
        IMPORTANT NOTES:
         - Congratulations! Your certificate and chain have been saved at:
           /etc/letsencrypt/live/apache1.example.com/fullchain.pem
           Your key file has been saved at:
           /etc/letsencrypt/live/apache1.example.com/privkey.pem
           Your cert will expire on 2019-10-07. To obtain a new or tweaked
           version of this certificate in the future, simply run certbot again
           with the "certonly" option. To non-interactively renew *all* of
           your certificates, run "certbot renew"
         - Your account credentials have been saved in your Certbot
           configuration directory at /etc/letsencrypt. You should make a
           secure backup of this folder now. This configuration directory will
           also contain certificates and private keys obtained by Certbot so
           making regular backups of this folder is ideal.
         - If you like Certbot, please consider supporting our work by:
        
           Donating to ISRG / Let's Encrypt:   https://letsencrypt.org/donate
           Donating to EFF:                    https://eff.org/donate-le
        
        
      6. Run certbot as root with the --nginx parameter in order to perform the auto-configuration of Let’s Encrypt for the second website. This is the second time we run certbot, therefore we are asked directly to select the website to configure.

        sudo certbot --nginx
        
          
        Saving debug log to /var/log/letsencrypt/letsencrypt.log
        Plugins selected: Authenticator nginx, Installer nginx
        
        Which names would you like to activate HTTPS for?
        - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
        1: apache1.example.com
        2: nginx1.example.com
        - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
        Select the appropriate numbers separated by commas and/or spaces, or leave input
        blank to select all options shown (Enter 'c' to cancel): 2
        Obtaining a new certificate
        Performing the following challenges:
        http-01 challenge for nginx1.example.com
        Waiting for verification...
        Cleaning up challenges
        Deploying Certificate to VirtualHost /etc/nginx/sites-enabled/nginx1.example.com
        
        Please choose whether or not to redirect HTTP traffic to HTTPS, removing HTTP access.
        - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
        1: No redirect - Make no further changes to the webserver configuration.
        2: Redirect - Make all requests redirect to secure HTTPS access. Choose this for
        new sites, or if you're confident your site works on HTTPS. You can undo this
        change by editing your web server's configuration.
        - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
        Select the appropriate number [1-2] then [enter] (press 'c' to cancel): 2
        Redirecting all traffic on port 80 to ssl in /etc/nginx/sites-enabled/nginx1.example.com
        
        - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
        Congratulations! You have successfully enabled https://nginx1.example.com
        
        You should test your configuration at:
        https://www.ssllabs.com/ssltest/analyze.html?d=nginx1.example.com
        - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
        
        IMPORTANT NOTES:
         - Congratulations! Your certificate and chain have been saved at:
           /etc/letsencrypt/live/nginx1.example.com/fullchain.pem
           Your key file has been saved at:
           /etc/letsencrypt/live/nginx1.example.com/privkey.pem
           Your cert will expire on 2019-10-07. To obtain a new or tweaked
           version of this certificate in the future, simply run certbot again
           with the "certonly" option. To non-interactively renew *all* of
           your certificates, run "certbot renew"
         - If you like Certbot, please consider supporting our work by:
        
           Donating to ISRG / Let's Encrypt:   https://letsencrypt.org/donate
           Donating to EFF:                    https://eff.org/donate-le
        
        
      7. After adding all websites, perform a dry run in order to test the renewal of the certificates. Check that all websites are updating successfully to ensure the automated facility will update the certificates without further effort.

        sudo certbot renew --dry-run
        
          
        Saving debug log to /var/log/letsencrypt/letsencrypt.log
        
        - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
        Processing /etc/letsencrypt/renewal/apache1.example.com.conf
        - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
        Cert not due for renewal, but simulating renewal for dry run
        Plugins selected: Authenticator nginx, Installer nginx
        Renewing an existing certificate
        Performing the following challenges:
        http-01 challenge for apache1.example.com
        Waiting for verification...
        Cleaning up challenges
        
        - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
        new certificate deployed with reload of nginx server; fullchain is
        /etc/letsencrypt/live/apache1.example.com/fullchain.pem
        - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
        
        - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
        Processing /etc/letsencrypt/renewal/nginx1.example.com.conf
        - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
        Cert not due for renewal, but simulating renewal for dry run
        Plugins selected: Authenticator nginx, Installer nginx
        Renewing an existing certificate
        Performing the following challenges:
        http-01 challenge for nginx1.example.com
        Waiting for verification...
        Cleaning up challenges
        
        - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
        new certificate deployed with reload of nginx server; fullchain is
        /etc/letsencrypt/live/nginx1.example.com/fullchain.pem
        - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
        
        - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
        ** DRY RUN: simulating 'certbot renew' close to cert expiry
        **          (The test certificates below have not been saved.)
        
        Congratulations, all renewals succeeded. The following certs have been renewed:
          /etc/letsencrypt/live/apache1.example.com/fullchain.pem (success)
          /etc/letsencrypt/live/nginx1.example.com/fullchain.pem (success)
        ** DRY RUN: simulating 'certbot renew' close to cert expiry
        **          (The test certificates above have not been saved.)
        - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
        
        IMPORTANT NOTES:
         - Your account credentials have been saved in your Certbot
           configuration directory at /etc/letsencrypt. You should make a
           secure backup of this folder now. This configuration directory will
           also contain certificates and private keys obtained by Certbot so
           making regular backups of this folder is ideal.
        
        

        Note

        The certbot package adds a systemd timer in order to activate the automated renewal of Let’s Encrypt certificates. You can view the details of this timer by running systemctl list-timers.

      8. The certbot tool edits and changes the NGINX configuration files of your websites. In doing so, certbot does not obey initial listen directive (listen 80 proxy_protocol;) and does not add the proxy_protocol parameter to the newly added listen 443 ssl; lines. You must edit the configuration files for each website and append “proxy_protocol” to each “listen 443 ssl;” line.

        sudo nano /etc/nginx/sites-enabled/apache1.example.com
        sudo nano /etc/nginx/sites-enabled/nginx1.example.com
        
          
        listen 443 ssl proxy_protocol; # managed by Certbot
        listen [::]:443 ssl proxy_protocol; # managed by Certbot
        
        

        Note

        Each website configuration file has two pairs of listen directives: HTTP and HTTPS, respectively. The first is the original pair for HTTP that was added in a previous section. The second pair was added by certbot for HTTPS. These are pairs because they they cover both IPv4 and IPv6. The notation [::] refers to IPv6. When adding the parameter proxy_protocol, add it before the ; on each line as shown above.

      9. Restart NGINX.

        sudo systemctl restart nginx
        

      Troubleshooting

      Browser Error “SSL_ERROR_RX_RECORD_TOO_LONG”

      You have configured Certbot and created the appropriate Let’s Encrypt configuration for each website. But when you access the website from your browser, you get the following error.

        
      Secure Connection Failed
      
      An error occurred during a connection to apache1.example.com. SSL received a record that exceeded the maximum permissible length. Error code: SSL_ERROR_RX_RECORD_TOO_LONG
      
          The page you are trying to view cannot be shown because the authenticity of the received data could not be verified.
          Please contact the website owners to inform them of this problem.
      
      

      This error is caused when the NGINX reverse proxy in the proxy container does not have the proxy_protocol parameter in the listen 443 directives. Without the parameter, the reverse proxy does not consume the PROXY protocol information before it performs the HTTPS work. It mistakenly passes the PROXY protocol information to the HTTPS module, hence the record too long error.

      Follow the instructions in the previous section and add proxy_protocol to all listen 443 directives. Finally, restart NGINX.

      Error “Unable to connect” or “This site can’t be reached”

      When you attempt to connect to the website from your local computer and receive Unable to connect or This site can’t be reached errors, it is likely the proxy devices have not been configured.

      Run the following command on the host to verify whether LXD is listening and is able to accept connections to ports 80 (HTTP) and 443 (HTTPS).

      sudo ss -ltp '( sport = :http || sport = :https )'
      

      Note

      The ss command is similar to netstat and lsof. It shows information about network connections. In this case, we use it to verify whether there is a service on ports 80 and 443, and which service it is. * -l, to display the listening sockets, * -t, to display only TCP sockets, * -p, to show which processes use those sockets, * ( sport = :http || sport = :https ), to show only ports 80 and 443 (HTTP and HTTPS, respectively).

      In the following output we can verify that both ports 80 and 443 (HTTP and HTTPS, respectively) are in the LISTEN state. In the last column we verify that the process listening is lxd itself.

        
      State     Recv-Q  Send-Q   Local Address:Port   Peer Address:Port
      LISTEN    0       128                  *:http              *:*       users:(("lxd",pid=1301,fd=7),("lxd",pid=1301,fd=5))
      LISTEN    0       128                  *:https             *:*       users:(("lxd",pid=1349,fd=7),("lxd",pid=1349,fd=5))
      
      

      If you see a process listed other than lxd, stop that service and restart the proxy container. By restarting the proxy container, LXD will apply the proxy devices again.

      The Apache access.log Shows the IP Address of the Proxy Container

      You have set up the apache1 container and verified that it is accessible from the internet. But the logs at /var/log/apache2/access.log still show the private IP address of the proxy container, either the private IPv4 (10.x.x.x) or the private IPv6 addresses. What went wrong?

      The default log formats for printing access logs in Apache only print the IP address of the host of the last hop (i.e. the proxy server). This is the %h format specifier as shown below.

        
      LogFormat "%v:%p %h %l %u %t "%r" %>s %O "%{Referer}i" "%{User-Agent}i"" vhost_combined
      LogFormat "%h %l %u %t "%r" %>s %O "%{Referer}i" "%{User-Agent}i"" combined
      LogFormat "%h %l %u %t "%r" %>s %O" common
      
      

      The %h must be manually replaced with the %a format specifier, which prints the value as returned by the real RemoteIP Apache module.

        
      LogFormat "%v:%p %a %l %u %t "%r" %>s %O "%{Referer}i" "%{User-Agent}i"" vhost_combined
      LogFormat "%a %l %u %t "%r" %>s %O "%{Referer}i" "%{User-Agent}i"" combined
      LogFormat "%a %l %u %t "%r" %>s %O" common
      
      
      1. Run the following command in the apache1 container to edit the configuration file httpd.conf and perform the change from %h to %a.

        sudo nano /etc/apache2/apache2.conf
        
      2. Reload the Apache web server service.

        sudo systemctl reload apache2
        

      Next Steps

      You have set up a reverse proxy to host many websites on the same server and installed each website in a separate container. You can install static or dynamic websites in the containers. For dynamic websites, you may need additional configuration; check the respective documentation for setup using a reverse proxy. In addition, you may also use NGINX as a reverse proxy for non-HTTP(S) services.

      More Information

      You may wish to consult the following resources for additional information on this topic. While these are provided in the hope that they will be useful, please note that we cannot vouch for the accuracy or timeliness of externally hosted materials.

      Find answers, ask questions, and help others.

      This guide is published under a CC BY-ND 4.0 license.



      Source link

      A Beginner's Guide to LXD: Setting Up an Apache Webserver In a Container


      Updated by Linode Contributed by Simos Xenitellis

      Access an Apache Web Server Inside a LXD Container

      What is LXD?

      LXD (pronounced “Lex-Dee”) is a system container manager build on top of LXC (Linux Containers) that is currently supported by Canonical. The goal of LXD is to provide an experience similar to a virtual machine but through containerization rather than hardware virtualization. Compared to Docker for delivering applications, LXD offers nearly full operating-system functionality with additional features such as snapshots, live migrations, and storage management.

      The main benefits of LXD are the support of high density containers and the performance it delivers compared to virtual machines. A computer with 2GB RAM can adequately support half a dozen containers. In addition, LXD officially supports the container images of major Linux distributions. We can choose the Linux distribution and version to run in the container.

      This guide covers how to install and setup LXD 3 on a Linode and how to setup an Apache Web server in a container.

      Note

      For simplicity, the term container is used throughout this guide to describe the LXD system containers.

      Before You Begin

      1. Complete the Getting Started guide. Select a Linode with at least 2GB of RAM memory, such as the Linode 2GB. Specify the Ubuntu 19.04 distribution. You may specify a different Linux distribution, as long as there is support for snap packages (snapd); see the More Information for more details.

      2. This guide will use sudo wherever necessary. Follow the Securing Your Server guide to create a limited (non-root) user account, harden SSH access, and remove unnecessary network services.

      3. Update your system:

        sudo apt update && sudo apt upgrade
        

      Configure the Snap Package Support

      LXD is available as a Debian package in the long-term support (LTS) versions of Ubuntu, such as Ubuntu 18.04 LTS. For other versions of Ubuntu and other distributions, LXD is available as a snap package. Snap packages are universal packages because there is a single package file that works on any supported Linux distributions. See the More Information section for more details on what a snap package is, what Linux distributions are supported, and how to set it up.

      1. Verify that snap support is installed correctly. The following command either shows that there are no snap packages installed, or that some are.

        snap list
        
          
        No snaps are installed yet. Try 'snap install hello-world'.
        
        
      2. View the details of the LXD snap package lxd. The output below shows that, currently, the latest version of LXD is 3.12 in the default stable channel. This channel is updated often with new features. There are also other channels such as the 3.0/stable channel which has the LTS LXD version (supported along with Ubuntu 18.04, until 2023) and the 2.0/stable channel (supported along with Ubuntu 16.04, until 2021). We will be using the latest version of LXD from the default stable channel.

        snap info lxd
        
          
        name:      lxd
        summary:   System container manager and API
        publisher: Canonical✓
        contact:   https://github.com/lxc/lxd/issues
        license:   Apache-2.0
        description: |
          **LXD is a system container manager**
        
          With LXD you can run hundreds of containers of a variety of Linux
          distributions, apply resource limits, pass in directories, USB devices
          or GPUs and setup any network and storage you want.
        
          LXD containers are lightweight, secure by default and a great
          alternative to running Linux virtual machines.
        
        
          **Run any Linux distribution you want**
        
          Pre-made images are available for Ubuntu, Alpine Linux, ArchLinux,
          CentOS, Debian, Fedora, Gentoo, OpenSUSE and more.
        
          A full list of available images can be [found
          here](https://images.linuxcontainers.org)
        
          Can't find the distribution you want? It's easy to make your own images
          too, either using our `distrobuilder` tool or by assembling your own image
          tarball by hand.
        
        
          **Containers at scale**
        
          LXD is network aware and all interactions go through a simple REST API,
          making it possible to remotely interact with containers on remote
          systems, copying and moving them as you wish.
        
          Want to go big? LXD also has built-in clustering support,
          letting you turn dozens of servers into one big LXD server.
        
        
          **Configuration options**
        
          Supported options for the LXD snap (`snap set lxd KEY=VALUE`):
           - criu.enable: Enable experimental live-migration support [default=false]
           - daemon.debug: Increases logging to debug level [default=false]
           - daemon.group: Group of users that can interact with LXD [default=lxd]
           - ceph.builtin: Use snap-specific ceph configuration [default=false]
           - openvswitch.builtin: Run a snap-specific OVS daemon [default=false]
        
          [Documentation](https://lxd.readthedocs.io)
        snap-id: J60k4JY0HppjwOjW8dZdYc8obXKxujRu
        channels:
          stable:        3.12        2019-04-16 (10601) 56MB -
          candidate:     3.12        2019-04-26 (10655) 56MB -
          beta:          ↑
          edge:          git-570aaa1 2019-04-27 (10674) 56MB -
          3.0/stable:    3.0.3       2018-11-26  (9663) 53MB -
          3.0/candidate: 3.0.3       2019-01-19  (9942) 53MB -
          3.0/beta:      ↑
          3.0/edge:      git-eaa62ce 2019-02-19 (10212) 53MB -
          2.0/stable:    2.0.11      2018-07-30  (8023) 28MB -
          2.0/candidate: 2.0.11      2018-07-27  (8023) 28MB -
          2.0/beta:      ↑
          2.0/edge:      git-c7c4cc8 2018-10-19  (9257) 26MB -
        
        
      3. Install the lxd snap package. Run the following command to install the snap package for LXD.

        sudo snap install lxd
        
          
        lxd 3.12 from Canonical✓ installed
        
        

      You can verify that the snap package has been installed by running snap list again. The core snap package is a prerequisite for any system with snap package support. When you install your first snap package, core is installed and shared among all other snap packages that will get installed in the future.

          snap list
      
      
        
      Name  Version  Rev    Tracking  Publisher   Notes
      core  16-2.38  6673   stable    canonical✓  core
      lxd   3.12     10601  stable    canonical✓  -
      
      

      Initialize LXD

      1. Add your non-root Unix user to the lxd group:

        sudo usermod -a -G lxd username
        

        Note

        By adding the non-root Unix user account to the lxd group, you are able to run any lxc commands without prepending sudo. Without this addition, you would have needed to prepend sudo to each lxc command.

      2. Start a new SSH session for the previous change to take effect. For example, log out and log in again.

      3. Verify the available free disk space:

        df -h /
        
          
        Filesystem      Size  Used Avail Use% Mounted on
        /dev/sda         49G  2.0G   45G   5% /
        
        

        In this case there is 45GB of free disk space. LXD requires at least 15GB of space for the storage needs of containers. We will allocate 15GB of space for LXD, leaving 30GB of free space for the needs of the server.

      4. Run lxd init to initialize LXD:

        sudo lxd init
        

        You will be prompted several times during the initialization process. Choose the defaults for all options.

          
        Would you like to use LXD clustering? (yes/no) [default=no]:
        Do you want to configure a new storage pool? (yes/no) [default=yes]:
        Name of the new storage pool [default=default]:
        Name of the storage backend to use (btrfs, ceph, dir, lvm, zfs) [default=zfs]:
        Create a new ZFS pool? (yes/no) [default=yes]:
        Would you like to use an existing block device? (yes/no) [default=no]:
        Size in GB of the new loop device (1GB minimum) [default=15GB]:
        Would you like to connect to a MAAS server? (yes/no) [default=no]:
        Would you like to create a new local network bridge? (yes/no) [default=yes]:
        What should the new bridge be called? [default=lxdbr0]:
        What IPv4 address should be used? (CIDR subnet notation, “auto” or “none”) [default=auto]:
        What IPv6 address should be used? (CIDR subnet notation, “auto” or “none”) [default=auto]:
        Would you like LXD to be available over the network? (yes/no) [default=no]:
        Would you like stale cached images to be updated automatically? (yes/no) [default=yes]
        Would you like a YAML "lxd init" preseed to be printed? (yes/no) [default=no]:
        
        

      Apache Web Server with LXD

      This section will create a container, install the Apache web server, and add the appropriate iptables rules in order to expose post 80.

      1. Launch a new container:

        lxc launch ubuntu:18.04 web
        
      2. Update the package list in the container.

        lxc exec web -- apt update
        
      3. Install the Apache in the LXD container.

        lxc exec web -- apt install apache2
        
      4. Get a shell in the LXD container.

        lxc exec web -- sudo --user ubuntu --login
        
      5. Edit the default web page for Apache to make a reference that it runs inside a LXD container.

        sudo nano /var/www/html/index.html
        

        Change the line It works! (line number 224) to It works inside a LXD container!. Then, save and exit.

      6. Exit back to the host. We have made all the necessary changes to the container.

        exit
        
      7. Add a LXD proxy device to redirect connections from the internet to port 80 (HTTP) on the server to port 80 at this container.

        sudo lxc config device add web myport80 proxy listen=tcp:0.0.0.0:80 connect=tcp:127.0.0.1:80
        

        Note

        In recent versions of LXD, you need to specify an IP address (such as 127.0.0.1) instead of a hostname (such as localhost). If your container already has a proxy device that uses hostnames, you can edit the container configuration to replace with IP addresses by running lxc config edit web.

      8. From your local computer, navigate to your Linode’s public IP address in a web browser. You should see the default Apache page:

        Web page of Apache server running in a container

      Common LXD Commands

      • List all containers:

        lxc list
        
          
        To start your first container, try: lxc launch ubuntu:18.04
        
        +------+-------+------+------+------+-----------+
        | NAME | STATE | IPV4 | IPV6 | TYPE | SNAPSHOTS |
        +------+-------+------+------+------+-----------+
        
        
      • List all available repositories of container images:

        lxc remote list
        
          
        +-----------------+------------------------------------------+---------------+-------------+--------+--------+
        |      NAME       |                   URL                    |   PROTOCOL    |  AUTH TYPE  | PUBLIC | STATIC |
        +-----------------+------------------------------------------+---------------+-------------+--------+--------+
        | images          | https://images.linuxcontainers.org       | simplestreams | none        | YES    | NO     |
        +-----------------+------------------------------------------+---------------+-------------+--------+--------+
        | local (default) | unix://                                  | lxd           | file access | NO     | YES    |
        +-----------------+------------------------------------------+---------------+-------------+--------+--------+
        | ubuntu          | https://cloud-images.ubuntu.com/releases | simplestreams | none        | YES    | YES    |
        +-----------------+------------------------------------------+---------------+-------------+--------+--------+
        | ubuntu-daily    | https://cloud-images.ubuntu.com/daily    | simplestreams | none        | YES    | YES    |
        +-----------------+------------------------------------------+---------------+-------------+--------+--------+
        
        

        The repository ubuntu has container images of Ubuntu versions. The images repository has container images of a large number of different Linux distributions. The ubuntu-daily has daily container images to be used for testing purposes. The local repository is the LXD server that we have just installed. It is not public and can be used to store your own container images.

      • List all available container images from a repository:

        lxc image list ubuntu:
        
          
        +------------------+--------------+--------+-----------------------------------------------+---------+----------+-------------------------------+
        |      ALIAS       | FINGERPRINT  | PUBLIC |                  DESCRIPTION                  |  ARCH   |   SIZE   |          UPLOAD DATE          |
        +------------------+--------------+--------+-----------------------------------------------+---------+----------+-------------------------------+
        | b (11 more)      | 5b72cf46f628 | yes    | ubuntu 18.04 LTS amd64 (release) (20190424)   | x86_64  | 180.37MB | Apr 24, 2019 at 12:00am (UTC) |
        +------------------+--------------+--------+-----------------------------------------------+---------+----------+-------------------------------+
        | c (5 more)       | 4716703f04fc | yes    | ubuntu 18.10 amd64 (release) (20190402)       | x86_64  | 313.29MB | Apr 2, 2019 at 12:00am (UTC)  |
        +------------------+--------------+--------+-----------------------------------------------+---------+----------+-------------------------------+
        | d (5 more)       | faef94acf5f9 | yes    | ubuntu 19.04 amd64 (release) (20190417)       | x86_64  | 322.56MB | Apr 17, 2019 at 12:00am (UTC) |
        +------------------+--------------+--------+-----------------------------------------------+---------+----------+-------------------------------+
        .....................................................................
        
        

        Note

        The first two columns for the alias and fingerprint provide an identifier that can be used to specify the container image when launching it.

        The output snippet shows the container images Ubuntu versions 18.04 LTS, 18.10, and 19.04. When creating a container we can just specify the short alias. For example, ubuntu:b means that the repository is ubuntu and the container image has the short alias b (for bionic, the codename of Ubuntu 18.04 LTS).

      • Get more information about a container image:

        lxc image info ubuntu:b
        
          
        Fingerprint: 5b72cf46f628b3d60f5d99af48633539b2916993c80fc5a2323d7d841f66afbe
        Size: 180.37MB
        Architecture: x86_64
        Public: yes
        Timestamps:
            Created: 2019/04/24 00:00 UTC
            Uploaded: 2019/04/24 00:00 UTC
            Expires: 2023/04/26 00:00 UTC
            Last used: never
        Properties:
            release: bionic
            version: 18.04
            architecture: amd64
            label: release
            serial: 20190424
            description: ubuntu 18.04 LTS amd64 (release) (20190424)
            os: ubuntu
        Aliases:
            - 18.04
            - 18.04/amd64
            - b
            - b/amd64
            - bionic
            - bionic/amd64
            - default
            - default/amd64
            - lts
            - lts/amd64
            - ubuntu
            - amd64
        Cached: no
        Auto update: disabled
        
        

        The output shows the details of the container image including all the available aliases. For Ubuntu 18.04 LTS, we can specify either b (for bionic, the codename of Ubuntu 18.04 LTS) or any other alias.

      • Launch a new container with the name mycontainer:

        lxc launch ubuntu:18.04 mycontainer
        
          
        Creating mycontainer
        Starting mycontainer
        
        
      • Check the list of containers to make sure the new container is running:

        lxc list
        
          
        +-------------+---------+-----------------------+---------------------------+------------+-----------+
        |    NAME     |  STATE  |         IPV4          |          IPV6             |    TYPE    | SNAPSHOTS |
        +-------------+---------+-----------------------+---------------------------+------------+-----------+
        | mycontainer | RUNNING | 10.142.148.244 (eth0) | fde5:5d27:...:1371 (eth0) | PERSISTENT | 0         |
        +-------------+---------+-----------------------+---------------------------+------------+-----------+
        
        
      • Execute basic commands in mycontainer:

        lxc exec mycontainer -- apt update
        lxc exec mycontainer -- apt upgrade
        

        Note

        The characters -- instruct the lxc command not to parse any more command-line parameters.

      • Open a shell session within mycontainer:

        lxc exec mycontainer -- sudo --login --user ubuntu
        
          
        To run a command as administrator (user "root"), use "sudo ".
        See "man sudo_root" for details.
        
        ubuntu@mycontainer:~$
        
        

        Note

        The Ubuntu container images have by default a non-root account with username ubuntu. This account can use sudo and does not require a password to perform administrative tasks.

        The sudo command provides a login to the existing account ubuntu.

      • View the container logs:

        lxc info mycontainer --show-log
        
      • Stop the container:

        lxc stop mycontainer
        
      • Remove the container:

        lxc delete mycontainer
        

        Note

        A container needs to be stopped before it can be deleted.

      Troubleshooting

      Error “unix.socket: connect: connection refused”

      When you run any lxc command, you get the following error:

          lxc list
      
      
        
      Error: Get http://unix.socket/1.0: dial unix /var/snap/lxd/common/lxd/unix.socket: connect: connection refused
      
      

      This happens when the LXD service is not currently running. By default, the LXD service is running as soon as it is configured successfully. See Initialize LXD to configure LXD.

      Error “unix.socket: connect: permission denied”

      When you run any lxc command, you get the following error:

          lxc list
      
      
        
      Error: Get http://unix.socket/1.0: dial unix /var/snap/lxd/common/lxd/unix.socket: connect: permission denied
      
      

      This happens when your limited user account is not a member of the lxd group, or you did not log out and log in again so that the new group membership to the lxd group gets updated.

      If your user account is ubuntu, the following command shows whether you are a member of the lxd group:

          groups ubuntu
      
      
        
      ubuntu : ubuntu sudo lxd
      
      

      In this example, we are members of the lxd group and we just need to log out and log in again. If you are not a member of the lxd group, see Initialize LXD on how to make your limited account a member of the lxd group.

      Next Steps

      If you plan to use a single website, then a single proxy device to the website container will suffice. If you plan to use multiple websites, you may install virtual hosts inside the website container. If instead you would like to setup multiple websites on their own container, then you will need to set up a reverse proxy in a container. In that case, the proxy device would direct to the reverse proxy container to direct the connections to the individual websites containers.

      More Information

      You may wish to consult the following resources for additional information on this topic. While these are provided in the hope that they will be useful, please note that we cannot vouch for the accuracy or timeliness of externally hosted materials.

      Find answers, ask questions, and help others.

      This guide is published under a CC BY-ND 4.0 license.



      Source link