One place for hosting & domains

      Apache

      Configure Apache with Salt Stack


      Updated by Linode Written by Linode

      Salt is a powerful configuration management tool. In this guide you will create Salt state files that are capable of installing and configuring Apache on Ubuntu 18.04, Debian 9, or CentOS 7.

      Before You Begin

      You will need at least two Linodes with Salt installed. If you have not already, read our Getting Started with Salt – Basic Installation and Setup Guide and follow the instructions for setting up a Salt master and minion.

      The following steps will be performed on your Salt master.

      Note

      The steps in this guide require root privileges. Be sure to run the steps below as root or with the sudo prefix. For more information on privileges, see our Users and Groups guide.

      Setting Up Your Salt Master and Managed Files

      Salt Master SLS Files

      1. Create the /srv/salt directory if it does not already exist:

        mkdir /srv/salt
        
      2. Create a Salt top file in /srv/salt that will be Salt’s entry point to the Apache configuration:

        /srv/salt/top.sls
        1
        2
        3
        4
        5
        6
        7
        8
        
        base:
          'G@os_family:Debian':
            - match: compound
            - apache-debian
        
          'G@os:CentOS':
            - match: compound
            - apache-centos

        This top file uses compound matching to target your minions by operating system using Salt Grains. This will allow Salt to choose the appropriate Apache configuration depending on the Linux distribution. These matchers could be extended to be even more specific. For instance, if you wanted to only target minions with the ID of web-server that are running on Ubuntu, you can type web* and G@os:Ubuntu.

      Pillar Files

      1. Create the /srv/pillar directory if it does not already exist:

        mkdir /srv/pillar
        
      2. Create a Pillar top file. This top file references the apache.sls Pillar file that you will create in the next step:

        /srv/pillar/top.sls
        1
        2
        3
        
        base:
          '*':
            - apache
      3. Create the apache.sls file that was referenced in the previous step. This file defines Pillar data that will be used inside our Apache state file in the next section, in this case your domain name. Replace example.com with your domain:

        /srv/pillar/apache.sls

      Website Files

      1. Create a directory for your website files in the /srv/salt directory. Replace example.com with your website domain name:

        mkdir /srv/salt/example.com
        

        This directory will be accessible from your Salt state files at salt://example.com.

      2. Create an index.html file for your website in the /srv/salt/example.com directory, substituting example.com for the folder name you chose in the previous step. You will use this file as a test to make sure your website is functioning correctly.

        /srv/salt/example.com/index.html
        1
        2
        3
        4
        5
        
        <html>
          <body>
            <h1>Server Up and Running!</h1>
          </body>
        </html>

      Configuration Files

      1. Create a folder for your additional configuration files at /srv/salt/files. These files will be accessible at salt://files.

        mkdir /srv/salt/files
        
      2. Create a file called tune_apache.conf in /srv/salt/files and paste in the following block:

        /srv/salt/files/tune_apache.conf
        1
        2
        3
        4
        5
        6
        7
        
        <IfModule mpm_prefork_module>
        StartServers 4
        MinSpareServers 20
        MaxSpareServers 40
        MaxClients 200
        MaxRequestsPerChild 4500
        </IfModule>

        This MPM prefork module provides additional tuning for your Apache installation. This file will be managed by Salt and installed into the appropriate configuration directory in a later step.

      3. If you will be installing Apache on a CentOS machine, create a file called include_sites_enabled.conf in /srv/salt/files and paste in the following:

        /srv/salt/files/include_sites_enabled.conf
        1
        
        IncludeOptional sites-enabled/*.conf

        This file will allow us to use file directories like those found on Debian installations to help organize the Apache configuration.

      Creating the Apache State File for Debian and Ubuntu

      Individual Steps

      This guide will be going through the process of creating the Apache for Debian and Ubuntu state file step by step. If you would like to view the entirety of the state file, you can view it at the end of this section.

      1. Create a state file named apache-debian.sls in /srv/salt and open it in a text editor of your choice.

      2. Instruct Salt to install the apache2 package and start the apache2 service:

        /srv/salt/apache-debian.sls
         1
         2
         3
         4
         5
         6
         7
         8
         9
        10
        11
        
        apache2:
          pkg.installed
        
        apache2 Service:
          service.running:
            - name: apache2
            - enable: True
            - require:
              - pkg: apache2
        
        ...

        Here Salt makes sure the apache2 package is installed with pkg.installed. Likewise, it ensures the apache2 service is running and enabled under service.running. Also under service.running, apache-debian.sls uses require to ensure that this command does not run before the apache2 package is installed. This require step will be repeated throughout apache-debain.sls.

        Lastly, a watch statement is employed to restart the apache2 service if your site’s configuration file changes. You will define that configuration file in a later step. Note that this configuration file is named using the domain you supplied when creating your Salt Pillar file in the first section. This Pillar data will be used throughout apache-debian.sls.

      3. Turn off KeepAlive:

        /srv/salt/apache-debian.sls
         1
         2
         3
         4
         5
         6
         7
         8
         9
        10
        11
        
        ...
        
        Turn Off KeepAlive:
          file.replace:
            - name: /etc/apache2/apache2.conf
            - pattern: 'KeepAlive On'
            - repl: 'KeepAlive Off'
            - show_changes: True
            - require:
              - pkg: apache2
        ...

        KeepAlive allows multiple requests to be sent over the same TCP connection. For the purpose of this guide KeepAlive will be disabled. To disable it, Salt is instructed to find the KeepAlive directive in /etc/apache2/apache2.conf by matching a pattern and replacing it with KeepAlive Off. show_changes instructs Salt to display any changes it has made during a highstate.

      4. Transfer tune_apache.conf to your minion and enable it:

        /srv/salt/apache-debian.sls
         1
         2
         3
         4
         5
         6
         7
         8
         9
        10
        11
        12
        13
        14
        15
        
        ...
        
        /etc/apache2/conf-available/tune_apache.conf:
          file.managed:
            - source: salt://files/tune_apache.conf
            - require:
              - pkg: apache2
        
        Enable tune_apache:
          apache_conf.enabled:
            - name: tune_apache
            - require:
              - pkg: apache2
        
        ...

        This step takes the tune_apache.conf file you created in the Configuration Files step and transfers it to your Salt minion. Then, Salt enables that configuration file with the apache_conf module.

      5. Create the necessary directories:

        /srv/salt/apache-debian.sls
         1
         2
         3
         4
         5
         6
         7
         8
         9
        10
        11
        12
        13
        14
        15
        
        ...
        
        /var/www/html/{{ pillar['domain'] }}:
          file.directory
        
        /var/www/html/{{ pillar['domain'] }}/log:
          file.directory
        
        /var/www/html/{{ pillar['domain'] }}/backups:
          file.directory
        
        /var/www/html/{{ pillar['domain'] }}/public_html:
          file.directory
        
        ...
      6. Disable the default virtual host configuration file:

        /srv/salt/apache-debian.sls
        1
        2
        3
        4
        5
        6
        7
        8
        
        ...
        
        000-default:
          apache_site.disabled:
            - require:
              - pkg: apache2
        
        ...

        This step uses Salt’s apache_site module to disable the default Apache virtual host configuration file, and is the same as running a2dissite on a Debian-based machine.

      7. Create your site’s virtual host configuration file:

        /srv/salt/apache-debian.sls
         1
         2
         3
         4
         5
         6
         7
         8
         9
        10
        11
        12
        13
        14
        15
        16
        
        ...
        
        /etc/apache2/sites-available/{{ pillar['domain'] }}.conf:
          apache.configfile:
            - config:
              - VirtualHost:
                  this: '*:80'
                  ServerName:
                    - {{ pillar['domain'] }}
                  ServerAlias:
                    - www.{{ pillar['domain'] }}
                  DocumentRoot: /var/www/html/{{ pillar['domain'] }}/public_html
                  ErrorLog: /var/www/html/{{ pillar['domain'] }}/log/error.log
                  CustomLog: /var/www/html/{{ pillar['domain'] }}/log/access.log combined
        
        ...

        This step uses Salt’s apache module, (not to be confused with the apache_site module used in the previous step), to create your site’s virtual host configuration file. The this variable signifies what would traditionally be include with VirtualHost within angle brackets in an Apache configuration file: <VirtualHost *:80>.

      8. Enable your new virtual host configuration file:

        /srv/salt/apache-debian.sls
        1
        2
        3
        4
        5
        6
        7
        8
        
        ...
        
        {{ pillar['domain'] }}:
          apache_site.enabled:
            - require:
              - pkg: apache2
        
        ...

        This step uses the same apache_site module you used to disable the default virtual host file to enable your newly created virtual host file. apache_site.enabled creates a symlink from /etc/apache2/sites-available/example.com.conf to /etc/apache2/sites-enabled/example.com.conf and is the same as running a2ensite on a Debian-based machine.

      9. Transfer your index.html website file to your minion:

        /srv/salt/apache-debian.sls
        1
        2
        3
        4
        5
        
        ...
        
        /var/www/html/{{ pillar['domain'] }}/public_html/index.html:
          file.managed:
            - source: salt://{{ pillar['domain'] }}/index.html

        Any changes made to your index.html file on your Salt master will be propagated to your minion.

        Note

        Since Salt is not watching configuration files for a change to trigger a restart for Apache, you may need to use the command below from your Salt master.

        salt '*' apache.signal restart
        

      Complete State File

      The complete apache-debian.sls file looks like this:

      /srv/salt/apache-debian.sls
       1
       2
       3
       4
       5
       6
       7
       8
       9
      10
      11
      12
      13
      14
      15
      16
      17
      18
      19
      20
      21
      22
      23
      24
      25
      26
      27
      28
      29
      30
      31
      32
      33
      34
      35
      36
      37
      38
      39
      40
      41
      42
      43
      44
      45
      46
      47
      48
      49
      50
      51
      52
      53
      54
      55
      56
      57
      58
      59
      60
      61
      62
      63
      64
      65
      66
      67
      68
      69
      
      apache2:
        pkg.installed
      
      apache2 Service:
        service.running:
          - name: apache2
          - enable: True
          - require:
            - pkg: apache2
      
      Turn Off KeepAlive:
        file.replace:
          - name: /etc/apache2/apache2.conf
          - pattern: 'KeepAlive On'
          - repl: 'KeepAlive Off'
          - show_changes: True
          - require:
            - pkg: apache2
      
      /etc/apache2/conf-available/tune_apache.conf:
        file.managed:
          - source: salt://files/tune_apache.conf
          - require:
            - pkg: apache2
      
      Enable tune_apache:
        apache_conf.enabled:
          - name: tune_apache
          - require:
            - pkg: apache2
      
      /var/www/html/{{ pillar['domain'] }}:
        file.directory
      
      /var/www/html/{{ pillar['domain'] }}/log:
        file.directory
      
      /var/www/html/{{ pillar['domain'] }}/backups:
        file.directory
      
      /var/www/html/{{ pillar['domain'] }}/public_html:
        file.directory
      
      000-default:
        apache_site.disabled:
          - require:
            - pkg: apache2
      
      /etc/apache2/sites-available/{{ pillar['domain'] }}.conf:
        apache.configfile:
          - config:
            - VirtualHost:
                this: '*:80'
                ServerName:
                  - {{ pillar['domain'] }}
                ServerAlias:
                  - www.{{ pillar['domain'] }}
                DocumentRoot: /var/www/html/{{ pillar['domain'] }}/public_html
                ErrorLog: /var/www/html/{{ pillar['domain'] }}/log/error.log
                CustomLog: /var/www/html/{{ pillar['domain'] }}/log/access.log combined
      
      {{ pillar['domain'] }}:
        apache_site.enabled:
          - require:
            - pkg: apache2
      
      /var/www/html/{{ pillar['domain'] }}/public_html/index.html:
        file.managed:
          - source: salt://{{ pillar['domain'] }}/index.html

      Creating an Apache State File for CentOS

      Individual Steps

      1. Create a file called apache-centos.sls in /srv/salt and open it in a text editor of your choice.

      2. On CentOS Apache is named httpd. Instruct Salt to install httpd and run the httpd service:

        /srv/salt/apache-centos.sls
         1
         2
         3
         4
         5
         6
         7
         8
         9
        10
        11
        12
        13
        
        httpd:
          pkg.installed
        
        httpd Service:
          service.running:
            - name: httpd
            - enable: True
            - require:
              - pkg: httpd
            - watch:
              - file: /etc/httpd/sites-available/{{ pillar['domain'] }}.conf
        
        ...

        Here Salt makes sure the httpd package is installed with pkg.installed. Likewise, it ensures the httpd service is running and enabled under service.running. Also under service.running, apache-debian.sls uses require to ensure that this command does not run before the httpd package is installed. This require step will be repeated throughout apache-centos.sls.

        Lastly, a watch statement is employed to restart the httpd service if your site’s configuration file changes. You will define that configuration file in a later step. Note that this configuration file is named using the domain you supplied when creating your Salt Pillar file in the first section. This Pillar data will be used throughout apache-centos.sls.

      3. Turn off KeepAlive:

        /srv/salt/apache-centos.sls
         1
         2
         3
         4
         5
         6
         7
         8
         9
        10
        11
        
        ...
        
        Turn Off KeepAlive:
          file.replace:
            - name: /etc/httpd/conf/httpd.conf
            - pattern: 'KeepAlive On'
            - repl: 'KeepAlive Off'
            - show_changes: True
            - require:
              - pkg: httpd
        ...

        KeepAlive allows multiple requests to be sent over the same TCP connection. For the purpose of this guide KeepAlive will be disabled. To disable it, Salt is instructed to find the KeepAlive directive in /etc/httpd/conf/httpd.conf by matching a pattern and replacing it with KeepAlive Off. show_changes instructs Salt to display any changes it has made during a highstate.

      4. Change the DocumentRoot:

        /srv/salt/apache-centos.sls
         1
         2
         3
         4
         5
         6
         7
         8
         9
        10
        11
        12
        
        ...
        
        Change DocumentRoot:
          file.replace:
            - name: /etc/httpd/conf/httpd.conf
            - pattern: 'DocumentRoot "/var/www/html"'
            - repl: 'DocumentRoot "/var/www/html/{{ pillar['domain'] }}/public_html"'
            - show_changes: True
            - require:
              - pkg: httpd
        
        ...

        Similar to the last step, in this step salt-centos.sls instructs Salt to search for the DocumentRoot directive in Apache’s httpd.conf file, and replaces that line with the new document root. This allows for the use of a Debian-style site directory architecture.

      5. Transfer the tune_apache.conf and include_sites_enabled.conf to your minion.

        /srv/salt/apache-centos.sls
         1
         2
         3
         4
         5
         6
         7
         8
         9
        10
        11
        12
        13
        14
        15
        
        ...
        
        /etc/httpd/conf.d/tune_apache.conf:
          file.managed:
            - source: salt://files/tune_apache.conf
            - require:
              - pkg: httpd
        
        /etc/httpd/conf.d/include_sites_enabled.conf:
          file.managed:
            - source: salt://files/include_sites_enabled.conf
            - require:
              - pkg: httpd
        
        ...
      6. Create the necessary directories:

        srv/salt/apache-centos.sls
         1
         2
         3
         4
         5
         6
         7
         8
         9
        10
        11
        12
        13
        14
        15
        16
        17
        18
        
        ...
        
        /etc/httpd/sites-available:
          file.directory
        
        /etc/httpd/sites-enabled:
          file.directory
        
        /var/www/html/{{ pillar['domain'] }}:
          file.directory
        
        /var/www/html/{{ pillar['domain'] }}/backups:
          file.directory
        
        /var/www/html/{{ pillar['domain'] }}/public_html:
          file.directory
        
        ...
      7. Create your site’s virtual host configuration file:

        /srv/salt/apache-centos.sls
         1
         2
         3
         4
         5
         6
         7
         8
         9
        10
        11
        12
        13
        14
        15
        16
        17
        
        ...
        
        /etc/httpd/sites-available/{{ pillar['domain'] }}.conf:
          apache.configfile:
            - config:
              - VirtualHost:
                  this: '*:80'
                  ServerName:
                    - {{ pillar['domain'] }}
                  ServerAlias:
                    - www.{{ pillar['domain'] }}
                  DocumentRoot: /var/www/html/{{ pillar['domain'] }}/public_html
          file.symlink:
            - target: /etc/httpd/sites-enabled/{{ pillar['domain'] }}.conf
            - force: True
        
        ...

        This step uses Salt’s apache module to create your site’s virtual host configuration file. The this variable signifies what would traditionally be include with VirtualHost within angle brackets in an Apache configuration file: <VirtualHost *:80>.

      8. Transfer your index.html website file to your minion:

        /srv/salt/apache-debian.sls
        1
        2
        3
        4
        5
        6
        7
        
        ...
        
        /var/www/html/{{ pillar['domain'] }}/public_html/index.html:
          file.managed:
            - source: salt://{{ pillar['domain'] }}/index.html
        
        ...

        Any changes made to your index.html file on your Salt master will be propigated to your minion.

      9. Configure your firewall to allow http and https traffic:

        /srv/salt/apache-centos.sls
        1
        2
        3
        4
        5
        6
        7
        8
        9
        
        ...
        
        Configure Firewall:
          firewalld.present:
            - name: public
            - ports:
              - 22/tcp
              - 80/tcp
              - 443/tcp

        Note

        It is imperative that you list all ports you need open to your machine in this section. Failure to list these ports will result in their closure by Salt.

      Complete State File

      The complete apache-centos.sls file looks like this:

      /srv/salt/apache-centos.sls
       1
       2
       3
       4
       5
       6
       7
       8
       9
      10
      11
      12
      13
      14
      15
      16
      17
      18
      19
      20
      21
      22
      23
      24
      25
      26
      27
      28
      29
      30
      31
      32
      33
      34
      35
      36
      37
      38
      39
      40
      41
      42
      43
      44
      45
      46
      47
      48
      49
      50
      51
      52
      53
      54
      55
      56
      57
      58
      59
      60
      61
      62
      63
      64
      65
      66
      67
      68
      69
      70
      71
      72
      73
      74
      75
      76
      77
      78
      79
      80
      81
      82
      
      httpd:
        pkg.installed
      
      httpd Service:
        service.running:
          - name: httpd
          - enable: True
          - require:
            - pkg: httpd
          - watch:
            - file: /etc/httpd/sites-available/{{ pillar['domain'] }}.conf
      
      Turn off KeepAlive:
        file.replace:
          - name: /etc/httpd/conf/httpd.conf
          - pattern: 'KeepAlive On'
          - repl: 'KeepAlive Off'
          - show_changes: True
          - require:
            - pkg: httpd
      
      Change DocumentRoot:
        file.replace:
          - name: /etc/httpd/conf/httpd.conf
          - pattern: 'DocumentRoot "/var/www/html"'
          - repl: 'DocumentRoot "/var/www/html/{{ pillar['domain'] }}/public_html"'
          - show_changes: True
          - require:
            - pkg: httpd
      
      /etc/httpd/conf.d/tune_apache.conf:
        file.managed:
          - source: salt://files/tune_apache.conf
          - require:
            - pkg: httpd
      
      /etc/httpd/conf.d/include_sites_enabled.conf:
        file.managed:
          - source: salt://files/include_sites_enabled.conf
          - require:
            - pkg: httpd
      
      /etc/httpd/sites-available:
        file.directory
      
      /etc/httpd/sites-enabled:
        file.directory
      
      /var/www/html/{{ pillar['domain'] }}:
        file.directory
      
      /var/www/html/{{ pillar['domain'] }}/backups:
        file.directory
      
      /var/www/html/{{ pillar['domain'] }}/public_html:
        file.directory
      
      /etc/httpd/sites-available/{{ pillar['domain'] }}.conf:
        apache.configfile:
          - config:
            - VirtualHost:
                this: '*:80'
                ServerName:
                  - {{ pillar['domain'] }}
                ServerAlias:
                  - www.{{ pillar['domain'] }}
                DocumentRoot: /var/www/html/{{ pillar['domain'] }}/public_html
        file.symlink:
          - target: /etc/httpd/sites-enabled/{{ pillar['domain'] }}.conf
          - force: True
      
      /var/www/html/{{ pillar['domain'] }}/public_html/index.html:
        file.managed:
          - source: salt://{{ pillar['domain'] }}/index.html
      
      Configure Firewall:
        firewalld.present:
          - name: public
          - ports:
            - 22/tcp
            - 80/tcp
            - 443/tcp

      Running the Apache State File

      On your Salt master, issue a highstate command:

      salt '*' state.apply
      

      After a few moments you should see a list of Salt commands and a summary of their successes. Navigate to your website’s domain name if you have your DNS set up already, or your website’s public IP address. You should see your index.html file. You have now used Salt to configure Apache. Visit the links in the section below for more information.

      More Information

      You may wish to consult the following resources for additional information on this topic. While these are provided in the hope that they will be useful, please note that we cannot vouch for the accuracy or timeliness of externally hosted materials.

      Find answers, ask questions, and help others.

      This guide is published under a CC BY-ND 4.0 license.



      Source link

      How To Install the Apache Web Server on Ubuntu 18.04 [Quickstart]


      Introduction

      The Apache HTTP server is the most widely-used web server in the world. It provides many powerful features, including dynamically loadable modules, robust media support, and extensive integration with other popular software.

      In this guide, we’ll explain how to install an Apache web server on your Ubuntu 18.04 server. For a more detailed version of this tutorial, please refer to How To Install the Apache Web Server on Ubuntu 18.04.

      Prerequisites

      Before you begin this guide, you should have the following:

      • An Ubuntu 18.04 server and a regular, non-root user with sudo privileges. Additionally, you will need to enable a basic firewall to block non-essential ports. You can learn how to configure a regular user account and set up a firewall for your server by following our initial server setup guide for Ubuntu 18.04.

      When you have an account available, log in as your non-root user to begin.

      Step 1 — Installing Apache

      Apache is available within Ubuntu’s default software repositories, so you can install it using conventional package management tools.

      Update your local package index:

      Install the apache2 package:

      Step 2 — Adjusting the Firewall

      Check the available ufw application profiles:

      Output

      Available applications: Apache Apache Full Apache Secure OpenSSH

      Let's enable the most restrictive profile that will still allow the traffic you've configured, permitting traffic on port 80 (normal, unencrypted web traffic):

      Verify the change:

      Output

      Status: active To Action From -- ------ ---- OpenSSH ALLOW Anywhere Apache ALLOW Anywhere OpenSSH (v6) ALLOW Anywhere (v6) Apache (v6) ALLOW Anywhere (v6)

      Step 3 — Checking your Web Server

      Check with the systemd init system to make sure the service is running by typing:

      • sudo systemctl status apache2

      Output

      ● apache2.service - The Apache HTTP Server Loaded: loaded (/lib/systemd/system/apache2.service; enabled; vendor preset: enabled) Drop-In: /lib/systemd/system/apache2.service.d └─apache2-systemd.conf Active: active (running) since Tue 2018-04-24 20:14:39 UTC; 9min ago Main PID: 2583 (apache2) Tasks: 55 (limit: 1153) CGroup: /system.slice/apache2.service ├─2583 /usr/sbin/apache2 -k start ├─2585 /usr/sbin/apache2 -k start └─2586 /usr/sbin/apache2 -k start

      Access the default Apache landing page to confirm that the software is running properly through your IP address:

      http://your_server_ip
      

      You should see the default Ubuntu 18.04 Apache web page:

      Apache default page

      When using the Apache web server, you can use virtual hosts (similar to server blocks in Nginx) to encapsulate configuration details and host more than one domain from a single server. We will set up a domain called example.com, but you should replace this with your own domain name. To learn more about setting up a domain name with DigitalOcean, see our introduction to DigitalOcean DNS.

      Create the directory for example.com, using the -p flag to create any necessary parent directories:

      sudo mkdir -p /var/www/example.com/html
      

      Assign ownership of the directory:

      • sudo chown -R $USER:$USER /var/www/example.com/html

      The permissions of your web roots should be correct if you haven't modified your unmask value, but you can make sure by typing:

      • sudo chmod -R 755 /var/www/example.com

      Create a sample index.html page using nano or your favorite editor:

      • nano /var/www/example.com/html/index.html

      Inside, add the following sample HTML:

      /var/www/example.com/html/index.html

      <html>
          <head>
              <title>Welcome to Example.com!</title>
          </head>
          <body>
              <h1>Success!  The example.com server block is working!</h1>
          </body>
      </html>
      

      Save and close the file when you are finished.

      Make a new virtual host file at /etc/apache2/sites-available/example.com.conf:

      • sudo nano /etc/apache2/sites-available/example.com.conf

      Paste in the following configuration block, updated for our new directory and domain name:

      /etc/apache2/sites-available/example.com.conf

      <VirtualHost *:80>
          ServerAdmin admin@example.com
          ServerName example.com
          ServerAlias www.example.com
          DocumentRoot /var/www/example.com/html
          ErrorLog ${APACHE_LOG_DIR}/error.log
          CustomLog ${APACHE_LOG_DIR}/access.log combined
      </VirtualHost>
      

      Save and close the file when you are finished.

      Enable the file with a2ensite:

      • sudo a2ensite example.com.conf

      Disable the default site defined in 000-default.conf:

      • sudo a2dissite 000-default.conf

      Test for configuration errors:

      • sudo apache2ctl configtest

      You should see the following output:

      Output

      Syntax OK

      Restart Apache to implement your changes:

      • sudo systemctl restart apache2

      Apache should now be serving your domain name. You can test this by navigating to http://example.com, where you should see something like this:

      Apache virtual host example

      Conclusion

      Now that you have your web server installed, you have many options for the type of content to serve and the technologies you want to use to create a richer experience.

      If you'd like to build out a more complete application stack, check out this article on how to configure a LAMP stack on Ubuntu 18.04.



      Source link

      How To Install Apache Kafka on Ubuntu 18.04


      The author selected the Free and Open Source Fund to receive a donation as part of the Write for DOnations program.

      Introduction

      Apache Kafka is a popular distributed message broker designed to efficiently handle large volumes of real-time data. A Kafka cluster is not only highly scalable and fault-tolerant, but it also has a much higher throughput compared to other message brokers such as ActiveMQ and RabbitMQ. Though it is generally used as a publish/subscribe messaging system, a lot of organizations also use it for log aggregation because it offers persistent storage for published messages.

      A publish/subscribe messaging system allows one or more producers to publish messages without considering the number of consumers or how they will process the messages. Subscribed clients are notified automatically about updates and the creation of new messages. This system is more efficient and scalable than systems where clients poll periodically to determine if new messages are available.

      In this tutorial, you will install and use Apache Kafka 1.1.0 on Ubuntu 18.04.

      Prerequisites

      To follow along, you will need:

      • One Ubuntu 18.04 server and a non-root user with sudo privileges. Follow the steps specified in this guide if you do not have a non-root user set up.
      • At least 4GB of RAM on the server. Installations without this amount of RAM may cause the Kafka service to fail, with the Java virtual machine (JVM) throwing an “Out Of Memory” exception during startup.
      • OpenJDK 8 installed on your server. To install this version, follow these instructions on installing specific versions of OpenJDK. Kafka is written in Java, so it requires a JVM; however, its startup shell script has a version detection bug that causes it to fail to start with JVM versions above 8.

      Step 1 — Creating a User for Kafka

      Since Kafka can handle requests over a network, you should create a dedicated user for it. This minimizes damage to your Ubuntu machine should the Kafka server be compromised. We will create a dedicated kafka user in this step, but you should create a different non-root user to perform other tasks on this server once you have finished setting up Kafka.

      Logged in as your non-root sudo user, create a user called kafka with the useradd command:

      The -m flag ensures that a home directory will be created for the user. This home directory, /home/kafka, will act as our workspace directory for executing commands in the sections below.

      Set the password using passwd:

      Add the kafka user to the sudo group with the adduser command, so that it has the privileges required to install Kafka's dependencies:

      Your kafka user is now ready. Log into this account using su:

      Now that we've created the Kafka-specific user, we can move on to downloading and extracting the Kafka binaries.

      Step 2 — Downloading and Extracting the Kafka Binaries

      Let's download and extract the Kafka binaries into dedicated folders in our kafka user's home directory.

      To start, create a directory in /home/kafka called Downloads to store your downloads:

      Use curl to download the Kafka binaries:

      • curl "http://www-eu.apache.org/dist/kafka/1.1.0/kafka_2.12-1.1.0.tgz" -o ~/Downloads/kafka.tgz

      Create a directory called kafka and change to this directory. This will be the base directory of the Kafka installation:

      • mkdir ~/kafka && cd ~/kafka

      Extract the archive you downloaded using the tar command:

      • tar -xvzf ~/Downloads/kafka.tgz --strip 1

      We specify the --strip 1 flag to ensure that the archive's contents are extracted in ~/kafka/ itself and not in another directory (such as ~/kafka/kafka_2.12-1.1.0/) inside of it.

      Now that we've downloaded and extracted the binaries successfully, we can move on configuring to Kafka to allow for topic deletion.

      Step 3 — Configuring the Kafka Server

      Kafka's default behavior will not allow us to delete a topic, the category, group, or feed name to which messages can be published. To modify this, let's edit the configuration file.

      Kafka's configuration options are specified in server.properties. Open this file with nano or your favorite editor:

      • nano ~/kafka/config/server.properties

      Let's add a setting that will allow us to delete Kafka topics. Add the following to the bottom of the file:

      ~/kafka/config/server.properties

      delete.topic.enable = true
      

      Save the file, and exit nano. Now that we've configured Kafka, we can move on to creating systemd unit files for running and enabling it on startup.

      Step 4 — Creating Systemd Unit Files and Starting the Kafka Server

      In this section, we will create systemd unit files for the Kafka service. This will help us perform common service actions such as starting, stopping, and restarting Kafka in a manner consistent with other Linux services.

      Zookeeper is a service that Kafka uses to manage its cluster state and configurations. It is commonly used in many distributed systems as an integral component. If you would like to know more about it, visit the official Zookeeper docs.

      Create the unit file for zookeeper:

      • sudo nano /etc/systemd/system/zookeeper.service

      Enter the following unit definition into the file:

      /etc/systemd/system/zookeeper.service

      [Unit]
      Requires=network.target remote-fs.target
      After=network.target remote-fs.target
      
      [Service]
      Type=simple
      User=kafka
      ExecStart=/home/kafka/kafka/bin/zookeeper-server-start.sh /home/kafka/kafka/config/zookeeper.properties
      ExecStop=/home/kafka/kafka/bin/zookeeper-server-stop.sh
      Restart=on-abnormal
      
      [Install]
      WantedBy=multi-user.target
      

      The [Unit] section specifies that Zookeeper requires networking and the filesystem to be ready before it can start.

      The [Service] section specifies that systemd should use the zookeeper-server-start.sh and zookeeper-server-stop.sh shell files for starting and stopping the service. It also specifies that Zookeeper should be restarted automatically if it exits abnormally.

      Next, create the systemd service file for kafka:

      • sudo nano /etc/systemd/system/kafka.service

      Enter the following unit definition into the file:

      /etc/systemd/system/kafka.service

      [Unit]
      Requires=zookeeper.service
      After=zookeeper.service
      
      [Service]
      Type=simple
      User=kafka
      ExecStart=/bin/sh -c '/home/kafka/kafka/bin/kafka-server-start.sh /home/kafka/kafka/config/server.properties > /home/kafka/kafka/kafka.log 2>&1'
      ExecStop=/home/kafka/kafka/bin/kafka-server-stop.sh
      Restart=on-abnormal
      
      [Install]
      WantedBy=multi-user.target
      

      The [Unit] section specifies that this unit file depends on zookeeper.service. This will ensure that zookeeper gets started automatically when the kafa service starts.

      The [Service] section specifies that systemd should use the kafka-server-start.sh and kafka-server-stop.sh shell files for starting and stopping the service. It also specifies that Kafka should be restarted automatically if it exits abnormally.

      Now that the units have been defined, start Kafka with the following command:

      • sudo systemctl start kafka

      To ensure that the server has started successfully, check the journal logs for the kafka unit:

      You should see output similar to the following:

      Output

      Jul 17 18:38:59 kafka-ubuntu systemd[1]: Started kafka.service.

      You now have a Kafka server listening on port 9092.

      While we have started the kafka service, if we were to reboot our server, it would not be started automatically. To enable kafka on server boot, run:

      • sudo systemctl enable kafka

      Now that we've started and enabled the services, let's check the installation.

      Step 5 — Testing the Installation

      Let's publish and consume a "Hello World" message to make sure the Kafka server is behaving correctly. Publishing messages in Kafka requires:

      • A producer, which enables the publication of records and data to topics.
      • A consumer, which reads messages and data from topics.

      First, create a topic named TutorialTopic by typing:

      • ~/kafka/bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic TutorialTopic

      You can create a producer from the command line using the kafka-console-producer.sh script. It expects the Kafka server's hostname, port, and a topic name as arguments.

      Publish the string "Hello, World" to the TutorialTopic topic by typing:

      • echo "Hello, World" | ~/kafka/bin/kafka-console-producer.sh --broker-list localhost:9092 --topic TutorialTopic > /dev/null

      Next, you can create a Kafka consumer using the kafka-console-consumer.sh script. It expects the ZooKeeper server's hostname and port, along with a topic name as arguments.

      The following command consumes messages from TutorialTopic. Note the use of the --from-beginning flag, which allows the consumption of messages that were published before the consumer was started:

      • ~/kafka/bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic TutorialTopic --from-beginning

      If there are no configuration issues, you should see Hello, World in your terminal:

      Output

      Hello, World

      The script will continue to run, waiting for more messages to be published to the topic. Feel free to open a new terminal and start a producer to publish a few more messages. You should be able to see them all in the consumer's output.

      When you are done testing, press CTRL+C to stop the consumer script. Now that we have tested the installation, let's move on to installing KafkaT.

      Step 6 — Install KafkaT (Optional)

      KafkaT is a tool from Airbnb that makes it easier for you to view details about your Kafka cluster and perform certain administrative tasks from the command line. Because it is a Ruby gem, you will need Ruby to use it. You will also need the build-essential package to be able to build the other gems it depends on. Install them using apt:

      • sudo apt install ruby ruby-dev build-essential

      You can now install KafkaT using the gem command:

      KafkaT uses .kafkatcfg as the configuration file to determine the installation and log directories of your Kafka server. It should also have an entry pointing KafkaT to your ZooKeeper instance.

      Create a new file called .kafkatcfg:

      Add the following lines to specify the required information about your Kafka server and Zookeeper instance:

      ~/.kafkatcfg

      {
        "kafka_path": "~/kafka",
        "log_path": "/tmp/kafka-logs",
        "zk_path": "localhost:2181"
      }
      

      You are now ready to use KafkaT. For a start, here's how you would use it to view details about all Kafka partitions:

      You will see the following output:

      Output

      Topic Partition Leader Replicas ISRs TutorialTopic 0 0 [0] [0] __consumer_offsets 0 0 [0] [0] ... ...

      You will see TutorialTopic, as well as __consumer_offsets, an internal topic used by Kafka for storing client-related information. You can safely ignore lines starting with __consumer_offsets.

      To learn more about KafkaT, refer to its GitHub repository.

      Step 7 — Setting Up a Multi-Node Cluster (Optional)

      If you want to create a multi-broker cluster using more Ubuntu 18.04 machines, you should repeat Step 1, Step 4, and Step 5 on each of the new machines. Additionally, you should make the following changes in the server.properties file for each:

      • The value of the broker.id property should be changed such that it is unique throughout the cluster. This property uniquely identifies each server in the cluster and can have any string as its value. For example, "server1", "server2", etc.

      • The value of the zookeeper.connect property should be changed such that all nodes point to the same ZooKeeper instance. This property specifies the Zookeeper instance's address and follows the <HOSTNAME/IP_ADDRESS>:<PORT> format. For example, "203.0.113.0:2181", "203.0.113.1:2181" etc.

      If you want to have multiple ZooKeeper instances for your cluster, the value of the zookeeper.connect property on each node should be an identical, comma-separated string listing the IP addresses and port numbers of all the ZooKeeper instances.

      Step 8 — Restricting the Kafka User

      Now that all of the installations are done, you can remove the kafka user's admin privileges. Before you do so, log out and log back in as any other non-root sudo user. If you are still running the same shell session you started this tutorial with, simply type exit.

      Remove the kafka user from the sudo group:

      To further improve your Kafka server's security, lock the kafka user's password using the passwd command. This makes sure that nobody can directly log into the server using this account:

      At this point, only root or a sudo user can log in as kafka by typing in the following command:

      In the future, if you want to unlock it, use passwd with the -u option:

      You have now successfully restricted the kafka user's admin privileges.

      Conclusion

      You now have Apache Kafka running securely on your Ubuntu server. You can make use of it in your projects by creating Kafka producers and consumers using Kafka clients, which are available for most programming languages. To learn more about Kafka, you can also consult its documentation.



      Source link