One place for hosting & domains

      Servers

      How To Install Nagios 4 and Monitor Your Servers on Ubuntu 18.04


      The author selected the Open Source Initiative to receive a donation as part of the Write for DOnations program.

      Introduction

      Nagios is a popular open-source monitoring system. It keeps an inventory of your servers and monitors them so you know your critical services are up and running. Using a monitoring system like Nagios is an essential tool for any production environment, because by monitoring uptime, CPU usage, or disk space, you can head off problems before they occur, or before your users call you.

      In this tutorial, you’ll install Nagios 4 and configure it so you can monitor host resources via Nagios’ web interface. You’ll also set up the Nagios Remote Plugin Executor (NRPE), which runs as an agent on remote hosts so you can monitor their resources.

      Prerequisites

      To follow this tutorial, you will need:

      • Two Ubuntu 18.04 servers set up by following our Initial Server Setup Guide for Ubuntu 18.04, including a non-root user with sudo privileges and a firewall configured with ufw. On one server, you will install Nagios; this tutorial will refer to this as the Nagios server. It will monitor your second server; this second server will be referred to as the second Ubuntu server.
      • The server that will run the Nagios server needs Apache and PHP installed. Follow this guide to configure those on one of your servers. You can skip the MySQL steps in that tutorial.

      Typically, Nagios runs behind a hardware firewall or VPN. If your Nagios server is exposed to the public internet, you should secure the Nagios web interface by installing a TLS/SSL certificate. This is optional but strongly encouraged. You can follow the Let’s Encrypt on Ubuntu 18.04 guide to obtain the free TLS/SSL certificate.

      This tutorial assumes that your servers have private networking enabled so that monitoring happens on the private network rather than the public network. If you don’t have private networking enabled, you can still follow this tutorial by replacing all the references to private IP addresses with public IP addresses.

      Step 1 — Installing Nagios 4

      There are multiple ways to install Nagios, but you’ll install Nagios and its components from source to ensure you get the latest features, security updates, and bug fixes.

      Log in to your server that runs Apache. In this tutorial, we’ll call this the Nagios server:

      • ssh sammy@your_nagios_server_ip

      Because you’re building Nagios and its components from source, you must install a few development libraries to complete the build, including compilers, development headers, and OpenSSL.

      Update your package lists to ensure you can download the latest versions of the prerequisites:

      Then install the required packages:

      • sudo apt install autoconf gcc make unzip libgd-dev libmcrypt-dev libssl-dev dc snmp libnet-snmp-perl gettext

      With the prerequisites installed, you can install Nagios itself. Download the source code for the latest stable release of Nagios Core. Go to the Nagios downloads page, and click the Skip to download link below the form. Copy the link address for the latest stable release so you can download it to your Nagios server.

      Download the release to your home directory with the curl command:

      • cd ~
      • curl -L -O https://github.com/NagiosEnterprises/nagioscore/archive/nagios-4.4.4.tar.gz

      Extract the Nagios archive:

      • tar zxf nagios-4.4.4.tar.gz

      Then change to the extracted directory:

      • cd nagioscore-nagios-4.4.4

      Before building Nagios, run the configure script and specify the Apache configs directory:

      • ./configure --with-httpd-conf=/etc/apache2/sites-enabled

      Note: If you want Nagios to send emails using Postfix, you must install Postfix and configure Nagios to use it by adding --with-mail=/usr/sbin/sendmail to the configure command. We won't cover Postfix in this tutorial, but if you choose to use Postfix and Nagios later, you'll need to reconfigure and reinstall Nagios to use Postfix support.

      You'll see the following output from the configure command:

      Output

      *** Configuration summary for nagios 4.4.4 2019-07-29 ***: General Options: ------------------------- Nagios executable: nagios Nagios user/group: nagios,nagios Command user/group: nagios,nagios Event Broker: yes Install ${prefix}: /usr/local/nagios Install ${includedir}: /usr/local/nagios/include/nagios Lock file: /run/nagios.lock Check result directory: /usr/local/nagios/var/spool/checkresults Init directory: /lib/systemd/system Apache conf.d directory: /etc/apache2/sites-enabled Mail program: /bin/mail Host OS: linux-gnu IOBroker Method: epoll Web Interface Options: ------------------------ HTML URL: http://localhost/nagios/ CGI URL: http://localhost/nagios/cgi-bin/ Traceroute (used by WAP): Review the options above for accuracy. If they look okay, type 'make all' to compile the main program and CGIs.

      Now compile Nagios with this command:

      Next create a nagios user and nagios group. They will be used to run the Nagios process:

      • sudo make install-groups-users

      Now run these make commands to install Nagios binary files, service files, and its sample configuration files:

      • sudo make install
      • sudo make install-daemoninit
      • sudo make install-commandmode
      • sudo make install-config

      You'll use Apache to serve Nagios' web interface, so run the following to install the Apache configuration files and configure its settings:

      • sudo make install-webconf

      Enable the Apache rewrite and cgi modules with the a2enmod command:

      • sudo a2enmod rewrite
      • sudo a2enmod cgi

      In order to issue external commands via the web interface to Nagios, add the web server user, www-data, to the nagios group:

      • sudo usermod -a -G nagios www-data

      Use the htpasswd command to create an admin user called nagiosadmin that can access the Nagios web interface:

      • sudo htpasswd -c /usr/local/nagios/etc/htpasswd.users nagiosadmin

      Enter a password at the prompt. Remember this password, as you will need it to access the Nagios web interface.

      Warning: If you create a user with a name other than nagiosadmin, you will need to edit /usr/local/nagios/etc/cgi.cfg and change all the nagiosadmin references to the user you created.

      Restart Apache to load the new Apache configuration:

      • sudo systemctl restart apache2

      You've now installed Nagios. But for this to work, it is necessary to install the Nagios Plugins, which you'll cover in the next step.

      Step 2 — Installing the Nagios Plugins

      Nagios needs plugins to operate properly. The official Nagios Plugins package contains over 50 plugins that allow you to monitor basic services such as uptime, disk usage, swap usage, NTP, and others.

      Let's install the the plugins bundle.

      You can find the latest version of the Nagios Plugins on the official site.

      Download it to your home directory with curl:

      • cd ~
      • curl -L -O https://nagios-plugins.org/download/nagios-plugins-2.2.1.tar.gz

      Extract the NRPE archive and navigate into the extracted directory:

      • tar zxf nagios-plugins-<^>2.2.1<^.tar.gz
      • cd nagios-plugins-2.2.1

      Next configure their installation:

      Now build and install the plugins:

      Now the plugins are installed, but you need one more plugin for monitoring remote servers. Let's install it next.

      Step 3 — Installing the check_nrpe Plugin

      Nagios monitors remote hosts using the Nagios Remote Plugin Executor, or NRPE. It consists of two pieces:

      • The check_nrpe plugin that the Nagios server uses.
      • The NRPE daemon, which runs on the remote hosts and sends data to the Nagios server.

      Let's install the check_nrpe plugin on our Nagios server.

      Find the download URL for the latest stable release of NRPE at the GitHub page.

      Download it to your home directory with curl:

      • cd ~
      • curl -L -O https://github.com/NagiosEnterprises/nrpe/releases/download/nrpe-3.2.1/nrpe-3.2.1.tar.gz

      Extract the NRPE archive:

      • tar zxf nrpe-3.2.1.tar.gz

      Then change to the extracted directory:

      Configure the check_nrpe plugin:

      Now build and install check_nrpe plugin:

      • make check_nrpe
      • sudo make install-plugin

      Let's configure the Nagios server next.

      Step 4 — Configuring Nagios

      Now let's perform the initial Nagios configuration, which involves editing some configuration files. You only need to perform this section once on your Nagios server.

      Open the main Nagios configuration file in your preferred text editor. Here, you'll use nano:

      • sudo nano /usr/local/nagios/etc/nagios.cfg

      Find this line in the file:

      /usr/local/nagios/etc/nagios.cfg

      ...
      #cfg_dir=/usr/local/nagios/etc/servers
      ...
      

      Uncomment this line by deleting the # character from the front of the line:

      /usr/local/nagios/etc/nagios.cfg

      cfg_dir=/usr/local/nagios/etc/servers
      

      Save and close nagios.cfg by pressing CTRL+X, followed by Y, and then ENTER (if you're using nano).

      Now create the directory that will store the configuration file for each server that you will monitor:

      • sudo mkdir /usr/local/nagios/etc/servers

      Open the Nagios contacts configuration in your text editor:

      • sudo nano /usr/local/nagios/etc/objects/contacts.cfg

      Find the email directive and replace its value with your own email address:

      /usr/local/nagios/etc/objects/contacts.cfg

      ...
      define contact{
              contact_name                    nagiosadmin             ; Short name of user
              use                             generic-contact         ; Inherit default values from generic-contact template (defined above)
              alias                           Nagios Admin            ; Full name of user
              email                           your_email@your_domain.com        ; <<***** CHANGE THIS TO YOUR EMAIL ADDRESS ******
      ...
      
      

      Save and exit the editor.

      Next, add a new command to your Nagios configuration that lets you use the check_nrpe command in Nagios service definitions. Open the file /usr/local/nagios/etc/objects/commands.cfg in your editor:

      • sudo nano /usr/local/nagios/etc/objects/commands.cfg

      Add the following to the end of the file to define a new command called check_nrpe:

      /usr/local/nagios/etc/objects/commands.cfg

      ...
      define command{
              command_name check_nrpe
              command_line $USER1$/check_nrpe -H $HOSTADDRESS$ -c $ARG1$
      }
      

      This defines the name and specifies the command-line options to execute the plugin.

      Save and exit the editor.

      Then start Nagios and enable it to start when the server boots:

      • sudo systemctl start nagios

      Nagios is now running, so let's log in to its web interface.

      Step 5 — Accessing the Nagios Web Interface

      Open your favorite web browser, and go to your Nagios server by visiting http://nagios_server_public_ip/nagios.

      Enter the login credentials for the web interface in the popup that appears. Use nagiosadmin for the username, and the password you created for that user.

      After authenticating, you will see the default Nagios home page. Click on the Hosts link in the left navigation bar to see which hosts Nagios is monitoring:

      Nagios Hosts Page

      As you can see, Nagios is monitoring only "localhost", or itself.

      Let's monitor our other server with Nagios,

      Step 6 — Installing Nagios Plugins and NRPE Daemon on a Host

      Let's add a new host so Nagios can monitor it. You'll install the Nagios Remote Plugin Executor (NRPE) on the remote host, install some plugins, and then configure the Nagios server to monitor this host.

      Log in to the second server, which we'll call the second Ubuntu server:

      • ssh sammy@your_monitored_server_ip

      First create a nagios user which will run the NRPE agent:

      You'll install NRPE from source, which means you'll need the same development libraries you installed on the Nagios server in Step 1. Update your package sources and install the NRPE prerequisites:

      • sudo apt update
      • sudo apt install autoconf gcc libmcrypt-dev make libssl-dev wget dc build-essential gettext

      NRPE requires that Nagios Plugins is installed on the remote host. Let's install this package from source.

      Find the latest release of Nagios Plugins from the downloads page.

      Download Nagios Plugins to your home directory with curl:

      • cd ~
      • curl -L -O https://nagios-plugins.org/download/nagios-plugins-2.2.1.tar.gz

      Extract the Nagios Plugins archive and change to the extracted directory:

      • tar zxf nagios-plugins-2.2.1.tar.gz
      • cd nagios-plugins-2.2.1

      Before building Nagios Plugins, configure them with the following command:

      Now compile the plugins:

      Then install them by running:

      Next, install NRPE daemon. Find the download URL for the latest stable release of NRPE at the GitHub page just like you did in Step 3. Download the latest stable release of NRPE to your monitored server's home directory with curl:

      • cd ~
      • curl -L -O https://github.com/NagiosEnterprises/nrpe/releases/download/nrpe-3.2.1/nrpe-3.2.1.tar.gz

      Extract the NRPE archive with this command:

      • tar zxf nrpe-3.2.1.tar.gz

      Then change to the extracted directory:

      Configure NRPE:

      Now build and install NRPE and its startup script with these commands:

      • make nrpe
      • sudo make install-daemon
      • sudo make install-config
      • sudo make install-init

      Now, let's update the NRPE configuration file and add some basic checks that Nagios can monitor.

      First, let's monitor the disk usage of this server. Use the df -h command to look for the root filesystem. You'll use this filesystem name in the NRPE configuration:

      You'll see output similar to this:

      Output

      Filesystem Size Used Avail Use% Mounted on /dev/vda1 25G 1.4G 23G 6% /

      Now open /usr/local/nagios/etc/nrpe.cfg file in your editor:

      • sudo nano /usr/local/nagios/etc/nrpe.cfg

      The NRPE configuration file is very long and full of comments. There are a few lines that you will need to find and modify:

      • server_address: Set to the private IP address of the monitored server.
      • allowed_hosts: Add the private IP address of your Nagios server to the comma-delimited list.
      • command[check_hda1]: Change /dev/hda1 to whatever your root filesystem is called.

      Locate these settings and alter them appropriately:

      /usr/local/nagios/etc/nrpe.cfg

      ...
      server_address=second_ubuntu_server_private_ip
      ...
      allowed_hosts=127.0.0.1,::1,your_nagios_server_private_ip
      ...
      command[check_vda1]=/usr/local/nagios/libexec/check_disk -w 20% -c 10% -p /dev/vda1
      ...
      

      Save and exit the editor. Now you can start NRPE:

      • sudo systemctl start nrpe.service

      Ensure that the service is running by checking its status:

      • sudo systemctl status nrpe.service

      You'll see the following output:

      Output

      ... Aug 01 06:28:31 client systemd[1]: Started Nagios Remote Plugin Executor. Aug 01 06:28:31 client nrpe[8021]: Starting up daemon Aug 01 06:28:31 client nrpe[8021]: Server listening on 0.0.0.0 port 5666. Aug 01 06:28:31 client nrpe[8021]: Server listening on :: port 5666. Aug 01 06:28:31 client nrpe[8021]: Listening for connections on port 5666 Aug 01 06:28:31 client nrpe[8021]: Allowing connections from: 127.0.0.1,::1,165.22.212.38

      Next, allow access to port 5666 through the firewall. If you are using UFW, configure it to allow TCP connections to port 5666 with the following command:

      You can learn more about UFW in How To Set Up a Firewall with UFW on Ubuntu 18.04.

      Now you can check the communication with the remote NRPE server. Run the following command on the Nagios server:

      • /usr/local/nagios/libexec/check_nrpe -H second_ubuntu_server_ip

      You'll see the following output:

      Output

      NRPE v3.2.1

      Repeat the steps in this section for each additional server you want to monitor.

      Once you are done installing and configuring NRPE on the hosts that you want to monitor, you will have to add these hosts to your Nagios server configuration before it will start monitoring them. Let's do that next.

      Step 7 — Monitoring Hosts with Nagios

      To monitor your hosts with Nagios, you'll add configuration files for each host specifying what you want to monitor. You can then view those hosts in the Nagios web interface.

      On your Nagios server, create a new configuration file for each of the remote hosts that you want to monitor in /usr/local/nagios/etc/servers/. Replace the highlighted word, monitored_server_host_name with the name of your host:

      • sudo nano /usr/local/nagios/etc/servers/your_monitored_server_host_name.cfg

      Add the following host definition, replacing the host_name value with your remote hostname, the alias value with a description of the host, and the address value with the private IP address of the remote host:

      /usr/local/nagios/etc/servers/your_monitored_server_host_name.cfg

      define host {
              use                             linux-server
              host_name                       your_monitored_server_host_name
              alias                           My client server
              address                         your_monitored_server_private_ip
              max_check_attempts              5
              check_period                    24x7
              notification_interval           30
              notification_period             24x7
      }
      

      With this configuration, Nagios will only tell you if the host is up or down. Let's add some services to monitor.

      First, add this block to monitor load average:

      /usr/local/nagios/etc/servers/your_monitored_server_host_name.cfg

      define service {
              use                             generic-service
              host_name                       your_monitored_server_host_name
              service_description             Load average
              check_command                   check_nrpe!check_load
      }
      

      The use generic-service directive tells Nagios to inherit the values of a service template called generic-service, which is predefined by Nagios.

      Next, add this block to monitor disk usage:

      /usr/local/nagios/etc/servers/your_monitored_server_host_name.cfg

      define service {
              use                             generic-service
              host_name                       your_monitored_server_host_name
              service_description             /dev/vda1 free space
              check_command                   check_nrpe!check_vda1
      }
      

      Now save and quit. Restart the Nagios service to put any changes into effect:

      • sudo systemctl restart nagios

      After several minutes, Nagios will check the new hosts and you'll see them in the Nagios web interface. Click on the Services link in the left navigation bar to see all of your monitored hosts and services.

      Nagios Services Page

      Conclusion

      You've installed Nagios on a server and configured it to monitor load average and disk usage of at least one remote machine.

      Now that you're monitoring a host and some of its services, you can start using Nagios to monitor your mission-critical services. You can use Nagios to set up notifications for critical events. For example, you can receive an email when your disk utilization reaches a warning or critical threshold, or a notification when your main website is down. This way you can resolve the situation promptly, or even before a problem occurs.



      Source link

      How To Configure a Galera Cluster with MariaDB on Debian 10 Servers


      The author selected the Free and Open Source Fund to receive a donation as part of the Write for DOnations program.

      Introduction

      Clustering adds high availability to your database by distributing changes to different servers. In the event that one of the instances fails, others are quickly available to continue serving.

      Clusters come in two general configurations, active-passive and active-active. In active-passive clusters, all writes are done on a single active server and then copied to one or more passive servers that are poised to take over only in the event of an active server failure. Some active-passive clusters also allow SELECT operations on passive nodes. In an active-active cluster, every node is read-write and a change made to one is replicated to all.

      MariaDB is an open source relational database system that is fully compatible with the popular MySQL RDBMS system. You can read the official documentation for MariaDB at this page. Galera is a database clustering solution that enables you to set up multi-master clusters using synchronous replication. Galera automatically handles keeping the data on different nodes in sync while allowing you to send read and write queries to any of the nodes in the cluster. You can learn more about Galera at the official documentation page.

      In this guide, you will configure an active-active MariaDB Galera cluster. For demonstration purposes, you will configure and test three Debian 10 servers that will act as nodes in the cluster. This is the smallest configurable cluster.

      Prerequisites

      To follow along, you will need a DigitalOcean account, in addition to the following:

      • Three Debian 10 servers with private networking enabled, each with a non-root user with sudo privileges.

      While the steps in this tutorial have been written for and tested against DigitalOcean Droplets, much of them should also be applicable to non-DigitalOcean servers with private networking enabled.

      Step 1 — Adding the MariaDB Repositories to All Servers

      In this step, you will add the relevant MariaDB package repositories to each of your three servers so that you will be able to install the right version of MariaDB used in this tutorial. Once the repositories are updated on all three servers, you will be ready to install MariaDB.

      One thing to note about MariaDB is that it originated as a drop-in replacement for MySQL, so in many configuration files and startup scripts, you’ll see mysql rather than mariadb. For consistency’s sake, we will use mysql in this guide where either could work.

      In this tutorial, you will use MariaDB version 10.4. Since this version isn’t included in the default Debian repositories, you’ll start by adding the external Debian repository maintained by the MariaDB project to all three of your servers.

      To add the repository, you will first need to install the dirmngr and software-properties-common packages. dirmngr is a server for managing repository certificates and keys. software-properties-common is a package that allows easy addition and updates of source repository locations. Install the two packages by running:

      • sudo apt install dirmngr software-properties-common

      Note: MariaDB is a well-respected provider, but not all external repositories are reliable. Be sure to install only from trusted sources.

      You’ll add the MariaDB repository key with the apt-key command, which the APT package manager will use to verify that the package is authentic:

      • sudo apt-key adv --recv-keys --keyserver hkp://keyserver.ubuntu.com:80 0xF1656F24C74CD1D8

      Once you have the trusted key in the database, you can add the repository with the following command:

      • sudo add-apt-repository 'deb [arch=amd64] http://nyc2.mirrors.digitalocean.com/mariadb/repo/10.4/debian buster main'

      After adding the repository, run apt update in order to include package manifests from the new repository:

      Once you have completed this step on your first server, repeat for your second and third servers.

      Now that you have successfully added the package repository on all three of your servers, you're ready to install MariaDB in the next section.

      Step 2 — Installing MariaDB on All Servers

      In this step, you will install the actual MariaDB packages on your three servers.

      Beginning with version 10.1, the MariaDB Server and MariaDB Galera Server packages are combined, so installing mariadb-server will automatically install Galera and several dependencies:

      • sudo apt install mariadb-server

      You will be asked to confirm whether you would like to proceed with the installation. Enter yes to continue with the installation.

      From MariaDB version 10.4 onwards, the root MariaDB user does not have a password by default. To set a password for the root user, start by logging into MariaDB:

      Once you're inside the MariaDB shell, change the password by executing the following statement:

      • set password = password("your_password");

      You will see the following output indicating that the password was set correctly:

      Output

      Query OK, 0 rows affected (0.001 sec)

      Exit the MariaDB shell by running the following command:

      If you would like to learn more about SQL or need a quick refresher, check out our MySQL tutorial.

      You now have all of the pieces necessary to begin configuring the cluster, but since you'll be relying on rsync in later steps, make sure it's installed:

      This will confirm that the newest version of rsync is already available or prompt you to upgrade or install it.

      Once you have installed MariaDB and set the root password on your first server, repeat these steps for your other two servers.

      Now that you have installed MariaDB successfully on each of the three servers, you can proceed to the configuration step in the next section.

      Step 3 — Configuring the First Node

      In this step you will configure your first node. Each node in the cluster needs to have a nearly identical configuration. Because of this, you will do all of the configuration on your first machine, and then copy it to the other nodes.

      By default, MariaDB is configured to check the /etc/mysql/conf.d directory to get additional configuration settings from files ending in .cnf. Create a file in this directory with all of your cluster-specific directives:

      • sudo nano /etc/mysql/conf.d/galera.cnf

      Add the following configuration into the file. The configuration specifies different cluster options, details about the current server and the other servers in the cluster, and replication-related settings. Note that the IP addresses in the configuration are the private addresses of your respective servers; replace the highlighted lines with the appropriate IP addresses.

      /etc/mysql/conf.d/galera.cnf

      [mysqld]
      binlog_format=ROW
      default-storage-engine=innodb
      innodb_autoinc_lock_mode=2
      bind-address=0.0.0.0
      
      # Galera Provider Configuration
      wsrep_on=ON
      wsrep_provider=/usr/lib/galera/libgalera_smm.so
      
      # Galera Cluster Configuration
      wsrep_cluster_name="test_cluster"
      wsrep_cluster_address="gcomm://First_Node_IP,Second_Node_IP,Third_Node_IP"
      
      # Galera Synchronization Configuration
      wsrep_sst_method=rsync
      
      # Galera Node Configuration
      wsrep_node_address="This_Node_IP"
      wsrep_node_name="This_Node_Name"
      
      • The first section modifies or re-asserts MariaDB/MySQL settings that will allow the cluster to function correctly. For example, Galera won’t work with MyISAM or similar non-transactional storage engines, and mysqld must not be bound to the IP address for localhost. You can learn about the settings in more detail on the Galera Cluster system configuration page.
      • The "Galera Provider Configuration" section configures the MariaDB components that provide a WriteSet replication API. This means Galera in your case, since Galera is a wsrep (WriteSet Replication) provider. You specify the general parameters to configure the initial replication environment. This doesn't require any customization, but you can learn more about Galera configuration options.
      • The "Galera Cluster Configuration" section defines the cluster, identifying the cluster members by IP address or resolvable domain name and creating a name for the cluster to ensure that members join the correct group. You can change the wsrep_cluster_name to something more meaningful than test_cluster or leave it as-is, but you must update wsrep_cluster_address with the private IP addresses of your three servers.
      • The "Galera Synchronization Configuration" section defines how the cluster will communicate and synchronize data between members. This is used only for the state transfer that happens when a node comes online. For your initial setup, you are using rsync, because it's commonly available and does what you'll need for now.
      • The "Galera Node Configuration" section clarifies the IP address and the name of the current server. This is helpful when trying to diagnose problems in logs and for referencing each server in multiple ways. The wsrep_node_address must match the address of the machine you're on, but you can choose any name you want in order to help you identify the node in log files.

      When you are satisfied with your cluster configuration file, copy the contents into your clipboard, save and close the file. With the nano text editor, you can do this by pressing CTRL+X, typing y, and pressing ENTER.

      Now that you have configured your first node successfully, you can move on to configuring the remaining nodes in the next section.

      Step 4 — Configuring the Remaining Nodes

      In this step, you will configure the remaining two nodes. On your second node, open the configuration file:

      • sudo nano /etc/mysql/conf.d/galera.cnf

      Paste in the configuration you copied from the first node, then update the Galera Node Configuration to use the IP address or resolvable domain name for the specific node you're setting up. Finally, update its name, which you can set to whatever helps you identify the node in your log files:

      /etc/mysql/conf.d/galera.cnf

      . . .
      # Galera Node Configuration
      wsrep_node_address="This_Node_IP"
      wsrep_node_name="This_Node_Name"
      . . .
      

      Save and exit the file.

      Once you have completed these steps, repeat them on the third node.

      You're almost ready to bring up the cluster, but before you do, make sure that the appropriate ports are open in your firewall.

      Step 5 — Opening the Firewall on Every Server

      In this step, you will configure your firewall so that the ports required for inter-node communication are open. On every server, check the status of the firewall by running:

      In this case, only SSH is allowed through:

      Output

      Status: active To Action From -- ------ ---- OpenSSH ALLOW Anywhere OpenSSH (v6) ALLOW Anywhere (v6)

      Since only SSH traffic is permitted in this case, you’ll need to add rules for MySQL and Galera traffic. If you tried to start the cluster, it would fail because of firewall rules.

      Galera can make use of four ports:

      • 3306 For MySQL client connections and State Snapshot Transfer that use the mysqldump method.
      • 4567 For Galera Cluster replication traffic. Multicast replication uses both UDP transport and TCP on this port.
      • 4568 For Incremental State Transfer.
      • 4444 For all other State Snapshot Transfer.

      In this example, you’ll open all four ports while you do your setup. Once you've confirmed that replication is working, you'd want to close any ports you're not actually using and restrict traffic to just servers in the cluster.

      Open the ports with the following command:

      • sudo ufw allow 3306,4567,4568,4444/tcp
      • sudo ufw allow 4567/udp

      Note: Depending on what else is running on your servers you might want to restrict access right away. The UFW Essentials: Common Firewall Rules and Commands guide can help with this.

      After you have configured your firewall on the first node, create the same firewall settings on the second and third node.

      Now that you have configured the firewalls successfully, you're ready to start the cluster in the next step.

      Step 6 — Starting the Cluster

      In this step, you will start your MariaDB cluster. To begin, you need to stop the running MariaDB service so that you can bring your cluster online.

      Stop MariaDB on All Three Servers

      Use the following command on all three servers to stop MariaDB so that you can bring them back up in a cluster:

      • sudo systemctl stop mysql

      systemctl doesn't display the outcome of all service management commands, so to be sure you succeeded, use the following command:

      • sudo systemctl status mysql

      If the last line looks something like the following, the command was successful:

      Output

      . . . Apr 26 03:34:23 galera-node-01 systemd[1]: Stopped MariaDB 10.4.4 database server.

      Once you've shut down mysql on all of the servers, you're ready to proceed.

      Bring Up the First Node

      To bring up the first node, you'll need to use a special startup script. The way you've configured your cluster, each node that comes online tries to connect to at least one other node specified in its galera.cnf file to get its initial state. Without using the galera_new_cluster script that allows systemd to pass the --wsrep-new-cluster parameter, a normal systemctl start mysql would fail because there are no nodes running for the first node to connect with.

      This command will not display any output on successful execution. When this script succeeds, the node is registered as part of the cluster, and you can see it with the following command:

      • mysql -u root -p -e "SHOW STATUS LIKE 'wsrep_cluster_size'"

      You will see the following output indicating that there is one node in the cluster:

      Output

      +--------------------+-------+ | Variable_name | Value | +--------------------+-------+ | wsrep_cluster_size | 1 | +--------------------+-------+

      On the remaining nodes, you can start mysql normally. They will search for any member of the cluster list that is online, so when they find one, they will join the cluster.

      Bring Up the Second Node

      Now you can bring up the second node. Start mysql:

      • sudo systemctl start mysql

      No output will be displayed on successful execution. You will see your cluster size increase as each node comes online:

      • mysql -u root -p -e "SHOW STATUS LIKE 'wsrep_cluster_size'"

      You will see the following output indicating that the second node has joined the cluster and that there are two nodes in total.

      Output

      +--------------------+-------+ | Variable_name | Value | +--------------------+-------+ | wsrep_cluster_size | 2 | +--------------------+-------+

      Bring Up the Third Node

      It's now time to bring up the third node. Start mysql:

      • sudo systemctl start mysql

      Run the following command to find the cluster size:

      • mysql -u root -p -e "SHOW STATUS LIKE 'wsrep_cluster_size'"

      You will see the following output, which indicates that the third node has joined the cluster and that the total number of nodes in the cluster is three.

      Output

      +--------------------+-------+ | Variable_name | Value | +--------------------+-------+ | wsrep_cluster_size | 3 | +--------------------+-------+

      At this point, the entire cluster is online and communicating successfully. Now, you can ensure the working setup by testing replication in the next section.

      Step 7 — Testing Replication

      You've gone through the steps up to this point so that your cluster can perform replication from any node to any other node, known as active-active replication. Follow the steps below to test and see if the replication is working as expected.

      Write to the First Node

      You'll start by making database changes on your first node. The following commands will create a database called playground and a table inside of this database called equipment.

      • mysql -u root -p -e 'CREATE DATABASE playground;
      • CREATE TABLE playground.equipment ( id INT NOT NULL AUTO_INCREMENT, type VARCHAR(50), quant INT, color VARCHAR(25), PRIMARY KEY(id));
      • INSERT INTO playground.equipment (type, quant, color) VALUES ("slide", 2, "blue");'

      In the previous command, the CREATE DATABASE statement creates a database named playground. The CREATE statement creates a table named equipment inside the playground database having an auto-incrementing identifier column called id and other columns. The type column, quant column, and color column are defined to store the type, quantity, and color of the equipment respectively. The INSERT statement inserts an entry of type slide, quantity 2, and color blue.

      You now have one value in your table.

      Read and Write on the Second Node

      Next, look at the second node to verify that replication is working:

      • mysql -u root -p -e 'SELECT * FROM playground.equipment;'

      If replication is working, the data you entered on the first node will be visible here on the second:

      Output

      +----+-------+-------+-------+ | id | type | quant | color | +----+-------+-------+-------+ | 1 | slide | 2 | blue | +----+-------+-------+-------+

      From this same node, you can write data to the cluster:

      • mysql -u root -p -e 'INSERT INTO playground.equipment (type, quant, color) VALUES ("swing", 10, "yellow");'

      Read and Write on the Third Node

      From the third node, you can read all of this data by querying the table again:

      • mysql -u root -p -e 'SELECT * FROM playground.equipment;'

      You will see the following output showing the two rows:

      Output

      +----+-------+-------+--------+ | id | type | quant | color | +----+-------+-------+--------+ | 1 | slide | 2 | blue | | 2 | swing | 10 | yellow | +----+-------+-------+--------+

      Again, you can add another value from this node:

      • mysql -u root -p -e 'INSERT INTO playground.equipment (type, quant, color) VALUES ("seesaw", 3, "green");'

      Read on the First Node:

      Back on the first node, you can verify that your data is available everywhere:

      • mysql -u root -p -e 'SELECT * FROM playground.equipment;'

      You will see the following output which indicates that the rows are available on the first node.

      Output

      +----+--------+-------+--------+ | id | type | quant | color | +----+--------+-------+--------+ | 1 | slide | 2 | blue | | 2 | swing | 10 | yellow | | 3 | seesaw | 3 | green | +----+--------+-------+--------+

      You've successfully verified that you can write to all of the nodes and that replication is being performed properly.

      Conclusion

      At this point, you have a working three-node Galera test cluster configured. If you plan on using a Galera cluster in a production situation, it’s recommended that you begin with no fewer than five nodes.

      Before production use, you may want to take a look at some of the other state snapshot transfer (sst) agents like xtrabackup, which allows you to set up new nodes very quickly and without large interruptions to your active nodes. This does not affect the actual replication, but is a concern when nodes are being initialized.



      Source link

      How To Configure a Galera Cluster with MariaDB on Debian 9 Servers


      The author selected the Free and Open Source Fund to receive a donation as part of the Write for DOnations program.

      Introduction

      Clustering adds high availability to your database by distributing changes to different servers. In the event that one of the instances fails, others are quickly available to continue serving.

      Clusters come in two general configurations, active-passive and active-active. In active-passive clusters, all writes are done on a single active server and then copied to one or more passive servers that are poised to take over only in the event of an active server failure. Some active-passive clusters also allow SELECT operations on passive nodes. In an active-active cluster, every node is read-write and a change made to one is replicated to all.

      MariaDB is an open source relational database system that is fully compatible with the popular MySQL RDBMS system. You can read the official documentation for MariaDB at this page. Galera is a database clustering solution that enables you to set up multi-master clusters using synchronous replication. Galera automatically handles keeping the data on different nodes in sync while allowing you to send read and write queries to any of the nodes in the cluster. You can learn more about Galera at the official documentation page.

      In this guide, you will configure an active-active MariaDB Galera cluster. For demonstration purposes, you will configure and test three Debian 9 Droplets that will act as nodes in the cluster. This is the smallest configurable cluster.

      Prerequisites

      To follow along, you will need a DigitalOcean account, in addition to the following:

      • Three Debian 9 Droplets with private networking enabled, each with a non-root user with sudo privileges.

      While the steps in this tutorial have been written for and tested against DigitalOcean Droplets, much of them should also be applicable to non-DigitalOcean servers with private networking enabled.

      Step 1 — Adding the MariaDB Repositories to All Servers

      In this step, you will add the relevant MariaDB package repositories to each of your three servers so that you will be able to install the right version of MariaDB used in this tutorial. Once the repositories are updated on all three servers, you will be ready to install MariaDB.

      One thing to note about MariaDB is that it originated as a drop-in replacement for MySQL, so in many configuration files and startup scripts, you’ll see mysql rather than mariadb. For consistency’s sake, we will use mysql in this guide where either could work.

      In this tutorial, you will use MariaDB version 10.4. Since this version isn’t included in the default Debian repositories, you’ll start by adding the external Debian repository maintained by the MariaDB project to all three of your servers.

      To add the repository, you will first need to install the dirmngr and software-properties-common packages. dirmngr is a server for managing repository certificates and keys. software-properties-common is a package that allows easy addition and updates of source repository locations. Install the two packages by running:

      • sudo apt install dirmngr software-properties-common

      Note: MariaDB is a well-respected provider, but not all external repositories are reliable. Be sure to install only from trusted sources.

      You’ll add the MariaDB repository key with the apt-key command, which the APT package manager will use to verify that the package is authentic:

      • sudo apt-key adv --recv-keys --keyserver hkp://keyserver.ubuntu.com:80 0xF1656F24C74CD1D8

      Once you have the trusted key in the database, you can add the repository with the following command:

      • sudo add-apt-repository 'deb [arch=amd64] http://nyc2.mirrors.digitalocean.com/mariadb/repo/10.4/debian stretch main'

      After adding the repository, run apt update in order to include package manifests from the new repository:

      Once you have completed this step on your first server, repeat for your second and third servers.

      Now that you have successfully added the package repository on all three of your servers, you're ready to install MariaDB in the next section.

      Step 2 — Installing MariaDB on All Servers

      In this step, you will install the actual MariaDB packages on your three servers.

      Beginning with version 10.1, the MariaDB Server and MariaDB Galera Server packages are combined, so installing mariadb-server will automatically install Galera and several dependencies:

      • sudo apt install mariadb-server

      You will be asked to confirm whether you would like to proceed with the installation. Enter yes to continue with the installation.

      From MariaDB version 10.4 onwards, the root MariaDB user does not have a password by default. To set a password for the root user, start by logging into MariaDB:

      Once you're inside the MariaDB shell, change the password by executing the following statement:

      • set password = password("your_password");

      You will see the following output indicating that the password was set correctly:

      Output

      Query OK, 0 rows affected (0.001 sec)

      Exit the MariaDB shell by running the following command:

      If you would like to learn more about SQL or need a quick refresher, check out our MySQL tutorial.

      You now have all of the pieces necessary to begin configuring the cluster, but since you'll be relying on rsync in later steps, make sure it's installed:

      This will confirm that the newest version of rsync is already available or prompt you to upgrade or install it.

      Once you have installed MariaDB and set the root password on your first server, repeat these steps for your other two servers.

      Now that you have installed MariaDB successfully on each of the three servers, you can proceed to the configuration step in the next section.

      Step 3 — Configuring the First Node

      In this step you will configure your first node. Each node in the cluster needs to have a nearly identical configuration. Because of this, you will do all of the configuration on your first machine, and then copy it to the other nodes.

      By default, MariaDB is configured to check the /etc/mysql/conf.d directory to get additional configuration settings from files ending in .cnf. Create a file in this directory with all of your cluster-specific directives:

      • sudo nano /etc/mysql/conf.d/galera.cnf

      Add the following configuration into the file. The configuration specifies different cluster options, details about the current server and the other servers in the cluster, and replication-related settings. Note that the IP addresses in the configuration are the private addresses of your respective servers; replace the highlighted lines with the appropriate IP addresses.

      /etc/mysql/conf.d/galera.cnf

      [mysqld]
      binlog_format=ROW
      default-storage-engine=innodb
      innodb_autoinc_lock_mode=2
      bind-address=0.0.0.0
      
      # Galera Provider Configuration
      wsrep_on=ON
      wsrep_provider=/usr/lib/galera/libgalera_smm.so
      
      # Galera Cluster Configuration
      wsrep_cluster_name="test_cluster"
      wsrep_cluster_address="gcomm://First_Node_IP,Second_Node_IP,Third_Node_IP"
      
      # Galera Synchronization Configuration
      wsrep_sst_method=rsync
      
      # Galera Node Configuration
      wsrep_node_address="This_Node_IP"
      wsrep_node_name="This_Node_Name"
      
      • The first section modifies or re-asserts MariaDB/MySQL settings that will allow the cluster to function correctly. For example, Galera won’t work with MyISAM or similar non-transactional storage engines, and mysqld must not be bound to the IP address for localhost. You can learn about the settings in more detail on the Galera Cluster system configuration page.
      • The "Galera Provider Configuration" section configures the MariaDB components that provide a WriteSet replication API. This means Galera in your case, since Galera is a wsrep (WriteSet Replication) provider. You specify the general parameters to configure the initial replication environment. This doesn't require any customization, but you can learn more about Galera configuration options.
      • The "Galera Cluster Configuration" section defines the cluster, identifying the cluster members by IP address or resolvable domain name and creating a name for the cluster to ensure that members join the correct group. You can change the wsrep_cluster_name to something more meaningful than test_cluster or leave it as-is, but you must update wsrep_cluster_address with the private IP addresses of your three servers.
      • The "Galera Synchronization Configuration" section defines how the cluster will communicate and synchronize data between members. This is used only for the state transfer that happens when a node comes online. For your initial setup, you are using rsync, because it's commonly available and does what you'll need for now.
      • The "Galera Node Configuration" section clarifies the IP address and the name of the current server. This is helpful when trying to diagnose problems in logs and for referencing each server in multiple ways. The wsrep_node_address must match the address of the machine you're on, but you can choose any name you want in order to help you identify the node in log files.

      When you are satisfied with your cluster configuration file, copy the contents into your clipboard, save and close the file. With the nano text editor, you can do this by pressing CTRL+X, typing y, and pressing ENTER.

      Now that you have configured your first node successfully, you can move on to configuring the remaining nodes in the next section.

      Step 4 — Configuring the Remaining Nodes

      In this step, you will configure the remaining two nodes. On your second node, open the configuration file:

      • sudo nano /etc/mysql/conf.d/galera.cnf

      Paste in the configuration you copied from the first node, then update the Galera Node Configuration to use the IP address or resolvable domain name for the specific node you're setting up. Finally, update its name, which you can set to whatever helps you identify the node in your log files:

      /etc/mysql/conf.d/galera.cnf

      . . .
      # Galera Node Configuration
      wsrep_node_address="This_Node_IP"
      wsrep_node_name="This_Node_Name"
      . . .
      

      Save and exit the file.

      Once you have completed these steps, repeat them on the third node.

      You're almost ready to bring up the cluster, but before you do, make sure that the appropriate ports are open in your firewall.

      Step 5 — Opening the Firewall on Every Server

      In this step, you will configure your firewall so that the ports required for inter-node communication are open. On every server, check the status of the firewall by running:

      In this case, only SSH is allowed through:

      Output

      Status: active To Action From -- ------ ---- OpenSSH ALLOW Anywhere OpenSSH (v6) ALLOW Anywhere (v6)

      Since only SSH traffic is permitted in this case, you’ll need to add rules for MySQL and Galera traffic. If you tried to start the cluster, it would fail because of firewall rules.

      Galera can make use of four ports:

      • 3306 For MySQL client connections and State Snapshot Transfer that use the mysqldump method.
      • 4567 For Galera Cluster replication traffic. Multicast replication uses both UDP transport and TCP on this port.
      • 4568 For Incremental State Transfer.
      • 4444 For all other State Snapshot Transfer.

      In this example, you’ll open all four ports while you do your setup. Once you've confirmed that replication is working, you'd want to close any ports you're not actually using and restrict traffic to just servers in the cluster.

      Open the ports with the following command:

      • sudo ufw allow 3306,4567,4568,4444/tcp
      • sudo ufw allow 4567/udp

      Note: Depending on what else is running on your servers you might want to restrict access right away. The UFW Essentials: Common Firewall Rules and Commands guide can help with this.

      After you have configured your firewall on the first node, create the same firewall settings on the second and third node.

      Now that you have configured the firewalls successfully, you're ready to start the cluster in the next step.

      Step 6 — Starting the Cluster

      In this step, you will start your MariaDB cluster. To begin, you need to stop the running MariaDB service so that you can bring your cluster online.

      Stop MariaDB on All Three Servers

      Use the following command on all three servers to stop MariaDB so that you can bring them back up in a cluster:

      • sudo systemctl stop mysql

      systemctl doesn't display the outcome of all service management commands, so to be sure you succeeded, use the following command:

      • sudo systemctl status mysql

      If the last line looks something like the following, the command was successful:

      Output

      . . . Apr 26 03:34:23 galera-node-01 systemd[1]: Stopped MariaDB 10.4.4 database server.

      Once you've shut down mysql on all of the servers, you're ready to proceed.

      Bring Up the First Node

      To bring up the first node, you'll need to use a special startup script. The way you've configured your cluster, each node that comes online tries to connect to at least one other node specified in its galera.cnf file to get its initial state. Without using the galera_new_cluster script that allows systemd to pass the --wsrep-new-cluster parameter, a normal systemctl start mysql would fail because there are no nodes running for the first node to connect with.

      This command will not display any output on successful execution. When this script succeeds, the node is registered as part of the cluster, and you can see it with the following command:

      • mysql -u root -p -e "SHOW STATUS LIKE 'wsrep_cluster_size'"

      You will see the following output indicating that there is one node in the cluster:

      Output

      +--------------------+-------+ | Variable_name | Value | +--------------------+-------+ | wsrep_cluster_size | 1 | +--------------------+-------+

      On the remaining nodes, you can start mysql normally. They will search for any member of the cluster list that is online, so when they find one, they will join the cluster.

      Bring Up the Second Node

      Now you can bring up the second node. Start mysql:

      • sudo systemctl start mysql

      No output will be displayed on successful execution. You will see your cluster size increase as each node comes online:

      • mysql -u root -p -e "SHOW STATUS LIKE 'wsrep_cluster_size'"

      You will see the following output indicating that the second node has joined the cluster and that there are two nodes in total.

      Output

      +--------------------+-------+ | Variable_name | Value | +--------------------+-------+ | wsrep_cluster_size | 2 | +--------------------+-------+

      Bring Up the Third Node

      It's now time to bring up the third node. Start mysql:

      • sudo systemctl start mysql

      Run the following command to find the cluster size:

      • mysql -u root -p -e "SHOW STATUS LIKE 'wsrep_cluster_size'"

      You will see the following output, which indicates that the third node has joined the cluster and that the total number nodes in the cluster is three.

      Output

      +--------------------+-------+ | Variable_name | Value | +--------------------+-------+ | wsrep_cluster_size | 3 | +--------------------+-------+

      At this point, the entire cluster is online and communicating successfully. Next, you can ensure the working setup by testing replication in the next section.

      Step 7 — Testing Replication

      You've gone through the steps up to this point so that your cluster can perform replication from any node to any other node, known as active-active replication. Follow the steps below to test and see if the replication is working as expected.

      Write to the First Node

      You'll start by making database changes on your first node. The following commands will create a database called playground and a table inside of this database called equipment.

      • mysql -u root -p -e 'CREATE DATABASE playground;
      • CREATE TABLE playground.equipment ( id INT NOT NULL AUTO_INCREMENT, type VARCHAR(50), quant INT, color VARCHAR(25), PRIMARY KEY(id));
      • INSERT INTO playground.equipment (type, quant, color) VALUES ("slide", 2, "blue");'

      In the previous command, the CREATE DATABASE statement creates a database named playground. The CREATE statement creates a table named equipment inside the playground database having an auto-incrementing identifier column called id and other columns. The type column, quant column, and color column are defined to store the type, quantity, and color of the equipment respectively. The INSERT statement inserts an entry of type slide, quantity 2, and color blue.

      You now have one value in your table.

      Read and Write on the Second Node

      Next, look at the second node to verify that replication is working:

      • mysql -u root -p -e 'SELECT * FROM playground.equipment;'

      If replication is working, the data you entered on the first node will be visible here on the second:

      Output

      +----+-------+-------+-------+ | id | type | quant | color | +----+-------+-------+-------+ | 1 | slide | 2 | blue | +----+-------+-------+-------+

      From this same node, you can write data to the cluster:

      • mysql -u root -p -e 'INSERT INTO playground.equipment (type, quant, color) VALUES ("swing", 10, "yellow");'

      Read and Write on the Third Node

      From the third node, you can read all of this data by querying the table again:

      • mysql -u root -p -e 'SELECT * FROM playground.equipment;'

      You will see the following output showing the two rows:

      Output

      +----+-------+-------+--------+ | id | type | quant | color | +----+-------+-------+--------+ | 1 | slide | 2 | blue | | 2 | swing | 10 | yellow | +----+-------+-------+--------+

      Again, you can add another value from this node:

      • mysql -u root -p -e 'INSERT INTO playground.equipment (type, quant, color) VALUES ("seesaw", 3, "green");'

      Read on the First Node:

      Back on the first node, you can verify that your data is available everywhere:

      • mysql -u root -p -e 'SELECT * FROM playground.equipment;'

      You will see the following output that indicates the rows are available on the first node.

      Output

      +----+--------+-------+--------+ | id | type | quant | color | +----+--------+-------+--------+ | 1 | slide | 2 | blue | | 2 | swing | 10 | yellow | | 3 | seesaw | 3 | green | +----+--------+-------+--------+

      You've successfully verified that you can write to all of the nodes and that replication is being performed properly.

      Conclusion

      At this point, you have a working three-node Galera test cluster configured. If you plan on using a Galera cluster in a production situation, it’s recommended that you begin with no fewer than five nodes.

      Before production use, you may want to take a look at some of the other state snapshot transfer (sst) agents like xtrabackup, which allows you to set up new nodes very quickly and without large interruptions to your active nodes. This does not affect the actual replication, but is a concern when nodes are being initialized.



      Source link