One place for hosting & domains

      Monitor

      How To Monitor Server Health with Checkmk on Ubuntu 18.04


      The author selected the Open Internet/Free Speech Fund to receive a donation as part of the Write for DOnations program.

      Introduction

      As a systems administrator, it’s a best practice to know the current state of your infrastructure and services. Ideally, you want to notice failing disks or application downtimes before your users do. Monitoring tools like Checkmk can help administrators detect these issues and maintain healthy servers.

      Generally, monitoring software can track your servers’ hardware, uptime, and service statuses, and it can raise alerts when something goes wrong. In a very basic scenario, a monitoring system would alert you if any services go down. In a more robust one, the notifications would come shortly after any suspicious signs arose, such as increased memory usage or an abnormal amount of TCP connections.

      There are many monitoring solutions available offering varying degrees of complexity and feature sets, both free and commercial. In many cases, the installation, configuration, and management of these tools is difficult and time-consuming.

      Checkmk, however, is a monitoring solution that is both robust and simpler to install. It is a self-contained software bundle that combines Nagios (a popular and open-source alerting service) with add-ons for gathering, monitoring, and graphing data. It also comes with Checkmk’s web interface — a comprehensive tool that addresses many of Nagios’s shortcomings. It offers a user-friendly dashboard, a full-featured notification system, and a repository of easy-to-install monitoring agents for many Linux distributions. If it weren’t for Checkmk’s web interface, we would have to use different views for different tasks and it wouldn’t be possible to configure all these features without resorting to extensive file modifications.

      In this guide we will set up Checkmk on an Ubuntu 18.04 server and monitor two separate hosts. We will monitor the Ubuntu server itself as well as a separate CentOS 7 server, but we could use the same approach to add any number of additional hosts to our monitoring configuration.

      Prerequisites

      • One Ubuntu 18.04 server with a regular, non-root user with sudo privileges. You can learn how to prepare your server by following this initial server setup tutorial.
      • One CentOS 7 server with a regular, non-root user with sudo privileges. To prepare this server you can follow this initial server setup tutorial.

      Step 1 — Installing Checkmk on Ubuntu

      In order to use our monitoring site, we first must install Checkmk on the Ubuntu server. This will give us all the tools we need. Checkmk provides official ready-to-use Ubuntu package files that we can use to install the software bundle.

      First, let’s update the packages list so that we have the most recent version of the repository listings:

      To browse the packages we can go to the package listing site. Ubuntu 18.04, among others, can be selected in the page menu.

      Now download the package:

      • wget https://checkmk.com/support/1.6.0p8/check-mk-raw-1.6.0p8_0.bionic_amd64.deb

      Then install the newly downloaded package:

      • sudo apt install -y ./check-mk-raw-1.6.0p8_0.bionic_amd64.deb

      This command will install the Checkmk package along with all necessary dependencies, including the Apache web server that is used to provide web access to the monitoring interface.

      After the installation completes, we now can access the omd command. Try it out:

      This omd command will output the following:

      Output

      Usage (called as root): omd help Show general help . . . General Options: -V <version> set specific version, useful in combination with update/create omd COMMAND -h, --help show available options of COMMAND

      The omd command can manage all Checkmk instances on our server. It can start and stop all the monitoring services at once, and we will use it to create our Checkmk instance. First, however, we have to update our firewall settings to allow outside access to the default web ports.

      Step 2 — Adjusting the Firewall Settings

      Before we’ll be able to work with Checkmk, it’s necessary to allow outside access to the web server in the firewall configuration. Assuming that you followed the firewall configuration steps in the prerequisites, you’ll have a UFW firewall set up to restrict access to your server.

      During installation, Apache registers itself with UFW to provide an easy way to enable or disable access to Apache through the firewall.

      To allow access to Apache, use the following command:

      Now verify the changes:

      You’ll see that Apache is listed among the allowed services:

      Output

      Status: active To Action From -- ------ ---- OpenSSH ALLOW Anywhere Apache ALLOW Anywhere OpenSSH (v6) ALLOW Anywhere (v6) Apache (v6) ALLOW Anywhere (v6)

      This will allow us to access the Checkmk web interface.

      In the next step, we’ll create the first Checkmk monitoring instance.

      Step 3 — Creating a Checkmk Monitoring Instance

      Checkmk uses the concept of instances, or individual installations, to isolate multiple Checkmk copies on a server. In most cases, only one copy of Checkmk is enough and that’s how we will configure the software in this guide.

      First we must give our new instance a name, and we will use monitoring throughout this text. To create the instance, type:

      • sudo omd create monitoring

      The omd tool will set up everything for us automatically. The command output will look similar to the following:

      Output

      Adding /opt/omd/sites/monitoring/tmp to /etc/fstab. Creating temporary filesystem /omd/sites/monitoring/tmp...OK Restarting Apache...OK Created new site monitoring with version 1.6.0p8.cre. The site can be started with omd start monitoring. The default web UI is available at http://your_ubuntu_server/monitoring/ The admin user for the web applications is cmkadmin with password: your-default-password (It can be changed with 'htpasswd -m ~/etc/htpasswd cmkadmin' as site user.) Please do a su - monitoring for administration of this site.

      In this output the URL address, default username, and password for accessing our monitoring interface are highlighted. The instance is now created, but it still needs to be started. To start the instance, type:

      • sudo omd start monitoring

      Now all the necessary tools and services will be started at once. At the end we we’ll see an output verifying that all our services have started successfully:

      Output

      Starting mkeventd...OK Starting rrdcached...OK Starting npcd...OK Starting nagios...OK Starting apache...OK Initializing Crontab...OK

      The instance is up and running.

      To access the Checkmk instance, open http://your_ubuntu_server_ip/monitoring/ in the web browser. You will be prompted for a password. Use the default credentials printed beforehand on the screen; we will change these defaults later on.

      The Checkmk screen opens with a dashboard, which shows all our services and server statuses in lists, and it uses practical graphs resembling the Earth. Straight after installation these are empty, but we will shortly make it display statuses for our services and systems.

      Blank Checkmk dashboard

      In the next step, we will change the default password to secure the site using this interface.

      Step 4 — Changing Your Administrative Password

      During installation, Checkmk generates a random password for the cmkadmin administrative user. This password is meant to be changed upon installation, and as such it is often short and not very secure. We can change this via the web interface.

      First, open the Users page from the WATO – Configuration menu on the left. The list will show all users that currently have access to the Checkmk site. On a fresh installation it will list only two users. The first one, automation, is intended for use with automated tools; the second is the cmkadmin user we used to log in to the site.

      List of Checkmk users

      Click on the pencil icon next to the cmkadmin user to change its details, including the password.

      Edit form for Checkmk admin user

      Update the password, add an admin email, and make any other desired changes.

      After saving the changes we will be asked to log in again using our new credentials. Do so and return to the dashboard, where there is one more thing we must do to fully apply our new configuration.

      Once again open the Users page from the WATO – Configuration menu on the left. The orange button in the top left corner labeled as 1 Change tells us that we have made some changes to the configuration of Checkmk, and that we need to save and activate them. This will happen every time we change the configuration of our monitoring system, not only after editing a user’s credentials. To save and activate pending changes we have to click on this button and agree to activate the listed changes using the Activate affected option on the following screen.

      List of Checkmk users after modifications
      Activate configuration changes confirmation screen
      Successfully activated configuration changes

      After activating the changes the new user’s data is written to the configuration files and it will be used by all the system’s components. Checkmk automatically takes care of notifying individual monitoring system components, reloading them when necessary, and managing all the needed configuration files.

      The Checkmk installation is now ready for use. In the next step, we will add the first host to our monitoring system.

      Step 5 — Monitoring the First Host

      We are now ready to monitor the first host. To accomplish this, we will first install check-mk-agent on the Ubuntu server. Then, we’ll restrict access to the monitoring data using xinetd.

      The components installed with Checkmk are responsible for receiving, storing, and presenting monitoring information. They do not provide the information itself.

      To gather the actual data, we will use Checkmk agent. Designed specifically for the job, Checkmk agent is capable of monitoring all vital system components at once and reporting that information back to the Checkmk instance.

      Installing the agent

      The first host we will monitor will be your_ubuntu_server—the server on which we have installed the Checkmk instance itself.

      To begin, we must install the Checkmk agent. Packages for all major distributions, including Ubuntu, are available directly from the web interface. Open the Monitoring Agents page from the WATO – Configuration menu on the left. You will see the available agent downloads with the most popular packages under the first section labeled Packaged agents.

      List of available packaged monitoring agents

      The package check-mk-agent_1.6.0p8-1_all.deb is the one suited for Debian based distributions, including Ubuntu. Copy the download link for that package from the web browser and use that address to download the package.

      • wget http://your_ubuntu_server_ip/monitoring/check_mk/agents/check-mk-agent_1.6.0p8-1_all.deb

      After downloading, install the package:

      • apt install -y ./check-mk-agent_1.6.0p8-1_all.deb

      Now verify that the agent has been successfully installed:

      The command will output a very long text that looks like gibberish but combines all vital information about the system in one place.

      Output

      <<<check_mk>>> Version: 1.6.0p8 AgentOS: linux . . . ["monitoring"] <<<job>>> <<<local>>>

      It is the output from this command that Checkmk uses to gather status data from monitored hosts. Now, we’ll restrict access to the monitoring data with xinetd.

      Restricting Access to Monitoring Data Using xinetd

      By default, the data from check_mk_agent is served using xinetd, a mechanism that outputs data on a certain network port upon accessing it. This means that we can access the check_mk_agent by using telnet to port 6556 (the default port for Checkmk) from any other computer on the internet unless our firewall configuration disallows it.

      It is not a good security policy to publish vital information about servers to anyone on the internet. We should allow only hosts that run Checkmk and are under our supervision to access this data, so that only our monitoring system can gather it.

      If you have followed the initial server setup tutorial including the steps about setting up a firewall, then access to Checkmk agent is by default blocked. It is, however, a good practice to enforce these access restrictions directly in the service configuration and not rely only on the firewall to guard it.

      To restrict access to the agent data, we have to edit the configuration file at /etc/xinetd.d/check_mk. Open the configuration file in your favorite editor. To use nano, type:

      • sudo nano /etc/xinetd.d/check_mk

      Locate this section:

      /etc/xinetd.d/check_mk

      . . .
      # configure the IP address(es) of your Nagios server here:
      #only_from      = 127.0.0.1 10.0.20.1 10.0.20.2
      . . .
      

      The only_from setting is responsible for restricting access to certain IP addresses. Because we are now working on monitoring the same server that Checkmk is running on, it is ok to allow only localhost to connect. Uncomment and update the configuration setting to:

      /etc/xinetd.d/check_mk

      . . .
      # configure the IP address(es) of your Nagios server here:
      only_from      = 127.0.0.1
      . . .
      

      Save and exit the file.

      The xinetd daemon has to be restarted for changes to take place. Do so now:

      • sudo systemctl restart xinetd

      Now our agent is up and running and restricted to accept only local connections. We can proceed to configure monitoring for that host using Checkmk.

      Configuring Host in Checkmk Web Interface

      First, to add a new host to monitor we have to go to the Hosts menu in the WATO – Configuration menu on the left. From here click Create new host. We will be asked for some information about the host.

      Creating a new host in Checkmk

      The Hostname is the familiar name that Checkmk will use for the monitoring. It may be a fully-qualified domain name, but it is not necessary. In this example, we will name the host monitoring, just like the name of the Checkmk instance itself. Because monitoring is not resolvable to our IP address, we also have to provide the IP address of our server. And since we are monitoring the local host, the IP will simply be 127.0.0.1. Check the IPv4 Address box to enable the manual IP input and enter the value in the text field.

      The default configuration of the Data Sources section relies on Checkmk agent to provide monitoring data, which is fine. The Networking Segment setting is used to denote hosts on remote networks, which are characterized by a higher expected latency that is not a sign of malfunction. Since this is a local host, the default setting is fine as well.

      To save the host and configure which services will be monitored, click the Save & go to services button.

      List of available services to monitor

      Checkmk will do an automatic inventory. That means it will gather the output from the agent and decipher it to know what kinds of services it can monitor. All available services for monitoring will be on the list, including CPU load, memory usage, and free space on disks.

      To enable monitoring of all discovered services, we have to click the Monitor button under the Undecided services (currently not monitored) section. This will refresh the page, but now all services will be listed under the Monitored services section, informing us that they are indeed being monitored.

      As was the case when changing our user password, these new changes must be saved and activated before they go live. Press the 2 changes button and accept the changes using the Activate affected button. After that, the host monitoring will be up and running.

      Now you are ready to work with your server data. Take a look at the main dashboard using the Overview/Main Overview menu item on the left.

      Working with Monitoring Data

      Now let’s take a look at the main dashboard using the Overview/Main Overview menu item on the left:

      Monitoring dashboard with all services healthy

      The Earth sphere is now fully green and the table says that one host is up with no problems. We can see the full host list, which now consists of a single host, in the Hosts/All hosts view (using the menu on the left).

      List of hosts with all services healthy

      There we will see how many services are in good health (shown in green), how many are failing, and how many are pending to be checked. After clicking on the hostname we will be able to see the list of all services with their full statuses and their Perf-O-Meters. Perf-O-Meter shows the performance of a single service relative to what Checkmk considers to be good health.

      Details of a host service status

      All services that return graphable data display a graph icon next to their name. We can use that icon to get access to graphs associated with the service. Since the host monitoring is fresh, there is almost nothing on the graphs—but after some time the graphs will provide valuable information on how our service performance changes over time.

      Graphs depicting CPU load on the server

      When any of these services fails or recovers, the information will be shown on the dashboard. For failing services a red error will be shown, and the problem will also be visible on the Earth graph.

      Dashboard with one host having problems

      After recovery, everything will be shown in green as working properly, but the event log on the right will contain information about past failures.

      Dashboard with one host recovered after problems

      Now that we have explored the dashboard a little, let’s add a second host to our monitoring instance.

      Step 6 — Monitoring a Second CentOS Host

      Monitoring gets really useful when you have multiple hosts. We will now add a second server to our Checkmk instance, this time running CentOS 7.

      As with our Ubuntu server, installing Checkmk agent is necessary to gather monitoring data on CentOS. This time, however, we will need an rpm package from the Monitoring Agents page in the web interface, called check-mk-agent-1.6.0p8-1.noarch.rpm.

      First, however, we must install xinetd, which by default is not available on the CentOS installation. Xinetd, we will remember, is a daemon that is responsible for making the monitoring data provided by check_mk_agent available over the network.

      On your CentOS server, first install xinetd:

      • sudo yum install -y xinetd

      Now we can download and install the monitoring agent package needed for our CentOS server:

      • sudo yum install -y http://your_ubuntu_server_ip/monitoring/check_mk/agents/check-mk-agent-1.6.0p8-1.noarch.rpm

      Just like before, we can verify that the agent is working properly by executing check_mk_agent:

      The output will be similar to that from the Ubuntu server. Now we will restrict access to the agent.

      Restricting Access

      This time we will not be monitoring a local host, so xinetd must allow connections coming from the Ubuntu server, where Checkmk is installed, to gather the data. To allow that, first open your configuration file:

      • sudo vi /etc/xinetd.d/check_mk

      Here you will see the configuration for your check_mk service, specifying how Checkmk agent can be accessed through the xinetd daemon. Find the following two commented lines:

      /etc/xinetd.d/check_mk

      . . .
      # configure the IP address(es) of your Nagios server here:
      #only_from      = 127.0.0.1 10.0.20.1 10.0.20.2
      . . .
      

      Now uncomment the second line and replace the local IP addresses with your_ubuntu_server_ip:

      /etc/xinetd.d/check_mk

      . . .
      # configure the IP address(es) of your Nagios server here:
      only_from      = your_ubuntu_server_ip
      . . .
      

      Save and exit the file by typing :x and then ENTER. Restart the xinetd service using:

      • sudo systemctl restart xinetd

      We can now proceed to configure Checkmk to monitor our CentOS 7 host.

      Configuring the New Host in Checkmk

      To add additional hosts to Checkmk, we use the Hosts menu just like before. This time we will name the host centos, configure its IP address, and choose WAN (high-latency) under the Networking Segment select box, since the host is on another network. If we skipped this and left it as local, Checkmk would soon alert us that the host is down, since it would expect it to respond to agent queries much quicker than is possible over the internet.

      Creating second host configuration screen

      Click Save & go to services, which will show services available for monitoring on the CentOS server. The list will be very similar to the one from the first host. Again, this time we also must click Monitor and then activate the changes using the orange button on the top left corner.

      After activating the changes, we can verify that the host is monitored on the All hosts page. Go there. Two hosts, monitoring and centos, will now be visible.

      List of hosts with two hosts being monitored

      You are now monitoring an Ubuntu server and a CentOS server with Checkmk. It is possible to monitor even more hosts. In fact, there is no upper limit other than server performance, which should not be a problem until your hosts number in the hundreds. Moreover, the procedure is the same for any other host. Checkmk agents in deb and rpm packages work on Ubuntu, CentOS, and the majority of other Linux distributions.

      Conclusion

      In this guide we set up two servers with two different Linux distributions: Ubuntu and CentOS. We then installed and configured Checkmk to monitor both servers, and explored Checkmk’s powerful web interface.

      Checkmk allows for the easy setup of a complete and versatile monitoring system, which packs all the hard work of manual configuration into an easy-to-use web interface full of options and features. With these tools it is possible to monitor multiple hosts; set up email, SMS, or push notifications for problems; set up additional checks for more services; monitor accessibility and performance, and so on.

      To learn more about Checkmk, make sure to visit the official documentation.



      Source link

      How To Monitor Your Managed PostgreSQL Database Using Nagios Core on Ubuntu 18.04


      The author selected the Free and Open Source Fund to receive a donation as part of the Write for DOnations program.

      Introduction

      Database monitoring is key to understanding how a database performs over time. It can help you uncover hidden usage problems and bottlenecks happening in your database. Implementing database monitoring systems can quickly turn out to be a long-term advantage, which will positively influence your infrastructure management process. You’ll be able to swiftly react to status changes of your database and will quickly be notified when monitored services return to normal functioning.

      Nagios Core is a popular monitoring system that you can use to monitor your managed database. The benefits of using Nagios for this task are its versatility—it’s easy to configure and use—a large repository of available plugins, and most importantly, integrated alerting.

      In this tutorial, you will set up PostgreSQL database monitoring in Nagios Core using the check_postgres Nagios plugin and set up Slack-based alerting. In the end, you’ll have a monitoring system in place for your managed PostgreSQL database, and will be notified of status changes of various functionality immediately.

      Prerequisites

      • An Ubuntu 18.04 server with root privileges, and a secondary, non-root account. You can set this up by following this initial server setup guide. For this tutorial the non-root user is sammy.

      • Nagios Core installed on your server. To achieve this, complete the first five steps of the How To Install Nagios 4 and Monitor Your Servers on Ubuntu 18.04 tutorial.

      • A DigitalOcean account and a PostgreSQL managed database provisioned from DigitalOcean with connection information available. Make sure that your server’s IP address is on the whitelist. To learn more about DigitalOcean Managed Databases, visit the product docs.

      • A Slack account with full access, added to a workspace where you’ll want to receive status updates.

      Step 1 — Installing check_postgres

      In this section, you’ll download the latest version of the check_postgres plugin from Github and make it available to Nagios Core. You’ll also install the PostgreSQL client (psql), so that check_postgres will be able to connect to your managed database.

      Start off by installing the PostgreSQL client by running the following command:

      • sudo apt install postgresql-client

      Next, you’ll download check_postgres to your home directory. First, navigate to it:

      Head over to the Github releases page and copy the link of the latest version of the plugin. At the time of writing, the latest version of check_postgres was 2.24.0; keep in mind that this will update, and where possible it's best practice to use the latest version.

      Now download it using curl:

      • curl -LO https://github.com/bucardo/check_postgres/releases/download/2.24.0/check_postgres-2.24.0.tar.gz

      Extract it using the following command:

      • tar xvf check_postgres-*.tar.gz

      This will create a directory with the same name as the file you have downloaded. That folder contains the check_postgres executable, which you'll need to copy to the directory where Nagios stores its plugins (usually /usr/local/nagios/libexec/). Copy it by running the following command:

      • sudo cp check_postgres-*/check_postgres.pl /usr/local/nagios/libexec/

      Next, you'll need to give the nagios user ownership of it, so that it can be run from Nagios:

      • sudo chown nagios:nagios /usr/local/nagios/libexec/check_postgres.pl

      check_postgres is now available to Nagios and can be used from it. However, it provides a lot of commands pertaining to different aspects of PostgreSQL, and for better service maintainability, it's better to break them up so that they can be called separately. You'll achieve this by creating a symlink to every check_postgres command in the plugin directory.

      Navigate to the directory where Nagios stores plugins by running the following command:

      • cd /usr/local/nagios/libexec

      Then, create the symlinks with:

      • sudo perl check_postgres.pl --symlinks

      The output will look like this:

      Output

      Created "check_postgres_archive_ready" Created "check_postgres_autovac_freeze" Created "check_postgres_backends" Created "check_postgres_bloat" Created "check_postgres_checkpoint" Created "check_postgres_cluster_id" Created "check_postgres_commitratio" Created "check_postgres_connection" Created "check_postgres_custom_query" Created "check_postgres_database_size" Created "check_postgres_dbstats" Created "check_postgres_disabled_triggers" Created "check_postgres_disk_space" Created "check_postgres_fsm_pages" Created "check_postgres_fsm_relations" Created "check_postgres_hitratio" Created "check_postgres_hot_standby_delay" Created "check_postgres_index_size" Created "check_postgres_indexes_size" Created "check_postgres_last_analyze" Created "check_postgres_last_autoanalyze" Created "check_postgres_last_autovacuum" Created "check_postgres_last_vacuum" Created "check_postgres_listener" Created "check_postgres_locks" Created "check_postgres_logfile" Created "check_postgres_new_version_bc" Created "check_postgres_new_version_box" Created "check_postgres_new_version_cp" Created "check_postgres_new_version_pg" Created "check_postgres_new_version_tnm" Created "check_postgres_pgagent_jobs" Created "check_postgres_pgb_pool_cl_active" Created "check_postgres_pgb_pool_cl_waiting" Created "check_postgres_pgb_pool_maxwait" Created "check_postgres_pgb_pool_sv_active" Created "check_postgres_pgb_pool_sv_idle" Created "check_postgres_pgb_pool_sv_login" Created "check_postgres_pgb_pool_sv_tested" Created "check_postgres_pgb_pool_sv_used" Created "check_postgres_pgbouncer_backends" Created "check_postgres_pgbouncer_checksum" Created "check_postgres_prepared_txns" Created "check_postgres_query_runtime" Created "check_postgres_query_time" Created "check_postgres_relation_size" Created "check_postgres_replicate_row" Created "check_postgres_replication_slots" Created "check_postgres_same_schema" Created "check_postgres_sequence" Created "check_postgres_settings_checksum" Created "check_postgres_slony_status" Created "check_postgres_table_size" Created "check_postgres_timesync" Created "check_postgres_total_relation_size" Created "check_postgres_txn_idle" Created "check_postgres_txn_time" Created "check_postgres_txn_wraparound" Created "check_postgres_version" Created "check_postgres_wal_files"

      Perl listed all the functions it created a symlink for. These can now be executed from the command line as usual.

      You've downloaded and installed the check_postgres plugin. You have also created symlinks to all the commands of the plugin, so that they can be used individually from Nagios. In the next step, you'll create a connection service file, which check_postgres will use to connect to your managed database.

      Step 2 — Configuring Your Database

      In this section, you will create a PostgreSQL connection service file containing the connection information of your database. Then, you will test the connection data by invoking check_postgres on it.

      The connection service file is by convention called pg_service.conf, and must be located under /etc/postgresql-common/. Create it for editing with your favorite editor (for example, nano):

      • sudo nano /etc/postgresql-common/pg_service.conf

      Add the following lines, replacing the highlighted placeholders with the actual values shown in your Managed Database Control Panel under the section Connection Details:

      /etc/postgresql-common/pg_service.conf

      [managed-db]
      host=host
      port=port
      user=username
      password=password
      dbname=defaultdb
      sslmode=require
      

      The connection service file can house multiple database connection info groups. The beginning of a group is signaled by putting its name in square brackets. After that comes the connection parameters (host, port, user, password, and so on), separated by new lines, which must be given a value.

      Save and close the file when you are finished.

      You'll now test the validity of the configuration by connecting to the database via check_postgres by running the following command:

      • ./check_postgres.pl --dbservice=managed-db --action=connection

      Here, you tell check_postgres which database connection info group to use with the parameter --dbservice, and also specify that it should only try to connect to it by specifying connection as the action.

      Your output will look similar to this:

      Output

      POSTGRES_CONNECTION OK: service=managed-db version 11.4 | time=0.10s

      This means that check_postgres succeeded in connecting to the database, according to the parameters from pg_service.conf. If you get an error, double check what you have just entered in that config file.

      You've created and filled out a PostgreSQL connection service file, which works as a connection string. You have also tested the connection data by running check_postgres on it and observing the output. In the next step, you will configure Nagios to monitor various parts of your database.

      Step 3 — Creating Monitoring Services in Nagios

      Now you will configure Nagios to watch over various metrics of your database by defining a host and multiple services, which will call the check_postgres plugin and its symlinks.

      Nagios stores your custom configuration files under /usr/local/nagios/etc/objects. New files you add there must be manually enabled in the central Nagios config file, located at /usr/local/nagios/etc/nagios.cfg. You'll now define commands, a host, and multiple services, which you'll use to monitor your managed database in Nagios.

      First, create a folder under /usr/local/nagios/etc/objects to store your PostgreSQL related configuration by running the following command:

      • sudo mkdir /usr/local/nagios/etc/objects/postgresql

      You'll store Nagios commands for check_nagios in a file named commands.cfg. Create it for editing:

      • sudo nano /usr/local/nagios/etc/objects/postgresql/commands.cfg

      Add the following lines:

      /usr/local/nagios/etc/objects/postgresql/commands.cfg

      define command {
          command_name           check_postgres_connection
          command_line           /usr/local/nagios/libexec/check_postgres_connection --dbservice=$ARG1$
      }
      
      define command {
          command_name           check_postgres_database_size
          command_line           /usr/local/nagios/libexec/check_postgres_database_size --dbservice=$ARG1$ --critical='$ARG2$'
      }
      
      define command {
          command_name           check_postgres_locks
          command_line           /usr/local/nagios/libexec/check_postgres_locks --dbservice=$ARG1$
      }
      
      define command {
          command_name           check_postgres_backends
          command_line           /usr/local/nagios/libexec/check_postgres_backends --dbservice=$ARG1$
      }
      

      Save and close the file.

      In this file, you define four Nagios commands that call different parts of the check_postgres plugin (checking connectivity, getting the number of locks and connections, and the size of the whole database). They all accept an argument that is passed to the --dbservice parameter, and specify which of the databases defined in pg_service.conf to connect to.

      The check_postgres_database_size command accepts a second argument that gets passed to the --critical parameter, which specifies the point at which the database storage is becoming full. Accepted values include 1 KB for a kilobyte, 1 MB for a megabyte, and so on, up to exabytes (EB). A number without a capacity unit is treated as being expressed in bytes.

      Now that the necessary commands are defined, you'll define the host (essentially, the database) and its monitoring services in a file named services.cfg. Create it using your favorite editor:

      • sudo nano /usr/local/nagios/etc/objects/postgresql/services.cfg

      Add the following lines, replacing db_max_storage_size with a value pertaining to the available storage of your database. It is recommended to set it to 90 percent of the storage size you have allocated to it:

      /usr/local/nagios/etc/objects/postgresql/services.cfg

      define host {
            use                    linux-server
            host_name              postgres
            check_command          check_postgres_connection!managed-db
      }
      
      define service {
            use                    generic-service
            host_name              postgres
            service_description    PostgreSQL Connection
            check_command          check_postgres_connection!managed-db
            notification_options   w,u,c,r,f,s
      }
      
      define service {
            use                    generic-service
            host_name              postgres
            service_description    PostgreSQL Database Size
            check_command          check_postgres_database_size!managed-db!db_max_storage_size
            notification_options   w,u,c,r,f,s
      }
      
      define service {
            use                    generic-service
            host_name              postgres
            service_description    PostgreSQL Locks
            check_command          check_postgres_locks!managed-db
            notification_options   w,u,c,r,f,s
      }
      
      define service {
            use                    generic-service
            host_name              postgres
            service_description    PostgreSQL Backends
            check_command          check_postgres_backends!managed-db
            notification_options   w,u,c,r,f,s
      }
      

      You first define a host, so that Nagios will know what entity the services relate to. Then, you create four services, which call the commands you just defined. Each one passes managed-db as the argument, detailing that the managed-db you defined in Step 2 should be monitored.

      Regarding notification options, each service specifies that notifications should be sent out when the service state becomes WARNING, UNKNOWN, CRITICAL, OK (when it recovers from downtime), when the service starts flapping, or when scheduled downtime starts or ends. Without explicitly giving this option a value, no notifications would be sent out (to available contacts) at all, except if triggered manually.

      Save and close the file.

      Next, you'll need to explicitly tell Nagios to read config files from this new directory, by editing the general Nagios config file. Open it for editing by running the following command:

      • sudo nano /usr/local/nagios/etc/nagios.cfg

      Find this highlighted line in the file:

      /usr/local/nagios/etc/nagios.cfg

      ...
      # directive as shown below:
      
      cfg_dir=/usr/local/nagios/etc/servers
      #cfg_dir=/usr/local/nagios/etc/printers
      ...
      

      Above it, add the following highlighted line:

      /usr/local/nagios/etc/nagios.cfg

      ...
      cfg_dir=/usr/local/nagios/etc/objects/postgresql
      cfg_dir=/usr/local/nagios/etc/servers
      ...
      

      Save and close the file. This line tells Nagios to load all config files from the /usr/local/nagios/etc/objects/postgresql directory, where your configuration files are located.

      Before restarting Nagios, check the validity of the configuration by running the following command:

      • sudo /usr/local/nagios/bin/nagios -v /usr/local/nagios/etc/nagios.cfg

      The end of the output will look similar to this:

      Output

      Total Warnings: 0 Total Errors: 0 Things look okay - No serious problems were detected during the pre-flight check

      This means that Nagios found no errors in the configuration. If it shows you an error, you'll also see a hint as to what went wrong, so you'll be able to fix the error more easily.

      To make Nagios reload its configuration, restart its service by running the following command:

      • sudo systemctl restart nagios

      You can now navigate to Nagios in your browser. Once it loads, press on the Services option from the left-hand menu. You'll see the postgres host and a list of services, along with their current statuses:

      PostgreSQL Monitoring Services - Pending

      They will all soon turn to green and show an OK status. You'll see the command output under the Status Information column. You can click on the service name and see detailed information about its status and availability.

      You've added check_postgres commands, a host, and multiple services to your Nagios installation to monitor your database. You've also checked that the services are working properly by examining them via the Nagios web interface. In the next step, you will configure Slack-based alerting.

      Step 4 — Configuring Slack Alerting

      In this section, you will configure Nagios to alert you about events via Slack, by posting them into desired channels in your workspace.

      Before you start, log in to your desired workspace on Slack and create two channels where you'll want to receive status messages from Nagios: one for host, and the other one for service notifications. If you wish, you can create only one channel where you'll receive both kinds of alerts.

      Then, head over to the Nagios app in the Slack App Directory and press on Add Configuration. You'll see a page for adding the Nagios Integration.

      Slack - Add Nagios Integration

      Press on Add Nagios Integration. When the page loads, scroll down and take note of the token, because you'll need it further on.

      Slack - Integration Token

      You'll now install and configure the Slack plugin (written in Perl) for Nagios on your server. First, install the required Perl prerequisites by running the following command:

      • sudo apt install libwww-perl libcrypt-ssleay-perl -y

      Then, download the plugin to your Nagios plugin directory:

      • sudo curl https://raw.githubusercontent.com/tinyspeck/services-examples/master/nagios.pl -o slack.pl

      Make it executable by running the following command:

      Now, you'll need to edit it to connect to your workspace using the token you got from Slack. Open it for editing:

      Find the following lines in the file:

      /usr/local/nagios/libexec/slack.pl

      ...
      my $opt_domain = "foo.slack.com"; # Your team's domain
      my $opt_token = "your_token"; # The token from your Nagios services page
      ...
      

      Replace foo.slack.com with your workspace domain and your_token with your Nagios app integration token, then save and close the file. The script will now be able to send proper requests to Slack, which you'll now test by running the following command:

      • ./slack.pl -field slack_channel=#your_channel_name -field HOSTALIAS="Test Host" -field HOSTSTATE="UP" -field HOSTOUTPUT="Host is UP" -field NOTIFICATIONTYPE="RECOVERY"

      Replace your_channel_name with the name of the channel where you'll want to receive status alerts. The script will output information about the HTTP request it made to Slack, and if everything went through correctly, the last line of the output will be ok. If you get an error, double check if the Slack channel you specified exists in the workspace.

      You can now head over to your Slack workspace and select the channel you specified. You'll see a test message coming from Nagios.

      Slack - Nagios Test Message

      This confirms that you have properly configured the Slack script. You'll now move on to configuring Nagios to alert you via Slack using this script.

      You'll need to create a contact for Slack and two commands that will send messages to it. You'll store this config in a file named slack.cfg, in the same folder as the previous config files. Create it for editing by running the following command:

      • sudo nano /usr/local/nagios/etc/objects/postgresql/slack.cfg

      Add the following lines:

      /usr/local/nagios/etc/objects/postgresql/slack.cfg

      define contact {
            contact_name                             slack
            alias                                    Slack
            service_notification_period              24x7
            host_notification_period                 24x7
            service_notification_options             w,u,c,f,s,r
            host_notification_options                d,u,r,f,s
            service_notification_commands            notify-service-by-slack
            host_notification_commands               notify-host-by-slack
      }
      
      define command {
            command_name     notify-service-by-slack
            command_line     /usr/local/nagios/libexec/slack.pl -field slack_channel=#service_alerts_channel
      }
      
      define command {
            command_name     notify-host-by-slack
            command_line     /usr/local/nagios/libexec/slack.pl -field slack_channel=#host_alerts_channel
      }
      

      Here you define a contact named slack, state that it can be contacted anytime and specify which commands to use for notifying service and host related events. Those two commands are defined after it and call the script you have just configured. You'll need to replace service_alerts_channel and host_alerts_channel with the names of the channels where you want to receive service and host messages, respectively. If preferred, you can use the same channel names.

      Similarly to the service creation in the last step, setting service and host notification options on the contact is crucial, because it governs what kind of alerts the contact will receive. Omitting those options would result in sending out notifications only when manually triggered from the web interface.

      When you are done with editing, save and close the file.

      To enable alerting via the slack contact you just defined, you'll need to add it to the admin contact group, defined in the contacts.cfg config file, located under /usr/local/nagios/etc/objects/. Open it for editing by running the following command:

      • sudo nano /usr/local/nagios/etc/objects/contacts.cfg

      Find the config block that looks like this:

      /usr/local/nagios/etc/objects/contacts.cfg

      define contactgroup {
      
          contactgroup_name       admins
          alias                   Nagios Administrators
          members                 nagiosadmin
      }
      

      Add slack to the list of members, like so:

      /usr/local/nagios/etc/objects/contacts.cfg

      define contactgroup {
      
          contactgroup_name       admins
          alias                   Nagios Administrators
          members                 nagiosadmin,slack
      }
      

      Save and close the file.

      By default when running scripts, Nagios does not make host and service information available via environment variables, which is what the Slack script requires in order to send meaningful messages. To remedy this, you'll need to set the enable_environment_macros setting in nagios.cfg to 1. Open it for editing by running the following command:

      • sudo nano /usr/local/nagios/etc/nagios.cfg

      Find the line that looks like this:

      /usr/local/nagios/etc/nagios.cfg

      enable_environment_macros=0
      

      Change the value to 1, like so:

      /usr/local/nagios/etc/nagios.cfg

      enable_environment_macros=1
      

      Save and close the file.

      Test the validity of the Nagios configuration by running the following command:

      • sudo /usr/local/nagios/bin/nagios -v /usr/local/nagios/etc/nagios.cfg

      The end of the output will look like:

      Output

      Total Warnings: 0 Total Errors: 0 Things look okay - No serious problems were detected during the pre-flight check

      Proceed to restart Nagios by running the following command:

      • sudo systemctl restart nagios

      To test the Slack integration, you'll send out a custom notification via the web interface. Reload the Nagios Services status page in your browser. Press on the PostgreSQL Backends service and press on Send custom service notification on the right when the page loads.

      Nagios - Custom Service Notification

      Type in a comment of your choice and press on Commit, and then press on Done. You'll immediately receive a new message in Slack.

      Slack - Status Alert From Nagios

      You have now integrated Slack with Nagios, so you'll receive messages about critical events and status changes immediately. You've also tested the integration by manually triggering an event from within Nagios.

      Conclusion

      You now have Nagios Core configured to watch over your managed PostgreSQL database and report any status changes and events to Slack, so you'll always be in the loop of what is happening to your database. This will allow you to swiftly react in case of an emergency, because you'll be getting the status feed in real time.

      If you'd like to learn more about the features of check_postgres, check out its docs, where you'll find a lot more commands that you can possibly use.

      For more information about what you can do with your PostgreSQL Managed Database, visit the product docs.



      Source link

      How To Install Nagios 4 and Monitor Your Servers on Ubuntu 18.04


      The author selected the Open Source Initiative to receive a donation as part of the Write for DOnations program.

      Introduction

      Nagios is a popular open-source monitoring system. It keeps an inventory of your servers and monitors them so you know your critical services are up and running. Using a monitoring system like Nagios is an essential tool for any production environment, because by monitoring uptime, CPU usage, or disk space, you can head off problems before they occur, or before your users call you.

      In this tutorial, you’ll install Nagios 4 and configure it so you can monitor host resources via Nagios’ web interface. You’ll also set up the Nagios Remote Plugin Executor (NRPE), which runs as an agent on remote hosts so you can monitor their resources.

      Prerequisites

      To follow this tutorial, you will need:

      • Two Ubuntu 18.04 servers set up by following our Initial Server Setup Guide for Ubuntu 18.04, including a non-root user with sudo privileges and a firewall configured with ufw. On one server, you will install Nagios; this tutorial will refer to this as the Nagios server. It will monitor your second server; this second server will be referred to as the second Ubuntu server.
      • The server that will run the Nagios server needs Apache and PHP installed. Follow this guide to configure those on one of your servers. You can skip the MySQL steps in that tutorial.

      Typically, Nagios runs behind a hardware firewall or VPN. If your Nagios server is exposed to the public internet, you should secure the Nagios web interface by installing a TLS/SSL certificate. This is optional but strongly encouraged. You can follow the Let’s Encrypt on Ubuntu 18.04 guide to obtain the free TLS/SSL certificate.

      This tutorial assumes that your servers have private networking enabled so that monitoring happens on the private network rather than the public network. If you don’t have private networking enabled, you can still follow this tutorial by replacing all the references to private IP addresses with public IP addresses.

      Step 1 — Installing Nagios 4

      There are multiple ways to install Nagios, but you’ll install Nagios and its components from source to ensure you get the latest features, security updates, and bug fixes.

      Log in to your server that runs Apache. In this tutorial, we’ll call this the Nagios server:

      • ssh sammy@your_nagios_server_ip

      Because you’re building Nagios and its components from source, you must install a few development libraries to complete the build, including compilers, development headers, and OpenSSL.

      Update your package lists to ensure you can download the latest versions of the prerequisites:

      Then install the required packages:

      • sudo apt install autoconf gcc make unzip libgd-dev libmcrypt-dev libssl-dev dc snmp libnet-snmp-perl gettext

      With the prerequisites installed, you can install Nagios itself. Download the source code for the latest stable release of Nagios Core. Go to the Nagios downloads page, and click the Skip to download link below the form. Copy the link address for the latest stable release so you can download it to your Nagios server.

      Download the release to your home directory with the curl command:

      • cd ~
      • curl -L -O https://github.com/NagiosEnterprises/nagioscore/archive/nagios-4.4.4.tar.gz

      Extract the Nagios archive:

      • tar zxf nagios-4.4.4.tar.gz

      Then change to the extracted directory:

      • cd nagioscore-nagios-4.4.4

      Before building Nagios, run the configure script and specify the Apache configs directory:

      • ./configure --with-httpd-conf=/etc/apache2/sites-enabled

      Note: If you want Nagios to send emails using Postfix, you must install Postfix and configure Nagios to use it by adding --with-mail=/usr/sbin/sendmail to the configure command. We won't cover Postfix in this tutorial, but if you choose to use Postfix and Nagios later, you'll need to reconfigure and reinstall Nagios to use Postfix support.

      You'll see the following output from the configure command:

      Output

      *** Configuration summary for nagios 4.4.4 2019-07-29 ***: General Options: ------------------------- Nagios executable: nagios Nagios user/group: nagios,nagios Command user/group: nagios,nagios Event Broker: yes Install ${prefix}: /usr/local/nagios Install ${includedir}: /usr/local/nagios/include/nagios Lock file: /run/nagios.lock Check result directory: /usr/local/nagios/var/spool/checkresults Init directory: /lib/systemd/system Apache conf.d directory: /etc/apache2/sites-enabled Mail program: /bin/mail Host OS: linux-gnu IOBroker Method: epoll Web Interface Options: ------------------------ HTML URL: http://localhost/nagios/ CGI URL: http://localhost/nagios/cgi-bin/ Traceroute (used by WAP): Review the options above for accuracy. If they look okay, type 'make all' to compile the main program and CGIs.

      Now compile Nagios with this command:

      Next create a nagios user and nagios group. They will be used to run the Nagios process:

      • sudo make install-groups-users

      Now run these make commands to install Nagios binary files, service files, and its sample configuration files:

      • sudo make install
      • sudo make install-daemoninit
      • sudo make install-commandmode
      • sudo make install-config

      You'll use Apache to serve Nagios' web interface, so run the following to install the Apache configuration files and configure its settings:

      • sudo make install-webconf

      Enable the Apache rewrite and cgi modules with the a2enmod command:

      • sudo a2enmod rewrite
      • sudo a2enmod cgi

      In order to issue external commands via the web interface to Nagios, add the web server user, www-data, to the nagios group:

      • sudo usermod -a -G nagios www-data

      Use the htpasswd command to create an admin user called nagiosadmin that can access the Nagios web interface:

      • sudo htpasswd -c /usr/local/nagios/etc/htpasswd.users nagiosadmin

      Enter a password at the prompt. Remember this password, as you will need it to access the Nagios web interface.

      Warning: If you create a user with a name other than nagiosadmin, you will need to edit /usr/local/nagios/etc/cgi.cfg and change all the nagiosadmin references to the user you created.

      Restart Apache to load the new Apache configuration:

      • sudo systemctl restart apache2

      You've now installed Nagios. But for this to work, it is necessary to install the Nagios Plugins, which you'll cover in the next step.

      Step 2 — Installing the Nagios Plugins

      Nagios needs plugins to operate properly. The official Nagios Plugins package contains over 50 plugins that allow you to monitor basic services such as uptime, disk usage, swap usage, NTP, and others.

      Let's install the the plugins bundle.

      You can find the latest version of the Nagios Plugins on the official site.

      Download it to your home directory with curl:

      • cd ~
      • curl -L -O https://nagios-plugins.org/download/nagios-plugins-2.2.1.tar.gz

      Extract the NRPE archive and navigate into the extracted directory:

      • tar zxf nagios-plugins-<^>2.2.1<^.tar.gz
      • cd nagios-plugins-2.2.1

      Next configure their installation:

      Now build and install the plugins:

      Now the plugins are installed, but you need one more plugin for monitoring remote servers. Let's install it next.

      Step 3 — Installing the check_nrpe Plugin

      Nagios monitors remote hosts using the Nagios Remote Plugin Executor, or NRPE. It consists of two pieces:

      • The check_nrpe plugin that the Nagios server uses.
      • The NRPE daemon, which runs on the remote hosts and sends data to the Nagios server.

      Let's install the check_nrpe plugin on our Nagios server.

      Find the download URL for the latest stable release of NRPE at the GitHub page.

      Download it to your home directory with curl:

      • cd ~
      • curl -L -O https://github.com/NagiosEnterprises/nrpe/releases/download/nrpe-3.2.1/nrpe-3.2.1.tar.gz

      Extract the NRPE archive:

      • tar zxf nrpe-3.2.1.tar.gz

      Then change to the extracted directory:

      Configure the check_nrpe plugin:

      Now build and install check_nrpe plugin:

      • make check_nrpe
      • sudo make install-plugin

      Let's configure the Nagios server next.

      Step 4 — Configuring Nagios

      Now let's perform the initial Nagios configuration, which involves editing some configuration files. You only need to perform this section once on your Nagios server.

      Open the main Nagios configuration file in your preferred text editor. Here, you'll use nano:

      • sudo nano /usr/local/nagios/etc/nagios.cfg

      Find this line in the file:

      /usr/local/nagios/etc/nagios.cfg

      ...
      #cfg_dir=/usr/local/nagios/etc/servers
      ...
      

      Uncomment this line by deleting the # character from the front of the line:

      /usr/local/nagios/etc/nagios.cfg

      cfg_dir=/usr/local/nagios/etc/servers
      

      Save and close nagios.cfg by pressing CTRL+X, followed by Y, and then ENTER (if you're using nano).

      Now create the directory that will store the configuration file for each server that you will monitor:

      • sudo mkdir /usr/local/nagios/etc/servers

      Open the Nagios contacts configuration in your text editor:

      • sudo nano /usr/local/nagios/etc/objects/contacts.cfg

      Find the email directive and replace its value with your own email address:

      /usr/local/nagios/etc/objects/contacts.cfg

      ...
      define contact{
              contact_name                    nagiosadmin             ; Short name of user
              use                             generic-contact         ; Inherit default values from generic-contact template (defined above)
              alias                           Nagios Admin            ; Full name of user
              email                           your_email@your_domain.com        ; <<***** CHANGE THIS TO YOUR EMAIL ADDRESS ******
      ...
      
      

      Save and exit the editor.

      Next, add a new command to your Nagios configuration that lets you use the check_nrpe command in Nagios service definitions. Open the file /usr/local/nagios/etc/objects/commands.cfg in your editor:

      • sudo nano /usr/local/nagios/etc/objects/commands.cfg

      Add the following to the end of the file to define a new command called check_nrpe:

      /usr/local/nagios/etc/objects/commands.cfg

      ...
      define command{
              command_name check_nrpe
              command_line $USER1$/check_nrpe -H $HOSTADDRESS$ -c $ARG1$
      }
      

      This defines the name and specifies the command-line options to execute the plugin.

      Save and exit the editor.

      Then start Nagios and enable it to start when the server boots:

      • sudo systemctl start nagios

      Nagios is now running, so let's log in to its web interface.

      Step 5 — Accessing the Nagios Web Interface

      Open your favorite web browser, and go to your Nagios server by visiting http://nagios_server_public_ip/nagios.

      Enter the login credentials for the web interface in the popup that appears. Use nagiosadmin for the username, and the password you created for that user.

      After authenticating, you will see the default Nagios home page. Click on the Hosts link in the left navigation bar to see which hosts Nagios is monitoring:

      Nagios Hosts Page

      As you can see, Nagios is monitoring only "localhost", or itself.

      Let's monitor our other server with Nagios,

      Step 6 — Installing Nagios Plugins and NRPE Daemon on a Host

      Let's add a new host so Nagios can monitor it. You'll install the Nagios Remote Plugin Executor (NRPE) on the remote host, install some plugins, and then configure the Nagios server to monitor this host.

      Log in to the second server, which we'll call the second Ubuntu server:

      • ssh sammy@your_monitored_server_ip

      First create a nagios user which will run the NRPE agent:

      You'll install NRPE from source, which means you'll need the same development libraries you installed on the Nagios server in Step 1. Update your package sources and install the NRPE prerequisites:

      • sudo apt update
      • sudo apt install autoconf gcc libmcrypt-dev make libssl-dev wget dc build-essential gettext

      NRPE requires that Nagios Plugins is installed on the remote host. Let's install this package from source.

      Find the latest release of Nagios Plugins from the downloads page.

      Download Nagios Plugins to your home directory with curl:

      • cd ~
      • curl -L -O https://nagios-plugins.org/download/nagios-plugins-2.2.1.tar.gz

      Extract the Nagios Plugins archive and change to the extracted directory:

      • tar zxf nagios-plugins-2.2.1.tar.gz
      • cd nagios-plugins-2.2.1

      Before building Nagios Plugins, configure them with the following command:

      Now compile the plugins:

      Then install them by running:

      Next, install NRPE daemon. Find the download URL for the latest stable release of NRPE at the GitHub page just like you did in Step 3. Download the latest stable release of NRPE to your monitored server's home directory with curl:

      • cd ~
      • curl -L -O https://github.com/NagiosEnterprises/nrpe/releases/download/nrpe-3.2.1/nrpe-3.2.1.tar.gz

      Extract the NRPE archive with this command:

      • tar zxf nrpe-3.2.1.tar.gz

      Then change to the extracted directory:

      Configure NRPE:

      Now build and install NRPE and its startup script with these commands:

      • make nrpe
      • sudo make install-daemon
      • sudo make install-config
      • sudo make install-init

      Now, let's update the NRPE configuration file and add some basic checks that Nagios can monitor.

      First, let's monitor the disk usage of this server. Use the df -h command to look for the root filesystem. You'll use this filesystem name in the NRPE configuration:

      You'll see output similar to this:

      Output

      Filesystem Size Used Avail Use% Mounted on /dev/vda1 25G 1.4G 23G 6% /

      Now open /usr/local/nagios/etc/nrpe.cfg file in your editor:

      • sudo nano /usr/local/nagios/etc/nrpe.cfg

      The NRPE configuration file is very long and full of comments. There are a few lines that you will need to find and modify:

      • server_address: Set to the private IP address of the monitored server.
      • allowed_hosts: Add the private IP address of your Nagios server to the comma-delimited list.
      • command[check_hda1]: Change /dev/hda1 to whatever your root filesystem is called.

      Locate these settings and alter them appropriately:

      /usr/local/nagios/etc/nrpe.cfg

      ...
      server_address=second_ubuntu_server_private_ip
      ...
      allowed_hosts=127.0.0.1,::1,your_nagios_server_private_ip
      ...
      command[check_vda1]=/usr/local/nagios/libexec/check_disk -w 20% -c 10% -p /dev/vda1
      ...
      

      Save and exit the editor. Now you can start NRPE:

      • sudo systemctl start nrpe.service

      Ensure that the service is running by checking its status:

      • sudo systemctl status nrpe.service

      You'll see the following output:

      Output

      ... Aug 01 06:28:31 client systemd[1]: Started Nagios Remote Plugin Executor. Aug 01 06:28:31 client nrpe[8021]: Starting up daemon Aug 01 06:28:31 client nrpe[8021]: Server listening on 0.0.0.0 port 5666. Aug 01 06:28:31 client nrpe[8021]: Server listening on :: port 5666. Aug 01 06:28:31 client nrpe[8021]: Listening for connections on port 5666 Aug 01 06:28:31 client nrpe[8021]: Allowing connections from: 127.0.0.1,::1,165.22.212.38

      Next, allow access to port 5666 through the firewall. If you are using UFW, configure it to allow TCP connections to port 5666 with the following command:

      You can learn more about UFW in How To Set Up a Firewall with UFW on Ubuntu 18.04.

      Now you can check the communication with the remote NRPE server. Run the following command on the Nagios server:

      • /usr/local/nagios/libexec/check_nrpe -H second_ubuntu_server_ip

      You'll see the following output:

      Output

      NRPE v3.2.1

      Repeat the steps in this section for each additional server you want to monitor.

      Once you are done installing and configuring NRPE on the hosts that you want to monitor, you will have to add these hosts to your Nagios server configuration before it will start monitoring them. Let's do that next.

      Step 7 — Monitoring Hosts with Nagios

      To monitor your hosts with Nagios, you'll add configuration files for each host specifying what you want to monitor. You can then view those hosts in the Nagios web interface.

      On your Nagios server, create a new configuration file for each of the remote hosts that you want to monitor in /usr/local/nagios/etc/servers/. Replace the highlighted word, monitored_server_host_name with the name of your host:

      • sudo nano /usr/local/nagios/etc/servers/your_monitored_server_host_name.cfg

      Add the following host definition, replacing the host_name value with your remote hostname, the alias value with a description of the host, and the address value with the private IP address of the remote host:

      /usr/local/nagios/etc/servers/your_monitored_server_host_name.cfg

      define host {
              use                             linux-server
              host_name                       your_monitored_server_host_name
              alias                           My client server
              address                         your_monitored_server_private_ip
              max_check_attempts              5
              check_period                    24x7
              notification_interval           30
              notification_period             24x7
      }
      

      With this configuration, Nagios will only tell you if the host is up or down. Let's add some services to monitor.

      First, add this block to monitor load average:

      /usr/local/nagios/etc/servers/your_monitored_server_host_name.cfg

      define service {
              use                             generic-service
              host_name                       your_monitored_server_host_name
              service_description             Load average
              check_command                   check_nrpe!check_load
      }
      

      The use generic-service directive tells Nagios to inherit the values of a service template called generic-service, which is predefined by Nagios.

      Next, add this block to monitor disk usage:

      /usr/local/nagios/etc/servers/your_monitored_server_host_name.cfg

      define service {
              use                             generic-service
              host_name                       your_monitored_server_host_name
              service_description             /dev/vda1 free space
              check_command                   check_nrpe!check_vda1
      }
      

      Now save and quit. Restart the Nagios service to put any changes into effect:

      • sudo systemctl restart nagios

      After several minutes, Nagios will check the new hosts and you'll see them in the Nagios web interface. Click on the Services link in the left navigation bar to see all of your monitored hosts and services.

      Nagios Services Page

      Conclusion

      You've installed Nagios on a server and configured it to monitor load average and disk usage of at least one remote machine.

      Now that you're monitoring a host and some of its services, you can start using Nagios to monitor your mission-critical services. You can use Nagios to set up notifications for critical events. For example, you can receive an email when your disk utilization reaches a warning or critical threshold, or a notification when your main website is down. This way you can resolve the situation promptly, or even before a problem occurs.



      Source link