One place for hosting & domains

      Ubuntu

      How To Install and Use ClickHouse on Ubuntu 20.04


      The author selected the Free and Open Source Fund to receive a donation as part of the Write for DOnations program.

      Introduction

      ClickHouse is an open source, column-oriented analytics database created by Yandex for OLAP and big data use cases. ClickHouse’s support for real-time query processing makes it suitable for applications that require sub-second analytical results. ClickHouse’s query language is a dialect of SQL that enables powerful declarative querying capabilities while offering familiarity and a smaller learning curve for the end user.

      Column-oriented databases store records in blocks grouped by columns instead of rows. By not loading data for columns absent in the query, column-oriented databases spend less time reading data while completing queries. As a result, these databases can compute and return results much faster than traditional row-based systems for certain workloads, such as OLAP.

      Online Analytics Processing (OLAP) systems allow for organizing large amounts of data and performing complex queries. They are capable of managing petabytes of data and returning query results quickly. In this way, OLAP is useful for work in areas like data science and business analytics.

      In this tutorial, you’ll install the ClickHouse database server and client on your machine. You’ll use the DBMS for typical tasks and optionally enable remote access from another server so that you’ll be able to connect to the database from another machine. Then you’ll test ClickHouse by modeling and querying example website-visit data.

      Prerequisites

      • One Ubuntu 20.04 server with a sudo enabled non-root user and firewall setup. The server should have at least 2GB of RAM. You can follow the Initial Server Setup tutorial to create the user and set up the firewall.

      • (Optional) A secondary Ubuntu 20.04 server with a sudo enabled non-root user and firewall setup.

      Step 1 — Installing ClickHouse

      In this section, you will install the ClickHouse server and client programs using apt.

      First, SSH into your server by running:

      Yandex maintains an APT repository that has the latest version of ClickHouse. Add the repository’s GPG key so that you’ll be able to securely download validated ClickHouse packages:

      • sudo apt-key adv --keyserver keyserver.ubuntu.com --recv E0C56BD4

      You will see output similar to the following:

      Output

      Executing: /tmp/apt-key-gpghome.JkkcKnBAFY/gpg.1.sh --keyserver keyserver.ubuntu.com --recv E0C56BD4 gpg: key C8F1E19FE0C56BD4: public key "ClickHouse Repository Key <milovidov@yandex-team.ru>" imported gpg: Total number processed: 1 gpg: imported: 1

      The output confirms it has successfully verified and added the key.

      Add the repository to your APT repositories list by executing:

      • echo "deb http://repo.yandex.ru/clickhouse/deb/stable/ main/" | sudo tee /etc/apt/sources.list.d/clickhouse.list

      Here you’ve piped the output of echo to sudo tee so that this output can print to a root-owned file.

      Now, update your packages:

      The clickhouse-server and clickhouse-client packages will now be available for installation. Install them with:

      • sudo apt install clickhouse-server clickhouse-client

      During the installation, you will be asked to set a password for the default ClickHouse user.

      You’ve installed the ClickHouse server and client successfully. You’re now ready to start the database service and ensure that it’s running correctly.

      Step 2 — Starting the Service

      The clickhouse-server package that you installed in the previous section creates a systemd service, which performs actions such as starting, stopping, and restarting the database server. systemd is an init system for Linux to initialize and manage services. In this section you’ll start the service and verify that it is running successfully.

      Start the clickhouse-server service by running:

      • sudo service clickhouse-server start

      The previous command will not display any output. To verify that the service is running successfully, execute:

      • sudo service clickhouse-server status

      You’ll see output similar to the following:

      Output

      ● clickhouse-server.service - ClickHouse Server (analytic DBMS for big data) Loaded: loaded (/etc/systemd/system/clickhouse-server.service; enabled; vendor preset: enabled) Active: active (running) since Wed 2020-09-16 05:18:54 UTC; 5s ago Main PID: 2697 (clickhouse-serv) Tasks: 46 (limit: 1137) Memory: 459.7M CGroup: /system.slice/clickhouse-server.service └─2697 /usr/bin/clickhouse-server --config=/etc/clickhouse-server/config.xml --pid-file=/run/clickhouse-server/clickhouse-server.pid

      The output notes that the server is running.

      You have successfully started the ClickHouse server and will now be able to use the clickhouse-client CLI program to connect to the server.

      Step 3 — Creating Databases and Tables

      In ClickHouse, you can create and delete databases by executing SQL statements directly in the interactive database prompt. Statements consist of commands following a particular syntax that tell the database server to perform a requested operation along with any data required. You create databases by using the CREATE DATABASE table_name syntax. To create a database, first start a client session by running the following command:

      • clickhouse-client --password

      You will be asked to enter the password you had set during the installation—enter it to successfully to start the client session.

      The previous command will log you in to the client prompt where you can run ClickHouse SQL statements to perform actions such as:

      • Creating, updating, and deleting databases, tables, indexes, partitions, and views.

      • Executing queries to retrieve data that is optionally filtered and grouped using various conditions.

      In this step, with the ClickHouse client ready for inserting data, you’re going to create a database and table. For the purposes of this tutorial, you’ll create a database named test, and inside that you’ll create a table named visits that tracks website-visit durations.

      Now that you’re inside the ClickHouse command prompt, create your test database by executing:

      You’ll see the following output that shows that you have created the database:

      Output

      CREATE DATABASE test Ok. 0 rows in set. Elapsed: 0.003 sec.

      A ClickHouse table is similar to tables in other relational databases; it holds a collection of related data in a structured format. You can specify columns along with their types, add rows of data, and execute different kinds of queries on tables.

      The syntax for creating tables in ClickHouse follows this example structure:

      CREATE TABLE table_name
      (
          column_name1 column_type [options],
          column_name2 column_type [options],
          ...
      ) ENGINE = engine
      

      The table_name and column_name values can be any valid ASCII identifiers. ClickHouse supports a wide range of column types; some of the most popular are:

      • UInt64: used for storing integer values in the range 0 to 18446744073709551615.

      • Float64: used for storing floating point numbers such as 2039.23, 10.5, etc.

      • String: used for storing variable length characters. It does not require a max-length attribute since it can store arbitrary lengths.

      • Date: used for storing dates that follow the YYYY-MM-DD format.

      • DateTime: used for storing dates coupled with time and follows the YYYY-MM-DD HH:MM:SS format.

      After the column definitions, you specify the engine used for the table. In ClickHouse, Engines determine the physical structure of the underlying data, the table’s querying capabilities, its concurrent access modes, and support for indexes. Different engine types are suitable for different application requirements. The most commonly used and widely applicable engine type is MergeTree.

      Now that you have an overview of table creation, you’ll create a table. Start by confirming the database you’ll be modifying:

      You will see the following output showing that you have switched to the test database from the default database:

      Output

      USE test Ok. 0 rows in set. Elapsed: 0.001 sec.

      The remainder of this guide will assume that you are executing statements within this database’s context.

      Create your visits table by running this command:

      • CREATE TABLE visits (
      • id UInt64,
      • duration Float64,
      • url String,
      • created DateTime
      • ) ENGINE = MergeTree()
      • PRIMARY KEY id
      • ORDER BY id;

      Here’s a breakdown of what the command does. You create a table named visits that has four columns:

      • id: The primary key column. Similarly to other RDBMS systems, a primary key column in ClickHouse uniquely identifies a row; each row should have a unique value for this column.

      • duration: A float column used to store the duration of each visit in seconds. float columns can store decimal values such as 12.50.

      • url: A string column that stores the URL visited, such as http://example.com.

      • created: A date and time column that tracks when the visit occurred.

      After the column definitions, you specify MergeTree as the storage engine for the table. The MergeTree family of engines is recommended for production databases due to its optimized support for large real-time inserts, overall robustness, and query support. Additionally, MergeTree engines support sorting of rows by primary key, partitioning of rows, and replicating and sampling data.

      If you intend to use ClickHouse for archiving data that is not queried often or for storing temporary data, you can use the Log family of engines to optimize for that use-case.

      After the column definitions, you’ll define other table-level options. The PRIMARY KEY clause sets id as the primary key column and the ORDER BY clause will store values sorted by the id column. A primary key uniquely identifies a row and is used for efficiently accessing a single row and efficient colocation of rows.

      On executing the create statement, you will see the following output:

      Output

      CREATE TABLE visits ( id UInt64, duration Float64, url String, created DateTime ) ENGINE = MergeTree() PRIMARY KEY id ORDER BY id Ok. 0 rows in set. Elapsed: 0.010 sec.

      In this section, you’ve created a database and a table to track website-visit data. In the next step, you’ll insert data into the table, update existing data, and delete that data.

      Step 4 — Inserting, Updating, and Deleting Data and Columns

      In this step, you’ll use your visits table to insert, update, and delete data. The following command is an example of the syntax for inserting rows into a ClickHouse table:

      INSERT INTO table_name VALUES (column_1_value, column_2_value, ....);
      

      Now, insert a few rows of example website-visit data into your visits table by running each of the following statements:

      • INSERT INTO visits VALUES (1, 10.5, 'http://example.com', '2019-01-01 00:01:01');
      • INSERT INTO visits VALUES (2, 40.2, 'http://example1.com', '2019-01-03 10:01:01');
      • INSERT INTO visits VALUES (3, 13, 'http://example2.com', '2019-01-03 12:01:01');
      • INSERT INTO visits VALUES (4, 2, 'http://example3.com', '2019-01-04 02:01:01');

      You’ll see the following output repeated for each insert statement.

      Output

      INSERT INTO visits VALUES Ok. 1 rows in set. Elapsed: 0.004 sec.

      The output for each row shows that you’ve inserted it successfully into the visits table.

      Now you’ll add an additional column to the visits table. When adding or deleting columns from existing tables, ClickHouse supports the ALTER syntax.

      For example, the basic syntax for adding a column to a table is as follows:

      ALTER TABLE table_name ADD COLUMN column_name column_type;
      

      Add a column named location that will store the location of the visits to a website by running the following statement:

      • ALTER TABLE visits ADD COLUMN location String;

      You’ll see output similar to the following:

      Output

      ALTER TABLE visits ADD COLUMN location String Ok. 0 rows in set. Elapsed: 0.014 sec.

      The output shows that you have added the location column successfully.

      As of version 19.13.3, ClickHouse doesn’t support updating and deleting individual rows of data due to implementation constraints. ClickHouse has support for bulk updates and deletes, however, and has a distinct SQL syntax for these operations to highlight their non-standard usage.

      The following syntax is an example for bulk updating rows:

      ALTER TABLE table_name UPDATE  column_1 = value_1, column_2 = value_2 ...  WHERE  filter_conditions;
      

      You’ll run the following statement to update the url column of all rows that have a duration of less than 15. Enter it into the database prompt to execute:

      • ALTER TABLE visits UPDATE url="http://example2.com" WHERE duration < 15;

      The output of the bulk update statement will be as follows:

      Output

      ALTER TABLE visits UPDATE url="http://example2.com" WHERE duration < 15 Ok. 0 rows in set. Elapsed: 0.003 sec.

      The output shows that your update query completed successfully. The 0 rows in set in the output denotes that the query did not return any rows; this will be the case for any update and delete queries.

      The example syntax for bulk deleting rows is similar to updating rows and has the following structure:

      ALTER TABLE table_name DELETE WHERE filter_conditions;
      

      To test deleting data, run the following statement to remove all rows that have a duration of less than 5:

      • ALTER TABLE visits DELETE WHERE duration < 5;

      The output of the bulk delete statement will be similar to:

      Output

      ALTER TABLE visits DELETE WHERE duration < 5 Ok. 0 rows in set. Elapsed: 0.003 sec.

      The output confirms that you have deleted the rows with a duration of less than five seconds.

      To delete columns from your table, the syntax would follow this example structure:

      ALTER TABLE table_name DROP COLUMN column_name;
      

      Delete the location column you added previously by running the following:

      • ALTER TABLE visits DROP COLUMN location;

      The DROP COLUMN output confirming that you have deleted the column will be as follows:

      Output

      ALTER TABLE visits DROP COLUMN location String Ok. 0 rows in set. Elapsed: 0.010 sec.

      Now that you’ve successfully inserted, updated, and deleted rows and columns in your visits table, you’ll move on to query data in the next step.

      Step 5 — Querying Data

      ClickHouse’s query language is a custom dialect of SQL with extensions and functions suited for analytics workloads. In this step, you’ll run selection and aggregation queries to retrieve data and results from your visits table.

      Selection queries allow you to retrieve rows and columns of data filtered by conditions that you specify, along with options such as the number of rows to return. You can select rows and columns of data using the SELECT syntax. The basic syntax for SELECT queries is:

      SELECT func_1(column_1), func_2(column_2) FROM table_name WHERE filter_conditions row_options;
      

      Execute the following statement to retrieve url and duration values for rows where the url is http://example.com.

      • SELECT url, duration FROM visits WHERE url="http://example2.com" LIMIT 2;

      You will see the following output:

      Output

      SELECT url, duration FROM visits WHERE url="http://example2.com" LIMIT 2 ┌─url─────────────────┬─duration─┐ │ http://example2.com │ 10.5 │ └─────────────────────┴──────────┘ ┌─url─────────────────┬─duration─┐ │ http://example2.com │ 13 │ └─────────────────────┴──────────┘ 2 rows in set. Elapsed: 0.013 sec.

      The output has returned two rows that match the conditions you specified. Now that you’ve selected values, you can move on to executing aggregation queries.

      Aggregation queries are queries that operate on a set of values and return single output values. In analytics databases, these queries are run frequently and are well optimized by the database. Some aggregate functions supported by ClickHouse are:

      • count: returns the count of rows matching the conditions specified.

      • sum: returns the sum of selected column values.

      • avg: returns the average of selected column values.

      Some ClickHouse-specific aggregate functions include:

      • uniq: returns an approximate number of distinct rows matched.

      • topK: returns an array of the most frequent values of a specific column using an approximation algorithm.

      To demonstrate the execution of aggregation queries, you’ll calculate the total duration of visits by running the sum query:

      • SELECT SUM(duration) FROM visits;

      You will see output similar to the following:

      Output

      SELECT SUM(duration) FROM visits ┌─SUM(duration)─┐ │ 63.7 │ └───────────────┘ 1 rows in set. Elapsed: 0.010 sec.

      Now, calculate the top two URLs by executing:

      • SELECT topK(2)(url) FROM visits;

      You will see output similar to the following:

      Output

      SELECT topK(2)(url) FROM visits ┌─topK(2)(url)──────────────────────────────────┐ │ ['http://example2.com','http://example1.com'] │ └───────────────────────────────────────────────┘ 1 rows in set. Elapsed: 0.010 sec.

      Now that you have successfully queried your visits table, you’ll delete tables and databases in the next step.

      Step 6 — Deleting Tables and Databases

      In this section, you’ll delete your visits table and test database.

      The syntax for deleting tables follows this example:

      DROP TABLE table_name;
      

      To delete the visits table, run the following statement:

      You will see the following output declaring that you’ve deleted the table successfully:

      output

      DROP TABLE visits Ok. 0 rows in set. Elapsed: 0.005 sec.

      You can delete databases using the DROP database table_name syntax. To delete the test database, execute the following statement:

      The resulting output shows that you’ve deleted the database successfully.

      Output

      DROP DATABASE test Ok. 0 rows in set. Elapsed: 0.003 sec.

      You’ve deleted tables and databases in this step. Now that you’ve created, updated, and deleted databases, tables, and data in your ClickHouse instance, you’ll enable remote access to your database server in the next section.

      Step 7 — Setting Up Firewall Rules (Optional)

      If you intend to only use ClickHouse locally with applications running on the same server, or do not have a firewall enabled on your server, you don’t need to complete this section. If instead, you’ll be connecting to the ClickHouse database server remotely, you should follow this step.

      Currently your server has a firewall enabled that disables your public IP address accessing all ports. You’ll complete the following two steps to allow remote access:

      • Modify ClickHouse’s configuration and allow it to listen on all interfaces.

      • Add a firewall rule allowing incoming connections to port 8123, which is the HTTP port that the ClickHouse server runs.

      If you are inside the database prompt, exit it by typing CTRL+D.

      Edit the configuration file by executing:

      • sudo nano /etc/clickhouse-server/config.xml

      Then uncomment the line containing <!-- <listen_host>0.0.0.0</listen_host> -->, like the following file:

      /etc/clickhouse-server/config.xml

      ...
       <interserver_http_host>example.yandex.ru</interserver_http_host>
          -->
      
          <!-- Listen specified host. use :: (wildcard IPv6 address), if you want to accept connections both with IPv4 and IPv6 from everywhere. -->
          <!-- <listen_host>::</listen_host> -->
          <!-- Same for hosts with disabled ipv6: -->
          <listen_host>0.0.0.0</listen_host>
      
          <!-- Default values - try listen localhost on ipv4 and ipv6: -->
          <!--
          <listen_host>::1</listen_host>
          <listen_host>127.0.0.1</listen_host>
          -->
      ...
      

      Save the file and exit. For the new configuration to apply restart the service by running:

      • sudo service clickhouse-server restart

      You won’t see any output from this command. ClickHouse’s server listens on port 8123 for HTTP connections and port 9000 for connections from clickhouse-client. Allow access to both ports for your second server’s IP address with the following command:

      • sudo ufw allow from second_server_ip/32 to any port 8123
      • sudo ufw allow from second_server_ip/32 to any port 9000

      You will see the following output for both commands that shows that you’ve enabled access to both ports:

      Output

      Rule added

      ClickHouse will now be accessible from the IP that you added. Feel free to add additional IPs such as your local machine’s address if required.

      To verify that you can connect to the ClickHouse server from the remote machine, first follow the steps in Step 1 of this tutorial on the second server and ensure that you have the clickhouse-client installed on it.

      Now that you have logged in to the second server, start a client session by executing:

      • clickhouse-client --host your_server_ip --password

      You will see the following output that shows that you have connected successfully to the server:

      Output

      ClickHouse client version 19.13.3.26 (official build). Password for user (default): Connecting to your_server_ip:9000 as user default. Connected to ClickHouse server version 19.13.3 revision 54425. hostname :)

      In this step, you’ve enabled remote access to your ClickHouse database server by adjusting your firewall rules.

      Conclusion

      You have successfully set up a ClickHouse database instance on your server and created a database and table, added data, performed queries, and deleted the database. Within ClickHouse’s documentation you can read about their benchmarks against other open-source and commercial analytics databases and general reference documents.

      Further features ClickHouse offers include distributed query processing across multiple servers to improve performance and protect against data loss by storing data over different shards.



      Source link

      How To Install and Configure Mahara on Ubuntu 18.04


      The author selected the COVID-19 Relief Fund to receive a donation as part of the Write for DOnations program.

      Introduction

      Mahara is an open-source ePortfolio and social networking system that bills itself as a “learner-centered personal learning environment.” Mahara is especially popular in academic institutions where it provides learners a platform to curate and share their knowledge and growth. In a remote learning environment, Mahara can help educators foster a digital classroom and organize a student’s progress. Unlike more a traditional Learning Management Systems (LMS), Mahara is more distributed and user-focused. This amount of user creativity contrasts other, more institutional systems.

      Mahara also has many non-student applications. You can use it to build a blog, a resume-builder, a file-repository, or a competency framework.

      In this guide, you will install Mahara and configure it to use a Postgres database as an authentication source. Upon completion, you will have administrative access to a server-based ePortfolio.

      Prerequisites

      Step 1 — Creating a Database and User for Mahara

      Your first step is to create a Postgres database and user for Mahara.

      From your server’s command line, switch to the postgres user:

      The -i switch provides an interactive shell, while sudo used with the -u switch allows you to switch users.

      Now create a Postgres user; make sure to replace mahara with your preferred username:

      Postgres will prompt you to Enter password for a new role:. role here refers to the new user you are trying to create. Provide a password of your choosing, and then reenter the password when prompted.

      Now create a database for your Mahara installation. Use the following command, replacing maharadb with a preferred name for your new database and mahara with the new user you just created:

      • createdb -O mahara -EUTF8 maharadb

      You can now exit from the postgres user:

      Note: It is recommended to use Postgres for your Mahara database, but if you decide to use MySQL, you’ll need to set up a new user and database for Mahara. To do so, log in to your MySQL root account using the following command:

      Provide your MySQL root password to gain access. Now create your Mahara database. Replace maharadb with another name:

      • create database maharadb character set UTF8;

      Next, create a new MySQL user for Mahara, set a password, and grant it access to your maharadb database. Remember to provide a strong password in place of password and replace mahara with your desired username:

      • grant all on maharadb .* to 'mahara '@'localhost' identified by 'password';

      You have now created a database for Mahara and created a role. Now you can install and configure Mahara.

      Step 2 — Downloading Mahara

      Now that you have a database created and configured for Mahara, let’s download Mahara itself. You can download a standard build from Mahara’s server, and you can find the latest Mahara build here.

      Once you have a .tar.gz link to the latest stable release, download it using the wget command. Make sure to replace /19.10/19.10.2/+download/mahara-19.10.2.tar.gz with the release you selected:

      • wget https://launchpad.net/mahara/19.10/19.10.2/+download/mahara-19.10.2.tar.gz

      This will download a compressed file labeled mahara-19.10.2.tar.gz. Extract the compressed file to create the Mahara directory structure:

      • tar xzvf mahara-19.10.2.tar.gz
      • ls

      The ls command checks the extraction. You will now see the new mahara-19.10.2 directory:

      Output

      mahara-19.10.2.tar.gz mahara-19.10.2

      Let’s copy Mahara to /var/www/mahara.your_domain. mahara.your_domain should match the directory you created during the Virtual Host configuration step in the Prerequisite section.

      Note: Mahara must install into an empty directory. If you have files here already, consider changing your DocumentRoot and using a new directory.

      Now copy the Mahara directory:

      • sudo cp -a mahara-19.10.2/. /var/www/mahara.your_domain/

      The -a switch ensures that the system copies the file permissions along with the files and folders. The . at the end of the source file ensures that everything in the source directory, including hidden files, gets copied to the destination directory.

      Mahara requires additional PHP extensions to function correctly.

      Run the following commands to install all the necessary dependencies:

      • sudo apt update
      • sudo apt install php-gd php-pgsql php-xmlrpc php-xml php-curl php-mbstring

      Type y for any prompts to finish the installation.

      You have installed Mahara on your server. Next, you will create the dataroot directory.

      Step 3 — Creating the Dataroot Directory

      Now that you have your Mahara code set up let’s create a dataroot directory. This is where Mahara will write uploaded files as well as some other files it needs to run. Therefore, the web server needs to be able to write to it.

      The dataroot directory should also be outside the directory where you have your Mahara code. You should avoid having it inside your public_html directory altogether.

      Let’s create the directory in /var/maharadata:

      • sudo mkdir /var/maharadata

      Update the ownership with chown:

      • sudo chown -R www-data:www-data /var/maharadata

      This allows the web server to write in the directory. Now we can finish configuring Mahara.

      Step 4 — Setting Up the Mahara Configuration File

      In this step you will make some final adjustments to Mahara’s config.php file. These adjustments will connect your database and encrypt your password.

      Switch to the Mahara code directory:

      • cd /var/www/mahara.your_domain/htdocs

      Create your config.php file by copying the config-dist.php file in the htdocs directory.

      Change directories and copy the file:

      • cp config-dist.php config.php

      Now open config.php using nano or your preferred text editor:

      Locate the following section in the file and make the required changes. Leave $cfg->dbtype as postgres unless you’re using MySQL, in which case change it to mysql. Change maharadb, mahara, and dbpassword to match the values you used in Step 1 for the database name, user, and password, respectively:

      ./htdocs/config.php

      ...
      $cfg->dbtype="postgres";
      $cfg->dbhost="localhost";
      $cfg->dbport   = null; // Change if you are using a non-standard port number for your database
      $cfg->dbname="maharadb";
      $cfg->dbuser="mahara";
      $cfg->dbpass="dbpassword";
      ...
      

      In the same file, locate the following section:

      /var/www/mahara/htdocs/config.php

      ...
      $cfg->dataroot="/path/to/uploaddir";
      ...
      

      Change it to point to Mahara’s dataroot directory as you set it in Step 3:

      /var/www/mahara/htdocs/config.php

      ...
      $cfg->dataroot="/var/maharadata";
      ...
      

      Finally, let’s update the passwordsaltmain with a random secret string. This encrypts stored user passwords. Locate and uncomment the following line:

      /var/www/mahara/htdocs/config.php

      ...
      // $cfg->passwordsaltmain = 'some long random string here with lots of characters';
      ...
      

      You can use the OpenSSL rand function to generate a random string to use as your secret salt string. Open a new terminal, connect to your server again, and run the following command to generate this string:

      The -base64 32 option ensures a Base64 encoded string that is 32 characters long. Update the value for passwordsaltmain to your_generated_salt:

      /var/www/mahara/htdocs/config.php

      ...
      $cfg->passwordsaltmain = 'your_generated_salt';
      ...
      

      Warning: Once set, you must not lose this string or you will need to reset all user passwords. It’s good practice to keep a secure backup of your config.php file

      Note: You may wish to set productionmode to false when using Mahara as a development or test site. Setting this to false will enable the on-screen display of warnings and error messages to aid in testing.

      Save and close the file.

      With Mahara configured, you are ready to serve Mahara using the Apache web server.

      Step 5 — Configuring Apache

      In the prerequisites, you configured a domain and pointed it to this server. You also installed Apache and set up HTTPS using Let’s Encrypt. Let’s now use Apache to serve Mahara.

      If your Virtual Host configuration file is named mahara.your_domain.conf Let’s Encrypt created a new configuration file called mahara.your_domain-le-ssl.conf that handles HTTPS requests for your domain.

      Open that file. Be sure to replace mahara.your_domain with the actual name of your file:

      • sudo nano /etc/apache2/sites-available/mahara.your_domain-le-ssl.conf

      The file will look similar to this:

      /etc/apache2/sites-available/your_domain-le-ssl.conf

      ...
      <IfModule mod_ssl.c>
      <VirtualHost *:443>
              ServerAdmin webmaster@localhost
              ServerName mahara.your_domain
              DocumentRoot /var/www/mahara.your_domain
              ErrorLog ${APACHE_LOG_DIR}/error.log
              CustomLog ${APACHE_LOG_DIR}/access.log combined
      
      SSLCertificateFile /etc/letsencrypt/live/mahara.your_domain/fullchain.pem
      SSLCertificateKeyFile /etc/letsencrypt/live/mahara.your_domain/privkey.pem
      Include /etc/letsencrypt/options-ssl-apache.conf
      </VirtualHost>
      </IfModule>
      ...
      

      The ServerName directive defines the base domain that should match for this virtual host definition. This should also be the domain name you chose to secure with an SSL certificate in the Prerequisites section.

      Note: Your Apache configuration should contain no ServerAliases. Mahara expects to be accessed through one url. If you use a server alias, you’ll experience issues like having to log in twice and breaking Single Sign On (SSO) features.

      However, if you do need an alias for some reason, you can set up a separate second VirtualHost directive in addition to the one above.

      Make the following additions to the file:

      /etc/apache2/sites-available/your_domain-le-ssl.conf

      ...
      <IfModule mod_ssl.c>
      <VirtualHost *:443>
              ServerAdmin webmaster@localhost
              ServerName mahara.your_domain
              DocumentRoot /var/www/mahara.your_domain/htdocs
      
              <Directory />
                  Options FollowSymLinks
                  AllowOverride None
              </Directory>
      
              <Directory /var/www/your_domain/htdocs>
                  Options Indexes FollowSymLinks MultiViews
                  AllowOverride None
                  Order allow,deny
                  allow from all
              </Directory>
      
              ErrorLog ${APACHE_LOG_DIR}/error.log
              CustomLog ${APACHE_LOG_DIR}/access.log combined
      
      SSLCertificateFile /etc/letsencrypt/live/mahara.your_domain/fullchain.pem
      SSLCertificateKeyFile /etc/letsencrypt/live/mahara.your_domain/privkey.pem
      Include /etc/letsencrypt/options-ssl-apache.conf
      </VirtualHost>
      </IfModule>
      ...
      

      First, you update the DocumentRoot to include /htdocs. Then you add two new blocks.

      The <Directory /> block refers to the filesystem root. It specifies directives applying to all files and directories anywhere on the server. Specifying directives for this block allows us to protect your server from intrusions by placing default directives such as AllowOverride None.

      The <Directory /var/www/mahara.your_domain/htdocs> block refers to the document root for Mahara. Directives declared here override ones declared on the <Directory /> block. These directives are the fundamental requirements for an Apache configuration to serve Mahara.

      Save and close the file.

      You are now ready to configure your PHP service.

      Step 6 — Adjusting PHP Settings

      Our final step is to update Apache’s php.ini file. This will change our logging verbosity and resize maximum upload and POST sizes.

      Open the file:

      • sudo nano /etc/php/7.2/apache2/php.ini

      This is a large file. Let’s go through the updates one by one.

      Find the log_errors line and make sure it matches the highlighted code:

      /etc/php/7.2/apache2/php.ini

      . . .
      log_errors = On
      . . .
      

      Setting log_errors to On will enable PHP to log errors encountered while running Mahara to aid in debugging.

      Now find the line beginning upload_max_filesize = 2M. Replace 2M with 50M:

      7.2/apache2/php.ini’>/etc/php/7.2/apache2/php.ini

      . . .
      upload_max_filesize = 50M
      . . .
      

      This will increase the max upload file size to 50 megabytes.

      Now find the line beginning post_max_size = 8M. Replace 8M with 100M:

      7.2/apache2/php.ini’>/etc/php/7.2/apache2/php.ini

      . . .
      post_max_size = 100M
      . . .
      

      upload_max_filesize accounts for the maximum size allowed for files posted, while post_max_size is the maximum size allowed for all POST body data. So you’ll always want to have this second number higher. These settings will enable Mahara to accept file uploads at the set maximums.

      Now move to the end of the file and add the following directives:

      /etc/php/7.2/apache2/php.ini

      ...
      register_globals = Off
      magic_quotes_runtime = Off
      magic_quotes_sybase = Off
      magic_quotes_gpc = Off
      allow_call_time_pass_reference = Off
      

      register_globals = Off disables register_globals. Enabling it has security implications as highlighted on PHP’s Manual.

      Setting magic_quotes_runtime to Off stops PHP from automatically escaping quotes with a backslash from any functions coming in from an external source (POST/GET). magic_quotes_runtime, magic_quotes_sybase, and magic_quotes_gpc are deprecated.

      allow_call_time_pass_reference is also deprecated.

      Save and close the file.

      Now restart Apache:

      • sudo systemctl restart apache2

      This completes your server setup. You are now ready to run the Mahara installer and enter your ePortfolio through a web browser.

      Step 7 — Running the Mahara Installer

      With Apache and PHP properly configured, let’s proceed to finish up Mahara’s installation through its web-based installer.

      In your browser, navigate to your server’s domain name. A screen will appear displaying Mahara’s licensing information and a button to continue installation.

      Mahara Install Screen

      Click the Install Mahara button and wait until the installer finishes performing all installations. Once done, scroll down. You will see a section that says Successfully installed Mahara. Continue. Click on Continue to proceed.

      Mahara will prompt you to enter a new password and a primary email address. Go ahead and submit the form.

      Mahara Welcome Screen

      You are ready to start using Mahara. But before you start exploring your new ePortfolio, you might want to set up a few final features.

      Set up Email

      For production settings, make sure that Mahara can send out emails. This is useful for sending out notifications such as confirmation emails after users register on the site. You can specify an outgoing SMTP server with admin settings. Click on the drop-down menu on the upper-right and choose Configure Site -> Site options -> Email.

      Set up a Cron Job

      Lastly, consider setting up a cron job to hit htdocs/lib/cron.php every minute. Without setting this up, you will find that RSS feeds will not update and some email notifications won’t get sent out. Go ahead and enter the following command. You’ll get an option to select the editor of your choice. In this example, we’re proceeding with the nano editor:

      Add the following line at the bottom of the file replacing mahara.your_domain with the name of the directory where you installed Mahara on Step 2:

      * * * * * php /var/www/mahara.your_domain/htdocs/lib/cron.php
      

      With your email and cron job configured, you are now ready to use Mahara.

      Conclusion

      Mahara is a powerful and popular platform. It is a great resource for teacher and learners, particularly within remote-learning environments. Mahara is also popular outside of education, where users create blogs and resume builders.

      Users can also choose from numerous themes while admins can explore a large library of plugins.

      After familiarizing yourself with the user interface, consider following the Next Steps article on Mahara’s website for more ideas.



      Source link

      How To Install Jitsi Meet on Ubuntu 20.04


      The author selected the Open Internet/Free Speech Fund to receive a donation as part of the Write for DOnations program.

      Introduction

      Jitsi Meet is an open source video-conferencing application based on WebRTC. A Jitsi Meet server provides multi-person video conference rooms that you can access using nothing more than your browser and provides comparable functionality to a Zoom or Skype conference call. The benefit of a Jitsi conference is that all your data only passes through your server and the end-to-end TLS encryption ensures that no one can snoop on the call. With Jitsi you can be sure that your private information stays that way.

      In this tutorial, you will install and configure a Jitsi Meet server on Ubuntu 20.04. The default configuration allows anyone to create a new conference room. This is not ideal for a server that is publicly available on the internet so you will also configure Jitsi Meet so that only registered users can create new conference rooms. After you have created the conference room any users can join as long as they have the unique address and the optional password.

      Prerequisites

      Before you begin this guide you’ll need the following:

      • One Ubuntu 20.04 server set up by following the Initial Server Setup with Ubuntu 20.04 tutorial, including a non-root sudo-enabled user. The size of the server you will need mostly depends on the available bandwidth and the number of participants you expect to be using the server. The following table will give you some idea of what is needed.
      • A domain name configured to point to your server. You can learn how to point domains to DigitalOcean Droplets by following the How To Set Up a Host Name with DigitalOcean tutorial. Throughout this guide, the example domain name jitsi.your_domain is used.

      When you are choosing a server to run your Jitsi Meet instance you will need to consider the system resources needed to host conference rooms. The following benchmark information was collected from a single-core virtual machine using high-quality video settings:

      CPU Server Bandwidth
      Two Participants 3% 30Kbps Up, 100Kbps Down
      Three Participants 15% 7Mbps Up, 6.5Mbps Down

      The jump in resource use between two and three participants is because Jitsi will route the call data directly between the clients when there are two of them. When more than two clients are present then call data is routed through the Jitsi Meet server.

      Log in to your server as the non-root, sudo-enabled user before starting Step 1.

      Step 1 — Setting the System Hostname

      In this step, you will change the system’s hostname to match the domain name that you intend to use for your Jitsi Meet instance and resolve that hostname to the localhost IP, 127.0.0.1. Jitsi Meet uses both of these settings when it installs and generates its configuration files.

      First, set the system’s hostname to the domain name that you will use for your Jitsi instance. The following command will set the current hostname and modify the /etc/hostname that holds the system’s hostname between reboots:

      • sudo hostnamectl set-hostname jitsi.your_domain

      The command that you ran breaks down as follows:

      • hostnamectl: A utility from the systemd tool suite to manage the system hostname.
      • set-hostname: Sets the system hostname.

      Check that this was successful by running the following:

      This will return the hostname you set with the hostnamectl command:

      Output

      jitsi.your-domain

      Next, you will set a local mapping of the server’s hostname to the loopback IP address, 127.0.0.1. Do this by opening the /etc/hosts with a text editor:

      Then, add the following line:

      /etc/hosts

      127.0.0.1 jitsi.your_domain
      

      This local mapping of your Jitsi Meet server’s domain name to 127.0.0.1 is important because your Jitsi Meet server uses several networked processes on your server that accept local connections on the 127.0.0.1 IP address from each other. These connections are authenticated and encrypted with a TLS certificate, which is registered to your domain name. Locally mapping the domain name to 127.0.0.1 makes it possible to use the TLS certificate for these local network connections.

      Your server now has the hostname that Jitsi requires when installed. In the next step, you will open the firewall ports that are needed by Jitsi and the TLS certificate installer.

      Step 2 — Configuring the Firewall

      When you followed the Initial Server Setup with Ubuntu 20.04 guide you enabled the UFW firewall and opened the SSH port. The Jitsi server needs some ports opened so that it can communicate with the call clients. Also, the TLS installation process needs to have a port open so that it can authenticate the certificate request.

      The ports that you will open are the following:

      • 80/tcp: Port used in the TLS certificate request.
      • 443/tcp: Port used for the conference room creation web page.
      • 4443/tcp 10000/udp: Ports that will transmit and receive the encrypted call traffic.

      Run the following ufw commands to open these ports:

      • sudo ufw allow 80/tcp
      • sudo ufw allow 443/tcp
      • sudo ufw allow 4443/tcp
      • sudo ufw allow 10000/udp

      Check that they were all added with the ufw status command:

      You will receive the following output if these ports are open:

      Output

      Status: active To Action From -- ------ ---- OpenSSH ALLOW Anywhere 80/tcp ALLOW Anywhere 443/tcp ALLOW Anywhere 4443/tcp ALLOW Anywhere 10000/udp ALLOW Anywhere OpenSSH (v6) ALLOW Anywhere (v6) 80/tcp (v6) ALLOW Anywhere (v6) 443/tcp (v6) ALLOW Anywhere (v6) 4443/tcp (v6) ALLOW Anywhere (v6) 10000/udp (v6) ALLOW Anywhere (v6)

      The server is now ready for the Jitsi installation, which you will complete in the next step.

      Step 3 — Installing Jitsi Meet

      In this step, you will add the Jitsi stable repository to your server and then install the Jitsi Meet package from that repository. This will ensure that you are always running the latest stable Jitsi Meet package.

      First, download the Jitsi GPG key with the wget downloading utility:

      • wget https://download.jitsi.org/jitsi-key.gpg.key

      The apt package manager will use this GPG key to validate the packages that you will download from the Jitsi repository.

      Next, add the GPG key you downloaded to apt’s keyring using the apt-key utility:

      • sudo apt-key add jitsi-key.gpg.key

      You can now delete the GPG key file as it is no longer needed with this command:

      Now, you will add the Jitsi repository to your server by creating a new sources file that contains the Jitsi repository. Open and create the new file:

      • sudo nano /etc/apt/sources.list.d/jitsi-stable.list

      Add this line to the file for the Jitsi repository:

      /etc/apt/sources.list.d/jitsi-stable.list

      deb https://download.jitsi.org stable/
      

      Save and exit the editor.

      Finally, perform a system update to collect the package list from the Jitsi repository and then install the jitsi-meet package:

      • sudo apt update
      • sudo apt install jitsi-meet

      During the installation of jitsi-meet you will be prompted to enter the domain name (for example, jitsi.your-domain) that you want to use for your Jitsi Meet instance.

      Image showing the jitsi-meet installation hostname dialog

      Note: You move the cursor from the hostname field to highlight the <OK> button with the TAB key. Press ENTER when <OK> is highlighted to submit the hostname.

      You will then be shown a new dialog box that asks if you want Jitsi to create and use a self-signed TLS certificate or use an existing one if you have one:

      Image showing the jitsi-meet installation certificate dialog

      If you do not have a TLS certificate for your Jitsi domain select the first, Generate a new self-signed certificate, option.

      Your Jitsi Meet instance is now installed using a self-signed TLS certificate. This will cause browser warnings so you will get a signed TLS certificate in the next step.

      Step 4 — Obtaining a Signed TLS Certificate

      Jitsi Meet uses TLS certificates to encrypt the call traffic so that no one can listen to your call as it travels over the internet. TLS certificates are the same certificates that are used by websites to enable HTTPS URLs.

      Jitsi Meet supplies a script to automatically download a TLS certificate for your domain. Run this certificate installation script provided by Jitsi Meet at /usr/share/jitsi-meet/scripts/install-letsencrypt-cert.sh with the following command:

      • sudo /usr/share/jitsi-meet/scripts/install-letsencrypt-cert.sh

      The script prints the following information when you run it and asks you to supply an email address:

      Output

      ------------------------------------------------------------------------- This script will: - Need a working DNS record pointing to this machine(for domain jitsi.your_domain) - Download certbot-auto from https://dl.eff.org to /usr/local/sbin - Install additional dependencies in order to request Let’s Encrypt certificate - If running with jetty serving web content, will stop Jitsi Videobridge - Configure and reload nginx or apache2, whichever is used - Configure the coturn server to use Let's Encrypt certificate and add required deploy hooks - Add command in weekly cron job to renew certificates regularly You need to agree to the ACME server's Subscriber Agreement (https://letsencrypt.org/documents/LE-SA-v1.1.1-August-1-2016.pdf) by providing an email address for important account notifications Enter your email and press [ENTER]:

      This email address will be submitted to the certificate issuer https://letsencrypt.org and will be used to notify you about security and other matters related to the certificate. You must enter an email address here to proceed with the installation.

      The script will complete the installation and configuration of an SSL certificate for your Jitsi server without needing any more user input.

      The default configuration for Jitsi Meet is that anyone visiting your Jitsi Meet server homepage can create a new conference room. This will use your server’s system resources to run the conference room and is not desirable for unauthorized users. In the next step, you will configure your Jitsi Meet instance to only allow registered users to create conference rooms.

      Step 5 — Locking Conference Creation

      In this step, you will configure your Jitsi Meet server to only allow registered users to create conference rooms. The files that you will edit were generated by the installer and are configured with your domain name.

      The variable jitsi.your_domain will be used in place of a domain name in the following examples.

      First, open /etc/prosody/conf.avail/jitsi.your_domain.cfg.lua with a text editor:

      • sudo nano /etc/prosody/conf.avail/your_domain.cfg.lua

      Edit this line:

      /etc/prosody/conf.avail/jitsi.your_domain.cfg.lua

              authentication = "anonymous"
      

      To

      /etc/prosody/conf.avail/jitsi.your_domain.cfg.lua

              authentication = "internal_plain"
      

      This configuration tells Jitsi Meet to force username and password authentication before allowing conference room creation by a new visitor.

      Then, in the same file, add the following section to the end of the file:

      /etc/prosody/conf.avail/your_domain.cfg.lua

      VirtualHost "guest.jitsi.your_domain"
          authentication = "anonymous"
          c2s_require_encryption = false
      

      This configuration allows anonymous users to join conference rooms that were created by an authenticated user. However, the guest must have a unique address and an optional password for the room to enter it.

      Here, you added guest. to the front of your domain name. For example, the correct name to put here for jitsi.your_domain is guest.jitsi.your_domain. The guest. hostname is only used internally by Jitsi Meet, you will never enter it into a browser or need to create a DNS record for it.

      Open another configuration file at /etc/jitsi/meet/jitsi.your_domain-config.js with a text editor:

      • sudo nano /etc/jitsi/meet/jitsi.your_domain-config.js

      Edit this line:

      /etc/jitsi/meet/your_domain-config.js

              // anonymousdomain: 'guest.jitsi.your_domain',
      

      To:

      /etc/jitsi/meet/your_domain-config.js

              anonymousdomain: 'guest.jitsi.your_domain',
      

      Again, using the guest.jitsi.your_domain hostname that you used previously. This configuration tells Jitsi Meet what internal hostname to use for the un-authenticated guests.

      Next, open /etc/jitsi/jicofo/sip-communicator.properties:

      • sudo nano /etc/jitsi/jicofo/sip-communicator.properties

      And add the following line to complete the configuration changes:

      /etc/jitsi/jicofo/sip-communicator.properties

      org.jitsi.jicofo.auth.URL=XMPP:jitsi.your_domain
      

      This configuration points one of the Jitsi Meet processes to the local server that performs the user authentication that is now required.

      Your Jitsi Meet instance is now configured so that only registered users can create conference rooms. After a conference room is created, anyone can join it without needing to be a registered user. All they will need is the unique conference room address and an optional password set by the room’s creator.

      Now that Jitsi Meet is configured to require authenticated users for room creation you need to register these users and their passwords. You will use the prosodyctl utility to do this.

      Run the following command to add a user to your server:

      • sudo prosodyctl register user your_domain password

      The user that you add here is not a system user. They will only be able to create a conference room and are not able to log in to your server via SSH.

      Finally, restart the Jitsi Meet processes to load the new configuration:

      • sudo systemctl restart prosody.service
      • sudo systemctl restart jicofo.service
      • sudo systemctl restart jitsi-videobridge2.service

      The Jitsi Meet instance will now request a username and password with a dialog box when a conference room is created.

      Image showing the Jitsi username and password box

      Your Jitsi Meet server is now set up and securely configured.

      Conclusion

      In this article, you deployed a Jitsi Meet server that you can use to host secure and private video conference rooms. You can extend your Jitsi Meet instance with instructions from the Jitsi Meet Wiki.



      Source link