One place for hosting & domains

      Cluster

      How to Back Up and Restore a Kubernetes Cluster on DigitalOcean Using Heptio Ark


      Introduction

      Heptio Ark is a convenient backup tool for Kubernetes clusters that compresses and backs up Kubernetes objects to object storage. It also takes snapshots of your cluster’s Persistent Volumes using your cloud provider’s block storage snapshot features, and can then restore your cluster’s objects and Persistent Volumes to a previous state.

      StackPointCloud’s DigitalOcean Ark Plugin allows you to use DigitalOcean block storage to snapshot your Persistent Volumes, and Spaces to back up your Kubernetes objects. When running a Kubernetes cluster on DigitalOcean, this allows you to quickly back up your cluster’s state and restore it should disaster strike.

      In this tutorial we’ll set up and configure the Ark client on a local machine, and deploy the Ark server into our Kubernetes cluster. We’ll then deploy a sample Nginx app that uses a Persistent Volume for logging, and simulate a disaster recovery scenario.

      Prerequisites

      Before you begin this tutorial, you should have the following available to you:

      On your local computer:

      In your DigitalOcean account:

      • A DigitalOcean Kubernetes cluster, or a Kubernetes cluster (version 1.7.5 or later) on DigitalOcean Droplets
      • A DNS server running inside of your cluster. If you are using DigitalOcean Kubernetes, this is running by default. To learn more about configuring a Kubernetes DNS service, consult Customizing DNS Service from the official Kuberentes documentation.
      • A DigitalOcean Space that will store your backed-up Kubernetes objects. To learn how to create a Space, consult the Spaces product documentation.
      • An access key pair for your DigitalOcean Space. To learn how to create a set of access keys, consult How to Manage Administrative Access to Spaces.
      • A personal access token for use with the DigitalOcean API. To learn how to create a personal access token, consult How to Create a Personal Access Token.

      Once you have all of this set up, you’re ready to begin with this guide.

      Step 1 — Installing the Ark Client

      The Heptio Ark backup tool consists of a client installed on your local computer and a server that runs in your Kubernetes cluster. To begin, we’ll install the local Ark client.

      In your web browser, navigate to the Ark GitHub repo releases page, find the latest release corresponding to your OS and system architecture, and copy the link address. For the purposes of this guide, we’ll use an Ubuntu 18.04 server on an x86-64 (or AMD64) processor as our local machine.

      Then, from the command line on your local computer, navigate to the temporary /tmp directory and cd into it:

      Use wget and the link you copied earlier to download the release tarball:

      • wget https://link_copied_from_release_page

      Once the download completes, extract the tarball using tar (note the filename may differ depending on the current release version and your OS):

      • tar -xvzf ark-v0.9.6-linux-amd64.tar.gz

      The /tmp directory should now contain the extracted ark binary as well as the tarball you just downloaded.

      Verify that you can run the ark client by executing the binary:

      You should see the following help output:

      Output

      Heptio Ark is a tool for managing disaster recovery, specifically for Kubernetes cluster resources. It provides a simple, configurable, and operationally robust way to back up your application state and associated data. If you're familiar with kubectl, Ark supports a similar model, allowing you to execute commands such as 'ark get backup' and 'ark create schedule'. The same operations can also be performed as 'ark backup get' and 'ark schedule create'. Usage: ark [command] Available Commands: backup Work with backups client Ark client related commands completion Output shell completion code for the specified shell (bash or zsh) create Create ark resources delete Delete ark resources describe Describe ark resources get Get ark resources help Help about any command plugin Work with plugins restic Work with restic restore Work with restores schedule Work with schedules server Run the ark server version Print the ark version and associated image . . .

      At this point you should move the ark executable out of the temporary /tmp directory and add it to your PATH. To add it to your PATH on an Ubuntu system, simply copy it to /usr/local/bin:

      • sudo mv ark /usr/local/bin/ark

      You're now ready to configure the Ark server and deploy it to your Kubernetes cluster.

      Step 2 — Installing and Configuring the Ark Server

      Before we deploy Ark into our Kubernetes cluster, we'll first create Ark's prerequisite objects. Ark's prerequisites consist of:

      • A heptio-ark Namespace

      • The ark Service Account

      • Role-based access control (RBAC) rules to grant permissions to the ark Service Account

      • Custom Resources (CRDs) for the Ark-specific resources: Backup, Schedule, Restore, Config

      A YAML file containing the specs for the above Kubernetes objects can be found in the official Ark Git repository. While still in the /tmp directory, download the Ark repo using git:

      • git clone https://github.com/heptio/ark.git

      Once downloaded, navigate into the ark directory:

      The prerequisite resources listed above can be found in the examples/common/00-prereqs.yaml YAML file. We'll create these resources in our Kubernetes cluster by using kubectl apply and passing in the file:

      • kubectl apply -f examples/common/00-prereqs.yaml

      You should see the following output:

      Output

      customresourcedefinition.apiextensions.k8s.io/backups.ark.heptio.com created customresourcedefinition.apiextensions.k8s.io/schedules.ark.heptio.com created customresourcedefinition.apiextensions.k8s.io/restores.ark.heptio.com created customresourcedefinition.apiextensions.k8s.io/configs.ark.heptio.com created customresourcedefinition.apiextensions.k8s.io/downloadrequests.ark.heptio.com created customresourcedefinition.apiextensions.k8s.io/deletebackuprequests.ark.heptio.com created customresourcedefinition.apiextensions.k8s.io/podvolumebackups.ark.heptio.com created customresourcedefinition.apiextensions.k8s.io/podvolumerestores.ark.heptio.com created customresourcedefinition.apiextensions.k8s.io/resticrepositories.ark.heptio.com created customresourcedefinition.apiextensions.k8s.io/backupstoragelocations.ark.heptio.com created namespace/heptio-ark created serviceaccount/ark created clusterrolebinding.rbac.authorization.k8s.io/ark created

      Now that we've created the necessary Ark Kubernetes objects in our cluster, we can download and install the Ark DigitalOcean Plugin, which will allow us to use DigitalOcean Spaces as a backupStorageProvider (for Kubernetes objects), and DigitalOcean Block Storage as a persistentVolumeProvider (for Persistent Volume backups).

      Move back out of the ark directory and fetch the plugin from StackPointCloud's repo using git:

      • cd ..
      • git clone https://github.com/StackPointCloud/ark-plugin-digitalocean.git

      Move into the plugin directory:

      • cd ark-plugin-digitalocean

      We'll now save the access keys for our DigitalOcean Space as a Kubernetes Secret. First, open up the examples/credentials-ark file using your favorite editor:

      • nano examples/credentials-ark

      Replace <AWS_ACCESS_KEY_ID> and <AWS_SECRET_ACCESS_KEY> with your Spaces access key and secret key:

      examples/credentials-ark

      [default]
      aws_access_key_id=your_spaces_access_key_here
      aws_secret_access_key=your_spaces_secret_key_here
      

      Save and close the file.

      Now, create the cloud-credentials Secret using kubectl, inserting your Personal Access Token using the digitalocean_token data item:

      • kubectl create secret generic cloud-credentials
      • --namespace heptio-ark
      • --from-file cloud=examples/credentials-ark
      • --from-literal digitalocean_token=your_personal_access_token

      You should see the following output:

      Output

      secret/cloud-credentials created

      To confirm that the cloud-credentials Secret was created successfully, you can describe it using kubectl:

      • kubectl describe secrets/cloud-credentials --namespace heptio-ark

      You should see the following output describing the cloud-credentials secret:

      Output

      Name: cloud-credentials Namespace: heptio-ark Labels: <none> Annotations: <none> Type: Opaque Data ==== cloud: 115 bytes digitalocean_token: 64 bytes

      We can now move on to creating an Ark Config object named default. To do this, we'll edit a YAML configuration file and then create the object in our Kubernetes cluster.

      Open examples/10-ark-config.yaml in your favorite editor:

      • nano examples/10-ark-config.yaml

      Insert your Space's name and region in the highlighted fields:

      examples/10-ark-config.yaml

      ---
      apiVersion: ark.heptio.com/v1
      kind: Config
      metadata:
        namespace: heptio-ark
        name: default
      persistentVolumeProvider:
        name: digitalocean
      backupStorageProvider:
        name: aws
        bucket: space_name_here
        config:
          region: space_region_here
          s3ForcePathStyle: "true"
          s3Url: https://space_region_here.digitaloceanspaces.com
      backupSyncPeriod: 30m
      gcSyncPeriod: 30m
      scheduleSyncPeriod: 1m
      restoreOnlyMode: false
      

      persistentVolumeProvider sets DigitalOcean Block Storage as the the provider for Persistent Volume backups. These will be Block Storage Volume Snapshots.

      backupStorageProvider sets DigitalOcean Spaces as the provider for Kubernetes object backups. Ark will create a tarball of all your Kubernetes objects (or some, depending on how you execute it), and upload this tarball to Spaces.

      When you're done, save and close the file.

      Create the object in your cluster using kubectl apply:

      • kubectl apply -f examples/10-ark-config.yaml

      You should see the following output:

      Output

      config.ark.heptio.com/default created

      At this point, we've finished configuring the Ark server and can create its Kubernetes deployment, found in the examples/20-deployment.yaml configuration file. Let's take a quick look at this file:

      • cat examples/20-deployment.yaml

      You should see the following text:

      examples/20-deployment.yaml

      ---
      apiVersion: apps/v1beta1
      kind: Deployment
      metadata:
        namespace: heptio-ark
        name: ark
      spec:
        replicas: 1
        template:
          metadata:
            labels:
              component: ark
            annotations:
              prometheus.io/scrape: "true"
              prometheus.io/port: "8085"
              prometheus.io/path: "/metrics"
          spec:
            restartPolicy: Always
            serviceAccountName: ark
            containers:
              - name: ark
                image: gcr.io/heptio-images/ark:latest
                command:
                  - /ark
                args:
                  - server
                volumeMounts:
                  - name: cloud-credentials
                    mountPath: /credentials
                  - name: plugins
                    mountPath: /plugins
                  - name: scratch
                    mountPath: /scratch
                env:
                  - name: AWS_SHARED_CREDENTIALS_FILE
                    value: /credentials/cloud
                  - name: ARK_SCRATCH_DIR
                    value: /scratch
                  - name: DIGITALOCEAN_TOKEN
                    valueFrom:
                      secretKeyRef:
                        key: digitalocean_token
                        name: cloud-credentials
            volumes:
              - name: cloud-credentials
                secret:
                  secretName: cloud-credentials
              - name: plugins
                emptyDir: {}
              - name: scratch
                emptyDir: {}
      

      We observe here that we're creating a Deployment called ark that consists of a single replica of the gcr.io/heptio-images/ark:latest container. The Pod is configured using the cloud-credentials secret we created earlier.

      Create the Deployment using kubectl apply:

      • kubectl apply -f examples/20-deployment.yaml

      You should see the following output:

      Output

      deployment.apps/ark created

      We can double check that the Deployment has been successfully created using kubectl get on the heptio-ark Namespace :

      • kubectl get deployments --namespace=heptio-ark

      You should see the following output:

      Output

      NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE ark 1 1 1 0 8m

      The Ark server Pod may not start correctly until you install the Ark DigitalOcean plugin. To install the ark-blockstore-digitalocean plugin, use the ark client we installed earlier:

      • ark plugin add quay.io/stackpoint/ark-blockstore-digitalocean:latest

      You can specify the kubeconfig to use with the --kubeconfig flag. If you don't use this flag, ark will check the KUBECONFIG environment variable and then fall back to the kubectl default (~/.kube/config).

      At this point Ark is running and fully configured, and ready to back up and restore your Kubernetes cluster objects and Persistent Volumes to DigitalOcean Spaces and Block Storage.

      In the next section, we'll run a quick test to make sure that the backup and restore functionality works as expected.

      Step 3 — Testing Backup and Restore Procedure

      Now that we've successfully installed and configured Ark, we can create a test Nginx Deployment and Persistent Volume, and run through a backup and restore drill to ensure that everything is working properly.

      The ark-plugin-digitalocean repository contains a sample Nginx deployment called nginx-pv.yaml.

      Let's take a quick look:

      • cat examples/nginx-pv.yaml

      You should see the following text:

      Output

      --- kind: PersistentVolumeClaim apiVersion: v1 metadata: name: nginx-logs namespace: nginx-example labels: app: nginx spec: storageClassName: do-block-storage accessModes: - ReadWriteOnce resources: requests: storage: 5Gi --- apiVersion: apps/v1beta1 kind: Deployment metadata: name: nginx-deployment namespace: nginx-example spec: replicas: 1 template: metadata: labels: app: nginx spec: volumes: - name: nginx-logs persistentVolumeClaim: claimName: nginx-logs containers: - image: nginx:1.7.9 name: nginx ports: - containerPort: 80 volumeMounts: - mountPath: "/var/log/nginx" name: nginx-logs readOnly: false --- apiVersion: v1 kind: Service metadata: labels: app: nginx name: my-nginx namespace: nginx-example spec: ports: - port: 80 targetPort: 80 selector: app: nginx type: LoadBalancer

      In this file, we observe specs for:

      • An Nginx Deployment consisting of a single replica of the nginx:1.7.9 container image
      • A 5Gi Persistent Volume Claim (called nginx-logs), using the do-block-storage StorageClass
      • A LoadBalancer Service that exposes port 80

      Create the deployment using kubectl apply:

      • kubectl apply -f examples/nginx-pv.yml

      You should see the following output:

      Output

      namespace/nginx-example created persistentvolumeclaim/nginx-logs created deployment.apps/nginx-deployment created service/my-nginx created

      Check that the Deployment succeeded:

      • kubectl get deployments --namespace=nginx-example

      You should see the following output:

      Output

      NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE nginx-deployment 1 1 1 1 1h

      Once Available reaches 1, fetch the Nginx load balancer’s external IP using kubectl get:

      • kubectl get services --namespace=nginx-example

      You should see both the internal CLUSTER-IP and EXTERNAL-IP for the my-nginx Service:

      Output

      NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE my-nginx LoadBalancer 10.32.27.0 203.0.113.0 80:30754/TCP 3m

      Note the EXTERNAL-IP and navigate to it using your web browser.

      You should see the following NGINX welcome page:

      Nginx Welcome Page

      This indicates that your Nginx Deployment and Service are up and running.

      Before we simulate our disaster scenario, let’s first check the Nginx access logs (stored on a Persistent Volume attached to the Nginx Pod):

      Fetch the Pod’s name using kubectl get:

      • kubectl get pods --namespace nginx-example

      Output

      NAME READY STATUS RESTARTS AGE nginx-deployment-77d8f78fcb-zt4wr 1/1 Running 0 29m

      Now, exec into the running Nginx container to get a shell inside of it:

      • kubectl exec -it nginx-deployment-77d8f78fcb-zt4wr --namespace nginx-example -- /bin/bash

      Once inside the Nginx container, cat the Nginx access logs:

      • cat /var/log/nginx/access.log

      You should see some Nginx access entries:

      Output

      10.244.17.1 - - [01/Oct/2018:21:47:01 +0000] "GET / HTTP/1.1" 200 612 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/203.0.113.11 Safari/537.36" "-" 10.244.17.1 - - [01/Oct/2018:21:47:01 +0000] "GET /favicon.ico HTTP/1.1" 404 570 "http://203.0.113.0/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/203.0.113.11 Safari/537.36" "-"

      Note these down (especially the timestamps), as we will use them to confirm the success of the restore procedure.

      We can now perform the backup procedure to copy all nginx Kubernetes objects to Spaces and take a Snapshot of the Persistent Volume we created when deploying Nginx.

      We'll create a backup called nginx-backup using the ark client:

      • ark backup create nginx-backup --selector app=nginx

      The --selector app=nginx instructs the Ark server to only back up Kubernetes objects with the app=nginx Label Selector.

      You should see the following output:

      Output

      Backup request "nginx-backup" submitted successfully. Run `ark backup describe nginx-backup` for more details.

      Running ark backup describe nginx-backup should provide the following output after a short delay:

      Output

      Name: nginx-backup Namespace: heptio-ark Labels: <none> Annotations: <none> Phase: Completed Namespaces: Included: * Excluded: <none> Resources: Included: * Excluded: <none> Cluster-scoped: auto Label selector: app=nginx Snapshot PVs: auto TTL: 720h0m0s Hooks: <none> Backup Format Version: 1 Started: 2018-09-26 00:14:30 -0400 EDT Completed: 2018-09-26 00:14:34 -0400 EDT Expiration: 2018-10-26 00:14:30 -0400 EDT Validation errors: <none> Persistent Volumes: pvc-e4862eac-c2d2-11e8-920b-92c754237aeb: Snapshot ID: 2eb66366-c2d3-11e8-963b-0a58ac14428b Type: ext4 Availability Zone: IOPS: <N/A>

      This output indicates that nginx-backup completed successfully.

      From the DigitalOcean Cloud Control Panel, navigate to the Space containing your Kubernetes backup files.

      You should see a new directory called nginx-backup containing the Ark backup files.

      Using the left-hand navigation bar, go to Images and then Snapshots. Within Snapshots, navigate to Volumes. You should see a Snapshot corresponding to the PVC listed in the above output.

      We can now test the restore procedure.

      Let's first delete the nginx-example Namespace. This will delete everything in the Namespace, including the Load Balancer and Persistent Volume:

      • kubectl delete namespace nginx-example

      Verify that you can no longer access Nginx at the Load Balancer endpoint, and that the nginx-example Deployment is no longer running:

      • kubectl get deployments --namespace=nginx-example

      Output

      No resources found.

      We can now perform the restore procedure, once again using the ark client:

      • ark restore create --from-backup nginx-backup

      Here we use create to create an Ark Restore object from the nginx-backup object.

      You should see the following output:

      Output

      • Restore request "nginx-backup-20180926143828" submitted successfully.
      • Run `ark restore describe nginx-backup-20180926143828` for more details.

      Check the status of the restored Deployment:

      • kubectl get deployments --namespace=nginx-example

      Output

      NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE nginx-deployment 1 1 1 1 1m

      Check for the creation of a Persistent Volume:

      • kubectl get pvc --namespace=nginx-example

      Output

      NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE nginx-logs Bound pvc-e4862eac-c2d2-11e8-920b-92c754237aeb 5Gi RWO do-block-storage 3m

      Navigate to the Nginx Service’s external IP once again to confirm that Nginx is up and running.

      Finally, check the logs on the restored Persistent Volume to confirm that the log history has been preserved post-restore.

      To do this, once again fetch the Pod’s name using kubectl get:

      • kubectl get pods --namespace nginx-example

      Output

      NAME READY STATUS RESTARTS AGE nginx-deployment-77d8f78fcb-zt4wr 1/1 Running 0 29m

      Then exec into it:

      • kubectl exec -it nginx-deployment-77d8f78fcb-zt4wr --namespace nginx-example -- /bin/bash

      Once inside the Nginx container, cat the Nginx access logs:

      • cat /var/log/nginx/access.log

      Output

      10.244.17.1 - - [01/Oct/2018:21:47:01 +0000] "GET / HTTP/1.1" 200 612 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/203.0.113.11 Safari/537.36" "-" 10.244.17.1 - - [01/Oct/2018:21:47:01 +0000] "GET /favicon.ico HTTP/1.1" 404 570 "http://203.0.113.0/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/203.0.113.11 Safari/537.36" "-"

      You should see the same pre-backup access attempts (note the timestamps), confirming that the Persistent Volume restore was successful. Note that there may be additional attempts in the logs if you visited the Nginx landing page after you performed the restore.

      At this point, we've successfully backed up our Kubernetes objects to DigitalOcean Spaces, and our Persistent Volumes using Block Storage Volume Snapshots. We simulated a disaster scenario, and restored service to the test Nginx application.

      Conclusion

      In this guide we installed and configured the Ark Kubernetes backup tool on a DigitalOcean-based Kubernetes cluster. We configured the tool to back up Kubernetes objects to DigitalOcean Spaces, and back up Persistent Volumes using Block Storage Volume Snapshots.

      Ark can also be used to schedule regular backups of your Kubernetes cluster. To do this, you can use the ark schedule command. It can also be used to migrate resources from one cluster to another. To learn more about these two use cases, consult the official Ark documentation.

      To learn more about DigitalOcean Spaces, consult the official Spaces documentation. To learn more about Block Storage Volumes, consult the Block Storage Volume documentation.

      This tutorial builds on the README found in StackPointCloud's ark-plugin-digitalocean GitHub repo.



      Source link

      How To Create a Multi-Node MySQL Cluster on Ubuntu 18.04


      Introduction

      The MySQL Cluster distributed database provides high availability and throughput for your MySQL database management system. A MySQL Cluster consists of one or more management nodes (ndb_mgmd) that store the cluster’s configuration and control the data nodes (ndbd), where cluster data is stored. After communicating with the management node, clients (MySQL clients, servers, or native APIs) connect directly to these data nodes.

      With MySQL Cluster there is typically no replication of data, but instead data node synchronization. For this purpose a special data engine must be used — NDBCluster (NDB). It’s helpful to think of the cluster as a single logical MySQL environment with redundant components. Thus, a MySQL Cluster can participate in replication with other MySQL Clusters.

      MySQL Cluster works best in a shared-nothing environment. Ideally, no two components should share the same hardware. For simplicity and demonstration purposes, we’ll limit ourselves to using only three servers. We will set up two servers as data nodes which sync data between themselves. The third server will be used for the Cluster Manager and also for the MySQL server/client. If you spin up additional servers, you can add more data nodes to the cluster, decouple the cluster manager from the MySQL server/client, and configure more servers as Cluster Managers and MySQL servers/clients.

      Prerequisites

      To complete this tutorial, you will need a total of three servers: two servers for the redundant MySQL data nodes (ndbd), and one server for the Cluster Manager (ndb_mgmd) and MySQL server/client (mysqld and mysql).

      In the same DigitalOcean data center, create the following Droplets with private networking enabled:

      Be sure to note down the private IP addresses of your three Droplets. In this tutorial our cluster nodes have the following private IP addresses:

      • 198.51.100.0 will be the first MySQL Cluster data node
      • 198.51.100.1 will be the second data node
      • 198.51.100.2 will be the Cluster Manager & MySQL server node

      Once you’ve spun up your Droplets, configured a non-root user, and noted down the IP addresses for the 3 nodes, you’re ready to begin with this tutorial.

      Step 1 — Installing and Configuring the Cluster Manager

      We’ll first begin by downloading and installing the MySQL Cluster Manager, ndb_mgmd.

      To install the Cluster Manager, we first need to fetch the appropriate .deb installer file from the the official MySQL Cluster download page.

      From this page, under Select Operating System, choose Ubuntu Linux. Then, under Select OS Version, choose Ubuntu Linux 18.04 (x86, 64-bit).

      Scroll down until you see DEB Package, NDB Management Server, and click on the Download link for the one that does not contain dbgsym (unless you require debug symbols). You will be brought to a Begin Your Download page. Here, right click on No thanks, just start my download. and copy the link to the .deb file.

      Now, log in to your Cluster Manager Droplet (in this tutorial, 198.51.100.2), and download this .deb file:

      • cd ~
      • wget https://dev.mysql.com/get/Downloads/MySQL-Cluster-7.6/mysql-cluster-community-management-server_7.6.6-1ubuntu18.04_amd64.deb

      Install ndb_mgmd using dpkg:

      • sudo dpkg -i mysql-cluster-community-management-server_7.6.6-1ubuntu18.04_amd64.deb

      We now need to configure ndb_mgmd before first running it; proper configuration will ensure correct synchronization and load distribution among the data nodes.

      The Cluster Manager should be the first component launched in any MySQL cluster. It requires a configuration file, passed in as an argument to its executable. We’ll create and use the following configuration file: /var/lib/mysql-cluster/config.ini.

      On the Cluster Manager Droplet, create the /var/lib/mysql-cluster directory where this file will reside:

      • sudo mkdir /var/lib/mysql-cluster

      Then create and edit the configuration file using your preferred text editor:

      • sudo nano /var/lib/mysql-cluster/config.ini

      Paste the following text into your editor:

      /var/lib/mysql-cluster/config.ini

      [ndbd default]
      # Options affecting ndbd processes on all data nodes:
      NoOfReplicas=2  # Number of replicas
      
      [ndb_mgmd]
      # Management process options:
      hostname=198.51.100.2 # Hostname of the manager
      datadir=/var/lib/mysql-cluster  # Directory for the log files
      
      [ndbd]
      hostname=198.51.100.0 # Hostname/IP of the first data node
      NodeId=2            # Node ID for this data node
      datadir=/usr/local/mysql/data   # Remote directory for the data files
      
      [ndbd]
      hostname=198.51.100.1 # Hostname/IP of the second data node
      NodeId=3            # Node ID for this data node
      datadir=/usr/local/mysql/data   # Remote directory for the data files
      
      [mysqld]
      # SQL node options:
      hostname=198.51.100.2 # In our case the MySQL server/client is on the same Droplet as the cluster manager
      

      After pasting in this text, being sure to replace the hostname values above with the correct IP addresses of the Droplets you’ve configured. Setting this hostname parameter is an important security measure that prevents other servers from connecting to the Cluster Manager.

      Save the file and close your text editor.

      This is a pared-down, minimal configuration file for a MySQL Cluster. You should customize the parameters in this file depending on your production needs. For a sample, fully configured ndb_mgmd configuration file, consult the MySQL Cluster documentation.

      In the above file you can add additional components like data nodes (ndbd) or MySQL server nodes (mysqld) by appending instances to the appropriate section.

      We can now start the manager by executing the ndb_mgmd binary and specifying its config file using the -f flag:

      • sudo ndb_mgmd -f /var/lib/mysql-cluster/config.ini

      You should see the following output:

      Output

      MySQL Cluster Management Server mysql-5.7.22 ndb-7.6.6 2018-07-25 21:48:39 [MgmtSrvr] INFO -- The default config directory '/usr/mysql-cluster' does not exist. Trying to create it... 2018-07-25 21:48:39 [MgmtSrvr] INFO -- Successfully created config directory

      This indicates that the MySQL Cluster Management server has successfully been installed and is now running on your Droplet.

      Ideally, we’d like to start the Cluster Management server automatically on boot. To do this, we’re going to create and enable a systemd service.

      Before we create the service, we need to kill the running server:

      Now, open and edit the following systemd Unit file using your favorite editor:

      • sudo nano /etc/systemd/system/ndb_mgmd.service

      Paste in the following code:

      /etc/systemd/system/ndb_mgmd.service

      [Unit]
      Description=MySQL NDB Cluster Management Server
      After=network.target auditd.service
      
      [Service]
      Type=forking
      ExecStart=/usr/sbin/ndb_mgmd -f /var/lib/mysql-cluster/config.ini
      ExecReload=/bin/kill -HUP $MAINPID
      KillMode=process
      Restart=on-failure
      
      [Install]
      WantedBy=multi-user.target
      

      Here, we’ve added a minimal set of options instructing systemd on how to start, stop and restart the ndb_mgmd process. To learn more about the options used in this unit configuration, consult the systemd manual.

      Save and close the file.

      Now, reload systemd’s manager configuration using daemon-reload:

      • sudo systemctl daemon-reload

      We’ll enable the service we just created so that the MySQL Cluster Manager starts on reboot:

      • sudo systemctl enable ndb_mgmd

      Finally, we’ll start the service:

      • sudo systemctl start ndb_mgmd

      You can verify that the NDB Cluster Management service is running:

      • sudo systemctl status ndb_mgmd

      You should see the following output:

      ● ndb_mgmd.service - MySQL NDB Cluster Management Server
         Loaded: loaded (/etc/systemd/system/ndb_mgmd.service; enabled; vendor preset: enabled)
         Active: active (running) since Thu 2018-07-26 21:23:37 UTC; 3s ago
        Process: 11184 ExecStart=/usr/sbin/ndb_mgmd -f /var/lib/mysql-cluster/config.ini (code=exited, status=0/SUCCESS)
       Main PID: 11193 (ndb_mgmd)
          Tasks: 11 (limit: 4915)
         CGroup: /system.slice/ndb_mgmd.service
                 └─11193 /usr/sbin/ndb_mgmd -f /var/lib/mysql-cluster/config.ini
      

      Which indicates that the ndb_mgmd MySQL Cluster Management server is now running as a systemd service.

      The final step for setting up the Cluster Manager is to allow incoming connections from other MySQL Cluster nodes on our private network.

      If you did not configure the ufw firewall when setting up this Droplet, you can skip ahead to the next section.

      We’ll add rules to allow local incoming connections from both data nodes:

      • sudo ufw allow from 198.51.100.0
      • sudo ufw allow from 198.51.100.1

      After entering these commands, you should see the following output:

      Output

      Rule added

      The Cluster Manager should now be up and running, and able to communicate with other Cluster nodes over the private network.

      Step 2 — Installing and Configuring the Data Nodes

      Note: All the commands in this section should be executed on both data nodes.

      In this step, we'll install the ndbd MySQL Cluster data node daemon, and configure the nodes so they can communicate with the Cluster Manager.

      To install the data node binaries we first need to fetch the appropriate .deb installer file from the official MySQL download page.

      From this page, under Select Operating System, choose Ubuntu Linux. Then, under Select OS Version, choose Ubuntu Linux 18.04 (x86, 64-bit).

      Scroll down until you see DEB Package, NDB Data Node Binaries, and click on the Download link for the one that does not contain dbgsym (unless you require debug symbols). You will be brought to a Begin Your Download page. Here, right click on No thanks, just start my download. and copy the link to the .deb file.

      Now, log in to your first data node Droplet (in this tutorial, 198.51.100.0), and download this .deb file:

      • cd ~
      • wget https://dev.mysql.com/get/Downloads/MySQL-Cluster-7.6/mysql-cluster-community-data-node_7.6.6-1ubuntu18.04_amd64.deb

      Before we install the data node binary, we need to install a dependency, libclass-methodmaker-perl:

      • sudo apt update
      • sudo apt install libclass-methodmaker-perl

      We can now install the data note binary using dpkg:

      • sudo dpkg -i mysql-cluster-community-data-node_7.6.6-1ubuntu18.04_amd64.deb

      The data nodes pull their configuration from MySQL’s standard location, /etc/my.cnf. Create this file using your favorite text editor and begin editing it:

      Add the following configuration parameter to the file:

      /etc/my.cnf

      [mysql_cluster]
      # Options for NDB Cluster processes:
      ndb-connectstring=198.51.100.2  # location of cluster manager
      

      Specifying the location of the Cluster Manager node is the only configuration needed for ndbd to start. The rest of the configuration will be pulled from the manager directly.

      Save and exit the file.

      In our example, the data node will find out that its data directory is /usr/local/mysql/data, per the manager's configuration. Before starting the daemon, we’ll create this directory on the node:

      • sudo mkdir -p /usr/local/mysql/data

      Now we can start the data node using the following command:

      You should see the following output:

      Output

      2018-07-18 19:48:21 [ndbd] INFO -- Angel connected to '198.51.100.2:1186' 2018-07-18 19:48:21 [ndbd] INFO -- Angel allocated nodeid: 2

      The NDB data node daemon has been successfully installed and is now running on your server.

      We also need to allow incoming connections from other MySQL Cluster nodes over the private network.

      If you did not configure the ufw firewall when setting up this Droplet, you can skip ahead to setting up the systemd service for ndbd.

      We’ll add rules to allow incoming connections from the Cluster Manager and other data nodes:

      • sudo ufw allow from 198.51.100.0
      • sudo ufw allow from 198.51.100.2

      After entering these commands, you should see the following output:

      Output

      Rule added

      Your MySQL data node Droplet can now communicate with both the Cluster Manager and other data node over the private network.

      Finally, we’d also like the data node daemon to start up automatically when the server boots. We’ll follow the same procedure used for the Cluster Manager, and create a systemd service.

      Before we create the service, we’ll kill the running ndbd process:

      Now, open and edit the following systemd Unit file using your favorite editor:

      • sudo nano /etc/systemd/system/ndbd.service

      Paste in the following code:

      /etc/systemd/system/ndbd.service

      [Unit]
      Description=MySQL NDB Data Node Daemon
      After=network.target auditd.service
      
      [Service]
      Type=forking
      ExecStart=/usr/sbin/ndbd
      ExecReload=/bin/kill -HUP $MAINPID
      KillMode=process
      Restart=on-failure
      
      [Install]
      WantedBy=multi-user.target
      

      Here, we’ve added a minimal set of options instructing systemd on how to start, stop and restart the ndbd process. To learn more about the options used in this unit configuration, consult the systemd manual.

      Save and close the file.

      Now, reload systemd’s manager configuration using daemon-reload:

      • sudo systemctl daemon-reload

      We’ll now enable the service we just created so that the data node daemon starts on reboot:

      • sudo systemctl enable ndbd

      Finally, we’ll start the service:

      • sudo systemctl start ndbd

      You can verify that the NDB Cluster Management service is running:

      • sudo systemctl status ndbd

      You should see the following output:

      Output

      ● ndbd.service - MySQL NDB Data Node Daemon Loaded: loaded (/etc/systemd/system/ndbd.service; enabled; vendor preset: enabled) Active: active (running) since Thu 2018-07-26 20:56:29 UTC; 8s ago Process: 11972 ExecStart=/usr/sbin/ndbd (code=exited, status=0/SUCCESS) Main PID: 11984 (ndbd) Tasks: 46 (limit: 4915) CGroup: /system.slice/ndbd.service ├─11984 /usr/sbin/ndbd └─11987 /usr/sbin/ndbd

      Which indicates that the ndbd MySQL Cluster data node daemon is now running as a systemd service. Your data node should now be fully functional and able to connect to the MySQL Cluster Manager.

      Once you’ve finished setting up the first data node, repeat the steps in this section on the other data node (198.51.100.1 in this tutorial).

      Step 3 — Configuring and Starting the MySQL Server and Client

      A standard MySQL server, such as the one available in Ubuntu's APT repository, does not support the MySQL Cluster engine NDB. This means we need to install the custom SQL server packaged with the other MySQL Cluster software we’ve installed in this tutorial.

      We’ll once again grab the MySQL Cluster Server binary from the official MySQL Cluster download page.

      From this page, under Select Operating System, choose Ubuntu Linux. Then, under Select OS Version, choose Ubuntu Linux 18.04 (x86, 64-bit).

      Scroll down until you see DEB Bundle, and click on the Download link (it should be the first one in the list). You will be brought to a Begin Your Download page. Here, right click on No thanks, just start my download. and copy the link to the .tar archive.

      Now, log in to the Cluster Manager Droplet (in this tutorial, 198.51.100.2), and download this .tar archive (recall that we are installing MySQL Server on the same node as our Cluster Manager – in a production setting you should run these daemons on different nodes):

      • cd ~
      • wget https://dev.mysql.com/get/Downloads/MySQL-Cluster-7.6/mysql-cluster_7.6.6-1ubuntu18.04_amd64.deb-bundle.tar

      We’ll now extract this archive into a directory called install. First, create the directory:

      Now extract the archive into this directory:

      • tar -xvf mysql-cluster_7.6.6-1ubuntu18.04_amd64.deb-bundle.tar -C install/

      Move into this directory, containing the extracted MySQL Cluster component binaries:

      Before we install the MySQL server binary, we need to install a couple of dependencies:

      • sudo apt update
      • sudo apt install libaio1 libmecab2

      Now, we need to install the MySQL Cluster dependencies, bundled in the tar archive we just extracted :

      • sudo dpkg -i mysql-common_7.6.6-1ubuntu18.04_amd64.deb
      • sudo dpkg -i mysql-cluster-community-client_7.6.6-1ubuntu18.04_amd64.deb
      • sudo dpkg -i mysql-client_7.6.6-1ubuntu18.04_amd64.deb
      • sudo dpkg -i mysql-cluster-community-server_7.6.6-1ubuntu18.04_amd64.deb

      When installing mysql-cluster-community-server, a configuration prompt should appear, asking you to set a password for the root account of your MySQL database. Choose a strong, secure password, and hit <Ok>. Re-enter this root password when prompted, and hit <Ok> once again to complete installation.

      We can now install the MySQL server binary using dpkg:

      • mysql-server_7.6.6-1ubuntu18.04_amd64.deb

      We now need to configure this MySQL server installation.

      The configuration for MySQL Server is stored in the default /etc/mysql/my.cnf file.

      Open this configuration file using your favorite editor:

      • sudo nano /etc/mysql/my.cnf

      You should see the following text:

      /etc/mysql/my.cnf

      # Copyright (c) 2015, 2016, Oracle and/or its affiliates. All rights reserved.
      #
      # This program is free software; you can redistribute it and/or modify
      # it under the terms of the GNU General Public License as published by
      # the Free Software Foundation; version 2 of the License.
      #
      # This program is distributed in the hope that it will be useful,
      # but WITHOUT ANY WARRANTY; without even the implied warranty of
      # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
      # GNU General Public License for more details.
      #
      # You should have received a copy of the GNU General Public License
      # along with this program; if not, write to the Free Software
      # Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA  02110-1301 USA
      
      #
      # The MySQL Cluster Community Server configuration file.
      #
      # For explanations see
      # http://dev.mysql.com/doc/mysql/en/server-system-variables.html
      
      # * IMPORTANT: Additional settings that can override those from this file!
      #   The files must end with '.cnf', otherwise they'll be ignored.
      #
      !includedir /etc/mysql/conf.d/
      !includedir /etc/mysql/mysql.conf.d/
      

      Append the following configuration to it:

      /etc/mysql/my.cnf

      . . .
      [mysqld]
      # Options for mysqld process:
      ndbcluster                      # run NDB storage engine
      
      [mysql_cluster]
      # Options for NDB Cluster processes:
      ndb-connectstring=198.51.100.2  # location of management server
      

      Save and exit the file.

      Restart the MySQL server for these changes to take effect:

      • sudo systemctl restart mysql

      MySQL by default should start automatically when your server reboots. If it doesn’t, the following command should fix this:

      • sudo systemctl enable mysql

      A SQL server should now be running on your Cluster Manager / MySQL Server Droplet.

      In the next step, we’ll run a few commands to verify that our MySQL Cluster installation is functioning as expected.

      Step 4 — Verifying MySQL Cluster Installation

      To verify your MySQL Cluster installation, log in to your Cluster Manager / SQL Server node.

      We’ll open the MySQL client from the command line and connect to the root account we just configured by entering the following command:

      Enter your password when prompted, and hit ENTER.

      You should see an output similar to:

      Output

      Welcome to the MySQL monitor. Commands end with ; or g. Your MySQL connection id is 3 Server version: 5.7.22-ndb-7.6.6 MySQL Cluster Community Server (GPL) Copyright (c) 2000, 2018, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or 'h' for help. Type 'c' to clear the current input statement. mysql>

      Once inside the MySQL client, run the following command:

      • SHOW ENGINE NDB STATUS G

      You should now see information about the NDB cluster engine, beginning with connection parameters:

      Output

      *************************** 1. row *************************** Type: ndbcluster Name: connection Status: cluster_node_id=4, connected_host=198.51.100.2, connected_port=1186, number_of_data_nodes=2, number_of_ready_data_nodes=2, connect_count=0 . . .

      This indicates that you’ve successfully connected to your MySQL Cluster.

      Notice here the number of ready_data_nodes: 2. This redundancy allows your MySQL cluster to continue operating even if one of the data nodes fails. It also means that your SQL queries will be load balanced across the two data nodes.

      You can try shutting down one of the data nodes to test cluster stability. The simplest test would be to restart the data node Droplet in order to fully test the recovery process. You should see the value of number_of_ready_data_nodes change to 1 and back up to 2 again as the node reboots and reconnects to the Cluster Manager.

      To exit the MySQL prompt, simply type quit or press CTRL-D.

      This is the first test that indicates that the MySQL cluster, server, and client are working. We'll now go through an additional test to confirm that the cluster is functioning properly.

      Open the Cluster management console, ndb_mgm using the command:

      You should see the following output:

      Output

      -- NDB Cluster -- Management Client -- ndb_mgm>

      Once inside the console enter the command SHOW and hit ENTER:

      You should see the following output:

      Output

      Connected to Management Server at: 198.51.100.2:1186 Cluster Configuration --------------------- [ndbd(NDB)] 2 node(s) id=2 @198.51.100.0 (mysql-5.7.22 ndb-7.6.6, Nodegroup: 0, *) id=3 @198.51.100.1 (mysql-5.7.22 ndb-7.6.6, Nodegroup: 0) [ndb_mgmd(MGM)] 1 node(s) id=1 @198.51.100.2 (mysql-5.7.22 ndb-7.6.6) [mysqld(API)] 1 node(s) id=4 @198.51.100.2 (mysql-5.7.22 ndb-7.6.6)

      The above shows that there are two data nodes connected with node-ids 2 and 3. There is also one management node with node-id 1 and one MySQL server with node-id 4. You can display more information about each id by typing its number with the command STATUS as follows:

      The above command shows you the status, MySQL version, and NDB version of node 2:

      Output

      Node 2: started (mysql-5.7.22 ndb-7.6.6)

      To exit the management console type quit, and then hit ENTER.

      The management console is very powerful and gives you many other options for administering the cluster and its data, including creating an online backup. For more information consult the official MySQL documentation.

      At this point, you’ve fully tested your MySQL Cluster installation. The concluding step of this guide shows you how to create and insert test data into this MySQL Cluster.

      Step 5 — Inserting Data into MySQL Cluster

      To demonstrate the cluster’s functionality, let's create a new table using the NDB engine and insert some sample data into it. Note that in order to use cluster functionality, the engine must be specified explicitly as NDB. If you use InnoDB (default) or any other engine, you will not make use of the cluster.

      First, let's create a database called clustertest with the command:

      • CREATE DATABASE clustertest;

      Next, switch to the new database:

      Now, create a simple table called test_table like this:

      • CREATE TABLE test_table (name VARCHAR(20), value VARCHAR(20)) ENGINE=ndbcluster;

      We have explicitly specified the engine ndbcluster in order to make use of the cluster.

      Now, we can start inserting data using this SQL query:

      • INSERT INTO test_table (name,value) VALUES('some_name','some_value');

      To verify that the data has been inserted, run the following select query:

      • SELECT * FROM test_table;

      When you insert data into and select data from an ndbcluster table, the cluster load balances queries between all the available data nodes. This improves the stability and performance of your MySQL database installation.

      You can also set the default storage engine to ndbcluster in the my.cnf file that we edited previously. If you do this, you won’t need to specify the ENGINE option when creating tables. To learn more, consult the MySQL Reference Manual.

      Conclusion

      In this tutorial, we’ve demonstrated how to set up and configure a MySQL Cluster on Ubuntu 18.04 servers. It’s important to note that this is a minimal, pared-down architecture used to demonstrate the installation procedure, and there are many advanced options and features worth learning about before deploying MySQL Cluster in production (for example, performing backups). To learn more, consult the official MySQL Cluster documentation.



      Source link

      How To Create a Kubernetes 1.11 Cluster Using Kubeadm on Ubuntu 18.04


      The author selected the Free and Open Source Fund to receive a donation as part of the Write for DOnations program.

      Introduction

      Kubernetes is a container orchestration system that manages containers at scale. Initially developed by Google based on its experience running containers in production, Kubernetes is open source and actively developed by a community around the world.

      Kubeadm automates the installation and configuration of Kubernetes components such as the API server, Controller Manager, and Kube DNS. It does not, however, create users or handle the installation of operating-system-level dependencies and their configuration. For these preliminary tasks, it is possible to use a configuration management tool like Ansible or SaltStack. Using these tools makes creating additional clusters or recreating existing clusters much simpler and less error prone.

      In this guide, you will set up a Kubernetes cluster from scratch using Ansible and Kubeadm, and then deploy a containerized Nginx application to it.

      Goals

      Your cluster will include the following physical resources:

      The master node (a node in Kubernetes refers to a server) is responsible for managing the state of the cluster. It runs Etcd, which stores cluster data among components that schedule workloads to worker nodes.

      Worker nodes are the servers where your workloads (i.e. containerized applications and services) will run. A worker will continue to run your workload once they’re assigned to it, even if the master goes down once scheduling is complete. A cluster’s capacity can be increased by adding workers.

      After completing this guide, you will have a cluster ready to run containerized applications, provided that the servers in the cluster have sufficient CPU and RAM resources for your applications to consume. Almost any traditional Unix application including web applications, databases, daemons, and command line tools can be containerized and made to run on the cluster. The cluster itself will consume around 300-500MB of memory and 10% of CPU on each node.

      Once the cluster is set up, you will deploy the web server Nginx to it to ensure that it is running workloads correctly.

      Prerequisites

      Step 1 — Setting Up the Workspace Directory and Ansible Inventory File

      In this section, you will create a directory on your local machine that will serve as your workspace. You will configure Ansible locally so that it can communicate with and execute commands on your remote servers. Once that’s done, you will create a hosts file containing inventory information such as the IP addresses of your servers and the groups that each server belongs to.

      Out of your three servers, one will be the master with an IP displayed as master_ip. The other two servers will be workers and will have the IPs worker_1_ip and worker_2_ip.

      Create a directory named ~/kube-cluster in the home directory of your local machine and cd into it:

      • mkdir ~/kube-cluster
      • cd ~/kube-cluster

      This directory will be your workspace for the rest of the tutorial and will contain all of your Ansible playbooks. It will also be the directory inside which you will run all local commands.

      Create a file named ~/kube-cluster/hosts using nano or your favorite text editor:

      • nano ~/kube-cluster/hosts

      Add the following text to the file, which will specify information about the logical structure of your cluster:

      ~/kube-cluster/hosts

      [masters]
      master ansible_host=master_ip ansible_user=root
      
      [workers]
      worker1 ansible_host=worker_1_ip ansible_user=root
      worker2 ansible_host=worker_2_ip ansible_user=root
      
      [all:vars]
      ansible_python_interpreter=/usr/bin/python3
      

      You may recall that inventory files in Ansible are used to specify server information such as IP addresses, remote users, and groupings of servers to target as a single unit for executing commands. ~/kube-cluster/hosts will be your inventory file and you’ve added two Ansible groups (masters and workers) to it specifying the logical structure of your cluster.

      In the masters group, there is a server entry named “master” that lists the master node’s IP (master_ip) and specifies that Ansible should run remote commands as the root user.

      Similarly, in the workers group, there are two entries for the worker servers (worker_1_ip and worker_2_ip) that also specify the ansible_user as root.

      The last line of the file tells Ansible to use the remote servers’ Python 3 interpreters for its management operations.

      Save and close the file after you’ve added the text.

      Having set up the server inventory with groups, let’s move on to installing operating system level dependencies and creating configuration settings.

      Step 2 — Creating a Non-Root User on All Remote Servers

      In this section you will create a non-root user with sudo privileges on all servers so that you can SSH into them manually as an unprivileged user. This can be useful if, for example, you would like to see system information with commands such as top/htop, view a list of running containers, or change configuration files owned by root. These operations are routinely performed during the maintenance of a cluster, and using a non-root user for such tasks minimizes the risk of modifying or deleting important files or unintentionally performing other dangerous operations.

      Create a file named ~/kube-cluster/initial.yml in the workspace:

      • nano ~/kube-cluster/initial.yml

      Next, add the following play to the file to create a non-root user with sudo privileges on all of the servers. A play in Ansible is a collection of steps to be performed that target specific servers and groups. The following play will create a non-root sudo user:

      ~/kube-cluster/initial.yml

      - hosts: all
        become: yes
        tasks:
          - name: create the 'ubuntu' user
            user: name=ubuntu append=yes state=present createhome=yes shell=/bin/bash
      
          - name: allow 'ubuntu' to have passwordless sudo
            lineinfile:
              dest: /etc/sudoers
              line: 'ubuntu ALL=(ALL) NOPASSWD: ALL'
              validate: 'visudo -cf %s'
      
          - name: set up authorized keys for the ubuntu user
            authorized_key: user=ubuntu key="{{item}}"
            with_file:
              - ~/.ssh/id_rsa.pub
      

      Here’s a breakdown of what this playbook does:

      • Creates the non-root user ubuntu.

      • Configures the sudoers file to allow the ubuntu user to run sudo commands without a password prompt.

      • Adds the public key in your local machine (usually ~/.ssh/id_rsa.pub) to the remote ubuntu user’s authorized key list. This will allow you to SSH into each server as the ubuntu user.

      Save and close the file after you’ve added the text.

      Next, execute the playbook by locally running:

      • ansible-playbook -i hosts ~/kube-cluster/initial.yml

      The command will complete within two to five minutes. On completion, you will see output similar to the following:

      Output

      PLAY [all] **** TASK [Gathering Facts] **** ok: [master] ok: [worker1] ok: [worker2] TASK [create the 'ubuntu' user] **** changed: [master] changed: [worker1] changed: [worker2] TASK [allow 'ubuntu' user to have passwordless sudo] **** changed: [master] changed: [worker1] changed: [worker2] TASK [set up authorized keys for the ubuntu user] **** changed: [worker1] => (item=ssh-rsa AAAAB3...) changed: [worker2] => (item=ssh-rsa AAAAB3...) changed: [master] => (item=ssh-rsa AAAAB3...) PLAY RECAP **** master : ok=5 changed=4 unreachable=0 failed=0 worker1 : ok=5 changed=4 unreachable=0 failed=0 worker2 : ok=5 changed=4 unreachable=0 failed=0

      Now that the preliminary setup is complete, you can move on to installing Kubernetes-specific dependencies.

      Step 3 — Installing Kubernetetes’ Dependencies

      In this section, you will install the operating-system-level packages required by Kubernetes with Ubuntu’s package manager. These packages are:

      • Docker – a container runtime. It is the component that runs your containers. Support for other runtimes such as rkt is under active development in Kubernetes.

      • kubeadm – a CLI tool that will install and configure the various components of a cluster in a standard way.

      • kubelet – a system service/program that runs on all nodes and handles node-level operations.

      • kubectl – a CLI tool used for issuing commands to the cluster through its API Server.

      Create a file named ~/kube-cluster/kube-dependencies.yml in the workspace:

      • nano ~/kube-cluster/kube-dependencies.yml

      Add the following plays to the file to install these packages to your servers:

      ~/kube-cluster/kube-dependencies.yml

      - hosts: all
        become: yes
        tasks:
         - name: install Docker
           apt:
             name: docker.io
             state: present
             update_cache: true
      
         - name: install APT Transport HTTPS
           apt:
             name: apt-transport-https
             state: present
      
         - name: add Kubernetes apt-key
           apt_key:
             url: https://packages.cloud.google.com/apt/doc/apt-key.gpg
             state: present
      
         - name: add Kubernetes' APT repository
           apt_repository:
            repo: deb http://apt.kubernetes.io/ kubernetes-xenial main
            state: present
            filename: 'kubernetes'
      
         - name: install kubelet
           apt:
             name: kubelet
             state: present
             update_cache: true
      
         - name: install kubeadm
           apt:
             name: kubeadm
             state: present
      
      - hosts: master
        become: yes
        tasks:
         - name: install kubectl
           apt:
             name: kubectl
             state: present
      

      The first play in the playbook does the following:

      • Installs Docker, the container runtime.

      • Installs apt-transport-https, allowing you to add external HTTPS sources to your APT sources list.

      • Adds the Kubernetes APT repository’s apt-key for key verification.

      • Adds the Kubernetes APT repository to your remote servers’ APT sources list.

      • Installs kubelet and kubeadm.

      The second play consists of a single task that installs kubectl on your master node.

      Save and close the file when you are finished.

      Next, execute the playbook by locally running:

      • ansible-playbook -i hosts ~/kube-cluster/kube-dependencies.yml

      On completion, you will see output similar to the following:

      Output

      PLAY [all] **** TASK [Gathering Facts] **** ok: [worker1] ok: [worker2] ok: [master] TASK [install Docker] **** changed: [master] changed: [worker1] changed: [worker2] TASK [install APT Transport HTTPS] ***** ok: [master] ok: [worker1] changed: [worker2] TASK [add Kubernetes apt-key] ***** changed: [master] changed: [worker1] changed: [worker2] TASK [add Kubernetes' APT repository] ***** changed: [master] changed: [worker1] changed: [worker2] TASK [install kubelet] ***** changed: [master] changed: [worker1] changed: [worker2] TASK [install kubeadm] ***** changed: [master] changed: [worker1] changed: [worker2] PLAY [master] ***** TASK [Gathering Facts] ***** ok: [master] TASK [install kubectl] ****** ok: [master] PLAY RECAP **** master : ok=9 changed=5 unreachable=0 failed=0 worker1 : ok=7 changed=5 unreachable=0 failed=0 worker2 : ok=7 changed=5 unreachable=0 failed=0

      After execution, Docker, kubeadm, and kubelet will be installed on all of the remote servers. kubectl is not a required component and is only needed for executing cluster commands. Installing it only on the master node makes sense in this context, since you will run kubectl commands only from the master. Note, however, that kubectl commands can be run from any of the worker nodes or from any machine where it can be installed and configured to point to a cluster.

      All system dependencies are now installed. Let’s set up the master node and initialize the cluster.

      Step 4 — Setting Up the Master Node

      In this section, you will set up the master node. Before creating any playbooks, however, it’s worth covering a few concepts such as Pods and Pod Network Plugins, since your cluster will include both.

      A pod is an atomic unit that runs one or more containers. These containers share resources such as file volumes and network interfaces in common. Pods are the basic unit of scheduling in Kubernetes: all containers in a pod are guaranteed to run on the same node that the pod is scheduled on.

      Each pod has its own IP address, and a pod on one node should be able to access a pod on another node using the pod’s IP. Containers on a single node can communicate easily through a local interface. Communication between pods is more complicated, however, and requires a separate networking component that can transparently route traffic from a pod on one node to a pod on another.

      This functionality is provided by pod network plugins. For this cluster, you will use Flannel, a stable and performant option.

      Create an Ansible playbook named master.yml on your local machine:

      • nano ~/kube-cluster/master.yml

      Add the following play to the file to initialize the cluster and install Flannel:

      ~/kube-cluster/master.yml

      - hosts: master
        become: yes
        tasks:
          - name: initialize the cluster
            shell: kubeadm init --pod-network-cidr=10.244.0.0/16 >> cluster_initialized.txt
            args:
              chdir: $HOME
              creates: cluster_initialized.txt
      
          - name: create .kube directory
            become: yes
            become_user: ubuntu
            file:
              path: $HOME/.kube
              state: directory
              mode: 0755
      
          - name: copy admin.conf to user's kube config
            copy:
              src: /etc/kubernetes/admin.conf
              dest: /home/ubuntu/.kube/config
              remote_src: yes
              owner: ubuntu
      
          - name: install Pod network
            become: yes
            become_user: ubuntu
            shell: kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/v0.9.1/Documentation/kube-flannel.yml >> pod_network_setup.txt
            args:
              chdir: $HOME
              creates: pod_network_setup.txt
      

      Here’s a breakdown of this play:

      • The first task initializes the cluster by running kubeadm init. Passing the argument --pod-network-cidr=10.244.0.0/16 specifies the private subnet that the pod IPs will be assigned from. Flannel uses the above subnet by default; we’re telling kubeadm to use the same subnet.

      • The second task creates a .kube directory at /home/ubuntu. This directory will hold configuration information such as the admin key files, which are required to connect to the cluster, and the cluster’s API address.

      • The third task copies the /etc/kubernetes/admin.conf file that was generated from kubeadm init to your non-root user’s home directory. This will allow you to use kubectl to access the newly-created cluster.

      • The last task runs kubectl apply to install Flannel. kubectl apply -f descriptor.[yml|json] is the syntax for telling kubectl to create the objects described in the descriptor.[yml|json] file. The kube-flannel.yml file contains the descriptions of objects required for setting up Flannel in the cluster.

      Save and close the file when you are finished.

      Execute the playbook locally by running:

      • ansible-playbook -i hosts ~/kube-cluster/master.yml

      On completion, you will see output similar to the following:

      Output

      PLAY [master] **** TASK [Gathering Facts] **** ok: [master] TASK [initialize the cluster] **** changed: [master] TASK [create .kube directory] **** changed: [master] TASK [copy admin.conf to user's kube config] ***** changed: [master] TASK [install Pod network] ***** changed: [master] PLAY RECAP **** master : ok=5 changed=4 unreachable=0 failed=0

      To check the status of the master node, SSH into it with the following command:

      Once inside the master node, execute:

      You will now see the following output:

      Output

      NAME STATUS ROLES AGE VERSION master Ready master 1d v1.11.1

      The output states that the master node has completed all initialization tasks and is in a Ready state from which it can start accepting worker nodes and executing tasks sent to the API Server. You can now add the workers from your local machine.

      Step 5 — Setting Up the Worker Nodes

      Adding workers to the cluster involves executing a single command on each. This command includes the necessary cluster information, such as the IP address and port of the master's API Server, and a secure token. Only nodes that pass in the secure token will be able join the cluster.

      Navigate back to your workspace and create a playbook named workers.yml:

      • nano ~/kube-cluster/workers.yml

      Add the following text to the file to add the workers to the cluster:

      ~/kube-cluster/workers.yml

      - hosts: master
        become: yes
        gather_facts: false
        tasks:
          - name: get join command
            shell: kubeadm token create --print-join-command
            register: join_command_raw
      
          - name: set join command
            set_fact:
              join_command: "{{ join_command_raw.stdout_lines[0] }}"
      
      
      - hosts: workers
        become: yes
        tasks:
          - name: join cluster
            shell: "{{ hostvars['master'].join_command }} >> node_joined.txt"
            args:
              chdir: $HOME
              creates: node_joined.txt
      

      Here's what the playbook does:

      • The first play gets the join command that needs to be run on the worker nodes. This command will be in the following format:kubeadm join --token <token> <master-ip>:<master-port> --discovery-token-ca-cert-hash sha256:<hash>. Once it gets the actual command with the proper token and hash values, the task sets it as a fact so that the next play will be able to access that info.

      • The second play has a single task that runs the join command on all worker nodes. On completion of this task, the two worker nodes will be part of the cluster.

      Save and close the file when you are finished.

      Execute the playbook by locally running:

      • ansible-playbook -i hosts ~/kube-cluster/workers.yml

      On completion, you will see output similar to the following:

      Output

      PLAY [master] **** TASK [get join command] **** changed: [master] TASK [set join command] ***** ok: [master] PLAY [workers] ***** TASK [Gathering Facts] ***** ok: [worker1] ok: [worker2] TASK [join cluster] ***** changed: [worker1] changed: [worker2] PLAY RECAP ***** master : ok=2 changed=1 unreachable=0 failed=0 worker1 : ok=2 changed=1 unreachable=0 failed=0 worker2 : ok=2 changed=1 unreachable=0 failed=0

      With the addition of the worker nodes, your cluster is now fully set up and functional, with workers ready to run workloads. Before scheduling applications, let's verify that the cluster is working as intended.

      Step 6 — Verifying the Cluster

      A cluster can sometimes fail during setup because a node is down or network connectivity between the master and worker is not working correctly. Let's verify the cluster and ensure that the nodes are operating correctly.

      You will need to check the current state of the cluster from the master node to ensure that the nodes are ready. If you disconnected from the master node, you can SSH back into it with the following command:

      Then execute the following command to get the status of the cluster:

      You will see output similar to the following:

      Output

      NAME STATUS ROLES AGE VERSION master Ready master 1d v1.11.1 worker1 Ready <none> 1d v1.11.1 worker2 Ready <none> 1d v1.11.1

      If all of your nodes have the value Ready for STATUS, it means that they're part of the cluster and ready to run workloads.

      If, however, a few of the nodes have NotReady as the STATUS, it could mean that the worker nodes haven't finished their setup yet. Wait for around five to ten minutes before re-running kubectl get nodes and inspecting the new output. If a few nodes still have NotReady as the status, you might have to verify and re-run the commands in the previous steps.

      Now that your cluster is verified successfully, let's schedule an example Nginx application on the cluster.

      Step 7 — Running An Application on the Cluster

      You can now deploy any containerized application to your cluster. To keep things familiar, let's deploy Nginx using Deployments and Services to see how this application can be deployed to the cluster. You can use the commands below for other containerized applications as well, provided you change the Docker image name and any relevant flags (such as ports and volumes).

      Still within the master node, execute the following command to create a deployment named nginx:

      • kubectl run nginx --image=nginx --port 80

      A deployment is a type of Kubernetes object that ensures there's always a specified number of pods running based on a defined template, even if the pod crashes during the cluster's lifetime. The above deployment will create a pod with one container from the Docker registry's Nginx Docker Image.

      Next, run the following command to create a service named nginx that will expose the app publicly. It will do so through a NodePort, a scheme that will make the pod accessible through an arbitrary port opened on each node of the cluster:

      • kubectl expose deploy nginx --port 80 --target-port 80 --type NodePort

      Services are another type of Kubernetes object that expose cluster internal services to clients, both internal and external. They are also capable of load balancing requests to multiple pods, and are an integral component in Kubernetes, frequently interacting with other components.

      Run the following command:

      This will output text similar to the following:

      Output

      NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 1d nginx NodePort 10.109.228.209 <none> 80:nginx_port/TCP 40m

      From the third line of the above output, you can retrieve the port that Nginx is running on. Kubernetes will assign a random port that is greater than 30000 automatically, while ensuring that the port is not already bound by another service.

      To test that everything is working, visit http://worker_1_ip:nginx_port or http://worker_2_ip:nginx_port through a browser on your local machine. You will see Nginx's familiar welcome page.

      If you would like to remove the Nginx application, first delete the nginx service from the master node:

      • kubectl delete service nginx

      Run the following to ensure that the service has been deleted:

      You will see the following output:

      Output

      NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 1d

      Then delete the deployment:

      • kubectl delete deployment nginx

      Run the following to confirm that this worked:

      Output

      No resources found.

      Conclusion

      In this guide, you've successfully set up a Kubernetes cluster on Ubuntu 18.04 using Kubeadm and Ansible for automation.

      If you're wondering what to do with the cluster now that it's set up, a good next step would be to get comfortable deploying your own applications and services onto the cluster. Here's a list of links with further information that can guide you in the process:

      • Dockerizing applications - lists examples that detail how to containerize applications using Docker.

      • Pod Overview - describes in detail how Pods work and their relationship with other Kubernetes objects. Pods are ubiquitous in Kubernetes, so understanding them will facilitate your work.

      • Deployments Overview - provides an overview of deployments. It is useful to understand how controllers such as deployments work since they are used frequently in stateless applications for scaling and the automated healing of unhealthy applications.

      • Services Overview - covers services, another frequently used object in Kubernetes clusters. Understanding the types of services and the options they have is essential for running both stateless and stateful applications.

      Other important concepts that you can look into are Volumes, Ingresses and Secrets, all of which come in handy when deploying production applications.

      Kubernetes has a lot of functionality and features to offer. The Kubernetes Official Documentation is the best place to learn about concepts, find task-specific guides, and look up API references for various objects.



      Source link