One place for hosting & domains

      How to Configure Load Balancing with TLS Encryption on a Kubernetes Cluster


      Updated by Linode Contributed by Linode

      This guide will use an example Kubernetes Deployment and Service to demonstrate how to route external traffic to a Kubernetes application over HTTPS. This is accomplished using the NGINX Ingress Controller, cert-manager and Linode NodeBalancers. The NGINX Ingress Controller uses Linode NodeBalancers, which are Linode’s load balancing service, to route a Kubernetes Service’s traffic to the appropriate backend Pods over HTTP and HTTPS. cert-manager creates a Transport Layer Security (TLS) certificate from the Let’s Encrypt certificate authority (CA) providing secure HTTPS access to a Kubernetes Service.

      Note

      Before you Begin

      1. This guide assumes that your Kubernetes cluster has the Linode Cloud Controller Manager (CCM) installed on your Kubernetes cluster. The Linode CCM is installed by default on clusters deployed with the Linode Kubernetes Engine and the Linode Terraform K8s module.

        Note

        The recommended way to deploy a Kubernetes cluster on Linode is using the Linode Kubernetes Engine (managed) or the Linode Terraform K8s module (unmanaged). However, to learn how to install the Linode CCM on a cluster that was not installed in the two ways mentioned above, see the Installing the Linode CCM on an Unmanaged Kubernetes Cluster guide.
      2. Install Helm 3, and kubectl to your local environment.

      3. Purchase a domain name from a reliable domain registrar. In a later section, you will use Linode’s DNS Manager, to create a new Domain and to add a DNS “A” record for two subdomains, one named blog and another named shop. Your subdomains will point to the example Kubernetes Services you will create in this guide. The example domain names used throughout this guide are blog.example.com and shop.example.com.

        Note

        Optionally, you can create a Wildcard DNS record, *.example.com and point your NodeBalancer’s external IP address to it. Using a Wildcard DNS record, will allow you to expose your Kubernetes services without requiring further configuration using the Linode DNS Manager.

      Create an Example Application

      The primary focus of this guide is to show you how to use the NGINX Ingress Controller and cert-manager to route traffic to a Kubernetes application over HTTPS. In this section, you will create two example applications that you will route external traffic to in a later section. The example application displays a page that returns information about the Deployment’s current backend Pod. This sample application is built using NGINX’s demo Docker image, nginxdemos/hello. You can replace the example applications used in this section with your own.

      Create your Application Service and Deployment

      Each example manifest file creates three Pods to serve the application.

      1. Using a text editor, create a new file named hello-one.yaml with the contents of the example file.

        hello-one.yaml
         1
         2
         3
         4
         5
         6
         7
         8
         9
        10
        11
        12
        13
        14
        15
        16
        17
        18
        19
        20
        21
        22
        23
        24
        25
        26
        27
        28
        29
        30
        31
        32
        
        apiVersion: v1
        kind: Service
        metadata:
          name: hello-one
        spec:
          type: ClusterIP
          ports:
          - port: 80
            targetPort: 80
          selector:
            app: hello-one
        ---
        apiVersion: apps/v1
        kind: Deployment
        metadata:
          name: hello-one
        spec:
          replicas: 3
          selector:
            matchLabels:
              app: hello-one
          template:
            metadata:
              labels:
                app: hello-one
            spec:
              containers:
              - name: hello-ingress
                image: nginxdemos/hello
                ports:
                - containerPort: 80
                
      2. Create a second Service and Deployment manifest file named hello-two.yaml with the contents of the example file.

        hello-two.yaml
         1
         2
         3
         4
         5
         6
         7
         8
         9
        10
        11
        12
        13
        14
        15
        16
        17
        18
        19
        20
        21
        22
        23
        24
        25
        26
        27
        28
        29
        30
        31
        32
        
        apiVersion: v1
        kind: Service
        metadata:
          name: hello-two
        spec:
          type: ClusterIP
          ports:
          - port: 80
            targetPort: 80
          selector:
            app: hello-two
        ---
        apiVersion: apps/v1
        kind: Deployment
        metadata:
          name: hello-two
        spec:
          replicas: 3
          selector:
            matchLabels:
              app: hello-two
          template:
            metadata:
              labels:
                app: hello-two
            spec:
              containers:
              - name: hello-ingress
                image: nginxdemos/hello
                ports:
                - containerPort: 80
                
      3. Use kubectl to create the Services and Deployments for your example applications.

        kubectl create -f hello-one.yaml
        kubectl create -f hello-two.yaml
        

        You should see a similar output:

          
        service/hello-one created
        deployment.apps/hello-one created
        service/hello-two created
        deployment.apps/hello-two created
            
        
      4. Verify that the Services are running.

        kubectl get svc
        

        You should see a similar output:

          
        NAME         TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)   AGE
        hello-one    ClusterIP   10.128.94.166            80/TCP    6s
        hello-two    ClusterIP   10.128.102.187           80/TCP    6s
        kubernetes   ClusterIP   10.128.0.1               443/TCP   18m
            
        

      Install the NGINX Ingress Controller

      In this section you will use Helm to install the NGINX Ingress Controller on your Kubernetes Cluster. Installing the NGINX Ingress Controller will create Linode NodeBalancers that your cluster can make use of to load balance traffic to your example application.

      Note

      1. Add the Google stable Helm charts repository to your Helm repos.

        helm repo add stable https://kubernetes-charts.storage.googleapis.com/
        
      2. Update your Helm repositories.

        helm repo update
        
      3. Install the NGINX Ingress Controller. This installation will result in a Linode NodeBalancer being created.

        helm install nginx-ingress stable/nginx-ingress
        

        You will see a similar output:

          
        NAME: nginx-ingress
        LAST DEPLOYED: Mon Jul 20 10:27:03 2020
        NAMESPACE: default
        STATUS: deployed
        REVISION: 1
        TEST SUITE: None
        NOTES:
        The nginx-ingress controller has been installed.
        It may take a few minutes for the LoadBalancer IP to be available.
        You can watch the status by running 'kubectl --namespace default get services -o wide -w nginx-ingress-controller'
        ...
           
        

      Create a Subdomain DNS Entries for your Example Applications

      Now that Linode NodeBalancers have been created by the NGINX Ingress Controller, you can point a subdomain DNS entries to the NodeBalancer’s public IPv4 address. Since this guide uses two example applications, it will require two subdomain entries.

      1. Access your NodeBalancer’s assigned external IP address.

        kubectl --namespace default get services -o wide -w nginx-ingress-controller
        

        The command will return a similar output:

          
        NAME                          TYPE           CLUSTER-IP      EXTERNAL-IP    PORT(S)                      AGE     SELECTOR
        my-nginx-ingress-controller   LoadBalancer   10.128.169.60   192.0.2.0   80:32401/TCP,443:30830/TCP   7h51m   app.kubernetes.io/component=controller,app=nginx-ingress,release=my-nginx-ingress
            
        
      2. Copy the IP address of the EXTERNAL IP field and navigate to Linode’s DNS manager and add two “A” records for the blog and shop subdomains. Ensure you point each record to the NodeBalancer’s IPv4 address you retrieved in the previous step.

      Now that your NGINX Ingress Controller has been deployed and your subdomains’ A records have been created, you are ready to enable HTTPS on each subdomain.

      Create a TLS Certificate Using cert-manager

      Note

      Before performing the commands in this section, ensure that your DNS has had time to propagate across the internet. This process can take a while. You can query the status of your DNS by using the following command, substituting blog.example.com for your domain.

      dig +short blog.example.com
      

      If successful, the output should return the IP address of your NodeBalancer.

      To enable HTTPS on your example application, you will create a Transport Layer Security (TLS) certificate from the Let’s Encrypt certificate authority (CA) using the ACME protocol. This will be facilitated by cert-manager, the native Kubernetes certificate management controller.

      In this section you will install cert-manager using Helm and the required cert-manager CustomResourceDefinitions (CRDs). Then, you will create a ClusterIssuer resource to assist in creating a cluster’s TLS certificate.

      Note

      If you would like a deeper dive into cert-manager, see our guide [’’]().

      Install cert-manager

      1. Install cert-manager’s CRDs.

        kubectl apply --validate=false -f https://github.com/jetstack/cert-manager/releases/download/v0.15.1/cert-manager.crds.yaml
        
      2. Create a cert-manager namespace.

        kubectl create namespace cert-manager
        
      3. Add the Helm repository which contains the cert-manager Helm chart.

        helm repo add jetstack https://charts.jetstack.io
        
      4. Update your Helm repositories.

        helm repo update
        
      5. Install the cert-manager Helm chart. These basic configurations should be sufficient for many use cases, however, additional cert-manager configurable parameters can be found in cert-manager’s official documentation.

        helm install 
        cert-manager jetstack/cert-manager 
        --namespace cert-manager 
        --version v0.15.1
        
      6. Verify that the corresponding cert-manager pods are now running.

        kubectl get pods --namespace cert-manager
        

        You should see a similar output:

          
        NAME                                       READY   STATUS    RESTARTS   AGE
        cert-manager-579d48dff8-84nw9              1/1     Running   3          1m
        cert-manager-cainjector-789955d9b7-jfskr   1/1     Running   3          1m
        cert-manager-webhook-64869c4997-hnx6n      1/1     Running   0          1m
            
        

        Note

        You should wait until all cert-manager pods are ready and running prior to proceeding to the next section.

      Create a ClusterIssuer Resource

      1. Create a manifest file named acme-issuer-prod.yaml that will be used to create a ClusterIssuer resource on your cluster. Ensure you replace [email protected] with your own email address.

        acme-issuer-prod.yaml
         1
         2
         3
         4
         5
         6
         7
         8
         9
        10
        11
        12
        13
        14
        15
        16
        
        apiVersion: cert-manager.io/v1alpha2
        kind: ClusterIssuer
        metadata:
          name: letsencrypt-prod
        spec:
          acme:
            email: [email protected]
            server: https://acme-v02.api.letsencrypt.org/directory
            privateKeySecretRef:
              name: letsencrypt-secret-prod
            solvers:
            - http01:
                ingress:
                  class: nginx
        
              
        • This manifest file creates a ClusterIssuer resource that will register an account on an ACME server. The value of spec.acme.server designates Let’s Encrypt’s production ACME server, which should be trusted by most browsers.

          Note

          Let’s Encrypt provides a staging ACME server that can be used to test issuing trusted certificates, while not worrying about hitting Let’s Encrypt’s production rate limits. The staging URL is https://acme-staging-v02.api.letsencrypt.org/directory.
        • The value of privateKeySecretRef.name provides the name of a secret containing the private key for this user’s ACME server account (this is tied to the email address you provide in the manifest file). The ACME server will use this key to identify you.

        • To ensure that you own the domain for which you will create a certificate, the ACME server will issue a challenge to a client. cert-manager provides two options for solving challenges, http01 and DNS01. In this example, the http01 challenge solver will be used and it is configured in the solvers array. cert-manager will spin up challenge solver Pods to solve the issued challenges and use Ingress resources to route the challenge to the appropriate Pod.

      2. Create the ClusterIssuer resource:

        kubectl create -f acme-issuer-prod.yaml
        

        You should see a similar output:

          
        clusterissuer.cert-manager.io/letsencrypt-prod created
              
        

      Enable HTTPS for your Application

      Create the Ingress Resource

      1. Create an Ingress resource manifest file named hello-app-ingress.yaml. If you assigned a different name to your ClusterIssuer, ensure you replace letsencrypt-prod with the name you used. Replace all hosts and host values with your own application’s domain name.

        hello-app-ingress.yaml
         1
         2
         3
         4
         5
         6
         7
         8
         9
        10
        11
        12
        13
        14
        15
        16
        17
        18
        19
        20
        21
        22
        23
        24
        25
        26
        27
        
        apiVersion: networking.k8s.io/v1beta1
        kind: Ingress
        metadata:
          name: hello-app-ingress
          annotations:
            kubernetes.io/ingress.class: "nginx"
            cert-manager.io/cluster-issuer: "letsencrypt-prod"
        spec:
          tls:
          - hosts:
            - blog.example.com
            - shop.example.com
            secretName: example-tls
          rules:
          - host: blog.example.com
            http:
              paths:
              - backend:
                  serviceName: hello-one
                  servicePort: 80
          - host: shop.example.com
            http:
              paths:
              - backend:
                  serviceName: hello-two
                  servicePort: 80
            

        This resource defines how traffic coming from the Linode NodeBalancers is handled. In this case, NGINX will accept these connections over port 80, diverting traffic to both of your services via their domain names. The tls section of the Ingress resource manifest handles routing HTTPS traffic to the hostnames that are defined.

      2. Create the Ingress resource.

        kubectl create -f hello-app-ingress.yaml
        

        You should see a similar output:

          
        ingress.networking.k8s.io/hello-app-ingress created
            
        
      3. Navigate to your app’s domain or if you have been following along with the example, navigate to blog.example.com and then, shop.example.com. You should see the demo NGINX page load and display information about the Pod being used to serve your request.

        The NGINX demo page loads with information about the Pod being used to serve your request

        Use your browser to view your TLS certificate. You should see that the certificate was issued by Let’s Encrypt Authority X3.

        Use your browser to view your TLS certificate.

      More Information

      You may wish to consult the following resources for additional information on this topic. While these are provided in the hope that they will be useful, please note that we cannot vouch for the accuracy or timeliness of externally hosted materials.

      This guide is published under a CC BY-ND 4.0 license.



      Source link