One place for hosting & domains

      Registry

      How to Set Up a Private Docker Registry with Linode Kubernetes Engine and Object Storage


      Updated by Leslie Salazar Contributed by Leslie Salazar

      Marquee image for How to Set Up a Private Docker Registry with Linode Kubernetes Engine and Object Storage

      Hosting a private Docker registry alongside your Kubernetes cluster allows you to securely manage your Docker images while also providing quick deployment of your apps. This guide will walk you through the steps needed to deploy a private Docker registry on a Linode Kubernetes Engine (LKE) cluster. At the end of this tutorial, you will be able to locally push and pull Docker images to your registry. Similarly, your LKE cluster’s pods will also be able to pull Docker images from the registry to complete their deployments.

      Before you Begin

      Note

      1. Deploy a LKE Cluster. This example was written using a node pool with two 2 GB nodes. Depending on the workloads you will be deploying on your cluster, you may consider using nodes with higher resources.

      2. Install Helm 3, kubectl, and Docker to your local environment.

        Note

      3. Ensure Object Storage is enabled on your Linode account, generate an Object Storage key pair and ensure you save it in a secure location. You will need the key pair for a later section in this guide. Finally create an Object Storage bucket to store your registry’s images. Throughout this guide, the example bucket name will be registry.

      4. Purchase a domain name from a reliable domain registrar. Using Linode’s DNS Manager, create a new Domain and add an DNS “A” record for a subdomain named registry. Your subdomain will host your Docker registry. This guide will use registry.example.com as the example domain.

        Note

        Optionally, you can create a Wildcard DNS record, *.example.com. In a later section, you will point your DNS A record to a Linode NodeBalancer’s external IP address. Using a Wildcard DNS record, will allow you to expose your Kubernetes services without requiring further configuration using the Linode DNS Manager.

      In this Guide

      In this guide you will:

      Install the NGINX Ingress Controller

      An Ingress is used to provide external routes, via HTTP or HTTPS, to your cluster’s services. An Ingress Controller, like the NGINX Ingress Controller, fulfills the requirements presented by the Ingress using a load balancer.

      In this section, you will install the NGINX Ingress Controller using Helm, which will create a Linode NodeBalancer to handle your cluster’s traffic.

      1. Add the Google stable Helm charts repository to your Helm repos:

        helm repo add stable https://kubernetes-charts.storage.googleapis.com/
        
      2. Update your Helm repositories:

        helm repo update
        
      3. Install the NGINX Ingress Controller. This installation will result in a Linode NodeBalancer being created.

        helm install nginx-ingress stable/nginx-ingress
        

        You will see a similar output after issuing the above command (the output has been truncated for brevity):

          
        NAME: my-nginx-ingress
        LAST DEPLOYED: Wed Apr  8 09:55:47 2020
        NAMESPACE: default
        STATUS: deployed
        REVISION: 1
        TEST SUITE: None
        NOTES:
        The nginx-ingress controller has been installed.
        It may take a few minutes for the LoadBalancer IP to be available.
        You can watch the status by running 'kubectl --namespace default get services -o wide -w my-nginx-ingress-controller'
        ...
            
        

        In the next section, you will use your Linode NodeBalancer’s external IP address to update your registry’s domain record.

      Update your Subdomain’s IP Address

      1. Access your NodeBalancer’s assigned external IP address.

        kubectl --namespace default get services -o wide -w nginx-ingress-controller
        

        The command will return a similar output:

          
        NAME                          TYPE           CLUSTER-IP      EXTERNAL-IP    PORT(S)                      AGE     SELECTOR
        my-nginx-ingress-controller   LoadBalancer   10.128.169.60   192.0.2.0   80:32401/TCP,443:30830/TCP   7h51m   app.kubernetes.io/component=controller,app=nginx-ingress,release=my-nginx-ingress
            
        
      2. Copy the IP address of the EXTERNAL IP field and navigate to Linode’s DNS manager and update your domain’s’ registry A record with the external IP address. Ensure that the entry’s TTL field is set to 5 minutes.

      Now that your NGINX Ingress Controller has been deployed and your subdomain’s A record has been updated, you are ready to enable HTTPS on your Docker registry.

      Enable HTTPS

      Note

      Before performing the commands in this section, ensure that your DNS has had time to propagate across the internet. This process can take several hours. You can query the status of your DNS by using the following command, substituting registry.example.com for your subdomain and domain.

      dig +short registry.example.com
      

      If successful, the output should return the IP address of your NodeBalancer.

      To enable HTTPS on your Docker registry, you will create a Transport Layer Security (TLS) certificate from the Let’s Encrypt certificate authority (CA) using the ACME protocol. This will be facilitated by cert-manager, the native Kubernetes certificate management controller.

      In this section you will install cert-manager using Helm and the required cert-manager CustomResourceDefinitions (CRDs). Then, you will create a ClusterIssuer and Certificate resource to create your cluster’s TLS certificate.

      Install cert-manager

      1. Install cert-manager’s CRDs.

        kubectl apply --validate=false -f https://github.com/jetstack/cert-manager/releases/download/v0.14.1/cert-manager.crds.yaml
        
      2. Create a cert-manager namespace.

        kubectl create namespace cert-manager
        
      3. Add the Helm repository which contains the cert-manager Helm chart.

        helm repo add jetstack https://charts.jetstack.io
        
      4. Update your Helm repositories.

        helm repo update
        
      5. Install the cert-manager Helm chart. These basic configurations should be sufficient for many use cases, however, additional cert-manager configurable parameters can be found in cert-manager’s official documentation.

        helm install 
        cert-manager jetstack/cert-manager 
        --namespace cert-manager 
        --version v0.14.1
        
      6. Verify that the corresponding cert-manager pods are now running.

        kubectl get pods --namespace cert-manager
        

        You should see a similar output:

          
        NAME                                       READY   STATUS    RESTARTS   AGE
        cert-manager-579d48dff8-84nw9              1/1     Running   3          1m
        cert-manager-cainjector-789955d9b7-jfskr   1/1     Running   3          1m
        cert-manager-webhook-64869c4997-hnx6n      1/1     Running   0          1m
            
        

      Create a ClusterIssuer Resource

      Now that cert-manager is installed and running on your cluster, you will need to create a ClusterIssuer resource which defines which CA can create signed certificates when a certificate request is received. A ClusterIssuer is not a namespaced resource, so it can be used by more than one namespace.

      1. Create a directory named registry to store all of your Docker registry’s related manifest files and move into the new directory.

        mkdir ~/registry && cd ~/registry
        
      2. Using the text editor of your choice, create a file named acme-issuer-prod.yaml with the example configurations. Replace the value of email with your own email address.

        ~/registry/acme-issuer-prod.yaml
         1
         2
         3
         4
         5
         6
         7
         8
         9
        10
        11
        12
        13
        14
        15
        
        apiVersion: cert-manager.io/v1alpha2
        kind: ClusterIssuer
        metadata:
          name: letsencrypt-prod
        spec:
          acme:
            email: [email protected]
            server: https://acme-v02.api.letsencrypt.org/directory
            privateKeySecretRef:
              name: letsencrypt-secret-prod
            solvers:
            - http01:
                ingress:
                  class: nginx
            
        • This manifest file creates a ClusterIssuer resource that will register an account on an ACME server. The value of spec.acme.server designates Let’s Encrypt’s production ACME server, which should be trusted by most browsers.

          Note

          Let’s Encrypt provides a staging ACME server that can be used to test issuing trusted certificates, while not worrying about hitting Let’s Encrypt’s production rate limits. The staging URL is https://acme-staging-v02.api.letsencrypt.org/directory.
        • The value of privateKeySecretRef.name provides the name of a secret containing the private key for this user’s ACME server account (this is tied to the email address you provide in the manifest file). The ACME server will use this key to identify you.

        • To ensure that you own the domain for which you will create a certificate, the ACME server will issue a challenge to a client. cert-manager provides two options for solving challenges, http01 and DNS01. In this example, the http01 challenge solver will be used and it is configured in the solvers array. cert-manager will spin up challenge solver Pods to solve the issued challenges and use Ingress resources to route the challenge to the appropriate Pod.

      3. Create the ClusterIssuer resource:

        kubectl create -f acme-issuer-prod.yaml
        

      Create a Certificate Resource

      After you have a ClusterIssuer resource, you can create a Certificate resource. This will describe your x509 public key certificate and will be used to automatically generate a CertificateRequest which will be sent to your ClusterIssuer.

      1. Using the text editor of your choice, create a file named certificate-prod.yaml with the example configurations. Replace the value of email with your own email address. Replace the value of spec.dnsNames with your own domain that you will use to host your Docker registry.

        ~/registry/certificate-prod.yaml
         1
         2
         3
         4
         5
         6
         7
         8
         9
        10
        11
        12
        13
        14
        
        apiVersion: cert-manager.io/v1alpha2
        kind: Certificate
        metadata:
          name: docker-registry-prod
        spec:
          secretName: letsencrypt-secret-prod
          duration: 2160h # 90d
          renewBefore: 360h # 15d
          issuerRef:
            name: letsencrypt-prod
            kind: ClusterIssuer
          dnsNames:
          - registry.example.com
            

        Note

        The configurations in this example create a Certificate that is valid for 90 days and renews 15 days before expiry.

      2. Create the Certificate resource:

        kubectl create -f certificate-prod.yaml
        
      3. Verify that the Certificate has been successfully issued:

        kubectl get certs
        

        When your certificate is ready, you should see a similar output:

          
        NAME                   READY   SECRET                    AGE
        docker-registry-prod   True    letsencrypt-secret-prod   42s
            
        

        All the necessary components are now in place to be able to enable HTTPS on your Docker registry. In the next section, you will complete the steps need to deploy your registry.

      Deploy your Docker Registry

      You will now complete the steps to deploy your Docker Registry to your Kubernetes cluster using a Helm chart. Prior to deploying your Docker registry, you will first need to create a username and password in order to enable basic authentication for your registry. This will allow you to restrict access to your Docker registry which will keep your images private. Since, your registry will require authentication, a Kubernetes secret will be added to your cluster in order to provide your cluster with your registry’s authentication credentials, so that it can pull images from it.

      Enable Basic Authentication

      To enabled basic access restriction for your Docker registry, you will use the htpasswd utility. This utility allows you to use a file to store usernames and passwords for basic HTTP authentication. This will require you to log into your Docker registry prior to being able to push or pull images from and to it.

      1. Install the htpasswd utility. This example is for an Ubuntu 18.04 instance, but you can use your system’s package manger to install it.

        sudo apt install apache2-utils -y
        
      2. Create a file to store your Docker registry’s username and password.

        touch my_docker_pass
        
      3. Create a username and password using htpasswd. Replace example_user with your own username. Follow the prompt to create a password.

        htpasswd -B my_docker_pass example_user
        
      4. View the contents of your password file.

        cat my_docker_pass
        

        Your output will resemble the following. You will need these values when deploying your registry in the Configure your Docker Registry section of the guide.

          
        example_user:$2y$05$8VhvzCVCB4txq8mNGh8eu.8GMyBEEeUInqQJHKJUD.KUwxastPG4m
          
        

      Grant your Cluster Access to your Docker Registry

      Your LKE Cluster will also need to authenticate to your Docker registry in order to pull images from it. In this section, you will create a Kubernetes Secret that you can use to grant your cluster’s kubelet with access to your registry’s images.

      1. Create a secret to store your registry’s authentication information. Replace the option values with your own registry’s details. The --docker-username and --docker-password should be the username and password that you used when generating credentials using the htpasswd utility.

        kubectl create secret docker-registry regcred 
          --docker-server=registry.example.com 
          --docker-username=example_user 
          --docker-password=3xampl3Passw0rd 
          [email protected]
        

      Configure your Docker Registry

      Before deploying the Docker Registry Helm chart to your cluster, you will define some configurations so that the Docker registry uses the NGINX Ingress controller, your registry Object Storage bucket, and your cert-manager created TLS certificate. See the Docker Registry Helm Chart’s official documentation for a full list of all available configurations.

      Note

      1. Create a new file named docker-configs.yaml using the example configurations. Ensure you replace the following values in your file:

        • ingress.hosts with your own Docker registry’s domain
        • ingress.tls.secretName with the secret name you used when creating your ClusterIssuer
        • ingress.tls.secretName.hosts with the domain for which you wish to secure with your TLS certificate.
        • secrets.s3.accessKey with the value of your Object Storage account’s access key and secrets.s3.secretKey with the corresponding secret key.
        • secrets.htpasswd with the value returned when you view the contents of your my_docker_pass file. However, ensure you do not remove the |- characters. This ensures that your YAML is properly formatted. See step 4 in the Enable Basic Authentication section for details on viewing the contents of your password file.
        • s3.region with your Object Storage bucket’s cluster region, s3.regionEndpoint with your Object Storage bucket’s region endpoint, and s3.bucket with your registry’s Object Storage bucket name.
        ~/registry/docker-configs.yaml
         1
         2
         3
         4
         5
         6
         7
         8
         9
        10
        11
        12
        13
        14
        15
        16
        17
        18
        19
        20
        21
        22
        23
        24
        25
        26
        27
        
        ingress:
          enabled: true
          hosts:
            - registry.example.com
          annotations:
            kubernetes.io/ingress.class: nginx
            cert-manager.io/cluster-issuer: letsencrypt-prod
            nginx.ingress.kubernetes.io/proxy-body-size: "0"
            nginx.ingress.kubernetes.io/proxy-read-timeout: "6000"
            nginx.ingress.kubernetes.io/proxy-send-timeout: "6000"
          tls:
            - secretName: letsencrypt-secret-prod
              hosts:
              - registry.example.com
        storage: s3
        secrets:
          htpasswd: |-
            example_user:$2y$05$8VhvzCVCB4txq8mNGh8eu.8GMyBEEeUInqQJHKJUD.KUwxastPG4m
          s3:
            accessKey: "myaccesskey"
            secretKey: "mysecretkey"
        s3:
          region: us-east-1
          regionEndpoint: us-east-1.linodeobjects.com/
          secure: true
          bucket: registry
              
        • The NGINX Ingress annotation nginx.ingress.kubernetes.io/proxy-body-size: "0" disables a maximum allowed size client request body check and ensures that you won’t receive a 413 error when pushing larger Docker images to your registry. The values for nginx.ingress.kubernetes.io/proxy-read-timeout: "6000" and nginx.ingress.kubernetes.io/proxy-send-timeout: "6000" are sane values to begin with, but may be adjusted as needed.
      2. Deploy your Docker registry using the configurations you created in the previous step:

        helm install docker-registry stable/docker-registry -f docker-configs.yaml
        
      3. Navigate to your registry’s domain and verify that your browser loads the TLS certificate.

        Verify that your Docker registry's site loads your TLS certificate

        You will interact with your registry via the Docker CLI, so you should not expect to see any content load on the page.

      Push an Image to your Docker Registry

      You are now ready to push and pull images to your Docker registry. In this section you will pull an existing image from Docker Hub and then push it to your registry. Then, in the next section, you will use your registry’s image to deploy an example static site.

      1. Use Docker to pull an image from Docker Hub. This example is using an image that was created following our Create and Deploy a Docker Container Image to a Kubernetes Cluster guide. The image will build a Hugo static site with some boiler plate content. However, you can use any image from Docker Hub that you prefer.

        sudo docker pull leslitagordita/hugo-site:v10
        
      2. Tag your local Docker image with your private registry’s hostname. This is required when pushing an image to a private registry and not the central Docker registry. Ensure that you replace registry.example.com with your own registry’s domain.

        sudo docker tag leslitagordita/hugo-site:v10 registry.example.com/leslitagordita/hugo-site:v10
        
      3. At this point, you have never authenticated to your private registry. You will need to log into it prior to pushing up any images. Issue the example command, replacing registry.example.com with your own registry’s URL. Follow the prompts to enter in the username and password you created in the Enable Basic Authentication section.

        sudo docker login registry.example.com
        
      4. Push the image to your registry. Ensure that you replace registry.example.com with your own registry’s domain.

        sudo docker push registry.example.com/leslitagordita/hugo-site:v10
        

        You should see a similar output when your image push is complete

          
        The push refers to repository [registry.example.com/leslitagordita/hugo-site]
        925cbd794bd8: Pushed
        b9fee92b7ac7: Pushed
        1658c062e6a8: Pushed
        21acf2dde3fe: Pushed
        588c407f9029: Pushed
        bcf2f368fe23: Pushed
        v10: digest: sha256:3db7ab6bc5a893375af6f7cf505bac2f4957d8a03701d7fd56853712b0900312 size: 1570
            
        

      Create a Test Deployment Using an Image from Your Docker Registry

      In this section, you will create a test deployment using the image that you pushed to your registry in the previous section. This will ensure that your cluster can authenticate to your Docker registry and pull images from it.

      1. Using Linode’s DNS manager to create a new subdomain A record to host your static site. The example will use static.example.com. When creating your record, assign your cluster’s NodeBalancer external IP address as the IP address. You can find the external IP address with the following command:

        kubectl --namespace default get services -o wide -w nginx-ingress-controller
        

        The command will return a similar output. Use the value of the EXTERNAL-IP field to create your static site’s new subdomain A record.

          
        NAME                          TYPE           CLUSTER-IP      EXTERNAL-IP    PORT(S)                      AGE     SELECTOR
        nginx-ingress-controller   LoadBalancer   10.128.169.60   192.0.2.0   80:32401/TCP,443:30830/TCP   7h51m   app.kubernetes.io/component=controller,app=nginx-ingress,release=nginx-ingress
            
        
      2. Using a text editor, create the static-site-test.yaml file with the example configurations. This file will create a deployment, service, and an ingress.

        ~/registry/staic-site-test.yaml
         1
         2
         3
         4
         5
         6
         7
         8
         9
        10
        11
        12
        13
        14
        15
        16
        17
        18
        19
        20
        21
        22
        23
        24
        25
        26
        27
        28
        29
        30
        31
        32
        33
        34
        35
        36
        37
        38
        39
        40
        41
        42
        43
        44
        45
        46
        47
        48
        49
        50
        51
        
        apiVersion: extensions/v1beta1
        kind: Ingress
        metadata:
          name: static-site-ingress
          annotations:
            kubernetes.io/ingress.class: nginx
            nginx.ingress.kubernetes.io/backend-protocol: "HTTP"
        spec:
          rules:
          - host: static.example.com
            http:
              paths:
              - path: /
                backend:
                  serviceName: static-site
                  servicePort: 80
        ---
        apiVersion: v1
        kind: Service
        metadata:
          name: static-site
        spec:
          type: NodePort
          ports:
          - port: 80
            targetPort: 80
          selector:
            app: static-site
        ---
        apiVersion: apps/v1
        kind: Deployment
        metadata:
          name: static-site
        spec:
          replicas: 3
          selector:
            matchLabels:
              app: static-site
          template:
            metadata:
              labels:
                app: static-site
            spec:
              containers:
              - name: static-site
                image: registry.example.com/leslitagordita/hugo-site:v10
                ports:
                - containerPort: 80
              imagePullSecrets:
              - name: regcred
              
        • In the Deployment section of the manifest, the imagePullSecrets field references the secret you created in the Grant your Cluster Access to your Docker Registry section. This secret contains the authentication credentials that your cluster’s kubelet can use to pull your private registry’s image.
        • The image field provides the image to pull from your Docker registry.
      3. Create the deployment.

        kubectl create -f static-site-test.yaml
        
      4. Open a browser and navigate to your site’s domain and view the example static site. Using our example, you would navigate to static.example.com. The example Hugo site should load.

      (Optional) Tear Down your Kubernetes Cluster

      To avoid being further billed for your Kubernetes cluster and NodeBlancer, delete your cluster using the Linode Cloud Manager. Similarly, to avoid being further billed for our registry’s Object Storage bucket, follow the steps in the cancel the Object Storage service on your account section of our How to Use Object Storage guide.

      This guide is published under a CC BY-ND 4.0 license.



      Source link

      How To Set Up a Private Docker Registry on Top of DigitalOcean Spaces and Use It with DigitalOcean Kubernetes


      The author selected the Free and Open Source Fund to receive a donation as part of the Write for DOnations program.

      Introduction

      A Docker registry is a storage and content delivery system for named Docker images, which are the industry standard for containerized applications. A private Docker registry allows you to securely share your images within your team or organization with more flexibility and control when compared to public ones. By hosting your private Docker registry directly in your Kubernetes cluster, you can achieve higher speeds, lower latency, and better availability, all while having control over the registry.

      The underlying registry storage is delegated to external drivers. The default storage system is the local filesystem, but you can swap this for a cloud-based storage driver. DigitalOcean Spaces is an S3-compatible object storage designed for developer teams and businesses that want a scalable, simple, and affordable way to store and serve vast amounts of data, and is very suitable for storing Docker images. It has a built-in CDN network, which can greatly reduce latency when frequently accessing images.

      In this tutorial, you’ll deploy your private Docker registry to your DigitalOcean Kubernetes cluster using Helm, backed up by DigitalOcean Spaces for storing data. You’ll create API keys for your designated Space, install the Docker registry to your cluster with custom configuration, configure Kubernetes to properly authenticate with it, and test it by running a sample deployment on the cluster. At the end of this tutorial, you’ll have a secure, private Docker registry installed on your DigitalOcean Kubernetes cluster.

      Prerequisites

      Before you begin this tutorial, you’ll need:

      • Docker installed on the machine that you’ll access your cluster from. For Ubuntu 18.04 visit How To Install and Use Docker on Ubuntu 18.04. You only need to complete the first step. Otherwise visit Docker’s website for other distributions.

      • A DigitalOcean Kubernetes cluster with your connection configuration configured as the kubectl default. Instructions on how to configure kubectl are shown under the Connect to your Cluster step shown when you create your cluster. To learn how to create a Kubernetes cluster on DigitalOcean, see Kubernetes Quickstart.

      • A DigitalOcean Space with API keys (access and secret). To learn how to create a DigitalOcean Space and API keys, see How To Create a DigitalOcean Space and API Key.

      • The Helm package manager installed on your local machine, and Tiller installed on your cluster. Complete steps 1 and 2 of the How To Install Software on Kubernetes Clusters with the Helm Package Manager. You only need to complete the first two steps.

      • The Nginx Ingress Controller and Cert-Manager installed on the cluster. For a guide on how to do this, see How to Set Up an Nginx Ingress with Cert-Manager on DigitalOcean Kubernetes.

      • A domain name with two DNS A records pointed to the DigitalOcean Load Balancer used by the Ingress. If you are using DigitalOcean to manage your domain’s DNS records, consult How to Manage DNS Records to create A records. In this tutorial, we’ll refer to the A records as registry.example.com and k8s-test.example.com.

      Step 1 — Configuring and Installing the Docker Registry

      In this step, you will create a configuration file for the registry deployment and install the Docker registry to your cluster with the given config using the Helm package manager.

      During the course of this tutorial, you will use a configuration file called chart_values.yaml to override some of the default settings for the Docker registry Helm chart. Helm calls its packages, charts; these are sets of files that outline a related selection of Kubernetes resources. You’ll edit the settings to specify DigitalOcean Spaces as the underlying storage system and enable HTTPS access by wiring up Let’s Encrypt TLS certificates.

      As part of the prerequisite, you would have created the echo1 and echo2 services and an echo_ingress ingress for testing purposes; you will not need these in this tutorial, so you can now delete them.

      Start off by deleting the ingress by running the following command:

      • kubectl delete -f echo_ingress.yaml

      Then, delete the two test services:

      • kubectl delete -f echo1.yaml && kubectl delete -f echo2.yaml

      The kubectl delete command accepts the file to delete when passed the -f parameter.

      Create a folder that will serve as your workspace:

      Navigate to it by running:

      Now, using your text editor, create your chart_values.yaml file:

      Add the following lines, ensuring you replace the highlighted lines with your details:

      chart_values.yaml

      ingress:
        enabled: true
        hosts:
          - registry.example.com
        annotations:
          kubernetes.io/ingress.class: nginx
          certmanager.k8s.io/cluster-issuer: letsencrypt-prod
          nginx.ingress.kubernetes.io/proxy-body-size: "30720m"
        tls:
          - secretName: letsencrypt-prod
            hosts:
              - registry.example.com
      
      storage: s3
      
      secrets:
        htpasswd: ""
        s3:
          accessKey: "your_space_access_key"
          secretKey: "your_space_secret_key"
      
      s3:
        region: your_space_region
        regionEndpoint: your_space_region.digitaloceanspaces.com
        secure: true
        bucket: your_space_name
      

      The first block, ingress, configures the Kubernetes Ingress that will be created as a part of the Helm chart deployment. The Ingress object makes outside HTTP/HTTPS routes point to internal services in the cluster, thus allowing communication from the outside. The overridden values are:

      • enabled: set to true to enable the Ingress.
      • hosts: a list of hosts from which the Ingress will accept traffic.
      • annotations: a list of metadata that provides further direction to other parts of Kubernetes on how to treat the Ingress. You set the Ingress Controller to nginx, the Let's Encrypt cluster issuer to the production variant (letsencrypt-prod), and tell the nginx controller to accept files with a max size of 30 GB, which is a sensible limit for even the largest Docker images.
      • tls: this subcategory configures Let's Encrypt HTTPS. You populate the hosts list that defines from which secure hosts this Ingress will accept HTTPS traffic with our example domain name.

      Then, you set the file system storage to s3 — the other available option would be filesystem. Here s3 indicates using a remote storage system compatible with the industry-standard Amazon S3 API, which DigitalOcean Spaces fulfills.

      In the next block, secrets, you configure keys for accessing your DigitalOcean Space under the s3 subcategory. Finally, in the s3 block, you configure the parameters specifying your Space.

      Save and close your file.

      Now, if you haven't already done so, set up your A records to point to the Load Balancer you created as part of the Nginx Ingress Controller installation in the prerequisite tutorial. To see how to set your DNS on DigitalOcean, see How to Manage DNS Records.

      Next, ensure your Space isn't empty. The Docker registry won't run at all if you don't have any files in your Space. To get around this, upload a file. Navigate to the Spaces tab, find your Space, click the Upload File button, and upload any file you'd like. You could upload the configuration file you just created.

      Empty file uploaded to empty Space

      Before installing anything via Helm, you need to refresh its cache. This will update the latest information about your chart repository. To do this run the following command:

      Now, you'll deploy the Docker registry chart with this custom configuration via Helm by running:

      • helm install stable/docker-registry -f chart_values.yaml --name docker-registry

      You'll see the following output:

      Output

      NAME: docker-registry ... NAMESPACE: default STATUS: DEPLOYED RESOURCES: ==> v1/ConfigMap NAME DATA AGE docker-registry-config 1 1s ==> v1/Pod(related) NAME READY STATUS RESTARTS AGE docker-registry-54df68fd64-l26fb 0/1 ContainerCreating 0 1s ==> v1/Secret NAME TYPE DATA AGE docker-registry-secret Opaque 3 1s ==> v1/Service NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE docker-registry ClusterIP 10.245.131.143 <none> 5000/TCP 1s ==> v1beta1/Deployment NAME READY UP-TO-DATE AVAILABLE AGE docker-registry 0/1 1 0 1s ==> v1beta1/Ingress NAME HOSTS ADDRESS PORTS AGE docker-registry registry.example.com 80, 443 1s NOTES: 1. Get the application URL by running these commands: https://registry.example.com/

      Helm lists all the resources it created as a result of the Docker registry chart deployment. The registry is now accessible from the domain name you specified earlier.

      You've configured and deployed a Docker registry on your Kubernetes cluster. Next, you will test the availability of the newly deployed Docker registry.

      Step 2 — Testing Pushing and Pulling

      In this step, you'll test your newly deployed Docker registry by pushing and pulling images to and from it. Currently, the registry is empty. To have something to push, you need to have an image available on the machine you're working from. Let's use the mysql Docker image.

      Start off by pulling mysql from the Docker Hub:

      Your output will look like this:

      Output

      Using default tag: latest latest: Pulling from library/mysql 27833a3ba0a5: Pull complete ... e906385f419d: Pull complete Digest: sha256:a7cf659a764732a27963429a87eccc8457e6d4af0ee9d5140a3b56e74986eed7 Status: Downloaded newer image for mysql:latest

      You now have the image available locally. To inform Docker where to push it, you'll need to tag it with the host name, like so:

      • sudo docker tag mysql registry.example.com/mysql

      Then, push the image to the new registry:

      • sudo docker push registry.example.com/mysql

      This command will run successfully and indicate that your new registry is properly configured and accepting traffic — including pushing new images. If you see an error, double check your steps against steps 1 and 2.

      To test pulling from the registry cleanly, first delete the local mysql images with the following command:

      • sudo docker rmi registry.example.com/mysql && sudo docker rmi mysql

      Then, pull it from the registry:

      • sudo docker pull registry.example.com/mysql

      This command will take a few seconds to complete. If it runs successfully, that means your registry is working correctly. If it shows an error, double check what you have entered against the previous commands.

      You can list Docker images available locally by running the following command:

      You'll see output listing the images available on your local machine, along with their ID and date of creation.

      Your Docker registry is configured. You've pushed an image to it and verified you can pull it down. Now let's add authentication so only certain people can access the code.

      Step 3 — Adding Account Authentication and Configuring Kubernetes Access

      In this step, you'll set up username and password authentication for the registry using the htpasswd utility.

      The htpasswd utility comes from the Apache webserver, which you can use for creating files that store usernames and passwords for basic authentication of HTTP users. The format of htpasswd files is username:hashed_password (one per line), which is portable enough to allow other programs to use it as well.

      To make htpasswd available on the system, you'll need to install it by running:

      • sudo apt install apache2-utils -y

      Note:
      If you're running this tutorial from a Mac, you'll need to use the following command to make htpasswd available on your machine:

      • docker run --rm -v ${PWD}:/app -it httpd htpasswd -b -c /app/htpasswd_file sammy password

      Create it by executing the following command:

      Add a username and password combination to htpasswd_file:

      • htpasswd -B htpasswd_file username

      Docker requires the password to be hashed using the bcrypt algorithm, which is why we pass the -B parameter. The bcrypt algorithm is a password hashing function based on Blowfish block cipher, with a work factor parameter, which specifies how expensive the hash function will be.

      Remember to replace username with your desired username. When run, htpasswd will ask you for the accompanying password and add the combination to htpasswd_file. You can repeat this command for as many users as you wish to add.

      Now, show the contents of htpasswd_file by running the following command:

      Select and copy the contents shown.

      To add authentication to your Docker registry, you'll need to edit chart_values.yaml and add the contents of htpasswd_file in the htpasswd variable.

      Open chart_values.yaml for editing:

      Find the line that looks like this:

      chart_values.yaml

        htpasswd: ""
      

      Edit it to match the following, replacing htpasswd_file_contents with the contents you copied from the htpasswd_file:

      chart_values.yaml

        htpasswd: |-
          htpasswd_file_contents
      

      Be careful with the indentation, each line of the file contents must have four spaces before it.

      Once you've added your contents, save and close the file.

      To propagate the changes to your cluster, run the following command:

      • helm upgrade docker-registry stable/docker-registry -f chart_values.yaml

      The output will be similar to that shown when you first deployed your Docker registry:

      Output

      Release "docker-registry" has been upgraded. Happy Helming! LAST DEPLOYED: ... NAMESPACE: default STATUS: DEPLOYED RESOURCES: ==> v1/ConfigMap NAME DATA AGE docker-registry-config 1 3m8s ==> v1/Pod(related) NAME READY STATUS RESTARTS AGE docker-registry-6c5bb7ffbf-ltnjv 1/1 Running 0 3m7s ==> v1/Secret NAME TYPE DATA AGE docker-registry-secret Opaque 4 3m8s ==> v1/Service NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE docker-registry ClusterIP 10.245.128.245 <none> 5000/TCP 3m8s ==> v1beta1/Deployment NAME READY UP-TO-DATE AVAILABLE AGE docker-registry 1/1 1 1 3m8s ==> v1beta1/Ingress NAME HOSTS ADDRESS PORTS AGE docker-registry registry.example.com 159.89.215.50 80, 443 3m8s NOTES: 1. Get the application URL by running these commands: https://registry.example.com/

      This command calls Helm and instructs it to upgrade an existing release, in your case docker-registry, with its chart defined in stable/docker-registry in the chart repository, after applying the chart_values.yaml file.

      Now, you'll try pulling an image from the registry again:

      • sudo docker pull registry.example.com/mysql

      The output will look like the following:

      Output

      Using default tag: latest Error response from daemon: Get https://registry.example.com/v2/mysql/manifests/latest: no basic auth credentials

      It correctly failed because you provided no credentials. This means that your Docker registry authorizes requests correctly.

      To log in to the registry, run the following command:

      • sudo docker login registry.example.com

      Remember to replace registry.example.com with your domain address. It will prompt you for a username and password. If it shows an error, double check what your htpasswd_file contains. You must define the username and password combination in the htpasswd_file, which you created earlier in this step.

      To test the login, you can try to pull again by running the following command:

      • sudo docker pull registry.example.com/mysql

      The output will look similar to the following:

      Output

      Using default tag: latest latest: Pulling from mysql Digest: sha256:f2dc118ca6fa4c88cde5889808c486dfe94bccecd01ca626b002a010bb66bcbe Status: Image is up to date for registry.example.com/mysql:latest

      You've now configured Docker and can log in securely. To configure Kubernetes to log in to your registry, run the following command:

      • sudo kubectl create secret generic regcred --from-file=.dockerconfigjson=/home/sammy/.docker/config.json --type=kubernetes.io/dockerconfigjson

      You will see the following output:

      Output

      secret/regcred created

      This command creates a secret in your cluster with the name regcred, takes the contents of the JSON file where Docker stores the credentials, and parses it as dockerconfigjson, which defines a registry credential in Kubernetes.

      You've used htpasswd to create a login config file, configured the registry to authenticate requests, and created a Kubernetes secret containing the login credentials. Next, you will test the integration between your Kubernetes cluster and registry.

      Step 4 — Testing Kubernetes Integration by Running a Sample Deployment

      In this step, you'll run a sample deployment with an image stored in the in-cluster registry to test the connection between your Kubernetes cluster and registry.

      In the last step, you created a secret, called regcred, containing login credentials for your private registry. It may contain login credentials for multiple registries, in which case you'll have to update the Secret accordingly.

      You can specify which secret Kubernetes should use when pulling containers in the pod definition by specifying imagePullSecrets. This step is necessary when the Docker registry requires authentication.

      You'll now deploy a sample Hello World image from your private Docker registry to your cluster. First, in order to push it, you'll pull it to your machine by running the following command:

      • sudo docker pull paulbouwer/hello-kubernetes:1.5

      Then, tag it by running:

      • sudo docker tag paulbouwer/hello-kubernetes:1.5 registry.example.com/paulbouwer/hello-kubernetes:1.5

      Finally, push it to your registry:

      • sudo docker push registry.example.com/paulbouwer/hello-kubernetes:1.5

      Delete it from your machine as you no longer need it locally:

      • sudo docker rmi registry.example.com/paulbouwer/hello-kubernetes:1.5

      Now, you'll deploy the sample Hello World application. First, create a new file, hello-world.yaml, using your text editor:

      Next, you'll define a Service and an Ingress to make the app accessible to outside of the cluster. Add the following lines, replacing the highlighted lines with your domains:

      hello-world.yaml

      apiVersion: extensions/v1beta1
      kind: Ingress
      metadata:
        name: hello-kubernetes-ingress
        annotations:
          kubernetes.io/ingress.class: nginx
          nginx.ingress.kubernetes.io/rewrite-target: /
      spec:
        rules:
        - host: k8s-test.example.com
          http:
            paths:
            - path: /
              backend:
                serviceName: hello-kubernetes
                servicePort: 80
      ---
      apiVersion: v1
      kind: Service
      metadata:
        name: hello-kubernetes
      spec:
        type: NodePort
        ports:
        - port: 80
          targetPort: 8080
        selector:
          app: hello-kubernetes
      ---
      apiVersion: apps/v1
      kind: Deployment
      metadata:
        name: hello-kubernetes
      spec:
        replicas: 3
        selector:
          matchLabels:
            app: hello-kubernetes
        template:
          metadata:
            labels:
              app: hello-kubernetes
          spec:
            containers:
            - name: hello-kubernetes
              image: registry.example.com/paulbouwer/hello-kubernetes:1.5
              ports:
              - containerPort: 8080
            imagePullSecrets:
            - name: regcred
      

      First, you define the Ingress for the Hello World deployment, which you will route through the Load Balancer that the Nginx Ingress Controller owns. Then, you define a service that can access the pods created in the deployment. In the actual deployment spec, you specify the image as the one located in your registry and set imagePullSecrets to regcred, which you created in the previous step.

      Save and close the file. To deploy this to your cluster, run the following command:

      • kubectl apply -f hello-world.yaml

      You'll see the following output:

      Output

      ingress.extensions/hello-kubernetes-ingress created service/hello-kubernetes created deployment.apps/hello-kubernetes created

      You can now navigate to your test domain — the second A record, k8s-test.example.com in this tutorial. You will see the Kubernetes Hello world! page.

      Hello World page

      The Hello World page lists some environment information, like the Linux kernel version and the internal ID of the pod the request was served from. You can also access your Space via the web interface to see the images you've worked with in this tutorial.

      If you want to delete this Hello World deployment after testing, run the following command:

      • kubectl delete -f hello-world.yaml

      You've created a sample Hello World deployment to test if Kubernetes is properly pulling images from your private registry.

      Conclusion

      You have now successfully deployed your own private Docker registry on your DigitalOcean Kubernetes cluster, using DigitalOcean Spaces as the storage layer underneath. There is no limit to how many images you can store, Spaces can extend infinitely, while at the same time providing the same security and robustness. In production, though, you should always strive to optimize your Docker images as much as possible, take a look at the How To Optimize Docker Images for Production tutorial.



      Source link

      How To Set Up a Private Docker Registry on Top of DigitalOcean Spaces and Use It with DO Kubernetes


      The author selected the Free and Open Source Fund to receive a donation as part of the Write for DOnations program.

      Introduction

      A Docker registry is a storage and content delivery system for named Docker images, which are the industry standard for containerized applications. A private Docker registry allows you to securely share your images within your team or organization with more flexibility and control when compared to public ones. By hosting your private Docker registry directly in your Kubernetes cluster, you can achieve higher speeds, lower latency, and better availability, all while having control over the registry.

      The underlying registry storage is delegated to external drivers. The default storage system is the local filesystem, but you can swap this for a cloud-based storage driver. DigitalOcean Spaces is an S3-compatible object storage designed for developer teams and businesses that want a scalable, simple, and affordable way to store and serve vast amounts of data, and is very suitable for storing Docker images. It has a built-in CDN network, which can greatly reduce latency when frequently accessing images.

      In this tutorial, you’ll deploy your private Docker registry to your DigitalOcean Kubernetes cluster using Helm, backed up by DigitalOcean Spaces for storing data. You’ll create API keys for your designated Space, install the Docker registry to your cluster with custom configuration, configure Kubernetes to properly authenticate with it, and test it by running a sample deployment on the cluster. At the end of this tutorial, you’ll have a secure, private Docker registry installed on your DigitalOcean Kubernetes cluster.

      Prerequisites

      Before you begin this tutorial, you’ll need:

      • Docker installed on the machine that you’ll access your cluster from. For Ubuntu 18.04 visit How To Install and Use Docker on Ubuntu 18.04. You only need to complete the first step. Otherwise visit Docker’s website for other distributions.

      • A DigitalOcean Kubernetes cluster with your connection configuration configured as the kubectl default. Instructions on how to configure kubectl are shown under the Connect to your Cluster step shown when you create your cluster. To learn how to create a Kubernetes cluster on DigitalOcean, see Kubernetes Quickstart.

      • A DigitalOcean Space with API keys (access and secret). To learn how to create a DigitalOcean Space and API keys, see How To Create a DigitalOcean Space and API Key.

      • The Helm package manager installed on your local machine, and Tiller installed on your cluster. Complete steps 1 and 2 of the How To Install Software on Kubernetes Clusters with the Helm Package Manager. You only need to complete the first two steps.

      • The Nginx Ingress Controller and Cert-Manager installed on the cluster. For a guide on how to do this, see How to Set Up an Nginx Ingress with Cert-Manager on DigitalOcean Kubernetes.

      • A domain name with two DNS A records pointed to the DigitalOcean Load Balancer used by the Ingress. If you are using DigitalOcean to manage your domain’s DNS records, consult How to Manage DNS Records to create A records. In this tutorial, we’ll refer to the A records as registry.example.com and k8s-test.example.com.

      Step 1 — Configuring and Installing the Docker Registry

      In this step, you will create a configuration file for the registry deployment and install the Docker registry to your cluster with the given config using the Helm package manager.

      During the course of this tutorial, you will use a configuration file called chart_values.yaml to override some of the default settings for the Docker registry Helm chart. Helm calls its packages, charts; these are sets of files that outline a related selection of Kubernetes resources. You’ll edit the settings to specify DigitalOcean Spaces as the underlying storage system and enable HTTPS access by wiring up Let’s Encrypt TLS certificates.

      As part of the prerequisite, you would have created the echo1 and echo2 services and an echo_ingress ingress for testing purposes; you will not need these in this tutorial, so you can now delete them.

      Start off by deleting the ingress by running the following command:

      • kubectl delete -f echo_ingress.yaml

      Then, delete the two test services:

      • kubectl delete -f echo1.yaml && kubectl delete -f echo2.yaml

      The kubectl delete command accepts the file to delete when passed the -f parameter.

      Create a folder that will serve as your workspace:

      Navigate to it by running:

      Now, using your text editor, create your chart_values.yaml file:

      Add the following lines, ensuring you replace the highlighted lines with your details:

      chart_values.yaml

      ingress:
        enabled: true
        hosts:
          - registry.example.com
        annotations:
          kubernetes.io/ingress.class: nginx
          certmanager.k8s.io/cluster-issuer: letsencrypt-prod
          nginx.ingress.kubernetes.io/proxy-body-size: "30720m"
        tls:
          - secretName: letsencrypt-prod
            hosts:
              - registry.example.com
      
      storage: s3
      
      secrets:
        htpasswd: ""
        s3:
          accessKey: "your_space_access_key"
          secretKey: "your_space_secret_key"
      
      s3:
        region: your_space_region
        regionEndpoint: your_space_region.digitaloceanspaces.com
        secure: true
        bucket: your_space_name
      

      The first block, ingress, configures the Kubernetes Ingress that will be created as a part of the Helm chart deployment. The Ingress object makes outside HTTP/HTTPS routes point to internal services in the cluster, thus allowing communication from the outside. The overridden values are:

      • enabled: set to true to enable the Ingress.
      • hosts: a list of hosts from which the Ingress will accept traffic.
      • annotations: a list of metadata that provides further direction to other parts of Kubernetes on how to treat the Ingress. You set the Ingress Controller to nginx, the Let's Encrypt cluster issuer to the production variant (letsencrypt-prod), and tell the nginx controller to accept files with a max size of 30 GB, which is a sensible limit for even the largest Docker images.
      • tls: this subcategory configures Let's Encrypt HTTPS. You populate the hosts list that defines from which secure hosts this Ingress will accept HTTPS traffic with our example domain name.

      Then, you set the file system storage to s3 — the other available option would be filesystem. Here s3 indicates using a remote storage system compatible with the industry-standard Amazon S3 API, which DigitalOcean Spaces fulfills.

      In the next block, secrets, you configure keys for accessing your DO Space under the s3 subcategory. Finally, in the s3 block, you configure the parameters specifying your Space.

      Save and close your file.

      Now, if you haven't already done so, set up your A records to point to the Load Balancer you created as part of the Nginx Ingress Controller installation in the prerequisite tutorial. To see how to set your DNS on DigitalOcean, see How to Manage DNS Records.

      Next, ensure your Space isn't empty. The Docker registry won't run at all if you don't have any files in your Space. To get around this, upload a file. Navigate to the Spaces tab, find your Space, click the Upload File button, and upload any file you'd like. You could upload the configuration file you just created.

      Empty file uploaded to empty Space

      Before installing anything via Helm, you need to refresh its cache. This will update the latest information about your chart repository. To do this run the following command:

      Now, you'll deploy the Docker registry chart with this custom configuration via Helm by running:

      • helm install stable/docker-registry -f chart_values.yaml --name docker-registry

      You'll see the following output:

      Output

      NAME: docker-registry ... NAMESPACE: default STATUS: DEPLOYED RESOURCES: ==> v1/ConfigMap NAME DATA AGE docker-registry-config 1 1s ==> v1/Pod(related) NAME READY STATUS RESTARTS AGE docker-registry-54df68fd64-l26fb 0/1 ContainerCreating 0 1s ==> v1/Secret NAME TYPE DATA AGE docker-registry-secret Opaque 3 1s ==> v1/Service NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE docker-registry ClusterIP 10.245.131.143 <none> 5000/TCP 1s ==> v1beta1/Deployment NAME READY UP-TO-DATE AVAILABLE AGE docker-registry 0/1 1 0 1s ==> v1beta1/Ingress NAME HOSTS ADDRESS PORTS AGE docker-registry registry.example.com 80, 443 1s NOTES: 1. Get the application URL by running these commands: https://registry.example.com/

      Helm lists all the resources it created as a result of the Docker registry chart deployment. The registry is now accessible from the domain name you specified earlier.

      You've configured and deployed a Docker registry on your Kubernetes cluster. Next, you will test the availability of the newly deployed Docker registry.

      Step 2 — Testing Pushing and Pulling

      In this step, you'll test your newly deployed Docker registry by pushing and pulling images to and from it. Currently, the registry is empty. To have something to push, you need to have an image available on the machine you're working from. Let's use the mysql Docker image.

      Start off by pulling mysql from the Docker Hub:

      Your output will look like this:

      Output

      Using default tag: latest latest: Pulling from library/mysql 27833a3ba0a5: Pull complete ... e906385f419d: Pull complete Digest: sha256:a7cf659a764732a27963429a87eccc8457e6d4af0ee9d5140a3b56e74986eed7 Status: Downloaded newer image for mysql:latest

      You now have the image available locally. To inform Docker where to push it, you'll need to tag it with the host name, like so:

      • sudo docker tag mysql registry.example.com/mysql

      Then, push the image to the new registry:

      • sudo docker push registry.example.com/mysql

      This command will run successfully and indicate that your new registry is properly configured and accepting traffic — including pushing new images. If you see an error, double check your steps against steps 1 and 2.

      To test pulling from the registry cleanly, first delete the local mysql images with the following command:

      • sudo docker rmi registry.example.com/mysql && sudo docker rmi mysql

      Then, pull it from the registry:

      • sudo docker pull registry.example.com/mysql

      This command will take a few seconds to complete. If it runs successfully, that means your registry is working correctly. If it shows an error, double check what you have entered against the previous commands.

      You can list Docker images available locally by running the following command:

      You'll see output listing the images available on your local machine, along with their ID and date of creation.

      Your Docker registry is configured. You've pushed an image to it and verified you can pull it down. Now let's add authentication so only certain people can access the code.

      Step 3 — Adding Account Authentication and Configuring Kubernetes Access

      In this step, you'll set up username and password authentication for the registry using the htpasswd utility.

      The htpasswd utility comes from the Apache webserver, which you can use for creating files that store usernames and passwords for basic authentication of HTTP users. The format of htpasswd files is username:hashed_password (one per line), which is portable enough to allow other programs to use it as well.

      To make htpasswd available on the system, you'll need to install it by running:

      • sudo apt install apache2-utils -y

      Note:
      If you're running this tutorial from a Mac, you'll need to use the following command to make htpasswd available on your machine:

      • docker run --rm -v ${PWD}:/app -it httpd htpasswd -b -c /app/htpasswd_file sammy password

      Create it by executing the following command:

      Add a username and password combination to htpasswd_file:

      • htpasswd -B htpasswd_file username

      Docker requires the password to be hashed using the bcrypt algorithm, which is why we pass the -B parameter. The bcrypt algorithm is a password hashing function based on Blowfish block cipher, with a work factor parameter, which specifies how expensive the hash function will be.

      Remember to replace username with your desired username. When run, htpasswd will ask you for the accompanying password and add the combination to htpasswd_file. You can repeat this command for as many users as you wish to add.

      Now, show the contents of htpasswd_file by running the following command:

      Select and copy the contents shown.

      To add authentication to your Docker registry, you'll need to edit chart_values.yaml and add the contents of htpasswd_file in the htpasswd variable.

      Open chart_values.yaml for editing:

      Find the line that looks like this:

      chart_values.yaml

        htpasswd: ""
      

      Edit it to match the following, replacing htpasswd_file_contents with the contents you copied from the htpasswd_file:

      chart_values.yaml

        htpasswd: |-
          htpasswd_file_contents
      

      Be careful with the indentation, each line of the file contents must have four spaces before it.

      Once you've added your contents, save and close the file.

      To propagate the changes to your cluster, run the following command:

      • helm upgrade docker-registry stable/docker-registry -f chart_values.yaml

      The output will be similar to that shown when you first deployed your Docker registry:

      Output

      Release "docker-registry" has been upgraded. Happy Helming! LAST DEPLOYED: ... NAMESPACE: default STATUS: DEPLOYED RESOURCES: ==> v1/ConfigMap NAME DATA AGE docker-registry-config 1 3m8s ==> v1/Pod(related) NAME READY STATUS RESTARTS AGE docker-registry-6c5bb7ffbf-ltnjv 1/1 Running 0 3m7s ==> v1/Secret NAME TYPE DATA AGE docker-registry-secret Opaque 4 3m8s ==> v1/Service NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE docker-registry ClusterIP 10.245.128.245 <none> 5000/TCP 3m8s ==> v1beta1/Deployment NAME READY UP-TO-DATE AVAILABLE AGE docker-registry 1/1 1 1 3m8s ==> v1beta1/Ingress NAME HOSTS ADDRESS PORTS AGE docker-registry registry.example.com 159.89.215.50 80, 443 3m8s NOTES: 1. Get the application URL by running these commands: https://registry.example.com/

      This command calls Helm and instructs it to upgrade an existing release, in your case docker-registry, with its chart defined in stable/docker-registry in the chart repository, after applying the chart_values.yaml file.

      Now, you'll try pulling an image from the registry again:

      • sudo docker pull registry.example.com/mysql

      The output will look like the following:

      Output

      Using default tag: latest Error response from daemon: Get https://registry.example.com/v2/mysql/manifests/latest: no basic auth credentials

      It correctly failed because you provided no credentials. This means that your Docker registry authorizes requests correctly.

      To log in to the registry, run the following command:

      • sudo docker login registry.example.com

      Remember to replace registry.example.com with your domain address. It will prompt you for a username and password. If it shows an error, double check what your htpasswd_file contains. You must define the username and password combination in the htpasswd_file, which you created earlier in this step.

      To test the login, you can try to pull again by running the following command:

      • sudo docker pull registry.example.com/mysql

      The output will look similar to the following:

      Output

      Using default tag: latest latest: Pulling from mysql Digest: sha256:f2dc118ca6fa4c88cde5889808c486dfe94bccecd01ca626b002a010bb66bcbe Status: Image is up to date for registry.example.com/mysql:latest

      You've now configured Docker and can log in securely. To configure Kubernetes to log in to your registry, run the following command:

      • sudo kubectl create secret generic regcred --from-file=.dockerconfigjson=/home/sammy/.docker/config.json --type=kubernetes.io/dockerconfigjson

      You will see the following output:

      Output

      secret/regcred created

      This command creates a secret in your cluster with the name regcred, takes the contents of the JSON file where Docker stores the credentials, and parses it as dockerconfigjson, which defines a registry credential in Kubernetes.

      You've used htpasswd to create a login config file, configured the registry to authenticate requests, and created a Kubernetes secret containing the login credentials. Next, you will test the integration between your Kubernetes cluster and registry.

      Step 4 — Testing Kubernetes Integration by Running a Sample Deployment

      In this step, you'll run a sample deployment with an image stored in the in-cluster registry to test the connection between your Kubernetes cluster and registry.

      In the last step, you created a secret, called regcred, containing login credentials for your private registry. It may contain login credentials for multiple registries, in which case you'll have to update the Secret accordingly.

      You can specify which secret Kubernetes should use when pulling containers in the pod definition by specifying imagePullSecrets. This step is necessary when the Docker registry requires authentication.

      You'll now deploy a sample Hello World image from your private Docker registry to your cluster. First, in order to push it, you'll pull it to your machine by running the following command:

      • sudo docker pull paulbouwer/hello-kubernetes:1.5

      Then, tag it by running:

      • sudo docker tag paulbouwer/hello-kubernetes:1.5 registry.example.com/paulbouwer/hello-kubernetes:1.5

      Finally, push it to your registry:

      • sudo docker push registry.example.com/paulbouwer/hello-kubernetes:1.5

      Delete it from your machine as you no longer need it locally:

      • sudo docker rmi registry.example.com/paulbouwer/hello-kubernetes:1.5

      Now, you'll deploy the sample Hello World application. First, create a new file, hello-world.yaml, using your text editor:

      Next, you'll define a Service and an Ingress to make the app accessible to outside of the cluster. Add the following lines, replacing the highlighted lines with your domains:

      hello-world.yaml

      apiVersion: extensions/v1beta1
      kind: Ingress
      metadata:
        name: hello-kubernetes-ingress
        annotations:
          kubernetes.io/ingress.class: nginx
          nginx.ingress.kubernetes.io/rewrite-target: /
      spec:
        rules:
        - host: k8s-test.example.com
          http:
            paths:
            - path: /
              backend:
                serviceName: hello-kubernetes
                servicePort: 80
      ---
      apiVersion: v1
      kind: Service
      metadata:
        name: hello-kubernetes
      spec:
        type: NodePort
        ports:
        - port: 80
          targetPort: 8080
        selector:
          app: hello-kubernetes
      ---
      apiVersion: apps/v1
      kind: Deployment
      metadata:
        name: hello-kubernetes
      spec:
        replicas: 3
        selector:
          matchLabels:
            app: hello-kubernetes
        template:
          metadata:
            labels:
              app: hello-kubernetes
          spec:
            containers:
            - name: hello-kubernetes
              image: registry.example.com/paulbouwer/hello-kubernetes:1.5
              ports:
              - containerPort: 8080
            imagePullSecrets:
            - name: regcred
      

      First, you define the Ingress for the Hello World deployment, which you will route through the Load Balancer that the Nginx Ingress Controller owns. Then, you define a service that can access the pods created in the deployment. In the actual deployment spec, you specify the image as the one located in your registry and set imagePullSecrets to regcred, which you created in the previous step.

      Save and close the file. To deploy this to your cluster, run the following command:

      • kubectl apply -f hello-world.yaml

      You'll see the following output:

      Output

      ingress.extensions/hello-kubernetes-ingress created service/hello-kubernetes created deployment.apps/hello-kubernetes created

      You can now navigate to your test domain — the second A record, k8s-test.example.com in this tutorial. You will see the Kubernetes Hello world! page.

      Hello World page

      The Hello World page lists some environment information, like the Linux kernel version and the internal ID of the pod the request was served from. You can also access your Space via the web interface to see the images you've worked with in this tutorial.

      If you want to delete this Hello World deployment after testing, run the following command:

      • kubectl delete -f hello-world.yaml

      You've created a sample Hello World deployment to test if Kubernetes is properly pulling images from your private registry.

      Conclusion

      You have now successfully deployed your own private Docker registry on your DigitalOcean Kubernetes cluster, using DigitalOcean Spaces as the storage layer underneath. There is no limit to how many images you can store, Spaces can extend infinitely, while at the same time providing the same security and robustness. In production, though, you should always strive to optimize your Docker images as much as possible, take a look at the How To Optimize Docker Images for Production tutorial.



      Source link