One place for hosting & domains

      Create

      Create and Deploy a Docker Container Image to a Kubernetes Cluster


      Updated by Linode

      Contributed by

      Linode

      Kubernetes and Docker

      Kubernetes is a system that automates the deployment, scaling, and management of containerized applications. Containerizing an application requires a base image that can be used to create an instance of a container. Once an application’s image exists, you can push it to a centralized container registry that Kubernetes can use to deploy container instances in a cluster’s pods.

      While Kubernetes supports several container runtimes, Docker is a very popular choice. Docker images are created using a Dockerfile that contains all commands, in their required order of execution, needed to build a given image. For example, a Dockerfile might contain instructions to install a specific operating system referencing another image, install an application’s dependencies, and execute configuration commands in the running container.

      Docker Hub is a centralized container image registry that can host your images and make them available for sharing and deployment. You can also find and use official Docker images and vendor specific images. When combined with a remote version control service, like GitHub, Docker Hub allows you to automate building container images and trigger actions for further automation with other services and tooling.

      Scope of This Guide

      This guide will show you how to package a Hugo static site in a Docker container image, host the image on Docker Hub, and deploy the container image on a Kubernetes cluster running on Linode. This example, is meant to demonstrate how applications can be containerized using Docker to leverage the deployment and scaling power of Kubernetes.

      Hugo is written in Go and is known for being extremely fast to compile sites, even very large ones. It is well-supported, well-documented, and has an active community. Some useful Hugo features include shortcodes, which are an easy way to include predefined templates inside of your Markdown, and built-in LiveReload web server, which allows you to preview your site changes locally as you make them.

      Note

      This guide was written using version 1.14 of Kubectl.

      Before You Begin

      1. Create a Kubernetes cluster with one worker node. This can be done in two ways:

        1. Deploy a Kubernetes cluster using kubeadm.
          • You will need to deploy two Linodes. One will serve as the master node and the other will serve as a worker node.
        2. Deploy a Kubernetes cluster using k8s-alpha CLI.
      2. Create a GitHub account if you don’t already have one.

      3. Create a Docker Hub account if you don’t already have one.

      Set up the Development Environment

      Development of your Hugo site and Docker image will take place locally on your personal computer. You will need to install Hugo, Docker CE, and Git, a version control software, on your personal computer to get started.

      1. Use the How to Install Git on Linux, Mac or Windows guide for the steps needed to install Git.

      2. Install Hugo. Hugo’s official documentation contains more information on installation methods, like Installing Hugo from Tarball. Below are installation instructions for common operating systems:

        • Debian/Ubuntu:

          sudo apt-get install hugo
          
        • Fedora, Red Hat and CentOS:

          sudo dnf install hugo
          
        • Mac, using Homebrew:

          brew install hugo
          
      3. These steps install Docker Community Edition (CE) using the official Ubuntu repositories. To install on another distribution, see the official installation page.

        1. Remove any older installations of Docker that may be on your system:

          sudo apt remove docker docker-engine docker.io
          
        2. Make sure you have the necessary packages to allow the use of Docker’s repository:

          sudo apt install apt-transport-https ca-certificates curl software-properties-common
          
        3. Add Docker’s GPG key:

          curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
          
        4. Verify the fingerprint of the GPG key:

          sudo apt-key fingerprint 0EBFCD88
          

          You should see output similar to the following:

            
          pub   4096R/0EBFCD88 2017-02-22
                  Key fingerprint = 9DC8 5822 9FC7 DD38 854A  E2D8 8D81 803C 0EBF CD88
          uid                  Docker Release (CE deb) 
          sub   4096R/F273FCD8 2017-02-22
          
          
        5. Add the stable Docker repository:

          sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"
          

          Note

          For Ubuntu 19.04 if you get an E: Package 'docker-ce' has no installation candidate error this is because the stable version of docker for is not yet available. Therefore, you will need to use the edge / test repository.

          sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable edge test"
          
        6. Update your package index and install Docker CE:

          sudo apt update
          sudo apt install docker-ce
          
        7. Add your limited Linux user account to the docker group:

          sudo usermod -aG docker $USER
          

          Note

          After entering the usermod command, you will need to close your SSH session and open a new one for this change to take effect.

        8. Check that the installation was successful by running the built-in “Hello World” program:

          docker run hello-world
          

      Create a Hugo Site

      Initialize the Hugo Site

      In this section you will use the Hugo CLI (command line interface) to create your Hugo site and initialize a Hugo theme. Hugo’s CLI provides several useful commands for common tasks needed to build, configure, and interact with your Hugo site.

      1. Create a new Hugo site on your local computer. This command will create a folder named example-site and scaffold Hugo’s directory structure inside it:

        hugo new site example-site
        
      2. Move into your Hugo site’s root directory:

        cd example-site
        
      3. You will use Git to add a theme to your Hugo site’s directory. Initialize your Hugo site’s directory as a Git repository:

        git init
        
      4. Install the Ananke theme as a submodule of your Hugo site’s Git repository. Git submodules allow one Git repository to be stored as a subdirectory of another Git repository, while still being able to maintain each repository’s version control information separately. The Ananke theme’s repository will be located in the ~/example-site/themes/ananke directory of your Hugo site.

        git submodule add https://github.com/budparr/gohugo-theme-ananke.git themes/ananke
        

        Note

        Hugo has many available themes that can be installed as a submodule of your Hugo site’s directory.
      5. Add the theme to your Hugo site’s configuration file. The configuration file (config.toml) is located at the root of your Hugo site’s directory.

        echo 'theme = "ananke"' >> config.toml
        

      Add Content to the Hugo Site

      You can now begin to add content to your Hugo site. In this section you will add a new post to your Hugo site and generate the corresponding static file by building the Hugo site on your local computer.

      1. Create a new content file for your site. This command will generate a Markdown file with an auto-populated date and title:

        hugo new posts/my-first-post.md
        
      2. You should see a similar output. Note that the file is located in the content/posts/ directory of your Hugo site:

          
        /home/username/example-site/content/posts/my-first-post.md created
            
        
      3. Open the Markdown file in the text editor of your choice to begin modifying its content; you can copy and paste the example snippet into your file, which contains an updated front matter section at the top and some example Markdown body text.

        Set your desired value for title. Then, set the draft state to false and add your content below the --- in Markdown syntax, if desired:

        /home/username/example-site/content/posts/my-first-post.md
         1
         2
         3
         4
         5
         6
         7
         8
         9
        10
        11
        12
        13
        
        ---
        title: "My First Post"
        date: 2019-05-07T11:25:11-04:00
        draft: false
        ---
        
        # Kubernetes Objects
        
        In Kubernetes, there are a number of objects that are abstractions of your Kubernetes system’s desired state. These objects represent your application, its networking, and disk resources – all of which together form your application. Kubernetes objects can describe:
        
        - Which containerized applications are running on the cluster
        - Application resources
        - Policies that should be applied to the application


        About front matter

        Front matter is a collection of metadata about your content, and it is embedded at the top of your file within opening and closing --- delimiters.

        Front matter is a powerful Hugo feature that provides a mechanism for passing data that is attached to a specific piece of content to Hugo’s rendering engine. Hugo accepts front matter in TOML, YAML, and JSON formats. In the example snippet, there is YAML front matter for the title, date, and draft state of the Markdown file. These variables will be referenced and displayed by your Hugo theme.

      4. Once you have added your content, you can preview your changes by building and serving the site using Hugo’s built-in webserver:

        hugo server
        
      5. You will see a similar output:

          
        &nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp| EN
        +------------------+----+
          Pages&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp| 11
          Paginator pages&nbsp&nbsp&nbsp&nbsp|  0
          Non-page files&nbsp&nbsp&nbsp&nbsp&nbsp|  0
          Static files&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp|  3
          Processed images&nbsp&nbsp&nbsp|  0
          Aliases&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp|  1
          Sitemaps&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp|  1
          Cleaned&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp|  0
        
        Total in 7 ms
        Watching for changes in /home/username/example-site/{content,data,layouts,static,themes}
        Watching for config changes in /home/username/example-site/config.toml
        Serving pages from memory
        Running in Fast Render Mode. For full rebuilds on change: hugo server --disableFastRender
        Web Server is available at http://localhost:1313/ (bind address 127.0.0.1)
        Press Ctrl+C to stop
        
        
      6. The output will provide a URL to preview your site. Copy and paste the URL into a browser to access the site. In the above example Hugo’s web server URL is http://localhost:1313/.

      7. When you are happy with your site’s content you can build the site:

        hugo -v
        

        Hugo will generate your site’s static HTML files and store them in a public directory that it will create inside your project. The static files that are generated by Hugo are the files that will be served to the internet through your Kubernetes cluster.

      8. View the contents of your site’s public directory:

        ls public
        

        Your output should resemble the following example. When you built the site, the Markdown file you created and edited in steps 6 and 7 was used to generate its corresponding static HTML file in the public/posts/my-first-post/index.html directory.

          
          404.html    categories  dist        images      index.html  index.xml   posts       sitemap.xml tags
            
        

      Version Control the Site with Git

      The example Hugo site was initialized as a local Git repository in the previous section. You can now version control all content, theme, and configuration files with Git. Once you have used Git to track your local Hugo site files, you can easily push them to a remote Git repository, like GitHub or GitLab. Storing your Hugo site files on a remote Git repository opens up many possibilities for collaboration and automating Docker image builds. This guide will not cover automated builds, but you can learn more about it on Docker’s official documentation.

      1. Add a .gitignore file to your Git repository. Any files or directories added to the .gitignore file will not be tracked by Git. The Docker image you will create in the next section will handle building your static site files. For this reason it is not necessary to track the public directory and its content.

        echo 'public/' >> .gitignore
        
      2. Display the state of your current working directory (root of your Hugo site):

        git status
        
      3. Stage all your files to be committed:

        git add -A
        
      4. Commit all your changes and add a meaningful commit message:

        git commit -m 'Add content, theme, and config files.'
        

        Note

        Any time you complete work related to one logical change to the Hugo site, you should make sure you commit the changes to your Git repository. Keeping your commits attached to small changes makes it easier to understand the changes and to roll back to previous commits, if necessary. See the Getting Started with Git guide for more information.

      Create a Docker Image

      Create the Dockerfile

      A Dockerfile contains the steps needed to build a Docker image. The Docker image provides the minimum set up and configuration necessary to deploy a container that satisfies its specific use case. The Hugo site’s minimum Docker container configuration requirements are an operating system, Hugo, the Hugo site’s content files, and the NGINX web server.

      1. In your Hugo site’s root directory, create and open a file named Dockerfile using the text editor of your choice. Add the following content to the file. You can read the Dockerfile comments to learn what each command will execute in the Docker container.

        Dockerfile
         1
         2
         3
         4
         5
         6
         7
         8
         9
        10
        11
        12
        13
        14
        15
        16
        17
        18
        19
        20
        21
        22
        23
        24
        
        #Install the container's OS.
        FROM ubuntu:latest as HUGOINSTALL
        
        # Install Hugo.
        RUN apt-get update
        RUN apt-get install hugo
        
        # Copy the contents of the current working directory to the hugo-site
        # directory. The directory will be created if it doesn't exist.
        COPY . /hugo-site
        
        # Use Hugo to build the static site files.
        RUN hugo -v --source=/hugo-site --destination=/hugo-site/public
        
        # Install NGINX and deactivate NGINX's default index.html file.
        # Move the static site files to NGINX's html directory.
        # This directory is where the static site files will be served from by NGINX.
        FROM nginx:stable-alpine
        RUN mv /usr/share/nginx/html/index.html /usr/share/nginx/html/old-index.html
        COPY --from=HUGOINSTALL /hugo-site/public/ /usr/share/nginx/html/
        
        # The container will listen on port 80 using the TCP protocol.
        EXPOSE 80
            
      2. Add a .dockerignore file to your Hugo repository. It is important to ensure that your images are as small as possible to reduce the time it takes to build, pull, push, and deploy the container. The .dockerignore file excludes files and directories that are not necessary for the function of your container or that may contain sensitive information that you do not want to included in the image. Since the Docker image will build the static Hugo site files, you can ignore the public/ directory. You can also exclude any Git related files and directories because they are not needed on the running container.

        echo -e "public/n.git/n.gitmodules/n.gitignore" >> .dockerignore
        
      3. Follow the steps 2 – 4 in the Version Control the Site with Git section to add any new files created in this section to your local git repository.

      Build the Docker Image

      You are now ready to build the Docker image. When Docker builds an image it incorporates the build context. A build context includes any files and directories located in the current working directory. By default, Docker assumes the current working directory is also the location of the Dockerfile.

      Note

      If you have not yet created a Docker Hub account, you will need to do so before proceeding with this section.
      1. Build the Docker image and add a tag mydockerhubusername/hugo-site:v1 to the image. Ensure you are in the root directory of your Hugo site. The tag will make it easy to reference a specific image version when creating your Kubernetes deployment manifest. Replace mydockerhubusername with your Docker Hub username and hugo-site with a Docker repository name you prefer.

        docker build -t mydockerhubusername/hugo-site:v1 .
        

        You should see a similar output. The entirety of the output has been removed for brevity:

          
        Sending build context to Docker daemon  3.307MB
        Step 1/10 : FROM ubuntu:latest as HUGOINSTALL
         ---> 94e814e2efa8
        Step 2/10 : ENV HUGO_VERSION=0.55.4
         ---> Using cache
         ---> e651df397e32
         ...
        
        Successfully built 50c590837916
        Successfully tagged hugo-k8s:v1
            
        
      2. View all locally available Docker images:

        docker images
        

        You should see the docker image hugo-site:v1 listed in the output:

          
        REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
        hugo-k8s            v1                  50c590837916        1 day ago          16.5MB
            
        


      Push your Hugo Site Repository to GitHub

      You can push your local Hugo site’s Git repository to GitHub in order to set up Docker automated builds. Docker automated builds will build an image using a external repository as the build context and automatically push the image to your Docker Hub repository. This step is not necessary to complete this guide.

      Host your Image on Docker Hub

      Hosting your Hugo site’s image on Docker Hub will enable you to use the image in a Kubernetes cluster deployment. You will also be able to share the image with collaborators and the rest of the Docker community.

      1. Log into your Docker Hub account via the command line on your local computer. Enter your username and password when prompted.

        docker login
        
      2. Push the local Docker image to Docker Hub. Replace mydockerhubusername/hugo-site:v1 with your image’s tag name.

        docker push mydockerhubusername/hugo-site:v1
        
      3. Navigate to Docker Hub to view your image on your account.

        The url for your image repository should be similar to the following: https://cloud.docker.com/repository/docker/mydockerhubusername/hugo-site. Replace the username and repository name with your own.

      Configure your Kubernetes Cluster

      This section will use kubectl to configure and manage your Kubernetes cluster. If your cluster was deployed using kubeadm, you will need to log into your master node to execute the kubectl commands in this section. If, instead, you used the k8s-alpha CLI you can run all commands from your local computer.

      In this section, you will create namespace, deployment, and service manifest files for your Hugo site deployment and apply them to your cluster with kubectl. Each manifest file creates different resources on the Kubernetes API that are used to create and the Hugo site’s pods on the worker nodes.

      Create the Namespace

      Namespaces provide a powerful way to logically partition your Kubernetes cluster and isolate components and resources to avoid collisions across the cluster. A common use-case is to encapsulate dev/testing/production environments with namespaces so that they can each utilize the same resource names across each stage of development.

      Namespaces add a layer of complexity to a cluster that may not always be necessary. It is important to keep this in mind when formulating the architecture for a project’s application. This example will create a namespace for demonstration purposes, but it is not a requirement. One situation where a namespace would be beneficial, in the context of this guide, would be if you were a developer and wanted to manage Hugo sites for several clients with a single Kubernetes cluster.

      1. Create a directory to store your Hugo site’s manifest files.

        mkdir -p clientx/k8s-hugo/
        
      2. Create the manifest file for your Hugo site’s namespace with the following content:

        clientx/k8s-hugo/ns-hugo-site.yaml
        1
        2
        3
        4
        5
        
        apiVersion: v1
        kind: Namespace
        metadata:
          name: hugo-site
              
        • The manifest file declares the version of the API in use, the kind of resource that is being defined, and metadata about the resource. All manifest files should provide this information.
        • The key-value pair name: hugo-site defines the namespace object’s unique name.
      3. Create the namespace from the ns-hugo-site.yaml manifest.

        kubectl create -f clientx/k8s-hugo/ns-hugo-site.yaml
        
      4. View all available namespaces in your cluster:

        kubectl get namespaces
        

        You should see the hugo-site namespace listed in the output:

          
        NAME          STATUS   AGE
        default       Active   1d
        hugo-site     Active   1d
        kube-public   Active   1d
        kube-system   Active   1d
            
        

      Create the Service

      The service will group together all pods for the Hugo site, expose the same port on all pods to the internet, and load balance site traffic between all pods. It is best to create a service prior to any controllers (like a deployment) so that the Kubernetes scheduler can distribute the pods for the service as they are created by the controller.

      The Hugo site’s service manifest file will use the NodePort method to get external traffic to the Hugo site service. NodePort opens a specific port on all the Nodes and any traffic that is sent to this port is forwarded to the service. Kubernetes will choose the port to open on the nodes if you do not provide one in your service manifest file. It is recommended to let Kubernetes handle the assignment. Kubernetes will choose a port in the default range, 30000-32767.

      Note

      The k8s-alpha CLI creates clusters that are pre-configured with useful Linode service integrations, like the Linode Cloud Controller Manager (CCM) which provides access to Linode’s load balancer service, NodeBalancers. In order to use Linode’s NodeBalancers you can use the LoadBalancer service type instead of NodePort in your Hugo site’s service manifest file. For more details, see the Kubernetes Cloud Controller Manager for Linode GitHub repository.
      1. Create the manifest file for your service with the following content.

        clientx/k8s-hugo/service-hugo.yaml
         1
         2
         3
         4
         5
         6
         7
         8
         9
        10
        11
        12
        13
        14
        
        apiVersion: v1
        kind: Service
        metadata:
          name: : hugo-site
          namespace: hugo-site
        spec:
          selector:
            app: hugo-site
          ports:
          - protocol: TCP
            port: 80
            targetPort: 80
          type: NodePort
            
        • The spec key defines the Hugo site service object’s desired behavior. It will create a service that exposes TCP port 80 on any pod with the app: hugo-site label.
        • The exposed container port is defined by the targetPort:80 key-value pair.
      2. Create the service for your hugo site:

        kubectl create -f clientx/k8s-hugo/service-hugo.yaml
        
      3. View the service and its corresponding information:

        kubectl get services -n hugo-site
        

        Your output will resemble the following:

          
        NAME        TYPE       CLUSTER-IP     EXTERNAL-IP   PORT(S)        AGE
        hugo-site   NodePort   10.108.110.6           80:30304/TCP   1d
            
        

      Create the Deployment

      A deployment is a controller that helps manage the state of your pods. The Hugo site deployment will define how many pods should be kept up and running with the Hugo site service and which container image should be used.

      1. Create the manifest file for your Hugo site’s deployment. Copy the following contents to your file.

        clientx/k8s-hugo/deployment.yaml
         1
         2
         3
         4
         5
         6
         7
         8
         9
        10
        11
        12
        13
        14
        15
        16
        17
        18
        19
        20
        21
        22
        
        apiVersion: apps/v1
        kind: Deployment
        metadata:
          name: hugo-site
          namespace: hugo-site
        spec:
          replicas: 3
          selector:
            matchLabels:
              app: hugo-site
          template:
            metadata:
              labels:
                app: hugo-site
            spec:
              containers:
              - name: hugo-site
                image: mydockerhubusername/hugo-site:v1
                imagePullPolicy: Always
                ports:
                - containerPort: 80
              
        • The deployment’s object spec states that the deployment should have 3 replica pods. This means at any given time the cluster will have 3 pods that run the Hugo site service.
        • The template field provides all the information needed to create actual pods.
        • The label app: hugo-site helps the deployment know which service pods to target.
        • The container field states that any containers connected to this deployment should use the Hugo site image mydockerhubusername/hugo-site:v1 that was created in the Build the Docker Image section of this guide.
        • imagePullPolicy: Always means that the container image will be pulled every time the pod is started.
        • containerPort: 80 states the port number to expose on the pod’s IP address. The system does not rely on this field to expose the container port, instead, it provides information about the network connections a container uses.
      2. Create the deployment for your hugo site:

        kubectl create -f clientx/k8s-hugo/deployment.yaml
        
      3. View the Hugo site’s deployment:

        kubectl get deployment hugo-site -n hugo-site
        

        Your output will resemble the following:

          
        NAME        READY   UP-TO-DATE   AVAILABLE   AGE
        hugo-site   3/3     3            3           1d
            
        

      View the Hugo Site

      After creating all required manifest files to configure your Hugo site’s Kubernetes cluster, you should be able to view the site using a worker node’s IP address and its exposed port.

      1. Get your worker node’s external IP address. Copy down the EXTERNAL-IP value for any worker node in the cluster:

        kubectl get nodes -o wide
        
      2. Access the hugo-site services to view its exposed port.

        kubectl get svc -n hugo-site
        

        The output will resemble the following. Copy down the listed port number in the 30000-32767 range.

          
        NAME        TYPE       CLUSTER-IP     EXTERNAL-IP   PORT(S)        AGE
        hugo-site   NodePort   10.108.110.6           80:30304/TCP   1d
            
        
      3. Open a browser window and enter in a worker node’s IP address and exposed port. An example url to your Hugo site would be, http://192.0.2.1:30304. Your Hugo site should appear.

        If desired, you can purchase a domain name and use Linode’s DNS Manager to assign a domain name to the cluster’s worker node IP address.

      Tear Down Your Cluster

      To avoid being further billed for your Kubernetes cluster, tear down your cluster’s Linodes. If you have Linodes that existed for only part a monthly billing cycle, you’ll be billed at the hourly rate for that service. See How Hourly Billing Works to learn more.

      Next Steps

      Now that you are familiar with basic Kubernetes concepts, like configuring pods, grouping resources, and deploying services, you can deploy a Kubernetes cluster on Linode for production use by using the steps in the following guides:

      More Information

      You may wish to consult the following resources for additional information on this topic. While these are provided in the hope that they will be useful, please note that we cannot vouch for the accuracy or timeliness of externally hosted materials.

      Find answers, ask questions, and help others.

      This guide is published under a CC BY-ND 4.0 license.



      Source link

      How To Create a Self-Signed SSL Certificate for Apache in Debian 10


      Introduction

      TLS, or transport layer security, and its predecessor SSL, which stands for secure sockets layer, are web protocols used to wrap normal traffic in a protected, encrypted wrapper.

      Using this technology, servers can send traffic safely between servers and clients without the possibility of messages being intercepted by outside parties. The certificate system also assists users in verifying the identity of the sites that they are connecting with.

      In this guide, we will show you how to set up a self-signed SSL certificate for use with an Apache web server on Debian 10.

      Note: A self-signed certificate will encrypt communication between your server and any clients. However, because it is not signed by any of the trusted certificate authorities included with web browsers, users cannot use the certificate to validate the identity of your server automatically.

      A self-signed certificate may be appropriate if you do not have a domain name associated with your server and for instances where an encrypted web interface is not user-facing. If you do have a domain name, in many cases it is better to use a CA-signed certificate. You can find out how to set up a free trusted certificate with the Let’s Encrypt project here.

      Prerequisites

      Before you begin, you should have a non-root user configured with sudo privileges. You can learn how to set up such a user account by following our Initial Server Setup with Debian 10.

      You will also need to have the Apache web server installed. If you would like to install an entire LAMP (Linux, Apache, MariaDB, PHP) stack on your server, you can follow our guide on setting up LAMP on Debian 10. If you just want the Apache web server, skip the steps pertaining to PHP and MariaDB.

      When you have completed these prerequisites, continue below.

      Step 1 — Creating the SSL Certificate

      TLS/SSL works by using a combination of a public certificate and a private key. The SSL key is kept secret on the server. It is used to encrypt content sent to clients. The SSL certificate is publicly shared with anyone requesting the content. It can be used to decrypt the content signed by the associated SSL key.

      We can create a self-signed key and certificate pair with OpenSSL in a single command:

      • sudo openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout /etc/ssl/private/apache-selfsigned.key -out /etc/ssl/certs/apache-selfsigned.crt

      You will be asked a series of questions. Before we go over that, let’s take a look at what is happening in the command we are issuing:

      • openssl: This is the basic command line tool for creating and managing OpenSSL certificates, keys, and other files.
      • req: This subcommand specifies that we want to use X.509 certificate signing request (CSR) management. The “X.509” is a public key infrastructure standard that SSL and TLS adheres to for its key and certificate management. We want to create a new X.509 cert, so we are using this subcommand.
      • -x509: This further modifies the previous subcommand by telling the utility that we want to make a self-signed certificate instead of generating a certificate signing request, as would normally happen.
      • -nodes: This tells OpenSSL to skip the option to secure our certificate with a passphrase. We need Apache to be able to read the file, without user intervention, when the server starts up. A passphrase would prevent this from happening because we would have to enter it after every restart.
      • -days 365: This option sets the length of time that the certificate will be considered valid. We set it for one year here.
      • -newkey rsa:2048: This specifies that we want to generate a new certificate and a new key at the same time. We did not create the key that is required to sign the certificate in a previous step, so we need to create it along with the certificate. The rsa:2048 portion tells it to make an RSA key that is 2048 bits long.
      • -keyout: This line tells OpenSSL where to place the generated private key file that we are creating.
      • -out: This tells OpenSSL where to place the certificate that we are creating.

      As we stated above, these options will create both a key file and a certificate. We will be asked a few questions about our server in order to embed the information correctly in the certificate.

      Fill out the prompts appropriately. The most important line is the one that requests the Common Name (e.g. server FQDN or YOUR name). You need to enter the domain name associated with your server or, more likely, your server’s public IP address.

      The entirety of the prompts will look something like this:

      Output

      Country Name (2 letter code) [AU]:US State or Province Name (full name) [Some-State]:New York Locality Name (eg, city) []:New York City Organization Name (eg, company) [Internet Widgits Pty Ltd]:Bouncy Castles, Inc. Organizational Unit Name (eg, section) []:Ministry of Water Slides Common Name (e.g. server FQDN or YOUR name) []:server_IP_address Email Address []:admin@your_domain.com

      Both of the files you created will be placed in the appropriate subdirectories under /etc/ssl.

      Step 2 — Configuring Apache to Use SSL

      We have created our key and certificate files under the /etc/ssl directory. Now we just need to modify our Apache configuration to take advantage of these.

      We will make a few adjustments to our configuration:

      1. We will create a configuration snippet to specify strong default SSL settings.
      2. We will modify the included SSL Apache Virtual Host file to point to our generated SSL certificates.
      3. (Recommended) We will modify the unencrypted Virtual Host file to automatically redirect requests to the encrypted Virtual Host.

      When we are finished, we should have a secure SSL configuration.

      Creating an Apache Configuration Snippet with Strong Encryption Settings

      First, we will create an Apache configuration snippet to define some SSL settings. This will set Apache up with a strong SSL cipher suite and enable some advanced features that will help keep our server secure. The parameters we will set can be used by any Virtual Hosts enabling SSL.

      Create a new snippet in the /etc/apache2/conf-available directory. We will name the file ssl-params.conf to make its purpose clear:

      • sudo nano /etc/apache2/conf-available/ssl-params.conf

      To set up Apache SSL securely, we will be using the recommendations by Remy van Elst on the Cipherli.st site. This site is designed to provide easy-to-consume encryption settings for popular software.

      The suggested settings on the site linked to above offer strong security. Sometimes, this comes at the cost of greater client compatibility. If you need to support older clients, there is an alternative list that can be accessed by clicking the link on the page labelled “Yes, give me a ciphersuite that works with legacy / old software.” That list can be substituted for the items copied below.

      The choice of which config you use will depend largely on what you need to support. They both will provide great security.

      For our purposes, we can copy the provided settings in their entirety. We will just make one small change to this and disable the Strict-Transport-Security header (HSTS).

      Preloading HSTS provides increased security, but can have far-reaching consequences if accidentally enabled or enabled incorrectly. In this guide, we will not enable the settings, but you can modify that if you are sure you understand the implications.

      Before deciding, take a moment to read up on HTTP Strict Transport Security, or HSTS, and specifically about the “preload” functionality.

      Paste the following configuration into the ssl-params.conf file we opened:

      /etc/apache2/conf-available/ssl-params.conf

      SSLCipherSuite EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH
      SSLProtocol All -SSLv2 -SSLv3 -TLSv1 -TLSv1.1
      SSLHonorCipherOrder On
      # Disable preloading HSTS for now.  You can use the commented out header line that includes
      # the "preload" directive if you understand the implications.
      # Header always set Strict-Transport-Security "max-age=63072000; includeSubDomains; preload"
      Header always set X-Frame-Options DENY
      Header always set X-Content-Type-Options nosniff
      # Requires Apache >= 2.4
      SSLCompression off
      SSLUseStapling on
      SSLStaplingCache "shmcb:logs/stapling-cache(150000)"
      # Requires Apache >= 2.4.11
      SSLSessionTickets Off
      

      Save and close the file when you are finished.

      Modifying the Default Apache SSL Virtual Host File

      Next, let’s modify /etc/apache2/sites-available/default-ssl.conf, the default Apache SSL Virtual Host file. If you are using a different server block file, substitute its name in the commands below.

      Before we go any further, let’s back up the original SSL Virtual Host file:

      • sudo cp /etc/apache2/sites-available/default-ssl.conf /etc/apache2/sites-available/default-ssl.conf.bak

      Now, open the SSL Virtual Host file to make adjustments:

      • sudo nano /etc/apache2/sites-available/default-ssl.conf

      Inside, with most of the comments removed, the Virtual Host block should look something like this by default:

      /etc/apache2/sites-available/default-ssl.conf

      <IfModule mod_ssl.c>
              <VirtualHost _default_:443>
                      ServerAdmin webmaster@localhost
      
                      DocumentRoot /var/www/html
      
                      ErrorLog ${APACHE_LOG_DIR}/error.log
                      CustomLog ${APACHE_LOG_DIR}/access.log combined
      
                      SSLEngine on
      
                      SSLCertificateFile      /etc/ssl/certs/ssl-cert-snakeoil.pem
                      SSLCertificateKeyFile /etc/ssl/private/ssl-cert-snakeoil.key
      
                      <FilesMatch ".(cgi|shtml|phtml|php)$">
                                      SSLOptions +StdEnvVars
                      </FilesMatch>
                      <Directory /usr/lib/cgi-bin>
                                      SSLOptions +StdEnvVars
                      </Directory>
      
              </VirtualHost>
      </IfModule>
      

      We will be making some minor adjustments to the file. We will set the normal things we’d want to adjust in a Virtual Host file (ServerAdmin email address, ServerName, etc.), and adjust the SSL directives to point to our certificate and key files. Again, if you’re using a different document root, be sure to update the DocumentRoot directive.

      After making these changes, your server block should look similar to this:

      /etc/apache2/sites-available/default-ssl.conf

      <IfModule mod_ssl.c>
              <VirtualHost _default_:443>
                      ServerAdmin your_email@example.com
                      ServerName server_domain_or_IP
      
                      DocumentRoot /var/www/html
      
                      ErrorLog ${APACHE_LOG_DIR}/error.log
                      CustomLog ${APACHE_LOG_DIR}/access.log combined
      
                      SSLEngine on
      
                      SSLCertificateFile      /etc/ssl/certs/apache-selfsigned.crt
                      SSLCertificateKeyFile /etc/ssl/private/apache-selfsigned.key
      
                      <FilesMatch ".(cgi|shtml|phtml|php)$">
                                      SSLOptions +StdEnvVars
                      </FilesMatch>
                      <Directory /usr/lib/cgi-bin>
                                      SSLOptions +StdEnvVars
                      </Directory>
      
              </VirtualHost>
      </IfModule>
      

      Save and close the file when you are finished.

      As it stands now, the server will provide both unencrypted HTTP and encrypted HTTPS traffic. For better security, it is recommended in most cases to redirect HTTP to HTTPS automatically. If you do not want or need this functionality, you can safely skip this section.

      To adjust the unencrypted Virtual Host file to redirect all traffic to be SSL encrypted, open the /etc/apache2/sites-available/000-default.conf file:

      • sudo nano /etc/apache2/sites-available/000-default.conf

      Inside, within the VirtualHost configuration blocks, add a Redirect directive, pointing all traffic to the SSL version of the site:

      /etc/apache2/sites-available/000-default.conf

      <VirtualHost *:80>
              . . .
      
              Redirect "/" "https://your_domain_or_IP/"
      
              . . .
      </VirtualHost>
      

      Save and close the file when you are finished.

      That’s all of the configuration changes you need to make to Apache. Next, we will discuss how to update firewall rules with ufw to allow encrypted HTTPS traffic to your server.

      Step 3 — Adjusting the Firewall

      If you have the ufw firewall enabled, as recommended by the prerequisite guides, you might need to adjust the settings to allow for SSL traffic. Fortunately, when installed on Debian 10, ufw comes loaded with app profiles which you can use to tweak your firewall settings

      We can see the available profiles by typing:

      You should see a list like this, with the following four profiles near the bottom of the output:

      Output

      Available applications: . . . WWW WWW Cache WWW Full WWW Secure . . .

      You can see the current setting by typing:

      If you allowed only regular HTTP traffic earlier, your output might look like this:

      Output

      Status: active To Action From -- ------ ---- OpenSSH ALLOW Anywhere WWW ALLOW Anywhere OpenSSH (v6) ALLOW Anywhere (v6) WWW (v6) ALLOW Anywhere (v6)

      To additionally let in HTTPS traffic, allow the "WWW Full" profile and then delete the redundant "WWW" profile allowance:

      • sudo ufw allow 'WWW Full'
      • sudo ufw delete allow 'WWW'

      Your status should look like this now:

      Output

      Status: active To Action From -- ------ ---- OpenSSH ALLOW Anywhere WWW Full ALLOW Anywhere OpenSSH (v6) ALLOW Anywhere (v6) WWW Full (v6) ALLOW Anywhere (v6)

      With your firewall configured to allow HTTPS traffic, you can move on to the next step where we’ll go over how to enable a few modules and configuration files to allow SSL to function properly.

      Step 4 — Enabling the Changes in Apache

      Now that we've made our changes and adjusted our firewall, we can enable the SSL and headers modules in Apache, enable our SSL-ready Virtual Host, and then restart Apache to put these changes into effect.

      Enable mod_ssl, the Apache SSL module, and mod_headers, which is needed by some of the settings in our SSL snippet, with the a2enmod command:

      • sudo a2enmod ssl
      • sudo a2enmod headers

      Next, enable your SSL Virtual Host with the a2ensite command:

      • sudo a2ensite default-ssl

      You will also need to enable your ssl-params.conf file, to read in the values you’ve set:

      At this point, the site and the necessary modules are enabled. We should check to make sure that there are no syntax errors in our files. Do this by typing:

      • sudo apache2ctl configtest

      If everything is successful, you will get a result that looks like this:

      Output

      Syntax OK

      As long as your output has Syntax OK in it, then your configuration file has no syntax errors and you can safely restart Apache to implement the changes:

      • sudo systemctl restart apache2

      With that, your self-signed SSL certificate is all set. You can now test that your server is correctly encrypting its traffic.

      Step 5 — Testing Encryption

      You’re now ready to test your SSL server.

      Open your web browser and type https:// followed by your server's domain name or IP into the address bar:

      https://server_domain_or_IP
      

      Because the certificate you created isn't signed by one of your browser's trusted certificate authorities, you will likely see a scary looking warning like the one below:

      Apache self-signed cert warning

      This is expected and normal. We are only interested in the encryption aspect of our certificate, not the third party validation of our host's authenticity. Click ADVANCED and then the link provided to proceed to your host anyways:

      Apache self-signed override

      You should be taken to your site. If you look in the browser address bar, you will see a lock with an "x" over it or another similar “not secure” notice. In this case, this just means that the certificate cannot be validated. It is still encrypting your connection.

      If you configured Apache to redirect HTTP to HTTPS, you can also check whether the redirect functions correctly:

      http://server_domain_or_IP
      

      If this results in the same icon, this means that your redirect worked correctly. However, the redirect you created earlier is only a temporary redirect. If you’d like to make the redirection to HTTPS permanent, continue on to the final step.

      Step 6 — Changing to a Permanent Redirect

      If your redirect worked correctly and you are sure you want to allow only encrypted traffic, you should modify the unencrypted Apache Virtual Host again to make the redirect permanent.

      Open your server block configuration file again:

      • sudo nano /etc/apache2/sites-available/000-default.conf

      Find the Redirect line we added earlier. Add permanent to that line, which changes the redirect from a 302 temporary redirect to a 301 permanent redirect:

      /etc/apache2/sites-available/000-default.conf

      <VirtualHost *:80>
              . . .
      
              Redirect permanent "/" "https://your_domain_or_IP/"
      
              . . .
      </VirtualHost>
      

      Save and close the file.

      Check your configuration for syntax errors:

      • sudo apache2ctl configtest

      If this command doesn’t report any syntax errors, restart Apache:

      • sudo systemctl restart apache2

      This will make the redirect permanent, and your site will only serve traffic over HTTPS.

      Conclusion

      You have configured your Apache server to use strong encryption for client connections. This will allow you serve requests securely, and will prevent outside parties from reading your traffic.



      Source link

      Create a TCP and UDP Client and Server using Go


      Updated by Linode Contributed by Mihalis Tsoukalos

      Go is a compiled, statically typed programming language developed by Google. Many modern applications, including Docker, Kubernetes, and Terraform, are written in Go. Go packages allow developers to organize and reuse Go code in a simple and maintainable manner.

      In this guide, you will use the net package, which is a part of Go’s standard library, to create TCP and UDP servers and clients. This guide is meant to provide instructional examples to help you become more familiar with the Go programming language.

      Scope of this Guide

      Throughout this guide you will create the following:

      • A TCP server and client. The TCP server accepts incoming messages from a TCP client and responds with the current date and time.
      • A UDP server and client. The UDP server accepts incoming messages from a UDP client and responds with a random number.
      • A concurrent TCP server that accepts incoming messages from several TCP clients and responds with the number of clients currently connected to it.

      Before You Begin

      1. If you are not familiar with using Go packages, review the Getting Started with Go Packages guide.

      2. Install Go on your computer if it is not already installed. You can follow our guide How to Install Go on Ubuntu for installation steps.

        This guide requires Go version 1.8 or higher. It is considered good practice to have the latest version of Go installed. You can check your Go version by executing the following command:

        go version
        

      Note

      This guide is written for a non-root user. Depending on the TCP/IP port number that you use when running the TCP and UDP servers, you may need to prefix commands with sudo. If you are not familiar with the sudo command, see the Users and Groups guide.

      Protocol Definitions

      Protocol Definition
      TCP (Transmission Control Protocol) TCP’s principal characteristic is that it is a reliable protocol by design. If there is no proof of a packet’s delivery, TCP will resend the packet. Some of the tasks TCP packets can be used for are establishing connections, transferring data, sending acknowledgements, and closing connections.
      IP (Internet Protocol) The IP protocol adheres to the end-to-end principle, which places all network intelligence in the end nodes and not in the intermediary nodes. This design favors a reduction in network complexity over reliability. For this reason, the Internet Protocol does not guarantee a reliable delivery of packets over a network. Instead, IP works together with TCP to reliably deliver packets over a network.
      UDP (User Datagram Protocol): UDP provides a simpler implementation of the transport layer protocol that, while less reliable than TCP, is much faster. UDP does not provide error checking, correction or packet retransmission, which makes it very fast. When speed is more important than reliability, UDP is generally chosen over TCP. UDP is commonly used for online gaming, video chatting, and other real-time applications.

      The net Package

      Go’s net package provides a portable interface for network I/O, including TCP/IP, UDP, domain name resolution, and Unix domain sockets. You will use this package to create TCP and UDP servers and clients in this guide.

      net Package Functions

      Use the table below as a quick reference for some of the net package functions used throughout this guide. To view all types and functions included in the net package, see Golang’s official documentation.

      Note

      All versions of net.Dial() and net.Listen() return data types that implement the io.Reader and io.Writer interfaces. This means that you can use regular File I/O functions to send and receive data from a TCP/IP connections.
      Type Function
      type Listener func Listen(network, address string) (Listener, error)

         • The network parameter defines the type of network to use and accepts values tcp, tcp4 (IPv4-only), tcp6 (IPv6-only), unix (Unix sockets), or unixpacket.

         • The address parameter defines the server address and port number that the server will listen on.

      type UDPConn func ListenUDP(network string, laddr *UDPAddr) (*UDPConn, error)

         • Used to create UDP servers.

         • The network parameter must be a UDP network name.

         • The laddr parameter defines the server address and port number that the server will listen on.

      func DialUDP(network string, laddr, raddr *UDPAddr) (*UDPConn, error)

         • Used to specify the kind of client you will create.

         • The network parameter must be a UDP network name.

         • The laddr is the listening address (server). If laddr is nil, a local address is automatically chosen.

         • raddr is the response address (client). If the IP field of raddr is nil or an unspecified IP address, the local system is assumed.

      type UDPAddr func ResolveUDPAddr(network, address string) (*UDPAddr, error)

         • This function returns the address of a UDP end point.

         • The network parameter must be a UDP network name.

         • The address parameter has the form host:port. The host must be a an IP address, or a host name that can be resolved to IP addresses.

      type TCPAddr func ResolveTCPAddr(network, address string) (*TCPAddr, error)

         • This function returns the address of a TCP end point.

         • The network parameter must be a TCP network name.

         • The address parameter has the form host:port. The host must be a an IP address, or a host name that can be resolved to IP addresses.

      type Conn func Dial(network, address string) (Conn, error)

         • This function connects to the address on the named network.

         • The network parameter can be tcp, tcp4 (IPv4-only), tcp6 (IPv6-only), udp, udp4 (IPv4-only), udp6 (IPv6-only), ip, ip4 (IPv4-only), ip6 (IPv6-only), unix, unixgram and unixpacket.

         • When using TCP or UDP networks, the address parameter has the form host:port. The host must be a an IP address, or a host name that can be resolved to IP addresses.

      type TCPConn func DialTCP(network string, laddr, raddr *TCPAddr) (*TCPConn, error)

         • This function connects to the address on the TCP networks.

         • The network parameter must be a TCP network name.

         • The laddr is the listening address (server). If laddr is nil, a local address is automatically chosen.

         • raddr is the response address (client). If the IP field of raddr is nil or an unspecified IP address, the local system is assumed.

      Create a TCP Client and Server

      In this section, you will create a generic TCP client and server using Go. After creating the client and server, you will run them to test their connection with each other.

      Note

      Create the TCP Client

      The TCP client that you will create in this section will allow you to interact with any TCP server.

      1. In your current working directory, create a file named tcpC.go with the following content:

        ./tcpC.go
         1
         2
         3
         4
         5
         6
         7
         8
         9
        10
        11
        12
        13
        14
        15
        16
        17
        18
        19
        20
        21
        22
        23
        24
        25
        26
        27
        28
        29
        30
        31
        32
        33
        34
        35
        36
        37
        38
        39
        
        package main
        
        import (
                "bufio"
                "fmt"
                "net"
                "os"
                "strings"
        )
        
        func main() {
                arguments := os.Args
                if len(arguments) == 1 {
                        fmt.Println("Please provide host:port.")
                        return
                }
        
                CONNECT := arguments[1]
                c, err := net.Dial("tcp", CONNECT)
                if err != nil {
                        fmt.Println(err)
                        return
                }
        
                for {
                        reader := bufio.NewReader(os.Stdin)
                        fmt.Print(">> ")
                        text, _ := reader.ReadString('n')
                        fmt.Fprintf(c, text+"n")
        
                        message, _ := bufio.NewReader(c).ReadString('n')
                        fmt.Print("->: " + message)
                        if strings.TrimSpace(string(text)) == "STOP" {
                                fmt.Println("TCP client exiting...")
                                return
                        }
                }
        }
            
        • This file creates the main package, which declares the main() function. The function will use the imported packages to create a TCP client.
        • The main() function gathers command line arguments in the arguments variable and makes sure that a value for host:port was sent.
        • The CONNECT variable stores the value of arguments[1]to be used in the net.Dial() call.
        • A call to net.Dial() begins the implementation of the TCP client and will connect you to the desired TCP server. The second parameter of net.Dial() has two parts; the first is the hostname or the IP address of the TCP server and the second is the port number the TCP server listens on.
        • bufio.NewReader(os.Stdin) and ReadString() is used to read user input. Any user input is sent to the TCP server over the network using Fprintf().
        • bufio reader and the bufio.NewReader(c).ReadString('n') statement read the TCP server’s response. The error variable is ignored here for simplicity.
        • The entire for loop that is used to read user input will only terminate when you send the STOP command to the TCP server.

      Create the TCP Server

      You are now ready to create the TCP server. The TCP server will return the current date and time to the TCP client using a single network packet.

      1. In your current working directory, create a file named tcpS.go with the following content:

        ./tcpS.go
         1
         2
         3
         4
         5
         6
         7
         8
         9
        10
        11
        12
        13
        14
        15
        16
        17
        18
        19
        20
        21
        22
        23
        24
        25
        26
        27
        28
        29
        30
        31
        32
        33
        34
        35
        36
        37
        38
        39
        40
        41
        42
        43
        44
        45
        46
        47
        48
        49
        50
        
        package main
        
        import (
                "bufio"
                "fmt"
                "net"
                "os"
                "strings"
                "time"
        )
        
        func main() {
                arguments := os.Args
                if len(arguments) == 1 {
                        fmt.Println("Please provide port number")
                        return
                }
        
                PORT := ":" + arguments[1]
                l, err := net.Listen("tcp", PORT)
                if err != nil {
                        fmt.Println(err)
                        return
                }
                defer l.Close()
        
                c, err := l.Accept()
                if err != nil {
                        fmt.Println(err)
                        return
                }
        
                for {
                        netData, err := bufio.NewReader(c).ReadString('n')
                        if err != nil {
                                fmt.Println(err)
                                return
                        }
                        if strings.TrimSpace(string(netData)) == "STOP" {
                                fmt.Println("Exiting TCP server!")
                                return
                        }
        
                        fmt.Print("-> ", string(netData))
                        t := time.Now()
                        myTime := t.Format(time.RFC3339) + "n"
                        c.Write([]byte(myTime))
                }
        }
            
        • This file creates the main package, which declares the main() function. The function will use the imported packages to create a TCP server.
        • The main() function gathers command line arguments in the arguments variable and includes error handling.
        • The net.Listen() function makes the program a TCP server. This functions returns a Listener variable, which is a generic network listener for stream-oriented protocols.
        • It is only after a successful call to Accept() that the TCP server can begin to interact with TCP clients.
        • The current implementation of the TCP server can only serve the first TCP client that connects to it, because the Accept() call is outside of the for loop. In the Create a Concurrent TCP Server section of this guide, you will see a TCP server implementation that can serve multiple TCP clients using Goroutines.
        • The TCP server uses regular File I/O functions to interact with TCP clients. This interaction takes place inside the for loop. Similarly to the TCP client, when the TCP server receives the STOP command from the TCP client, it will terminate.

      Test the TCP Client and Server

      You can now test your TCP client and server. You will need to execute the TCP server first so that the TCP client has somewhere it can connect to.

      1. Run your TCP server. From the directory containing the tcpS.go file, run the following command:

        go run tcpS.go 1234
        

        The server will listen on port number 1234. You will not see any output as a result of this command.

      2. Open a second shell session to execute the TCP client and to interact with the TCP server. Run the following command:

        go run tcpC.go 127.0.0.1:1234
        

        Note

        If the TCP server is not running on the expected TCP port, you will get the following error message from tcpC.go:

        dial tcp [::1]:1234: connect: connection refused
        
      3. You will see a >> prompt waiting for you to enter some text. Type in Hello! to receive a response from the TCP server:

        Hello!
        

        You should see a similar output:

          
        >> Hello!
        ->: 2019-05-23T19:43:21+03:00
            
        
      4. Send the STOP command to exit the TCP client and server:

        STOP
        

        You should see a similar output in the client:

          
        >> STOP
        ->: TCP client exiting...
            
        

        The output on the TCP server side will resemble the following:

          
        -> Hello!
        Exiting TCP server!
            
        

      Note

      The TCP server waits before writing back to the TCP client, whereas the client writes to the TCP server first and then waits to receive an answer. This behavior is part of the protocol definition that governs a TCP or a UDP connection. In this example, you have implemented an unofficial protocol that is based on TCP.

      Create a UDP Client and Server

      In this section, you will create a UDP client and server. After creating the client and server, you will run them both to test their connection with each other. A UDP client can be generic and can communicate with multiple UDP servers. On the other hand, a UDP server cannot be completely generic, because it typically implements a specific functionality. In the case of our UDP server example, it will return random numbers to UDP clients that connect to it.

      Create the UDP Client

      The UDP client that you will create in this section will allow you to interact with any UDP server.

      1. In your current working directory, create a file named udpC.go with the following content:

        ./udpC.go
         1
         2
         3
         4
         5
         6
         7
         8
         9
        10
        11
        12
        13
        14
        15
        16
        17
        18
        19
        20
        21
        22
        23
        24
        25
        26
        27
        28
        29
        30
        31
        32
        33
        34
        35
        36
        37
        38
        39
        40
        41
        42
        43
        44
        45
        46
        47
        48
        49
        50
        51
        52
        53
        54
        
        package main
        
        import (
                "bufio"
                "fmt"
                "net"
                "os"
                "strings"
        )
        
        func main() {
                arguments := os.Args
                if len(arguments) == 1 {
                        fmt.Println("Please provide a host:port string")
                        return
                }
                CONNECT := arguments[1]
        
                s, err := net.ResolveUDPAddr("udp4", CONNECT)
                c, err := net.DialUDP("udp4", nil, s)
                if err != nil {
                        fmt.Println(err)
                        return
                }
        
                fmt.Printf("The UDP server is %sn", c.RemoteAddr().String())
                defer c.Close()
        
                for {
                        reader := bufio.NewReader(os.Stdin)
                        fmt.Print(">> ")
                        text, _ := reader.ReadString('n')
                        data := []byte(text + "n")
                        _, err = c.Write(data)
                        if strings.TrimSpace(string(data)) == "STOP" {
                                fmt.Println("Exiting UDP client!")
                                return
                        }
        
                        if err != nil {
                                fmt.Println(err)
                                return
                        }
        
                        buffer := make([]byte, 1024)
                        n, _, err := c.ReadFromUDP(buffer)
                        if err != nil {
                                fmt.Println(err)
                                return
                        }
                        fmt.Printf("Reply: %sn", string(buffer[0:n]))
                }
        }
              
        • This file creates the main package, which declares the main() function. The function will use the imported packages to create a UDP client.
        • The main() function gathers command line arguments in the arguments variable and includes error handling.
        • Regular File I/O functions are used by the UDP client to interact with the UDP server. The client will terminate when you send the STOP command to the UDP server. This is not part of the UDP protocol, but is used in the example to provide the client with a way to exit.
        • A UDP end point address is returned by the net.ResolveUDPAddr() function. The UDP end point is of type UDPAddr and contains IP and port information.
        • The connection to the UDP server is established with the use of the net.DialUDP() function.
        • bufio.NewReader(os.Stdin) and ReadString() is used to read user input.
        • The ReadFromUDP() function reads a packet from the server connection and will return if it encounters an error.

      Create the UDP Server

      You are now ready to create the UDP server. You will write the UDP server code to respond to any connected client with random numbers.

      1. In your current working directory, create a file named udps.go with the following content:

        ./udpS.go
         1
         2
         3
         4
         5
         6
         7
         8
         9
        10
        11
        12
        13
        14
        15
        16
        17
        18
        19
        20
        21
        22
        23
        24
        25
        26
        27
        28
        29
        30
        31
        32
        33
        34
        35
        36
        37
        38
        39
        40
        41
        42
        43
        44
        45
        46
        47
        48
        49
        50
        51
        52
        53
        54
        55
        56
        57
        58
        59
        
        package main
        
        import (
                "fmt"
                "math/rand"
                "net"
                "os"
                "strconv"
                "strings"
                "time"
        )
        
        func random(min, max int) int {
                return rand.Intn(max-min) + min
        }
        
        func main() {
                arguments := os.Args
                if len(arguments) == 1 {
                        fmt.Println("Please provide a port number!")
                        return
                }
                PORT := ":" + arguments[1]
        
                s, err := net.ResolveUDPAddr("udp4", PORT)
                if err != nil {
                        fmt.Println(err)
                        return
                }
        
                connection, err := net.ListenUDP("udp4", s)
                if err != nil {
                        fmt.Println(err)
                        return
                }
        
                defer connection.Close()
                buffer := make([]byte, 1024)
                rand.Seed(time.Now().Unix())
        
                for {
                        n, addr, err := connection.ReadFromUDP(buffer)
                        fmt.Print("-> ", string(buffer[0:n-1]))
        
                        if strings.TrimSpace(string(buffer[0:n])) == "STOP" {
                                fmt.Println("Exiting UDP server!")
                                return
                        }
        
                        data := []byte(strconv.Itoa(random(1, 1001)))
                        fmt.Printf("data: %sn", string(data))
                        _, err = connection.WriteToUDP(data, addr)
                        if err != nil {
                                fmt.Println(err)
                                return
                        }
                }
        }
            
        • This file creates the main package, which declares the main() function. The function will use the imported packages to create a UDP server.
        • The main() function gathers command line arguments in the arguments variable and includes error handling.
        • The net.ListenUDP() function tells the application to listen for incoming UDP connections, which are served inside the for loop. This is the function call that makes the program a UDP server.
        • The ReadFromUDP() and WriteToUDP() functions are used to read data from a UDP connection and write data to a UDP connection, respectively. A byte slice is stored in the data variable and used to write the desired data. The buffer variable also stores a byte slice and is used to read data.
        • Since UDP is a stateless protocol, each UDP client is served and then the connection closes automatically. The UDP server program will only exit when it receives the STOP keyword from a UDP client. Otherwise, the server program will continue to wait for more UDP connections from other clients.

      Test the UDP Client and Server

      You can now test your UDP client and server. You will need to execute the UDP server first so that the UDP client has somewhere it can connect to.

      1. Run your UDP server. From the directory containing the udpS.go file, run the following command:

        go run udpS.go 1234
        

        The server will listen on port number 1234. You will not see any output as a result of this command.

      2. Open a second shell session to execute the UDP client and to interact with the UDP server. Run the following command:

        go run udpC.go 127.0.0.1:1234
        
      3. You will see a >> prompt waiting for you to enter some text. Type in Hello! to receive a response from the UDP server:

        Hello!
        

        You should see a similar output:

          
        The UDP server is 127.0.0.1:1234
        >> Hello!
        Reply: 82
            
        
      4. Send the STOP command to exit the UDP client and server:

        You should see a similar output on the client side:

          
        >> STOP
        Exiting UDP client!
            
        

        The output on the UDP server side will be as follows:

          
        -> STOP
        Exiting UDP server!
            
        

      Create a Concurrent TCP Server

      This section demonstrates the implementation of a concurrent TCP server. The benefit of a concurrent TCP server is that it can serve multiple clients. In Go, this is accomplished by creating a separate Goroutine to serve each TCP client.

      The example TCP server keeps a running count of the number of TCP clients it has served so far. The counter increases by one each time a new TCP client connects to the TCP server. The current value of that counter is returned to each TCP client.

      1. In your current working directory, create a file named concTCP.go with the following content:

        ./concTCP.go
         1
         2
         3
         4
         5
         6
         7
         8
         9
        10
        11
        12
        13
        14
        15
        16
        17
        18
        19
        20
        21
        22
        23
        24
        25
        26
        27
        28
        29
        30
        31
        32
        33
        34
        35
        36
        37
        38
        39
        40
        41
        42
        43
        44
        45
        46
        47
        48
        49
        50
        51
        52
        53
        54
        55
        56
        57
        58
        59
        
        package main
        
        import (
                "bufio"
                "fmt"
                "net"
                "os"
                "strconv"
                "strings"
        )
        
        var count = 0
        
        func handleConnection(c net.Conn) {
                fmt.Print(".")
                for {
                        netData, err := bufio.NewReader(c).ReadString('n')
                        if err != nil {
                                fmt.Println(err)
                                return
                        }
        
                        temp := strings.TrimSpace(string(netData))
                        if temp == "STOP" {
                                break
                        }
                        fmt.Println(temp)
                        counter := strconv.Itoa(count) + "n"
                        c.Write([]byte(string(counter)))
                }
                c.Close()
        }
        
        func main() {
                arguments := os.Args
                if len(arguments) == 1 {
                        fmt.Println("Please provide a port number!")
                        return
                }
        
                PORT := ":" + arguments[1]
                l, err := net.Listen("tcp4", PORT)
                if err != nil {
                        fmt.Println(err)
                        return
                }
                defer l.Close()
        
                for {
                        c, err := l.Accept()
                        if err != nil {
                                fmt.Println(err)
                                return
                        }
                        go handleConnection(c)
                        count++
                }
        }
              
        • This file creates the main package, which declares the handleConnection() and main() functions.
        • The main() function will use the imported packages to create a concurrent TCP server. It gathers command line arguments in the arguments variable and includes error handling.
        • Each TCP client is served by a separate Goroutine that executes the handleConnection() function. This means that while a TCP client is served, the TCP server is free to interact with more TCP clients. TCP clients are connected using the Accept() function.
        • Although the Accept() function can be executed multiple times, the net.Listen() function needs to be executed only once. For this reason the net.Listen() function remains outside of the for loop.
        • The for loop in the main() function is endless because TCP/IP servers usually run nonstop. However, if the handleConnection() function receives the STOP message, the Goroutine that runs it will exit and the related TCP connection will close.

      Test the Concurrent TCP Server

      In this section, you will test the concurrent TCP server using the netcat command line utility.

      1. Run your concurrent TCP server. From the directory containing the concTCP.go file, run the following command:

        go run concTCP.go 1234
        

        The command creates a TCP server that listens on port number 1234. You can use any port number, however, ensure it is not already in use and that you have the required privileges. Reference the list of well-known TCP and UDP ports, if needed.

      2. Use netcat to establish a connection with the TCP server. By default, netcat will establish a TCP connection with a remote host on the specified port number.

        nc 127.0.0.1 1234
        
      3. After issuing the previous command, you will not see any change in your output. Type Hello! to send a packet to the TCP server:

        Hello!
        

        The TCP server will return the number of current client connections as its response. Since this is your first connection established with the TCP server, you should expect an output of 1.

          
        Hello!
        1
            
        

        If you’d like, you can open a new shell session and use netcat to establish a second connection with the TCP server by repeating Step 2. When you send the server a second Hello!, you should receive a response of 2 this time.

      4. You can also connect to the TCP server using the TCP client you created in the Create the TCP Client section of the guide. Ensure you are in the directory containing the tcpC.go file and issue the following command:

        go run tcpC.go 127.0.0.1:1234
        
      5. You will see a >> prompt waiting for you to enter some text. Type in Hello! to receive a response from the TCP server:

        Hello!
        

        You should see a similar output indicating 3 client connections:

          
        >> Hello!
        ->: 3
            
        
      6. Send the STOP command to exit the TCP client:

        You should see a similar output on the client:

          
        >> STOP
        ->: TCP client exiting...
              
        

        The output on the TCP server side will be as follows:

          
        .Hello!
        .Hello!
        .Hello!
              
        

        Note

        From the shell session running the TCP server, type CTRL-c to interrupt program execution and then, CTRL-D to close all client connections and to stop the TCP server.

      More Information

      You may wish to consult the following resources for additional information on this topic. While these are provided in the hope that they will be useful, please note that we cannot vouch for the accuracy or timeliness of externally hosted materials.

      Find answers, ask questions, and help others.

      This guide is published under a CC BY-ND 4.0 license.



      Source link