One place for hosting & domains

      Static

      How to Deploy a Static Site on Linode Kubernetes Engine


      Updated by Linode Contributed by Linode

      Note

      Linode Kubernetes Engine (LKE) is currently in Private Beta, and you may not have access to LKE through the Cloud Manager or other tools. To request access to the Private Beta, sign up here. Beta access awards you $100/month in free credits for the duration of the beta, which is automatically applied to your account when an LKE cluster is in use. Additionally, you will have access to the Linode Green Light community, a new program connecting beta users with our product and engineering teams.

      Because LKE is in Beta, there may be breaking changes to how you access and manage LKE. This guide will be updated to reflect these changes if and when they occur.

      Linode Kubernetes Engine (LKE) allows you to easily create, scale, and manage Kubernetes clusters to meet your application’s demands, reducing the often complicated cluster set-up process to just a few clicks. Linode manages your Kubernetes master node, and you select how many Linodes you want to add as worker nodes to your cluster.

      Deploying a static site using an LKE cluster is a great example to follow when learning Kubernetes. A container image for a static site can be written in less than ten lines, and only one container image is needed, so it’s less complicated to deploy a static site on Kubernetes than some other applications that require multiple components.

      Caution

      Following the instructions in this guide will create billable resources on your account in the form of Linodes and NodeBalancers. You will be billed an hourly rate for the time that these resources exist on your account. Be sure to follow the tear-down section at the end of this guide if you do not wish to continue using these resources.

      In this Guide

      This guide will show you how to:

      Before You Begin

      • You should have a working knowledge of Kubernetes’ key concepts, including master and worker nodes, Pods, Deployments, and Services. For more information on Kubernetes, see our Beginner’s Guide to Kubernetes series.

      • You will also need to prepare your workstation with some prerequisite software:

      • Finally, you will need to create a cluster on LKE, if you do not already have one:

      Install kubectl

      You should have kubectl installed on your local workstation. kubectl is the command line interface for Kubernetes, and allows you to remotely connect to your Kubernetes cluster to perform tasks.

      macOS:

      Install via Homebrew:

      brew install kubernetes-cli
      

      If you don’t have Homebrew installed, visit the Homebrew home page for instructions. Alternatively, you can manually install the binary; visit the Kubernetes documentation for instructions.

      Linux:

      1. Download the latest kubectl release:

        curl -LO https://storage.googleapis.com/kubernetes-release/release/$(curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt)/bin/linux/amd64/kubectl
        
      2. Make the downloaded file executable:

        chmod +x ./kubectl
        
      3. Move the command into your PATH:

        sudo mv ./kubectl /usr/local/bin/kubectl
        

      Note

      Windows:

      Visit the Kubernetes documentation for a link to the most recent Windows release.

      Install Git

      To perform some of the commands in this guide you will need to have Git installed on your workstation. Git is a version control system that allows you to save your codebase in various states to ease development and deployment. Follow our How to Install Git on Linux, Mac or Windows guide for instructions on how to install Git.

      Install Docker

      These steps install Docker Community Edition (CE) using the official Ubuntu repositories. To install on another distribution, or to install on Mac or Windows, see the official installation page.

      1. Remove any older installations of Docker that may be on your system:

        sudo apt remove docker docker-engine docker.io
        
      2. Make sure you have the necessary packages to allow the use of Docker’s repository:

        sudo apt install apt-transport-https ca-certificates curl software-properties-common
        
      3. Add Docker’s GPG key:

        curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
        
      4. Verify the fingerprint of the GPG key:

        sudo apt-key fingerprint 0EBFCD88
        

        You should see output similar to the following:

          
        pub   4096R/0EBFCD88 2017-02-22
                Key fingerprint = 9DC8 5822 9FC7 DD38 854A  E2D8 8D81 803C 0EBF CD88
        uid                  Docker Release (CE deb) 
        sub   4096R/F273FCD8 2017-02-22
        
        
      5. Add the stable Docker repository:

        sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"
        

        Note

        For Ubuntu 19.04, if you get an E: Package 'docker-ce' has no installation candidate error, this is because the stable version of docker is not yet available. Therefore, you will need to use the edge / test repository.

        sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable edge test"
        
      6. Update your package index and install Docker CE:

        sudo apt update
        sudo apt install docker-ce
        
      7. Add your limited Linux user account to the docker group:

        sudo usermod -aG docker $USER
        

        Note

        After entering the usermod command, you will need to close your SSH session and open a new one for this change to take effect.

      8. Check that the installation was successful by running the built-in “Hello World” program:

        docker run hello-world
        

      Sign up for a Docker Hub Account

      You will use Docker Hub to store your Docker image. If you don’t already have a Docker Hub account, create one now.

      Install Hugo

      A static site generator (SSG) is usually a command line tool that takes text files written in a markup language like Markdown, applies a stylized template to the content, and produces valid HTML, CSS, and JavaScript files. Static sites are prized for their simplicity and speed, as they do not generally have to interact with a database.

      The Linode documentation website, and this guide, employ Hugo. Hugo is a powerful and fast SSG written in the Go programming language, but you can choose one that best suits your needs by reading our How to Choose a Static Site Generator guide.

      The steps in this guide are generally the same across SSGs: install a static site generator, create some content in a text file, and then generate your site’s HTML through a build process.

      To download and install Hugo, you can use a package manager.

      • For Debian and Ubuntu:

        sudo apt-get install hugo
        
      • For Red Hat, Fedora, and CentOS:

        sudo dnf install hugo
        
      • For macOS, use Homebrew:

        brew install hugo
        
      • For Windows, use Chocolatey:

        choco install hugo
        

      For more information on downloading Hugo, you can visit the official Hugo website.

      Create a Static Site Using Hugo

      In this section you will create a static site on your workstation using Hugo.

      1. Use Hugo to scaffold a new site. This command will create a new directory with the name you provide, and inside that directory it will create the default Hugo directory structure and configuration files:

        hugo new site lke-example
        
      2. Move into the new directory:

        cd lke-example
        
      3. Initialize the directory as a Git repository. This will allow you to track changes to your website and save it in version control.

        git init
        
      4. Hugo allows for custom themes. For the sake of this example, you will install the Ananke theme as a Git submodule.

        git submodule add https://github.com/budparr/gohugo-theme-ananke.git themes/ananke
        

        Note

        Git submodules allow you to include one Git repository within another, each maintaining their own version history. To view a collection of Hugo themes, visit the Hugo theme collection.
      5. In the text editor of your choice, open the config.toml file and add the following line to the end:

        theme = "ananke"
        

        This line instructs Hugo to search for a folder named ananke in the themes directory and applies the templating it finds to the static site.

      6. Add an example first post to your Hugo site:

        hugo new posts/first_post.md
        

        This will create a Markdown file in the content/posts/ directory with the name first_post.md. You will see output like the following:

          
        /Users/linode/k8s/lke/lke-example/content/posts/first_post.md created
        
        
      7. Open the first_post.md file in the text editor of your choosing. You will see a few lines of front matter, a format Hugo uses for extensible metadata, at the top of the file:

        lke-example/content/posts/first_post.md
        1
        2
        3
        4
        5
        
        ---
        title: "First_post"
        date: 2019-07-29T14:22:04-04:00
        draft: false
        ---

        Change the title to your desired value, and change draft to false. Then, add some example Markdown text to the bottom of the file, like the example below:

        lke-example/content/posts/first_post.md
         1
         2
         3
         4
         5
         6
         7
         8
         9
        10
        11
        12
        13
        
        ---
        title: "First Post About LKE Clusters"
        date: 2019-07-29T14:22:04-04:00
        draft: false
        ---
        
        ## LKE Clusters
        
        Linode Kubernetes Engine (LKE) clusters are:
        
        - Fast
        - Affordable
        - Scalable
      8. You can preview your changes by starting the local Hugo server:

        hugo server
        

        You should see output like the following:

          
                           | EN
        +------------------+----+
          Pages              |  8
          Paginator pages    |  0
          Non-page files     |  0
          Static files       |  3
          Processed images   |  0
          Aliases            |  0
          Sitemaps           |  1
          Cleaned            |  0
        
        Total in 6 ms
        Watching for changes in /Users/linode/k8s/lke/lke-example/{content,data,layouts,static,themes}
        Watching for config changes in /Users/linode/k8s/lke/lke-example/config.toml
        Serving pages from memory
        Running in Fast Render Mode. For full rebuilds on change: hugo server --disableFastRender
        Web Server is available at http://localhost:1313/ (bind address 127.0.0.1)
        Press Ctrl+C to stop
        
        
      9. Visit the URL that Hugo is running on. In the above example, the URL is http://localhost:1313. This server automatically updates whenever you make a change to a file in the Hugo site directory. To stop this server, enter CTRL-C on your keyboard in your terminal.

      10. When you are satisfied with your static site, you can generate the HTML, CSS, and JavaScript for your site by building the site:

        hugo -v
        

        Hugo creates the site’s files in the public/ directory. View the files by listing them:

        ls public
        
      11. You can build the site at any time from your source Markdown content files, so it’s common practice to keep built files out of a Git repository. This practice keeps the size of the repository to a minimum.

        You can instruct Git to ignore certain files within a repository by adding them to a .gitignore file. Add the public/ directory to your .gitignore file to exclude these files from the repository:

        echo 'public/' >> .gitignore
        
      12. Add and commit the source files to the Git repository:

        git add .
        git commit -m "Initial commit. Includes all of the source files, configuration, and first post."
        

        You are now ready to create a Docker image from the static site you’ve just created.

      Create a Docker Image

      In this section you will create a Docker container for your static site, which you will then run on your LKE cluster. Before deploying it on your cluster, you’ll test its functionality on your workstation.

      1. In your Hugo static site folder, create a new text file named Dockerfile and open it in the text editor of your choosing. A Dockerfile tells Docker how to create the container.

      2. Add the following contents to the Dockerfile. Each command has accompanying comments that describe their function:

        lke-example/Dockerfile
         1
         2
         3
         4
         5
         6
         7
         8
         9
        10
        11
        12
        13
        14
        15
        16
        17
        18
        19
        20
        21
        22
        23
        
        # Install the latest Debain operating system.
        FROM debian:latest as HUGO
        
        # Install Hugo.
        RUN apt-get update -y
        RUN apt-get install hugo -y
        
        # Copy the contents of the current working directory to the
        # static-site directory.
        COPY . /static-site
        
        # Command Hugo to build the static site from the source files,
        # setting the destination to the public directory.
        RUN hugo -v --source=/static-site --destination=/static-site/public
        
        # Install NGINX, remove the default NGINX index.html file, and
        # copy the built static site files to the NGINX html directory.
        FROM nginx:stable-alpine
        RUN mv /usr/share/nginx/html/index.html /usr/share/nginx/html/old-index.html
        COPY --from=HUGO /static-site/public/ /usr/share/nginx/html/
        
        # Instruct the container to listen for requests on port 80 (HTTP).
        EXPOSE 80

        Save the Dockerfile and return to the command prompt.

      3. Create a new text file named .dockerignore in your Hugo static site folder and add the following lines:

        lke-example/.dockerignore
        1
        2
        3
        4
        
        public/
        .git/
        .gitmodules/
        .gitignore

        Note

        This file, similar to the .gitignore file you created in the previous section, allows you to ignore certain files within the working directory that you would like to leave out of the container. Because you want the container to be the smallest size possible, the .dockerignore file will include the public/ folder and some hidden folders that Git creates.

      4. Run the Docker build command. Replace mydockerhubusername with your Docker Hub username. The period at the end of the command tells Docker to use the current directory as its build context.

        docker build -t mydockerhubusername/lke-example:v1 .
        

        Note

        In the example below, the container image is named lke-example and has been given a version tag of v1. Feel free to change these values.

      5. Docker will download the required Debian and NGINX images, as well as install Hugo into the image. Once complete, you should see output similar to the following:

          
        Successfully built 320ae416c940
        Successfully tagged mydockerhubusername/lke-example:v1
        
        
      6. You can view the image by listing all local images:

        docker images
        
          
        REPOSITORY                       TAG   IMAGE ID       CREATED             SIZE
        mydockerhubusername/lke-example  v1    320ae416c940   About an hour ago   20.8MB
        
        

      Test the Docker Image

      1. You can test your new image by creating a container with it locally. To do so, enter the following run command:

        docker run -p 8080:80 -d mydockerhubusername/lke-example:v1
        

        The -p flag instructs Docker to forward port 8080 on localhost to port 80 on the container. The -d flag instructs Docker to run in detached mode so that you are returned to the command prompt once the container initializes.

      2. Once the container has started, open your browser and navigate to localhost:8080. You should see your static site.

      3. You can stop the running container by finding the ID of the container and issuing the stop command. To find the ID of the container, use the ps command:

        docker ps
        

        You should see a list of actively running containers, similar to the following:

          
        b4a7b959a6c7        mydockerhubusername/lke-example:v1         "nginx -g 'daemon of…"   5 hours ago         Up 5 hours          0.0.0.0:8080->80/tcp        romantic_mahavira
        
        
      4. Note the random string of numbers and letters next to the image name. In the above example, the string is b4a7b959a6c7. Issue the stop command, supplying the string of numbers and letters:

        docker stop b4a7b959a6c7
        

      Upload the Image to Docker Hub

      1. Now that you have a working container image, you can push that image to Docker Hub. First, log in to Docker Hub from your workstation’s terminal:

        docker login
        
      2. Next, push the image, with version tag, to Docker Hub, using the push command:

        docker push mydockerhubusername/lke-example:v1
        

        You can now view your image on Docker Hub as a repository. To view all of your repositories, navigate to the Docker Hub repository listing page.

      3. Lastly, add the Dockerfile and .dockerignore file to your Git repository:

        git add .
        git commit -m "Add Dockerfile and .dockerignore."
        

        You are now ready to deploy the container to your LKE cluster.

      Deploying the Container to LKE

      In this section, you will create a Deployment from the container you created in the previous section, and a Service to load balance the deployment.

      1. Begin by navigating to a location outside of your static site directory. You will not need your static site directory for the remainder of this guide.

        cd ..
        
      2. Create a new directory to house your Kubernetes manifests, and move into that directory:

        mkdir manifests && cd manifests
        

      Create a Deployment

      1. In the text editor of your choice, create a new YAML manifest file for your Deployment. Name the file static-site-deployment.yaml, save it to your manifests directory, and enter the contents of this snippet:

        manifests/static-site-deployment.yaml
         1
         2
         3
         4
         5
         6
         7
         8
         9
        10
        11
        12
        13
        14
        15
        16
        17
        18
        19
        20
        21
        22
        
        apiVersion: apps/v1
        kind: Deployment
        metadata:
          name: static-site-deployment
          labels:
            app: static-site
        spec:
          replicas: 3
          selector:
            matchLabels:
              app: static-site
          template:
            metadata:
              labels:
                app: static-site
            spec:
              containers:
              - name: static-site
                image: mydockerhubusername/lke-example:v1
                imagePullPolicy: Always
                ports:
                - containerPort: 80
        • In this example the number of replica Pods is set to 3 on line 8. This value can be changed to meet the needs of your website.
        • The spec.containers.image field on line 19 should be changed to match the name of the container image you pushed to Docker Hub. Be sure to include the proper version tag at the end of the container name.
        • imagePullPolicy: Always on line 20 ensures that each time a Pod is created, the most recent version of the container image will be pulled from Docker Hub.
      2. Once you have a Deployment manifest, you can apply the deployment to the LKE cluster with kubectl:

        kubectl apply -f static-site-deployment.yaml
        
      3. You can check on the progress of your Deployment by listing the available pods:

        kubectl get pods
        

        If your Deployment was successful, you should see output like the following:

          
        NAME                                    READY   STATUS   RESTARTS   AGE
        static-site-deployment-cdb88b5bb-7pbjc  1/1     Running  0          1h
        static-site-deployment-cdb88b5bb-gx9h5  1/1     Running  0          1h
        static-site-deployment-cdb88b5bb-lzdvh  1/1     Running  0          1h
        
        

      Create a Service

      1. Create a Service manifest file to provide load balancing for the deployment. Load balancing ensures that traffic is balanced efficiently across multiple backend nodes, improving site performance and ensuring that your static site will be accessible should a node go down.

        Specifically, the Service manifest that will be used in this guide will trigger the creation of a Linode NodeBalancer.

        Note

      2. Name the file static-site-service.yaml, save it to your manifests directory, and enter the contents of this snippet:

        manifests/static-site-service.yaml
         1
         2
         3
         4
         5
         6
         7
         8
         9
        10
        11
        12
        13
        14
        15
        16
        17
        18
        
        apiVersion: v1
        kind: Service
        metadata:
          name: static-site-service
          annotations:
            service.beta.kubernetes.io/linode-loadbalancer-throttle: "4"
          labels:
            app: static-site
        spec:
          type: LoadBalancer
          ports:
          - name: http
            port: 80
            protocol: TCP
            targetPort: 80
          selector:
            app: static-site
          sessionAffinity: None
      3. Once you’ve created your Service manifest file, you can apply it to the LKE cluster:

        kubectl apply -f static-site-service.yaml
        
      4. You can check on the status of your Service by listing the Services currently running on your server:

        kubectl get services
        

        You should see output similar to the following:

          
        NAME                 TYPE          CLUSTER-IP     EXTERNAL-IP      PORT(S)        AGE
        kubernetes           ClusterIP     10.128.0.1     <none>           443/TCP        20h
        static-site-service  LoadBalancer  10.128.99.240  192.0.2.1        80:32648/TCP   100m
        
        
      5. Note the external IP address of the Service you created. This is the IP address of the NodeBalancer, and you can use it to view your static site.

      6. In the above example, the IP address is 192.0.2.1. Navigate to the external IP address in the browser of your choice to view your static site. You should see the same content as when you tested your Docker image on your workstation.

      Next Steps

      If you’d like to continue using the static site that you created in this guide, you may want to assign a domain to it. Review the DNS Records: An Introduction and DNS Manager guides for help with setting up DNS. When setting up your DNS record, use the external IP address that you noted at the end of the previous section.

      If you would rather not continue using the cluster you just created, review the tear-down section to remove the billable Linode resources that were generated.

      Tear Down your LKE Cluster and NodeBalancer

      • To remove the NodeBalancer you created, all you need to do is delete the underlying Service. From your workstation:

        kubectl delete service static-site-service
        

        Alternatively, you can use the manifest file you created to delete the Service. From your workstation:

        kubectl delete -f static-site-service.yaml
        
      • To remove the LKE Cluster and the associated nodes from your account, navigate to the Linode Cloud Manager:

        1. Click on the Kubernetes link in the sidebar. A new page with a table which lists your clusters will appear.

        2. Click on the more options elipsis next to the cluster you would like to delete, and select Delete.

        3. You will be prompted to enter the name of the cluster to confirm the action. Enter the cluster name and click Delete.

      • Lastly, remove the KUBECONFIG line you added to your Bash profile to remove the LKE cluster from your available contexts.

      More Information

      You may wish to consult the following resources for additional information on this topic. While these are provided in the hope that they will be useful, please note that we cannot vouch for the accuracy or timeliness of externally hosted materials.

      Find answers, ask questions, and help others.

      This guide is published under a CC BY-ND 4.0 license.



      Source link

      Host a Static Site using Linode Object Storage


      Updated by Linode

      Contributed by

      Linode

      Note

      Object Storage is currently in a closed early access Beta, and you may not have access to Object Storage through the Cloud Manager or other tools. To gain access to the Early Access Program (EAP), open up a Customer Support ticket noting that you’d like to be included in the program, or e-mail objbeta@linode.com – beta access is completely free.

      Additionally, because Object Storage is in Beta, there may be breaking changes to how you access and manage Object Storage. This guide will be updated to reflect these changes if and when they occur.

      Why Host a Static Site on Object Storage?

      Static site generators are a popular solution for creating simple, fast, flexible, and attractive websites that are easy to update. You can contribute new pages and content to a static site in two steps:

      1. First, write the content for your site’s new page using Markdown, an easy-to-learn and light-weight markup language.

      2. Then, tell your static site generator to compile your Markdown (along with other relevant assets, like CSS styling, images, and JavaScript) into static HTML files.

      The second compilation step only needs to happen once for each time that you update your content. This is in contrast with a dynamic website framework like WordPress or Drupal, which will reference a relational database and compile your HTML every time a visitor loads your site.

      Benefits of Hosting on Object Storage

      Traditionally, these static HTML files would be served by a web server (like NGINX or Apache) running on a Linode. Using Object Storage to host your static site files means you do not have to worry about maintaining your site’s infrastructure. It is no longer necessary to perform typical server maintenance tasks, like software upgrades, web server configuration, and security upkeep.

      Object Storage provides an HTTP REST gateway to objects, which means a unique URL over HTTP is available for every object. Once your static site is built, making it available publicly over the Internet is as easy uploading files to an Object Storage bucket.

      Object Storage Hosting Workflow

      At a high-level, the required steps to host a static site using Object Storage are:

      1. Install the static site generator of your choice to your local computer.

      2. Create the desired content and build the site (using your static site generator).

      3. Upload the static files to your Object Storage bucket to make the content publicly available over the Internet.

      This guide will use Hugo to demonstrate how to create a static site and host it on Linode Object Storage. However, there are many other static site generators to choose from–Jekyll and Gatsby are popular choices, and the general steps outlined in this guide could be adapted to them. For more information on choosing a static site generator, see the How to Choose a Static Site Generator guide.

      Before You Begin

      1. Read the How to Use Linode Object Storage guide to familiarize yourself with Object Storage on Linode. Specifically, be sure that you have:

        • Created your Object Storage access and secret keys.
        • Installed and configure the s3cmd tool.
      2. Install and configure Git on your local computer.

      Install the Hugo Static Site Generator

      Hugo is written in Go and is known for being extremely fast to compile sites, even very large ones. It is well-supported, well-documented, and has an active community. Some useful Hugo features include shortcodes, which are an easy way to include predefined templates inside of your Markdown, and built-in LiveReload web server, which allows you to preview your site changes locally as you make them.

      1. Install Hugo on your computer:

        macOS:

        Linux/Ubuntu:

        • Determine your Linux kernel’s architecture:

          uname -r
          

          Your output will resemble the following:

            
          4.9.0-8-amd64
          
          
        • Navigate to Hugo’s GitHub releases page and download the appropriate version for your platform. This example command downloads version 0.55, but a newer release may be available:

          wget https://github.com/gohugoio/hugo/releases/download/v0.55.0/hugo_0.55.0_Linux-64bit.deb
          
        • Install the package using dpkg:

          sudo dpkg -i hugo*.deb
          
      2. Verify that Hugo is installed. You should see output indicating your installed Hugo’s version number:

        hugo version
        

      Create a Hugo Site

      In this section, you will use the Hugo CLI (command line interface) to create your Hugo site, initialize a Hugo theme, and add content to your site. Hugo’s CLI provides several useful commands for common tasks needed to build, configure, and interact with your Hugo site.

      1. Create a new Hugo site on your local computer. This command will create a folder named example-site and scaffold Hugo’s directory structure inside it:

        hugo new site example-site
        
      2. Move into your Hugo site’s root directory:

        cd example-site
        

        Note

        All commands in this section of the guide should be issued from your site’s root directory.

      3. You will use Git to add a theme to your Hugo site’s directory. Initialize your Hugo site’s directory as a Git repository:

        git init
        
      4. Install the Ananke theme as a submodule of your Hugo site’s Git repository. Git submodules allow one Git repository to be stored as a subdirectory of another Git repository, while still being able to maintain each repository’s version control information separately. The Ananke theme’s repository will be located in the ~/example-site/themes/ananke directory of your Hugo site.

        git submodule add https://github.com/budparr/gohugo-theme-ananke.git themes/ananke
        

        Note

        Hugo has many available themes that can be installed as a submodule of your Hugo site’s directory.
      5. Add the theme to your Hugo site’s configuration file. The configuration file (config.toml) is located at the root of your Hugo site’s directory.

        echo 'theme = "ananke"' >> config.toml
        
      6. Create a new content file for your site. This command will generate a Markdown file with an auto-populated date and title:

        hugo new posts/my-first-post.md
        
      7. You should see a similar output. Note that the file is located in the content/posts/ directory of your Hugo site:

          
        /home/username/example-site/content/posts/my-first-post.md created
        
        
      8. Open the Markdown file in the text editor of your choice to begin modifying its content; you can copy and paste the example snippet into your file, which contains an updated front matter section at the top and some example Markdown body text.

        Set your desired value for title. Then, set the draft state to false and add your content below the --- in Markdown syntax, if desired:

        /home/username/example-site/content/posts/my-first-post.md
         1
         2
         3
         4
         5
         6
         7
         8
         9
        10
        11
        12
        13
        14
        
        ---
        title: "My First Post"
        date: 2019-04-11T11:25:11-04:00
        draft: false
        ---
        
        # Host a Static Site on Linode Object Storage
        
        There are many benefits to using a static site generator. Here is a list of a few of them:
        
        - Run your own website without having to manage a Linode.
        - You don't need to worry about running a web server like Apache or NGINX.
        - Static website performance is typically very fast.
        - Use Git to version control your static website's content.


        About front matter

        Front matter is a collection of metadata about your content, and it is embedded at the top of your file within opening and closing --- delimiters.

        Front matter is a powerful Hugo feature that provides a mechanism for passing data that is attached to a specific piece of content to Hugo’s rendering engine. Hugo accepts front matter in TOML, YAML, and JSON formats. In the example snippet, there is YAML front matter for the title, date, and draft state of the Markdown file. These variables will be referenced and displayed by your Hugo theme.

      9. Once you have added your content, you can preview your changes by building and serving the site using Hugo’s built-in webserver:

        hugo server
        
      10. You will see a similar output:

          
        &nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp| EN
        +------------------+----+
          Pages&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp| 11
          Paginator pages&nbsp&nbsp&nbsp&nbsp|  0
          Non-page files&nbsp&nbsp&nbsp&nbsp&nbsp|  0
          Static files&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp|  3
          Processed images&nbsp&nbsp&nbsp|  0
          Aliases&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp|  1
          Sitemaps&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp|  1
          Cleaned&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp|  0
        
        Total in 7 ms
        Watching for changes in /home/username/example-site/{content,data,layouts,static,themes}
        Watching for config changes in /home/username/example-site/config.toml
        Serving pages from memory
        Running in Fast Render Mode. For full rebuilds on change: hugo server --disableFastRender
        Web Server is available at http://localhost:1313/ (bind address 127.0.0.1)
        Press Ctrl+C to stop
        
        
      11. The output will provide a URL to preview your site. Copy and paste the URL into a browser to access the site. In the above example Hugo’s web server URL is http://localhost:1313/.

      12. When you are happy with your site’s content you can build your site:

        hugo -v
        

        Hugo will generate your site’s static HTML files and store them in a public directory that it will create inside your project. The static files that are generated by Hugo are the files that you will upload to your Object Storage bucket to make your site accessible via the Internet.

      13. View the contents of your site’s public directory:

        ls public
        

        Your output should resemble the following example. When you built the site, the Markdown file you created and edited in steps 6 and 7 was used to generate its corresponding static HTML file in the public/posts/my-first-post/index.html directory.

          
          404.html    categories  dist        images      index.html  index.xml   posts       sitemap.xml tags
            
        


        Track your Static Site Files with Git

        It’s not necessary to version control your site files in order to host them on Object Storage, but we still recommended that you do so:

        1. Display the state of your current working directory (root of your Hugo site):

          git status
          
        2. Stage all your files to be committed:

          git add -A
          
        3. Commit all your changes and add a meaningful commit message:

          git commit -m 'Add my first post.'
          

        Once you have used Git to track your local Hugo site files, you can easily push them to a remote Git repository, like GitHub or GitLab. Storing your static site files on a remote Git repository opens up many possibilities for collaboration and automating your static site’s deployment to Linode Object Storage. To learn more about Git, see the Getting Started with Git guide.

      Upload your Static Site to Linode Object Storage

      Before proceeding with this section ensure that you have already created your Object Storage access and secret keys and have installed the s3cmd tool.

      1. Create a new Object Storage bucket; prepend s3:// to the beginning of the bucket’s name:

        s3cmd mb s3://my-bucket
        

        Note

        Buckets names must be unique within the Object Storage cluster. You might find the bucket name my-bucket is already in use by another Linode customer, in which case you will need to choose a new bucket name.

      2. Initialize your Object Storage bucket as a website. You must tell your bucket which files to serve as the index page and the error page for your static site. This is done with the --ws-index and --ws-error options:

        s3cmd ws-create --ws-index=index.html --ws-error=404.html s3://my-bucket
        

        In our Hugo example, the site’s index file is index.html and the error file is 404.html. Whenever a user visits your static site’s URL, the Object Storage service will serve the index.html page. If a site visitor tries to access an invalid path, they will be presented with the 404.html page.

      3. The command will return the following message:

          
            Bucket 's3://my-bucket/': website configuration created.
              
        
      4. Display information about your Object Storage’s website configuration to obtain your site’s URL:

        s3cmd ws-info s3://my-bucket
        
      5. You should see a similar output. Be sure to take note of your Object Storage bucket’s URL:

          
              Bucket s3://my-bucket/: Website configuration
        Website endpoint: http://website-us-east-1.linodeobjects.com/
        Index document:   index.html
        Error document:   404.html
            
        

        Note

        The Linode Object Storage early access Beta provides SSL enabled by default. This means you can access your Object Storage bucket using https, as well.

      6. Use s3cmd’s sync command to upload the contents of your static site’s public directory to your Object Storage bucket. This step will make your site available publicly on the Internet. Ensure you are in your site’s root directory on your computer (e.g. /home/username/example-site):

        s3cmd --no-mime-magic --acl-public --delete-removed --delete-after sync public/ s3://my-bucket
        
        Option                          Description
        no-mime-magic Tells Object Storage not to use file signatures when guessing the object’s MIME-type.
        acl-public Sets the access level control of the objects to public.
        delete-removed Deletes any destination objects with no corresponding source file.
        delete-after Deletes destination files that are no longer found at the source after all files are uploaded to the bucket.
      7. Use a browser to navigate to your Object Storage bucket’s URL to view your Hugo site:

        Hugo Index Page

        Note

        It may take a minute or two after your s3cmd sync completes for the page to appear at your bucket’s website URL.

      8. If needed, you can continue to update your static site locally and upload any changes using s3cmd’s sync command from step 3 of this section.

      (Optional) Next Steps

      After uploading your static site to Linode Object Storage, you may want to use a custom domain for your site. To do this, you can add a CNAME entry to your domain’s DNS records that aliases it to your Object Storage bucket’s website URL. To learn about managing DNS records on Linode, see the DNS Manager and DNS Records: An Introduction guides.

      As noted before, it’s possible to trigger automatic deployments to the Object Storage service when you push new content updates to GitHub or GitLab. This is done by leveraging a CI/CD (continuous integration/continuous delivery) tool like Travis CI. Essentially, you would build your Hugo site within the Travis environment and then run the s3cmd sync command from it to your bucket.

      More Information

      You may wish to consult the following resources for additional information on this topic. While these are provided in the hope that they will be useful, please note that we cannot vouch for the accuracy or timeliness of externally hosted materials.

      Find answers, ask questions, and help others.

      This guide is published under a CC BY-ND 4.0 license.



      Source link

      Automate Static Site Deployments with Salt, Git, and Webhooks


      Updated by Linode Contributed by Nathan Melehan

      Use promo code DOCS10 for $10 credit on a new account.

      This guide will walk through the deployment of a static site using SaltStack, which is a flexible configuration management system. The configuration files created for Salt will be version controlled using Git. Updates to your static site’s code will be automatically communicated to the production system using webhooks, an event notification system for the web.

      Setting up these mechanisms offers an array of benefits:

      • Using webhooks will keep your production website in sync with your development without any actions needed on your part.

      • Using Salt provides an extensible, reliable way to alter your production systems and minimize human error.

      • Version controlling your configuration management helps you track or revert the changes you’ve made to your systems and collaborate with others on your deployments.

      Development and Deployment Workflow

      The static site generator used in this guide is Hugo, a fast framework written in Go. Static site generators compile markdown or other content files into HTML files. This guide can easily be adapted to other frameworks.

      Two Git repositories will be created: one will track changes to the Hugo site, and the other will track Salt’s configuration files. Remote repositories will be created for both on GitHub.

      Two Linodes will be created: one will act as the Salt master, and the other as the Salt minion. This guide was tested under Debian 9, but the instructions may work with other distributions as well. The Salt minion will run the production webserver which serves the Hugo site, and the master will configure the minion’s software. The minion will also run a webhook server which will receive code update notifications from GitHub.

      It is possible to run Salt in a masterless mode, but using a Salt master will make it easier to expand on your deployment in the future.

      Note

      The workflow described in this guide is similar to how Linode’s own Guides & Tutorials website is developed and deployed.

      Before You Begin

      Set Up the Development Environment

      Development of your Hugo site and your Salt formula will take place on your personal computer. Some software will need to be installed on your computer first:

      1. Install Git using one of the methods in Linode’s guide. If you have a Mac, use the Homebrew method, as it will also be used to install Hugo.

      2. Install Hugo. The Hugo documentation has a full list of installation methods, and instructions for some popular platforms are as follows:

        • Debian/Ubuntu:

          sudo apt-get install hugo
          
        • Fedora, Red Hat and CentOS:

          sudo dnf install hugo
          
        • Mac, using Homebrew:

          brew install hugo
          
        • Windows, using Chocolatey

          choco install hugo -confirm
          

      Deploy the Linodes

      1. Follow the Getting Started guide and deploy two Linodes running Debian 9.

      2. In the settings tab of your Linodes’ dashboards, label one of the Linodes as salt-master and the other as salt-minion. This is not required, but it will help keep track of which Linode serves which purpose.

      3. Complete the Securing Your Server guide on each Linode to create a limited Linux user account with sudo privileges, harden SSH access, and remove unnecessary network services.

        Note

        This guide is written for a non-root user. Commands that require elevated privileges are prefixed with sudo. If you’re not familiar with the sudo command, visit our Users and Groups guide.

        All configuration files should be edited with elevated privileges. Remember to include sudo before running your text editor.

      4. Configure DNS for your site by adding a domain zone and setting up reverse DNS on your Salt minion’s IP address.

      Set Up the Salt Master and Salt Minion

      Before you can start setting up the Salt formulas for the minion, you first need to install the Salt software on the master and minion and set up communication between them.

      1. Log into the Salt master Linode via SSH and run the Salt installation bootstrap script:

        wget -O bootstrap-salt.sh https://bootstrap.saltstack.com
        sudo sh bootstrap-salt.sh -M -N
        

        Note

        The -M option tells the script to install the Salt master software, and the -N option tells the script to not install the minion software.

      2. Log into the Salt minion Linode via SSH and set the hostname. This guide uses hugo-webserver as the example hostname:

        sudo hostnamectl set-hostname hugo-webserver
        

        Note

        This step needs to be completed before installing Salt on the minion, as Salt will use your hostname to generate the minion’s Salt ID.

      3. Edit the minion’s /etc/hosts file and append a new line for your hostname after the localhost line; replace 192.0.2.3 with your minion’s public IP address:

        /etc/hosts
        1
        2
        3
        
        127.0.0.1       localhost
        192.0.2.3       hugo-webserver
        # [...]
      4. Run the bootstrap script on the minion:

        wget -O bootstrap-salt.sh https://bootstrap.saltstack.com
        sudo sh bootstrap-salt.sh
        
      5. Edit /etc/salt/minion on the Salt minion. Uncomment the line that begins with #master: and enter your Salt master’s IP after the colon (in place of 192.0.2.2):

        /etc/salt/minion
        1
        2
        3
        
        # [...]
        master: 192.0.2.2
        # [...]

        Note

        Linode does not charge for traffic within a datacenter across private IP addresses. If your Salt master and minion are in the same datacenter, and both have a private IP addresses, you can use your Salt master’s private IP address in this step to avoid incurring data traffic charges.

      6. Restart Salt on the minion:

        sudo systemctl restart salt-minion
        

      Salt Minion Authentication

      The minion should now be able to find the master, but it has not yet been authenticated to communicate with the master. Salt uses public-private keypairs to authenticate minions to masters.

      1. On the master, list fingerprints for all the master’s local keys, accepted minion keys, and unaccepted keys:

        sudo salt-key --finger-all
        

        The output should resemble:

          
        Local Keys:
        master.pem:  fe:1f:e8:3d:26:83:1c:...
        master.pub:  2b:93:72:b3:3a:ae:cb:...
        Unaccepted Keys:
        hugo-webserver:  29:d8:f3:ed:91:9b:51:...
        
        

        Note

        The example fingerprints in this section have been truncated for brevity.

      2. Copy the fingerprint for master.pub from the output of salt-key --finger-all. On your Salt minion, open /etc/salt/minion in a text editor. Uncomment the line that begins with #master_finger: and enter the value for your master.pub after the colon in single-quotes:

        /etc/salt/minion
        1
        2
        3
        
        # [...]
        master_finger: '0f:d6:5f:5e:f3:4f:d3:...'
        # [...]
      3. Restart Salt on the minion:

        sudo systemctl restart salt-minion
        
      4. View the minion’s local key fingerprint:

        sudo salt-call key.finger --local
        
          
        local:
            29:d8:f3:ed:91:9b:51:...
        
        

        Compare the output’s listed fingerprint to the fingerprints listed by the Salt master for any Unaccepted Keys. This is the output of salt-key --finger-all run on the master in the beginning of this section.

      5. After verifying, that the minion’s fingerprint is the same as the fingerprint detected by the Salt master, run the following command on the master to accept the minion’s key:

        sudo salt-key -a hugo-webserver
        
      6. From the master, verify that the minion is running:

        sudo salt-run manage.up
        

        You can also run a Salt test ping from the master to the minion:

        sudo salt 'hugo-webserver' test.ping
        
          
        hugo-webserver:
            True
        
        

      Initialize the Salt Minion’s Formula

      The Salt minion is ready to be configured by the master. These configurations will be written in a Salt formula which will be hosted on GitHub.

      1. On your computer, create a new directory to hold your minion’s formula and change to that directory:

        mkdir hugo-webserver-salt-formula
        cd hugo-webserver-salt-formula
        
      2. Inside the formula directory, create a new hugo directory to hold your webserver’s configuration:

        mkdir hugo
        
      3. Inside the hugo directory, create a new install.sls file:

        hugo-webserver-salt-formula/hugo/install.sls
        1
        2
        3
        
        nginx_pkg:
          pkg.installed:
            - name: nginx

        Note

        Salt configurations are declared in YAML– a markup language that incorporates whitespace/indentation in its syntax. Be sure to use the same indentation as the snippets presented in this guide.

        A .sls file is a SaLt State file. Salt states describe the state a minion should be in after the state is applied to it: e.g., all the software that should be installed, all the services that should be run, and so on.

        The above snippet says that a package with name nginx (i.e. the NGINX web server) should be installed via the distribution’s package manager. Salt knows how to negotiate software installation via the built-in package manager for various distributions. Salt also knows how to install software via NPM and other package managers.

        The string nginx_pkg is the ID for the state component, pkg is the name of the Salt module used, and pkg.installed is referred to as a function declaration. The component ID is arbitrary, so you can name it however you prefer.

        Note

        If you were to name the ID to be the same as the relevant installed package, then you do not need to specify the - name option, as it will be inferred from the ID. For example, this snippet also installs NGINX:

        hugo-webserver-salt-formula/hugo/install.sls

        The same name/ID convention is true for other Salt modules.

      4. Inside the hugo directory, create a new service.sls file:

        hugo-webserver-salt-formula/hugo/service.sls
        1
        2
        3
        4
        5
        6
        
        nginx_service:
          service.running:
            - name: nginx
            - enable: True
            - require:
              - pkg: nginx_pkg

        This state says that the nginx service should be immediately run and be enabled to run at boot. For a Debian 9 system, Salt will set the appropriate systemd configurations to enable the service. Salt also supports other init systems.

        The require lines specify that this state component should not be applied until after the nginx_pkg component has been applied.

        Note

        Unless specified by a require declaration, Salt makes no guarantees about the order that different components are applied. The order that components are listed in a state file does not necessarily correspond with the order that they are applied.

      5. Inside the hugo directory, create a new init.sls file with the following contents:

        hugo-webserver-salt-formula/hugo/init.sls
        1
        2
        3
        
        include:
          - hugo.install
          - hugo.service

        Using the include declaration in this way simply concatenates the install.sls and service.sls files into a single combined state file.

        Right now, these state files only install and enable NGINX. More functionality will be enabled later in this guide.

        The install and service states will not be applied to the minion on their own–instead, only the combined init state will ever be applied. In Salt, when a file named init.sls exists inside a directory, Salt will refer to that particular state by the name of the directory it belongs to (i.e. hugo in our example).

        Note

        The organization of the state files used here is not mandated by Salt. Salt does not place restrictions on how you organize your states. This specific structure is presented as an example of a best practice.

      Push the Salt Formula to GitHub

      1. Inside your hugo-webserver-salt-formula directory on your computer, initialize a new Git repository:

        cd ~/hugo-webserver-salt-formula
        git init
        
      2. Stage the files you just created:

        git add .
        
      3. Review the staged files:

        git status
        
          
        On branch master
        No commits yet
        Changes to be committed:
          (use "git rm --cached ..." to unstage)
        
          new file:   hugo/init.sls
          new file:   hugo/install.sls
          new file:   hugo/service.sls
        
        
      4. Commit the files:

        git commit -m "Initial commit"
        
      5. Log into the GitHub website in your browser and navigate to the Create a New Repository page.

      6. Create a new public repository with the name hugo-webserver-salt-formula:

        GitHub New Repository - Add New Salt Formula Repo

      7. Copy the HTTPS URL for your new repository:

        GitHub New Repository - New Salt Formula Repo

      8. In your local Salt formula repository, add the GitHub repository as the origin remote and push your new files to it. Replace github-username with your GitHub user:

        git remote add origin https://github.com/github-username/hugo-webserver-salt-formula.git
        git push -u origin master
        

        Note

        If you haven’t pushed anything else to your GitHub account from the command line before, you may be prompted to authenticate with GitHub. If you have two-factor authentication enabled for your account, you will need to create and use a personal access token.
      9. If you navigate back to your hugo-webserver-salt-formula repository on GitHub and refresh the page, you should now see your new files.

      Enable GitFS on the Salt Master

      Update your Salt master to serve the new formula from GitHub:

      1. Salt requires that you install a Python interface to Git to use GitFS. On the Salt master Linode:

        sudo apt-get install python-git
        
      2. Open /etc/salt/master in a text editor. Uncomment the fileserver_backend declaration and enter roots and gitfs in the declaration list:

        /etc/salt/master
        1
        2
        3
        
        fileserver_backend:
          - roots
          - gitfs

        roots refers to Salt files stored on the master’s filesystem. While the Hugo webserver Salt formula is stored on GitHub, the Salt Top file will be stored on the master. The Top file is how Salt maps states to the minions they will be applied to.

      3. In the same file, uncomment the gitfs_remotes declaration and enter your Salt formula’s repository URL:

        /etc/salt/master
        1
        2
        
        gitfs_remotes:
          - https://github.com/your_github_user/hugo-webserver-salt-formula.git
      4. Uncomment the gitfs_provider declaration and set its value to gitpython:

        /etc/salt/master
        1
        
        gitfs_provider: gitpython

      Apply the Formula’s State to the Minion

      1. In /etc/salt/master, uncomment the file_roots declaration and set the following values:

        /etc/salt/master
        1
        2
        3
        
        file_roots:
          base:
            - /srv/salt/

        file_roots specifies where state files are kept on the Master’s filesystem. This is referenced when - roots is declared in the fileserver_backend section. base refers to a Salt environment, which is a tree of state files that can be applied to minions. This guide will only use the base environment, but other environments could be created for development, QA, and so on.

      2. Restart Salt on the master to enable the changes in /etc/salt/master:

        sudo systemctl restart salt-master
        
      3. Create the /srv/salt directory on the Salt master:

        sudo mkdir /srv/salt
        
      4. Create a new top.sls file in /srv/salt:

        /srv/salt/top.sls
        1
        2
        3
        
        base:
          'hugo-webserver':
            - hugo

        This is Salt’s Top file, and the snippet declares that the hugo-webserver minion should receive the init.sls state from the hugo directory (from your GitHub-hosted Salt formula).

      5. Tell Salt to apply states from the Top file to the minion:

        sudo salt 'hugo-webserver' state.apply
        

        Salt as refers to this command as a highstate. Running a highstate can take a bit of time to complete, and the output of the command will describe what actions were taken on the minion. The output will also show if any actions failed.

        Note

        If you see an error similar to:

          
        No matching sls found for 'hugo' in env 'base'
        
        

        Try running this command to manually fetch the Salt formula from GitHub, then run the state.apply command again:

        sudo salt-run fileserver.update
        

        Salt’s GitFS fetches files from remotes periodically, and this period can be configured.

      6. If you visit your domain name in a web browser, you should now see NGINX’s default test page served by the Salt minion.

      Initialize the Hugo Site

      1. On your computer, create a new Hugo site. Make sure you are not running this command in your hugo-webserver-salt-formula directory:

        hugo new site example-hugo-site
        
      2. Navigate to the new Hugo site directory and initialize a Git repository:

        cd example-hugo-site
        git init
        
      3. Install a theme into the themes/ directory. This guide uses the Cactus theme:

        git submodule add https://github.com/digitalcraftsman/hugo-cactus-theme.git themes/hugo-cactus-theme
        
      4. The theme comes with some example content. Copy it into the root of your site so that it can be viewed:

        cp -r themes/hugo-cactus-theme/exampleSite/ .
        
      5. Edit the baseurl, themesDir, and name options in config.toml as follows; replace example.com with your own domain and Your Name with your own name:

        example-hugo-site/config.toml
        1
        2
        3
        4
        5
        6
        
        # [...]
        baseURL = "http://example.com"
        # [...]
        themesDir = "themes"
        # [...]
          name = "Your Name"
      6. Run the Hugo development server on your computer:

        hugo server
        

        The output from this command will end with a line like:

          
        Web Server is available at http://localhost:1313/ (bind address 127.0.0.1)
        
        
      7. If you view the URL from this output in a browser, you can see your new Hugo site:

        New Hugo Site - Development Server

      8. Enter CTRL-C in the terminal session on your computer to stop the Hugo development server. Open the .gitignore file and make sure public/ is listed. The default .gitignore from the Cactus theme should look like:

        example-hugo-site/config.toml

        The public directory is the result of Hugo compiling the Markdown content files into HTML. These files can be regenerated by anyone who downloads your site code, so they won’t be checked into version control.

      Push the Hugo Site to GitHub

      1. In the Hugo site directory, commit the new site files:

        git add .
        git commit -m "Initial commit"
        
      2. Create a new public repository on GitHub named example-hugo-site and copy the repository’s HTTPS URL.

      3. In the site directory, add the GitHub repository as the origin remote and push your new files to it; replace github-username with your GitHub user:

        git remote add origin https://github.com/github-username/example-hugo-site.git
        git push -u origin master
        

      Deploy the Hugo Site

      The Salt minion’s formula needs to be updated in order to serve the Hugo site. Specifically, the formula will need to have states which:

      • Install Git and clone the Hugo site repository from GitHub.

      • Install Hugo and build the HTML files from the markdown content.

      • Update the NGINX configuration to serve the built site.

      Some of the new state components will refer to data stored in Salt Pillar. Pillar is a Salt system that stores private data and other parameters that you don’t want to list in your formulas. The Pillar data will be kept as a file on the Salt master and not checked into version control.

      Note

      There are methods for securely checking this data into version control or using other backends to host the data, but those strategies are outside the scope of this guide.

      Pillar data is injected into state files with Salt’s Jinja templating feature. State files are first evaluated as Jinja templates and then as YAML afterwards.

      Install Git and Hugo

      In your local Salt formula’s repository, edit the install.sls file to append the git_pkg and hugo_pkg states:

      hugo-webserver-salt-formula/hugo/install.sls
       1
       2
       3
       4
       5
       6
       7
       8
       9
      10
      11
      
      # [...]
      
      git_pkg:
        pkg.installed:
          - name: git
      
      hugo_pkg:
        pkg.installed:
          - name: hugo
          - sources:
            - hugo: https://github.com/gohugoio/hugo/releases/download/v{{ pillar['hugo_deployment_data']['hugo_version'] }}/hugo_{{ pillar['hugo_deployment_data']['hugo_version'] }}_Linux-64bit.deb

      The first state component installs Git, and the second component installs Hugo. The second component’s sources declaration specifies that the package should be downloaded from Hugo’s GitHub repository (instead of from the distribution package manager).

      The {{ }} syntax that appears in {{ pillar['hugo_deployment_data']['hugo_version'] }} is a Jinja substitution statement. pillar['hugo_deployment_data']['hugo_version'] returns the value of the hugo_version key from a dictionary named hugo_deployment_data in Pillar. Keeping the Hugo version in Pillar lets you update Hugo without needing to update your formulas.

      Clone the Hugo Site Git Repository

      Create a new config.sls file in your local Salt formula repository’s hugo directory:

      hugo-webserver-salt-formula/hugo/config.sls
       1
       2
       3
       4
       5
       6
       7
       8
       9
      10
      11
      12
      13
      14
      15
      16
      17
      18
      19
      20
      21
      22
      
      hugo_group:
        group.present:
          - name: {{ pillar['hugo_deployment_data']['group'] }}
      
      hugo_user:
        user.present:
          - name: {{ pillar['hugo_deployment_data']['user'] }}
          - gid: {{ pillar['hugo_deployment_data']['group'] }}
          - home: {{ pillar['hugo_deployment_data']['home_dir'] }}
          - createhome: True
          - require:
            - group: hugo_group
      
      hugo_site_repo:
        cmd.run:
          - name: git clone --recurse-submodules https://github.com/{{ pillar['hugo_deployment_data']['github_account'] }}/{{ pillar['hugo_deployment_data']['site_repo_name'] }}.git
          - cwd: {{ pillar['hugo_deployment_data']['home_dir'] }}
          - runas: {{ pillar['hugo_deployment_data']['user'] }}
          - creates: {{ pillar['hugo_deployment_data']['home_dir'] }}/{{ pillar['hugo_deployment_data']['site_repo_name'] }}
          - require:
            - pkg: git_pkg
            - user: hugo_user

      The final hugo_site_repo component in this snippet is responsible for cloning the example Hugo site repository from GitHub. This cloned repo is placed in the home directory of a system user that Salt creates in the preceding components. The clone command also recursively downloads the Cactus theme submodule.

      Note

      The - creates declaration tells Salt that running the cmd command module will result in the creation of the file that’s specified. If the state is applied again later, Salt will check if that file already exists. If it exists, Salt will not run the module again.

      The require declarations in each component ensure that:

      • The clone is not run until the system user and home directory have been created, and until the software package for Git has been installed.
      • The user is not created until the group it belongs to is created.

      Instead of hard-coding the parameters for the user, group, home directory, GitHub account, and repository name, these are retrieved from Pillar.

      Configure NGINX

      1. Append the following states to your config.sls:

        hugo-webserver-salt-formula/hugo/config.sls
         1
         2
         3
         4
         5
         6
         7
         8
         9
        10
        11
        12
        13
        14
        15
        16
        17
        18
        19
        20
        21
        22
        23
        24
        25
        26
        27
        28
        29
        30
        31
        32
        33
        34
        
        nginx_default:
          file.absent:
            - name: '/etc/nginx/sites-enabled/default'
            - require:
              - pkg: nginx_pkg
        
        nginx_config:
          file.managed:
            - name: /etc/nginx/sites-available/hugo_site
            - source: salt://hugo/files/hugo_site
            - user: root
            - group: root
            - mode: 0644
            - template: jinja
            - require:
              - pkg: nginx_pkg
        
        nginx_symlink:
          file.symlink:
            - name: /etc/nginx/sites-enabled/hugo_site
            - target: /etc/nginx/sites-available/hugo_site
            - user: root
            - group: root
            - require:
              - file: nginx_config
        
        nginx_document_root:
          file.directory:
            - name: {{ pillar['hugo_deployment_data']['nginx_document_root'] }}/{{ pillar['hugo_deployment_data']['site_repo_name'] }}
            - user: {{ pillar['hugo_deployment_data']['user'] }}
            - group: {{ pillar['hugo_deployment_data']['group'] }}
            - dir_mode: 0755
            - require:
              - user: hugo_user
        • The nginx_default component removes the symlink in sites-enabled for the default NGINX config, which disables that configuration.
        • nginx_config and nginx_symlink then create a new configuration file in sites-available and a symlink to it in sites-enabled.
        • The nginx_document_root component creates the directory that NGINX will serve your Hugo site files from (when filled in with Pillar data, this will directory will look like /var/www/example-hugo-site).
      2. The - source: salt://hugo/files/hugo_site declaration in nginx_config refers to an NGINX configuration file that doesn’t exist in your repository yet. Create the files/ directory:

        cd ~/hugo-webserver-salt-formula/hugo
        mkdir files
        
      3. Create the hugo_site file inside files/:

        hugo-webserver-salt-formula/hugo/files/hugo_site
         1
         2
         3
         4
         5
         6
         7
         8
         9
        10
        11
        12
        13
        
        server {
            listen 80;
            listen [::]:80;
            server_name {{ pillar['hugo_deployment_data']['domain_name'] }};
        
            root {{ pillar['hugo_deployment_data']['nginx_document_root'] }}/{{ pillar['hugo_deployment_data']['site_repo_name'] }};
        
            index index.html index.htm index.nginx-debian.html;
        
            location / {
                try_files $uri $uri/ = /404.html;
            }
        }

        The nginx_config component that manages this file also listed the - template: jinja declaration, so the source file is interpreted as a Jinja template. The source file is able to substitute values from Pillar using the Jinja substitution syntax.

      4. Replace the content of your service.sls with this snippet:

        hugo-webserver-salt-formula/hugo/service.sls
        1
        2
        3
        4
        5
        6
        7
        8
        
        nginx_service:
          service.running:
            - name: nginx
            - enable: True
            - require:
              - file: nginx_symlink
            - watch:
              - file: nginx_config

        The nginx_service component now requires nginx_symlink instead of nginx_pkg. Without this change, the service may be enabled and run before the new NGINX configuration is set up. The - watch declaration also instructs NGINX to restart whenever a change to nginx_config is made.

      Build Hugo

      1. Append a build_script state to config.sls:

        hugo-webserver-salt-formula/hugo/config.sls
         1
         2
         3
         4
         5
         6
         7
         8
         9
        10
        11
        12
        13
        14
        15
        16
        17
        18
        19
        
        build_script:
          file.managed:
            - name: {{ pillar['hugo_deployment_data']['home_dir'] }}/deploy.sh
            - source: salt://hugo/files/deploy.sh
            - user: {{ pillar['hugo_deployment_data']['user'] }}
            - group: {{ pillar['hugo_deployment_data']['group'] }}
            - mode: 0755
            - template: jinja
            - require:
              - user: hugo_user
          cmd.run:
            - name: ./deploy.sh
            - cwd: {{ pillar['hugo_deployment_data']['home_dir'] }}
            - runas: {{ pillar['hugo_deployment_data']['user'] }}
            - creates: {{ pillar['hugo_deployment_data']['nginx_document_root'] }}/{{ pillar['hugo_deployment_data']['site_repo_name'] }}/index.html
            - require:
              - file: build_script
              - cmd: hugo_site_repo
              - file: nginx_document_root

        This state uses more than one module. The first module will download the deploy.sh file from the salt master and place it on the minion. This script will be responsible for compiling your Hugo site files. The second module then calls that script. The first module is listed as a requirement of the second module, along with the Git clone command, and the creation of the document root folder.

        Note

        The - creates option in the second module ensures that Salt doesn’t rebuild Hugo if the state is re-applied to the minion.

      2. Create the deploy.sh script in files/:

        hugo-webserver-salt-formula/hugo/files/deploy.sh
        1
        2
        3
        4
        
        #!/bin/bash
        
        cd {{ pillar['hugo_deployment_data']['site_repo_name'] }}
        hugo --destination={{ pillar['hugo_deployment_data']['nginx_document_root'] }}/{{ pillar['hugo_deployment_data']['site_repo_name'] }}

        Hugo’s build function is called with NGINX’s document root as the destination for the built files.

      3. Update init.sls to include the new config.sls file:

        hugo-webserver-salt-formula/hugo/init.sls
        1
        2
        3
        4
        
        include:
          - hugo.install
          - hugo.config
          - hugo.service

      Push the Salt Formula Updates to GitHub

      Your state files should now have these contents: init.sls, install.sls, config.sls, service.sls.

      The files present in your Salt formula repository should be:

        
      hugo
      ├── config.sls
      ├── files
      │   ├── deploy.sh
      │   └── hugo_site
      ├── init.sls
      ├── install.sls
      └── service.sls
      
      
      1. Stage all the changes you made to your local Salt formula files in the previous steps and then commit the changes:

        cd ~/hugo-webserver-salt-formula
        git add .
        git commit -m "Deploy the Hugo site"
        
      2. Push the commit to your GitHub repository:

        git push origin master
        

      Create the Salt Pillar File

      1. Open /etc/salt/master on the Salt master in a text editor. Uncomment the pillar_roots section:

        /etc/salt/master
        1
        2
        3
        
        pillar_roots:
          base:
            - /srv/pillar

        pillar_roots performs an analogous function to file_roots: it specifies where Pillar data is stored on the master’s filesystem.

      2. Restart Salt on the master to enable the changes in /etc/salt/master:

        sudo systemctl restart salt-master
        
      3. Create the /srv/pillar directory on the Salt master:

        sudo mkdir /srv/pillar
        
      4. Create an example-hugo-site.sls file in /srv/pillar to contain the Pillar data for the minion. This file uses the same YAML syntax as other state files. Replace the values for github_account and domain_name with your GitHub account and your site’s domain name:

        /srv/pillar/example-hugo-site.sls
        1
        2
        3
        4
        5
        6
        7
        8
        9
        
        hugo_deployment_data:
          hugo_version: 0.49
          group: hugo
          user: hugo
          home_dir: /home/hugo
          github_account: your_github_user
          site_repo_name: example-hugo-site
          nginx_document_root: /var/www
          domain_name: yourdomain.com
      5. Create a top.sls file in /srv/pillar. Similar to the Top file in your state tree, the Pillar’s Top file maps Pillar data to minions:

        /srv/pillar/top.sls
        1
        2
        3
        
        base:
          'hugo-webserver':
            - example-hugo-site

      Apply State Updates to the Minion

      On the Salt master, apply the new states to all minions:

      sudo salt '*' state.apply
      

      Note

      In this guide there is only one minion, but Salt can use shell-style globbing and regular expressions to match against minion IDs when you have more than one. For example, this command would run a highstate on all minions whose IDs begin with hugo:

      sudo salt 'hugo*' state.apply
      

      If no changes are made, try manually fetching the Salt formula updates from GitHub and then run the state.apply command again:

      sudo salt-run fileserver.update
      

      When the operation finishes, your Hugo site should now be visible at your domain.

      Deploy Site Updates with Webhooks

      Your site is now deployed to production, but there is no automatic mechanism in place yet for updating the production server when you update your Hugo site’s content. To update the production server, your minion will need to:

      1. Pull the latest changes pushed to the master branch of your Hugo site repository on GitHub.

      2. Run the Hugo build process with the new content.

      The deploy.sh script can be altered to pull changes from GitHub. These script changes will be made in the Salt formula repository. Then, we’ll set up webhooks to notify the Salt minion that updates have been made to the Hugo site.

      Webhooks are HTTP POST requests specifically designed and sent by systems to communicate some kind of significant event. A webhook server listens for these requests and then takes some action when it receives one. For example, a GitHub repository can be configured to send webhook notifications whenever a push is made to the repository. This is the kind of notification we’ll configure, and the Salt minion will run a webhook server to receive them. Other event notifications can also be set up on GitHub.

      Set Up a Webhook Server on the Salt Minion

      1. In your local Salt formula repository, append a new webhook_pkg state to your install.sls that installs the webhook server package by adnanh:

        hugo-webserver-salt-formula/hugo/install.sls
        1
        2
        3
        
        webhook_pkg:
          pkg.installed:
            - name: webhook

        Note

        The webhook server written in Go by adnanh is a popular implementation of the concept, but it’s possible to write other HTTP servers that parse webhook payloads.

      2. Append two new components to your config.sls:

        hugo-webserver-salt-formula/hugo/config.sls
         1
         2
         3
         4
         5
         6
         7
         8
         9
        10
        11
        12
        13
        14
        15
        16
        17
        18
        19
        20
        21
        22
        23
        24
        25
        26
        
        webhook_systemd_unit:
          file.managed:
            - name: '/etc/systemd/system/webhook.service'
            - source: salt://hugo/files/webhook.service
            - user: root
            - group: root
            - mode: 0644
            - template: jinja
            - require:
              - pkg: webhook_pkg
          module.run:
            - name: service.systemctl_reload
            - onchanges:
              - file: webhook_systemd_unit
        
        webhook_config:
          file.managed:
            - name: '/etc/webhook.conf'
            - source: salt://hugo/files/webhook.conf
            - user: root
            - group: {{ pillar['hugo_deployment_data']['group'] }}
            - mode: 0640
            - template: jinja
            - require:
              - pkg: webhook_pkg
              - group: hugo_group

        The first state creates a systemd unit file for the webhook service. The second state creates a webhook configuration. The webhook server reads the configuration and generates a webhook URL from it.

      3. Create a webhook.service file in your repository’s files/ directory:

        hugo-webserver-salt-formula/hugo/files/webhook.service
         1
         2
         3
         4
         5
         6
         7
         8
         9
        10
        
        [Unit]
        Description=Small server for creating HTTP endpoints (hooks)
        Documentation=https://github.com/adnanh/webhook/
        
        [Service]
        User={{ pillar['hugo_deployment_data']['user'] }}
        ExecStart=/usr/bin/webhook -nopanic -hooks /etc/webhook.conf
        
        [Install]
        WantedBy=multi-user.target
      4. Create a webhook.conf file in your repository’s files/ directory:

        hugo-webserver-salt-formula/hugo/files/webhook.conf
         1
         2
         3
         4
         5
         6
         7
         8
         9
        10
        11
        12
        13
        14
        15
        16
        17
        18
        19
        20
        21
        22
        23
        24
        25
        26
        27
        28
        29
        30
        31
        32
        33
        34
        35
        36
        37
        
        [
          {
            "id": "github_push",
            "execute-command": "{{ pillar['hugo_deployment_data']['home_dir'] }}/deploy.sh",
            "command-working-directory": "{{ pillar['hugo_deployment_data']['home_dir'] }}",
            "trigger-rule":
            {
              "and":
              [
                {
                  "match":
                  {
                    "type": "payload-hash-sha1",
                    "secret": "{{ pillar['hugo_deployment_data']['webhook_secret'] }}",
                    "parameter":
                    {
                      "source": "header",
                      "name": "X-Hub-Signature"
                    }
                  }
                },
                {
                  "match":
                  {
                    "type": "value",
                    "value": "refs/heads/master",
                    "parameter":
                    {
                      "source": "payload",
                      "name": "ref"
                    }
                  }
                }
              ]
            }
          }
        ]

        This configuration sets up a URL named http://example.com:9000/hooks/github_push, where the last component of the URL is derived from the value of the configuration’s id.

        Note

        The webhook server runs on port 9000 and places your webhooks inside a hooks/ directory by default.

        When a POST request is sent to the URL:

        • The webhook server checks if the header and payload data from the request satisfies the rules in the trigger-rule dictionary, which are:

          • That the SHA1 hash of the server’s webhook secret matches the secret in the request headers. This prevents people who don’t know your webhook secret from triggering the webhook’s action.
          • The ref parameter in the payload matches refs/heads/master. This ensures that only pushes to the master branch trigger the action.
        • If the rules are satisfied, then the command listed in execute-command is run, which is the deploy.sh script.

        Note

        Further documentation on the webhook configuration options can be reviewed on the project’s GitHub repository.
      5. Append a new webhook_service state to your service.sls that enables and starts the webhook server:

        hugo-webserver-salt-formula/hugo/service.sls
        1
        2
        3
        4
        5
        6
        7
        
        webhook_service:
          service.running:
            - name: webhook
            - enable: True
            - watch:
              - file: webhook_config
              - module: webhook_systemd_unit
      6. Update the deploy.sh script so that it pulls changes from master before building the site:

        hugo-webserver-salt-formula/hugo/files/deploy.sh
        1
        2
        3
        4
        5
        
        #!/bin/bash
        
        cd {{ pillar['hugo_deployment_data']['site_repo_name'] }}
        git pull origin master
        hugo --destination={{ pillar['hugo_deployment_data']['nginx_document_root'] }}//{{ pillar['hugo_deployment_data']['site_repo_name'] }}
      7. Your state files should now have these contents: init.sls (unchanged), install.sls, config.sls, service.sls. Save the changes made to your Salt files, then commit and push them to GitHub:

        cd ~/hugo-webserver-salt-formula
        git add .
        git commit -m "Webhook server states"
        git push origin master
        
      8. On the Salt master, add a webhook_secret to the example-hugo-site.sls Pillar. Your secret should be a complex, random alphanumeric string.

        /srv/pillar/example-hugo-site.sls
        1
        2
        3
        
        hugo_deployment_data:
          # [...]
          webhook_secret: your_webhook_secret
      9. From the Salt master, apply the formula updates to the minion:

        sudo salt-run fileserver.update
        sudo salt 'hugo-webserver' state.apply
        
      10. Your webhook server should now be running on the minion. If you run a curl against it, you should see:

        curl http://example.com:9000/hooks/github_push
        
          
        Hook rules were not satisfied.⏎
        
        

      Configure a Webhook on GitHub

      1. Visit your example Hugo site repository on GitHub and navigate to the Webhooks section of the Settings tab. Click on the Add webhook button:

        GitHub - Add Webhook Button

      2. Fill in the form:

        • Enter http://example.com:9000/hooks/github_push for the payload URL (substitute example.com for your own domain).

        • Select application/json for the content type.

        • Paste in the webhook secret that you previously added to Salt Pillar.

        The webhook is configured to notify on push events by default. Keep this option selected.

        GitHub - New Webhook Configuration

      3. Click the green Add webhook button to complete the setup.

      Update the Hugo Site

      1. In your local Hugo site repository, create a new post using Hugo’s archetypes feature:

        hugo new post/test-post.md
        
      2. This command creates a new partially filled in markdown document in content/post/. Open this file in your editor, remove the draft: true line from the frontmatter, and add some body text:

        example-hugo-site/content/post/test-post.md
        1
        2
        3
        4
        5
        6
        
        ---
        title: "Test Post"
        date: 2018-10-19T11:39:15-04:00
        ---
        
        Test post body text
      3. If you run hugo server in the repository directory, you can see the new post:

        Hugo Home Page - Test Post

      4. Commit and push the new post to GitHub:

        cd ~/example-hugo-site
        git add .
        git commit -m "Test post"
        git push origin master
        
      5. Visit your domain in your browser; your test post should automatically appear.

        Note

        If your post does not appear, review the Recent Deliveries section at the bottom of your webhook configuration page on GitHub:

        GitHub Webhook - Recent Deliveries

        If you click on a delivery, full information about the request headers and payload and the server response are shown, and these may provide some troubleshooting information. Editing the webhook.service file so that it starts the service in verbose mode may help.

      Next Steps

      The current Salt configuration can be used as a foundation for more complex deployments:

      • Host multiple Hugo sites by updating Pillar with further GitHub repositories.

      • Host different kinds of static sites by changing the Salt formula to support them.

      • Load balance your site by creating more minions and apply the same Pillar data and Salt states to them. Then, set up a NodeBalancer to direct traffic to the minions.

      • Set up a separate development branch and development server with Salt’s environments feature.

      More Information

      You may wish to consult the following resources for additional information on this topic. While these are provided in the hope that they will be useful, please note that we cannot vouch for the accuracy or timeliness of externally hosted materials.

      Join our Community

      Find answers, ask questions, and help others.

      This guide is published under a CC BY-ND 4.0 license.



      Source link