One place for hosting & domains

      Manage

      How To Use Git to Manage Your Writing Project


      Introduction

      Version control isn’t just for code. It’s for anything you want to track, including content. Using Git to manage your next writing project gives you the ability to view multiple drafts at the same time, see differences between those drafts, and even roll back to a previous version. And if you’re comfortable doing so, you can then share your work with others on GitHub or other central Git repositories.

      In this tutorial you’ll use Git to manage a small Markdown document. You’ll store an initial version, commit it, make changes, view the difference between those changes, and review the previous version. When you’re done, you’ll have a workflow you can apply to your own writing projects.

      Prerequisites

      Step 1 — Creating a Workspace for Your Writing Project

      To manage your changes, you’ll create a local Git repository. A Git repository lives inside of an existing directory, so start by creating a new directory for your article:

      Switch to the new article directory:

      The git init command creates a new empty Git repository in the current directory. Execute that command now:

      You’ll see the following output which confirms your repository was created:

      Output

      Initialized empty Git repository in /Users/sammy/article/.git/

      The .gitignore file lets you tell Git that some files should be ignored. You can use this to ignore temporary files your text editor might create, or operating systems files. On macOS, for example, the Finder application creates .DS_Store files in directories. Create a .gitignore file that ignores them:

      Add the following lines to the file:

      .gitignore

      # Ignore Finder files
      .DS_store
      

      The first line is a comment, which will help you identify what you’re ignoring in the future. The second line specifies the file to ignore.

      Save the file and exit the editor.

      As you discover more files you want to ignore, open the .gitignore file and add a new line for each file or directory you want to ignore.

      Now that your repository is configured, you can start working.

      Step 2 — Saving Your Initial Draft

      Git only knows about files you tell it about. Just because a file exists in the directory holding the repository doesn’t mean Git will track its changes. You have to add a file to the repository and then commit the changes.

      Create a new Markdown file called article.md:

      Add some text to the file:

      article.md

      # How To Use Git to Manage Your Writing Project
      
      ### Introduction
      
      Version control isn't just for code. It's for anything you want to track, including content. Using Git to manage your next writing project gives you the ability to view multiple drafts at the same time,  see differences between those drafts, and even roll back to a previous version. And if you're comfortable doing so, you can then share your work with others on GitHub or other central git repositories.
      
      In this tutorial you'll use Git to manage a small Markdown document. You'll store an initial version, commit it, make changes, view the difference between those changes, and review the previous version. When you're done, you'll have a workflow you can apply to your own writing projects.
      

      Save the changes and exit the editor.

      The git status command will show you the state of your repository. It will show you what files need to be added so Git can track them. Run this command:

      You’ll see this output:

      Output

      On branch master No commits yet Untracked files: (use "git add <file>..." to include in what will be committed) .gitignore article.md nothing added to commit but untracked files present (use "git add" to track)

      In the output, the Untracked files section shows the files that Git isn’t looking at. These files need to be added to the repository so Git can watch them for changes. Use the git add command to do this:

      • git add .gitignore
      • git add article.md

      Now run git status to verify those files have been added:

      Output

      On branch master No commits yet Changes to be committed: (use "git rm --cached <file>..." to unstage) new file: .gitignore new file: article.md

      Both files are now listed in the Changes to be committed section. Git knows about them, but it hasn’t created a snapshot of the work yet. Use the git commit command to do that.

      When you create a new commit, you need to provide a commit message. A good commit message states what your changes are. When you’re working with others, the more detailed your commit messages are, the better.

      Use the command git commit to commit your changes:

      • git commit -m "Add gitignore file and initial version of article"

      The output of the command shows that the files were committed:

      Output

      [master (root-commit) 95fed84] Add gitignore file and initial version of article 2 files changed, 9 insertions(+) create mode 100644 .gitignore create mode 100644 article.md

      Use the git status command to see the state of the repository:

      The output shows there are no changes that need to be added or committed.

      Output

      On branch master nothing to commit, working tree clean

      Now let’s look at how to work with changes.

      Step 3 — Saving Revisions

      You’ve added your initial version of the article. Now you’ll add more text so you can see how to manage changes with Git.

      Open the article in your editor:

      Add some more text to the end of the file:

      ## Prerequisites
      
      * Git installed on your local computer. The tutorial [How to Contribute to Open Source: Getting Started with Git](https://www.digitalocean.com/community/tutorials/how-to-contribute-to-open-source-getting-started-with-git) walks you through installing Git and covers some background information you may find useful. 
      

      Save the file.

      Use the git status command to see where things stand in your repository:

      The output shows there are changes:

      Output

      On branch master Changes not staged for commit: (use "git add <file>..." to update what will be committed) (use "git checkout -- <file>..." to discard changes in working directory) modified: article.md no changes added to commit (use "git add" and/or "git commit -a")

      As expected, the article.md file has changes.

      Use git diff to see what they are:

      The output shows the lines you’ve added:

      diff --git a/article.md b/article.md
      index 77b081c..ef6c301 100644
      --- a/article.md
      +++ b/article.md
      @@ -5,3 +5,7 @@
       Version control isn't just for code. It's for anything you want to track, including content. Using Git to manage your next writing project gives you the ability to view multiple drafts at the same time,  see differences between those drafts, and even roll back to a previous version. And if you're comfortable doing so, you can then share your work with others on GitHub or other central git repositories.
      
       In this tutorial you'll use Git to manage a small Markdown document. You'll store an initial version, commit it, make changes, view the difference between those changes, and review the previous version. When you're done, you'll have a workflow you can apply to your own writing projects.
      +
      +## Prerequisites
      +
      +* Git installed on your local computer. The tutorial [How to Contribute to Open Source: Getting Started with Git](https://www.digitalocean.com/community/tutorials/how-to-contribute-to-open-source-getting-started-with-git) walks you through installing Git and covers some background information you may find useful. 
      

      In the output, lines starting with a plus (+) sign are lines you added. Lines that were removed would show up with a minus (-) sign. Lines that were unchanged would have neither of these characters in front.

      Using git diff and git status is a helpful way to see what you’ve changed. You can also save the diff to a file so you can view it later with the following command:

      • git diff article.md > article_diff.diff

      Using the .diff extension will help your text editor apply the proper syntax highlighting.

      Saving the changes to your repository is a two-step process. First, add the article.md file again, and then commit. Git wants you to explicitly tell it which files go in every commit, so even though you added the file before, you have to add it again. Note that the output from the git status command reminds you of that.

      Add the file and then commit the changes, providing a commit message:

      • git add article.md
      • git commit -m "add prerequisites section"

      The output verifies that the commit worked:

      Output

      [master 1fbfc21] add prerequisites section 1 file changed, 4 insertions(+)

      Use git status to see your repository status. You’ll see that there’s nothing else to do.

      Output

      On branch master nothing to commit, working tree clean

      Continue this process as you revise your article. Make changes, verify them, add the file, and commit the changes with a detailed message. Commit your changes as often or as little as you feel comfortable. You might perform a commit after you finish each draft, or right before you do a major rework of your article’s structure.

      If you send a draft of a document to someone else and they make changes to it, take their copy and replace your file with theirs. Then use git diff to see the changes they made quickly. Git will see the changes whether you typed them in directly or replaced the file with one you downloaded from the web, email, or elsewhere.

      Now let’s look at managing the versions of your article.

      Step 4 — Managing Changes

      Sometimes it’s helpful to look at a previous version of a document. Whenever you’ve used git commit, you’ve supplied a helpful message that summarizes what you’ve done.

      The git log command shows you the commit history of your repository. Every change you’ve committed has an entry in the log.

      Output

      commit 1fbfc2173f3cec0741e0a6b21803fbd0be511bc4 Author: Sammy Shark <sammy@digitalocean> Date: Thu Sep 19 16:35:41 2019 -0500 add prerequisites section commit 95fed849b0205c49eda994fff91ec03642d59c79 Author: Sammy Shark <sammy@digitalocean> Date: Thu Sep 19 16:32:34 2019 -0500 Add gitignore file and initial version of article

      Each commit has a specific identifier. You use this number to reference a specific commit’s changes. You only need the first several characters of the identifier though. The git log --oneline command gives you a condensed version of the log with shorter identifiers:

      Output

      1fbfc21 add prerequisites section 95fed84 Add gitignore file and initial version of article

      To view the initial version of your file, use git show and the commit identifier. The identifiers in your repository will be different than the ones in these examples.

      • git show 95fed84 article.md

      The output shows the commit detail, as well as the changes that happened during that commit:

      Output

      commit 95fed849b0205c49eda994fff91ec03642d59c79 Author: Sammy Shark <sammy@digitalocean> Date: Thu Sep 19 16:32:34 2019 -0500 Add gitignore file and initial version of article diff --git a/article.md b/article.md new file mode 100644 index 0000000..77b081c --- /dev/null +++ b/article.md @@ -0,0 +1,7 @@ +# How To Use Git to Manage Your Writing Project + +### Introduction + +Version control isn't just for code. It's for anything you want to track, including content. Using Git to manage your next writing project gives you the ability to view multiple drafts at the same time, see differences between those drafts, and even roll back to a previous version. And if you're comfortable doing so, you can then share your work with others on GitHub or other central git repositories. + +In this tutorial you'll use Git to manage a small Markdown document. You'll store an initial version, commit it, make changes, view the difference between those changes, and review the previous version. When you're done, you'll have a workflow you can apply to your own writing projects.

      To see the file itself, modify the command slightly. Instead of a space between the commit identifier and the file, replace with :./ like this:

      • git show 95fed84:./article.md

      You’ll see the content of that file, at that revision:

      Output

      # How To Use Git to Manage Your Writing Project ### Introduction Version control isn't just for code. It's for anything you want to track, including content. Using Git to manage your next writing project gives you the ability to view multiple drafts at the same time, see differences between those drafts, and even roll back to a previous version. And if you're comfortable doing so, you can then share your work with others on GitHub or other central git repositories. In this tutorial you'll use Git to manage a small Markdown document. You'll store an initial version, commit it, make changes, view the difference between those changes, and review the previous version. When you're done, you'll have a workflow you can apply to your own writing projects.

      You can save that output to a file if you need it for something else:

      • git show 95fed84:./article.md > old_article.md

      As you make more changes, your log will grow, and you’ll be able to review all of the changes you’ve made to your article over time.

      Conclusion

      In this tutorial you used a local Git repository to track the changes in your writing project. You can use this approach to manage individual articles, all the posts for your blog, or even your next novel. And if you push your repository to GitHub, you can invite others to help you edit your work.



      Source link

      How to Manage DigitalOcean and Kubernetes Infrastructure with Pulumi


      The author selected the Diversity in Tech Fund to receive a donation as part of the Write for DOnations program.

      Introduction

      Pulumi is a tool for creating, deploying, and managing infrastructure using code written in general purpose programming languages. It supports automating all of DigitalOcean’s managed services—such as Droplets, managed databases, DNS records, and Kubernetes clusters—in addition to application configuration. Deployments are performed from an easy-to-use command-line interface that also integrates with a wide variety of popular CI/CD systems.

      Pulumi supports multiple languages but in this tutorial you will use TypeScript, a statically typed version of JavaScript that uses the Node.js runtime. This means you will get IDE support and compile-time checking that will help to ensure you’ve configured the right resources, used correct slugs, etc., while still being able to access any NPM modules for utility tasks.

      In this tutorial, you will provision a DigitalOcean Kubernetes cluster, a load balanced Kubernetes application, and a DigitalOcean DNS domain that makes your application available at a stable domain name of your choosing. This can all be provisioned in 60 lines of infrastructure-as-code and a single pulumi up command-line gesture. After this tutorial, you’ll be ready to productively build powerful cloud architectures using Pulumi infrastructure-as-code that leverages the full surface area of DigitalOcean and Kubernetes.

      Prerequisites

      To follow this tutorial, you will need:

      • A DigitalOcean Account to deploy resources to. If you do not already have one, register here.
      • A DigitalOcean API Token to perform automated deployments. Generate a personal access token here and keep it handy as you’ll use it in Step 2.
      • Because you’ll be creating and using a Kubernetes cluster, you’ll need to install kubectl. Don’t worry about configuring it further — you’ll do that later.
      • You will write your infrastructure-as-code in TypeScript, so you will need Node.js 8 or later. Download it here or install it using your system’s package manager.
      • You’ll use Pulumi to deploy infrastructure, so you’ll need to install the open source Pulumi SDK.
      • To perform the optional Step 5, you will need a domain name configured to use DigitalOcean nameservers. This guide explains how to do this for your registrar of choice.

      Step 1 — Scaffolding a New Project

      The first step is to create a directory that will store your Pulumi project. This directory will contain the source code for your infrastructure definitions, in addition to metadata files describing the project and its NPM dependencies.

      First, create the directory:

      Next, move in to the newly created directory:

      From now on, run commands from your newly created do-k8s directory.

      Next, create a new Pulumi project. There are different ways to accomplish this, but the easiest way is to use the pulumi new command with the typescript project template. This command will first prompt you to log in to Pulumi so that your project and deployment state are saved, and will then create a simple TypeScript project in the current directory:

      Here you have passed the -y option to the new command which tells it to accept default project options. For example, the project name is taken from the current directory’s name, and so will be do-k8s. If you’d like to use different options for your project name, simply elide the -y.

      After running the command, list the contents of the directory with ls:

      The following files will now be present:

      Output

      Pulumi.yaml index.ts node_modules package-lock.json package.json tsconfig.json

      The primary file you’ll be editing is index.ts. Although this tutorial only uses this single file, you can organize your project any way you see fit using Node.js modules. This tutorial also describes one step at a time, leveraging the fact that Pulumi can detect and incrementally deploy only what has changed. If you prefer, you can just populate the entire program, and deploy it all in one go using pulumi up.

      Now that you’ve scaffolded your new project, you are ready to add the dependencies needed to follow the tutorial.

      Step 2 — Adding Dependencies

      The next step is to install and add dependencies on the DigitalOcean and Kubernetes packages. First, install them using NPM:

      This will download the NPM packages, Pulumi plugins, and save them as dependencies.

      Next, open the index.ts file with your favorite editor. This tutorial will use nano:

      Replace the contents of your index.ts with the following:

      index.ts

      import * as digitalocean from "@pulumi/digitalocean";
      import * as kubernetes from "@pulumi/kubernetes";
      

      This makes the full contents of these packages available to your program. If you type "digitalocean." using an IDE that understands TypeScript and Node.js, you should see a list of DigitalOcean resources supported by this package, for instance.

      Save and close the file after adding the content.

      Note: We will be using a subset of what’s available in those packages. For complete documentation of resources, properties, and associated APIs, please refer to the relevant API documentation for the @pulumi/digitalocean and @pulumi/kubernetes packages.

      Next, you will configure your DigitalOcean token so that Pulumi can provision resources in your account:

      • pulumi config set digitalocean:token YOUR_TOKEN_HERE --secret

      Notice the --secret flag, which uses Pulumi’s encryption service to encrypt your token, ensuring that it is stored in cyphertext. If you prefer, you can use the DIGITALOCEAN_TOKEN environment variable instead, but you’ll need to remember to set it every time you update your program, whereas using configuration automatically stores and uses it for your project.

      In this step you added the necessary dependencies and configured your API token with Pulumi so that you can provision your Kubernetes cluster.

      Step 3 — Provisioning a Kubernetes Cluster

      Now you’re ready to create a DigitalOcean Kubernetes cluster. Get started by reopening the index.ts file:

      Add these lines at the end of your index.ts file:

      index.ts

      …
      const cluster = new digitalocean.KubernetesCluster("do-cluster", {
          region: digitalocean.Regions.SFO2,
          version: "latest",
          nodePool: {
              name: "default",
              size: digitalocean.DropletSlugs.DropletS2VPCU2GB,
              nodeCount: 3,
          },
      });
      
      export const kubeconfig = cluster.kubeConfigs[0].rawConfig;
      

      This new code allocates an instance of digitalocean.KubernetesCluster and sets a number of properties on it. This includes using the sfo2 region slug, the latest supported version of Kubernetes, the s-2vcpu-2gb Droplet size slug, and states your desired count of three Droplet instances. Feel free to change any of these, but be aware that DigitalOcean Kubernetes is only available in certain regions at the time of this writing. You can refer to the product documentation for updated information about region availability.

      For a complete list of properties you can configure on your cluster, please refer to the KubernetesCluster API documentation.

      The final line in that code snippet exports the resulting Kubernetes cluster’s kubeconfig file so that it’s easy to use. Exported variables are printed to the console and also accessible to tools. You will use this momentarily to access our cluster from standard tools like kubectl.

      Now you’re ready to deploy your cluster. To do so, run pulumi up:

      This command takes the program, generates a plan for creating the infrastructure described, and carries out a series of steps to deploy those changes. This works for the initial creation of infrastructure in addition to being able to diff and update your infrastructure when subsequent updates are made. In this case, the output will look something like this:

      Output

      Previewing update (dev): Type Name Plan + pulumi:pulumi:Stack do-k8s-dev create + └─ digitalocean:index:KubernetesCluster do-cluster create Resources: + 2 to create Do you want to perform this update? yes > no details

      This says that proceeding with the update will create a single Kubernetes cluster named do-cluster. The yes/no/details prompt allows us to confirm that this is the desired outcome before any changes are actually made. If you select details, a full list of resources and their properties will be shown. Choose yes to begin the deployment:

      Output

      Updating (dev): Type Name Status + pulumi:pulumi:Stack do-k8s-dev created + └─ digitalocean:index:KubernetesCluster do-cluster created Outputs: kubeconfig: "…" Resources: + 2 created Duration: 6m5s Permalink: https://app.pulumi.com/…/do-k8s/dev/updates/1

      It takes a few minutes to create the cluster, but then it will be up and running, and the full kubeconfig will be printed out to the console. Save the kubeconfig to a file:

      • pulumi stack output kubeconfig > kubeconfig.yml

      And then use it with kubectl to perform any Kubernetes command:

      • KUBECONFIG=./kubeconfig.yml kubectl get nodes

      You will receive output similar to the following:

      Output

      NAME STATUS ROLES AGE VERSION default-o4sj Ready <none> 4m5s v1.14.2 default-o4so Ready <none> 4m3s v1.14.2 default-o4sx Ready <none> 3m37s v1.14.2

      At this point you’ve set up infrastructure-as-code and have a repeatable way to bring up and configure new DigitalOcean Kubernetes clusters. In the next step, you will build on top of this to define the Kubernetes infrastructure in code and learn how to deploy and manage them similarly.

      Step 4 — Deploying an Application to Your Cluster

      Next, you will describe a Kubernetes application’s configuration using infrastructure-as-code. This will consist of three parts:

      1. A Provider object, which tells Pulumi to deploy Kubernetes resources to the DigitalOcean cluster, rather than the default of whatever kubectl is configured to use.
      2. A Kubernetes Deployment, which is the standard Kubernetes way of deploying a Docker container image that is replicated across any number of Pods.
      3. A Kubernetes Service, which is the standard way to tell Kubernetes to load balance access across a target set of Pods (in this case, the Deployment above).

      This is a fairly standard reference architecture for getting up and running with a load balanced service in Kubernetes.

      To deploy all three of these, open your index.ts file again:

      Once the file is open, append this code to the end of the file:

      index.ts

      …
      const provider = new kubernetes.Provider("do-k8s", { kubeconfig })
      
      const appLabels = { "app": "app-nginx" };
      const app = new kubernetes.apps.v1.Deployment("do-app-dep", {
          spec: {
              selector: { matchLabels: appLabels },
              replicas: 5,
              template: {
                  metadata: { labels: appLabels },
                  spec: {
                      containers: [{
                          name: "nginx",
                          image: "nginx",
                      }],
                  },
              },
          },
      }, { provider });
      const appService = new kubernetes.core.v1.Service("do-app-svc", {
          spec: {
              type: "LoadBalancer",
              selector: app.spec.template.metadata.labels,
              ports: [{ port: 80 }],
          },
      }, { provider });
      
      export const ingressIp = appService.status.loadBalancer.ingress[0].ip;
      

      This code is similar to standard Kubernetes configuration, and the behavior of objects and their properties is equivalent, except that it’s written in TypeScript alongside your other infrastructure declarations.

      Save and close the file after making the changes.

      Just like before, run pulumi up to preview and then deploy the changes:

      After selecting yes to proceed, the CLI will print out detailed status updates, including diagnostics around Pod availability, IP address allocation, and more. This will help you understand why your deployment might be taking time to complete or getting stuck.

      The full output will look something like this:

      Output

      Updating (dev): Type Name Status pulumi:pulumi:Stack do-k8s-dev + ├─ pulumi:providers:kubernetes do-k8s created + ├─ kubernetes:apps:Deployment do-app-dep created + └─ kubernetes:core:Service do-app-svc created Outputs: + ingressIp : "157.230.199.202" Resources: + 3 created 2 unchanged Duration: 2m52s Permalink: https://app.pulumi.com/…/do-k8s/dev/updates/2

      After this completes, notice that the desired number of Pods are running:

      • KUBECONFIG=./kubeconfig.yml kubectl get pods

      Output

      NAME READY STATUS RESTARTS AGE do-app-dep-vyf8k78z-758486ff68-5z8hk 1/1 Running 0 1m do-app-dep-vyf8k78z-758486ff68-8982s 1/1 Running 0 1m do-app-dep-vyf8k78z-758486ff68-94k7b 1/1 Running 0 1m do-app-dep-vyf8k78z-758486ff68-cqm4c 1/1 Running 0 1m do-app-dep-vyf8k78z-758486ff68-lx2d7 1/1 Running 0 1m

      Similar to how the program exports the cluster’s kubeconfig file, this program also exports the Kubernetes service’s resulting load balancer’s IP address. Use this to curl the endpoint and see that it is up and running:

      • curl $(pulumi stack output ingressIp)

      Output

      <!DOCTYPE html> <html> <head> <title>Welcome to nginx!</title> <style> body { width: 35em; margin: 0 auto; font-family: Tahoma, Verdana, Arial, sans-serif; } </style> </head> <body> <h1>Welcome to nginx!</h1> <p>If you see this page, the nginx web server is successfully installed and working. Further configuration is required.</p> <p>For online documentation and support please refer to <a href="http://nginx.org/">nginx.org</a>.<br/> Commercial support is available at <a href="http://nginx.com/">nginx.com</a>.</p> <p><em>Thank you for using nginx.</em></p> </body> </html>

      From here, you can easily edit and redeploy your application infrastructure. For example, try changing the replicas: 5 line to say replicas: 7, and then rerun pulumi up:

      Notice that it just shows what has changed, and that selecting details displays the precise diff:

      Output

      Previewing update (dev): Type Name Plan Info pulumi:pulumi:Stack do-k8s-dev ~ └─ kubernetes:apps:Deployment do-app-dep update [diff: ~spec] Resources: ~ 1 to update 4 unchanged Do you want to perform this update? details pulumi:pulumi:Stack: (same) [urn=urn:pulumi:dev::do-k8s::pulumi:pulumi:Stack::do-k8s-dev] ~ kubernetes:apps/v1:Deployment: (update) [id=default/do-app-dep-vyf8k78z] [urn=urn:pulumi:dev::do-k8s::kubernetes:apps/v1:Deployment::do-app-dep] [provider=urn:pulumi:dev::do-k8s::pulumi:providers:kubernetes::do-k8s::80f36105-337f-451f-a191-5835823df9be] ~ spec: { ~ replicas: 5 => 7 }

      Now you have both a fully functioning Kubernetes cluster and a working application. With your application up and running, you may want to configure a custom domain to use with your application. The next step will guide you through configuring DNS with Pulumi.

      Step 5 — Creating a DNS Domain (Optional)

      Although the Kubernetes cluster and application are up and running, the application’s address is dependent upon the whims of automatic IP address assignment by your cluster. As you adjust and redeploy things, this address might change. In this step, you will see how to assign a custom DNS name to the load balancer IP address so that it’s stable even as you subsequently change your infrastructure.

      Note: To complete this step, ensure you have a domain using DigitalOcean’s DNS nameservers, ns1.digitalocean.com, ns2.digitalocean.com, and ns3.digitalocean.com. Instructions to configure this are available in the Prerequisites section.

      To configure DNS, open the index.ts file and append the following code to the end of the file:

      index.ts

      …
      const domain = new digitalocean.Domain("do-domain", {
          name: "your_domain",
          ipAddress: ingressIp,
      });
      

      This code creates a new DNS entry with an A record that refers to your Kubernetes service’s IP address. Replace your_domain in this snippet with your chosen domain name.

      It is common to want additional sub-domains, like www, to point at the web application. This is easy to accomplish using a DigitalOcean DNS record. To make this example more interesting, also add a CNAME record that points www.your_domain.com to your_domain.com:

      index.ts

      …
      const cnameRecord = new digitalocean.DnsRecord("do-domain-cname", {
          domain: domain.name,
          type: "CNAME",
          name: "www",
          value: "@",
      });
      

      Save and close the file after making these changes.

      Finally, run pulumi up to deploy the DNS changes to point at your existing application and cluster:

      Output

      Updating (dev): Type Name Status pulumi:pulumi:Stack do-k8s-dev + ├─ digitalocean:index:Domain do-domain created + └─ digitalocean:index:DnsRecord do-domain-cname created Resources: + 2 created 5 unchanged Duration: 6s Permalink: https://app.pulumi.com/…/do-k8s/dev/updates/3

      After the DNS changes have propagated, you will be able to access your content at your custom domain:

      You will receive output similar to the following:

      Output

      <!DOCTYPE html> <html> <head> <title>Welcome to nginx!</title> <style> body { width: 35em; margin: 0 auto; font-family: Tahoma, Verdana, Arial, sans-serif; } </style> </head> <body> <h1>Welcome to nginx!</h1> <p>If you see this page, the nginx web server is successfully installed and working. Further configuration is required.</p> <p>For online documentation and support please refer to <a href="http://nginx.org/">nginx.org</a>.<br/> Commercial support is available at <a href="http://nginx.com/">nginx.com</a>.</p> <p><em>Thank you for using nginx.</em></p> </body> </html>

      With that, you have successfully set up a new DigitalOcean Kubernetes cluster, deployed a load balanced Kubernetes application to it, and given that application’s load balancer a stable domain name using DigitalOcean DNS, all in 60 lines of code and a pulumi up command.

      The next step will guide you through removing the resources if you no longer need them.

      Step 6 — Removing the Resources (Optional)

      Before concluding the tutorial, you may want to destroy all of the resources created above. This will ensure you don’t get charged for resources that aren’t being used. If you prefer to keep your application up and running, feel free to skip this step.

      Run the following command to destroy the resources. Be careful using this, as it cannot be undone!

      Just as with the up command, destroy displays a preview and prompt before taking action:

      Output

      Previewing destroy (dev): Type Name Plan - pulumi:pulumi:Stack do-k8s-dev delete - ├─ digitalocean:index:DnsRecord do-domain-cname delete - ├─ digitalocean:index:Domain do-domain delete - ├─ kubernetes:core:Service do-app-svc delete - ├─ kubernetes:apps:Deployment do-app-dep delete - ├─ pulumi:providers:kubernetes do-k8s delete - └─ digitalocean:index:KubernetesCluster do-cluster delete Resources: - 7 to delete Do you want to perform this destroy? yes > no details

      Assuming this is what you want, select yes and watch the deletions occur:

      Output

      Destroying (dev): Type Name Status - pulumi:pulumi:Stack do-k8s-dev deleted - ├─ digitalocean:index:DnsRecord do-domain-cname deleted - ├─ digitalocean:index:Domain do-domain deleted - ├─ kubernetes:core:Service do-app-svc deleted - ├─ kubernetes:apps:Deployment do-app-dep deleted - ├─ pulumi:providers:kubernetes do-k8s deleted - └─ digitalocean:index:KubernetesCluster do-cluster deleted Resources: - 7 deleted Duration: 7s Permalink: https://app.pulumi.com/…/do-k8s/dev/updates/4

      At this point, nothing remains: the DNS entries are gone and the Kubernetes cluster—along with the application running inside of it—are gone. The permalink is still available, so you can still go back and see the full history of updates for this stack. This could help you recover if the destruction was a mistake, since the service keeps full state history for all resources.

      If you’d like to destroy your project in its entirety, remove the stack:

      You will receive output asking you to confirm the deletion by typing in the stack’s name:

      Output

      This will permanently remove the 'dev' stack! Please confirm that this is what you'd like to do by typing ("dev"):

      Unlike the destroy command, which deletes the cloud infrastructure resources, the removal of a stack erases completely the full history of your stack from Pulumi’s purview.

      Conclusion

      In this tutorial, you’ve deployed DigitalOcean infrastructure resources—a Kubernetes cluster and a DNS domain with A and CNAME records—in addition to the Kubernetes application configuration that uses this cluster. You have done so using infrastructure-as-code written in a familiar programming language, TypeScript, that works with existing editors, tools, and libraries, and leverages existing communities and packages. You’ve done it all using a single command line workflow for doing deployments that span your application and infrastructure.

      From here, there are a number of next steps you might take:

      The entire sample from this tutorial is available on GitHub. For extensive details about how to use Pulumi infrastructure-as-code in your own projects today, check out the Pulumi Documentation, Tutorials, or Getting Started guides. Pulumi is open source and free to use.



      Source link

      How To Deploy and Manage Your DNS using OctoDNS on Ubuntu 18.04


      The author selected the Electronic Frontier Foundation to receive a donation as part of the Write for DOnations program.

      Introduction

      OctoDNS is an infrastructure-as-code tool that allows you to deploy and manage your DNS zones using standard software development principles, including version control, testing, and automated deployment. OctoDNS was created by GitHub and is written in Python.

      Using OctoDNS eliminates many of the pitfalls of manual DNS management, as zone files are stored in a structured format (YAML). This allows you to deploy zones to multiple DNS providers simultaneously, identify syntax errors, and push out your DNS configuration automatically, reducing the risk of human error. Another common usage of OctoDNS is to synchronize your DNS configuration between different providers, such as a testing and production system, or between live and failover environments.

      OctoDNS is similar to DNSControl, which is an equivalent tool created by Stack Exchange and written in Go. Unlike OctoDNS, DNSControl uses a JavaScript-based configuration language for defining DNS zones, which allows you to use advanced programmatic features such as loops to specify multiple similar records within the same zone. The article How to Deploy and Manage Your DNS Using DNSControl on Ubuntu 18.04 covers the basic setup and configuration of DNSControl.

      In this tutorial, you’ll install and configure OctoDNS, create a basic DNS configuration, and begin deploying DNS records to a live provider. As part of this tutorial, we will use DigitalOcean as the example DNS provider. If you wish to use a different provider, the setup is very similar. When you’re finished, you’ll be able to manage and test your DNS configuration in a safe, offline environment, and then automatically deploy it to production.

      Prerequisites

      Before you begin this guide you’ll need the following:

      • One Ubuntu 18.04 server set up by following the Initial Server Setup with Ubuntu 18.04, including a sudo non-root user and enabled firewall to block non-essential ports. your-server-ipv4-address and your-server-ipv6-address refer to the IP addresses of the server where you’re hosting your website or domain.
      • A fully registered domain name with DNS hosted by a supported provider. This tutorial will use your-domain throughout and DigitalOcean as the service provider.
      • A DigitalOcean API key (Personal Access Token) with read and write permissions. To create one, visit How to Create a Personal Access Token.

      Once you have these ready, log in to your server as your non-root user to begin.

      Step 1 — Installing OctoDNS

      OctoDNS is distributed as a Python pip package, and runs in a Python Virtual Environment (virtualenv), so you’ll start this step by installing the packages required for this. A virtualenv is an isolated Python environment that can have its own libraries and configuration, separate from the main system-wide Python installation. Python and virtualenv are available within Ubuntu’s default software repositories, making it possible to install using conventional package management tools.

      Begin by updating the local package index to reflect any new upstream changes:

      Then, install the python and virtualenv packages:

      • sudo apt install python virtualenv

      After confirming the installation, apt will download and install Python, virtualenv, and all of their required dependencies.

      Next, you'll create the required directories for OctoDNS, where your DNS and program configuration will be stored. Start by creating the ~/octodns and ~/octodns/config directories:

      • mkdir ~/octodns ~/octodns/config

      Now move into ~/octodns:

      Next, you need to create the Python Virtual Environment—an isolated Python environment with its own libraries and configuration to run OctoDNS in:

      Activate your environment with the following command:

      This will output something similar to the following:

      Output

      Running virtualenv with interpreter /usr/bin/python2 New python executable in /home/user/octodns/env/bin/python2 Also creating executable in /home/user/octodns/env/bin/python Installing setuptools, pkg_resources, pip, wheel...done.

      Your Bash shell prompt will now also be prefixed with the name of the virtual environment. This shows that you are currently operating within the virtualenv:

      (env) user@digitalocean:~/octodns$
      

      If you wish to exit the virtualenv, you can use the deactivate command at any time. However, you should stay in your virtualenv to continue with this tutorial.

      Now that you've installed and configured Python and virtualenv, you can install OctoDNS. OctoDNS is distributed as a Python pip package, which is the standard package-management tool for Python packages and libraries.

      You can install the OctoDNS pip package using the following command within your virtualenv:

      Once this is complete, you can check the installed version to make sure that everything is working:

      Your output will look similar to the following:

      Output

      octoDNS 0.9.6

      If you see a octodns-sync: command not found error, double-check that you're still inside your virtualenv.

      Now that you've installed OctoDNS, you can create the required configuration files to connect OctoDNS to your DNS provider to allow it to make changes to your DNS records.

      Step 2 — Configuring OctoDNS

      In this step, you'll create the required configuration files for OctoDNS, and connect it to your DNS provider so that it can begin to make live changes to your DNS records.

      Note: This tutorial will focus on the initial setup of OctoDNS; however for production use it is recommended to store your OctoDNS configuration in a version control system (VCS) such as Git. The advantages of this include full version control, integration with CI/CD for testing, seamlessly rolling-back deployments, and so on.

      Firstly, you need to configure the config.yaml file, which defines the DNS zones for OctoDNS to manage, and allows it to authenticate to your DNS provider and make changes.

      The format of config.yaml differs slightly depending on the DNS provider that you are using. Please see the Supported Providers list in the official OctoDNS documentation to find the configuration for your own provider. When viewing this hyperlink, the configuration details are presented as a code comment in the actual Python code for your provider, which is linked in the 'Provider' column of the table. Once you have found the Python code for your provider, such as cloudflare.py or route53.py, the relevant code comment can be found directly under the class ProviderNameProvider. For example:

      Excerpt of octodns/provider/route53.py

      class Route53Provider(BaseProvider):
          '''
          AWS Route53 Provider
          route53:
              class: octodns.provider.route53.Route53Provider
              # The AWS access key id
              access_key_id:
              # The AWS secret access key
              secret_access_key:
              # The AWS session token (optional)
              # Only needed if using temporary security credentials
              session_token:
      

      Move into the ~/octodns/config directory:

      Then create and open config.yaml for editing:

      Add the sample config.yaml configuration for your DNS provider to the file. If you're using DigitalOcean as your DNS provider, you can use the following:

      ~/octodns/config/config.yaml

      ---
      providers:
        config:
          class: octodns.provider.yaml.YamlProvider
          directory: ./config
          default_ttl: 300
          enforce_order: True
        digitalocean:
          class: octodns.provider.digitalocean.DigitalOceanProvider
          token: your-digitalocean-oauth-token
      
      zones:
        your-domain.:
          sources:
            - config
          targets:
            - digitalocean
      

      This file tells OctoDNS which DNS providers you want it to connect to, and which DNS zones it should manage for those providers.

      You'll need to provide some form of authentication for your DNS provider. This is usually an API key or OAuth token.

      If you do not wish to store your access token in plain text in the configuration file, you can instead pass it as an environment variable when the program runs. To do this, you should use the following token: line instead in config.yaml:

      ~/octodns/config/config.yaml

      token: env/DIGITALOCEAN_OAUTH_TOKEN
      

      Then, before running OctoDNS, set the relevant environment variable to your access token, and OctoDNS will read it from there when run:

      • export DIGITALOCEAN_OAUTH_TOKEN=your-digitalocean-oauth-token

      Warning: This token will grant access to your DNS provider account, so you should protect it as you would a password. Also, ensure that if you're using a version control system, either the file containing the token is excluded (e.g. using .gitignore), or is securely encrypted in some way.

      If you're using DigitalOcean as your DNS provider, you can use the required OAuth token in your DigitalOcean account settings that you generated as part of the prerequisites.

      If you have multiple different DNS providers—for example, for multiple domain names, or delegated DNS zones—you can define these all in the same config.yaml file.

      You've set up the initial OctoDNS configuration file to allow the program to authenticate to your DNS provider and make changes. Next you'll create the configuration for your DNS zones.

      Step 3 — Creating a DNS Configuration File

      In this step, you'll create an initial DNS configuration file, which will contain the DNS records for your domain name or delegated DNS zone.

      Each DNS zone that you want to manage using OctoDNS has its own file, for example your-domain.yaml. In this file, the DNS records for the zone are defined using YAML.

      To begin, move into the ~/octodns/config directory:

      Then create and open your-domain.yaml for editing:

      Add the following sample configuration to the file:

      ~/octodns/config/your-domain.yaml

      ---
      '':
        - type: A
          value: your-server-ipv4-address
      
      www:
        - type: A
          value: your-server-ipv4-address
      

      This sample file defines a DNS zone for your-domain with two A records, pointing to the IPv4 address that you're hosting your domain or website on. One A record is for the root domain (e.g. your-domain), and the other is for the www subdomain (e.g. www.your-domain).

      Once complete, save and close the file.

      You've set up a basic DNS zone configuration file for OctoDNS, with two basic A records pointing to the IPv4 address of your domain or website. Next, you'll expand the file with some useful DNS records.

      Step 4 — Populating Your DNS Configuration File

      Next, you can populate the DNS configuration file with a practical set of DNS records for your website or service, using the YAML structured configuration language.

      Unlike traditional BIND zone files, where DNS records are written in a raw, line-by-line format, DNS records within OctoDNS are defined as YAML keys and subkeys with a number of associated values, as shown briefly in Step 3.

      The top-level key is usually the 'name', which is essentially the record identifier. www, subdomain1, and mail are all examples of DNS 'name'. In OctoDNS, there are two special-use names, which are '', for the root record (usually referred to as @), and '*', for wildcard records. A required value of each key (DNS record) is type. This defines which type of DNS record you are defining within that YAML top-level key. A type exists for each of the standard DNS record types, including A, AAAA, MX, TXT, NS, CNAME, and so on. A full list of available record types is available in the Records section of the OctoDNS documentation.

      The values for your DNS records are defined either directly as values to the top-level keys (if you only have one value), or as a list (if you have multiple values, e.g. multiple IP addresses or MX addresses).

      For example, to define a single value, you could use the following configuration:

      ~/octodns/config/your-domain.yaml

      'www':
        type: A
        value: 203.0.113.1
      

      Alternatively, to define multiple values for a single record:

      ~/octodns/config/your-domain.yaml

      'www':
        type: A
        values:
        - 203.0.113.1
        - 203.0.113.2
      

      The syntax for setting DNS records varies slightly for each record type. Following are some examples for the most common record types:

      A records:

      Purpose: To point to an IPv4 address.

      Syntax:

      'name':
        type: A
        value: ipv4-address
      

      Example:

      'www':
        type: A
        value: your-server-ipv4-address
      

      AAAA records:

      Purpose: To point to an IPv6 address.

      Syntax:

      'name':
        type: AAAA
        value: ipv6-address
      

      Example:

      'www':
        type: AAAA
        value: your-server-ipv6-address
      

      CNAME records:

      Purpose: To make your domain/subdomain an alias of another.

      Syntax:

      'name':
        type: CNAME
        value: fully-qualified-domain-name
      

      Example:

      'www':
        type: CNAME
        value: www.example.org
      

      MX records:

      Purpose: To direct email to specific servers/addresses.

      Syntax:

      'name':
        type: MX
        value:
          exchange: mail-server
          preference: priority-value
      

      Note that a trailing . must be included if there are any dots in the MX value.

      Example:

      '':
        type: MX
        value:
          exchange: mail.your-domain.
          preference: 10
      

      TXT records:

      Purpose: To add arbitrary plain text, often used for configurations without their own dedicated record type.

      Syntax:

      'name':
        type: TXT
        value: content
      

      Example:

      '':
        type: TXT
        value: This is a TXT record.
      

      In order to begin adding DNS records for your domain or delegated DNS zone, edit your DNS configuration file:

      • cd ~/octodns/config
      • nano your-domain.yaml

      Next, you can begin populating your DNS zone using the syntax described in the previous list, as well as the Records section of the official OctoDNS documentation.

      For reference, the code block here contains a full sample configuration for an initial DNS setup:

      ~/octodns/config/your-domain.yaml

      ---
      '':
        - type: A
          value: your-server-ipv4-address
      
        - type: AAAA
          value: your-server-ipv6-address
      
        - type: MX
          value:
            exchange: mail.your-domain.
            preference: 10
      
        - type: TXT
          value: v=spf1 -all
      
      _dmarc:
        type: TXT
        value: v=DMARC1; p=reject; rua=mailto:abuse@your-domain; aspf=s; adkim=s;
      
      mail:
        - type: A
          value: your-server-ipv4-address
      
        - type: AAAA
          value: your-server-ipv6-address
      
      www:
        - type: A
          value: your-server-ipv4-address
      
        - type: AAAA
          value: your-server-ipv6-address
      

      Once you have completed your initial DNS configuration, save and close the file.

      In this step, you set up the initial DNS configuration file, containing your DNS records. Next, you will test the configuration and deploy it.

      Step 5 — Testing and Deploying Your DNS Configuration

      In this step, you will run a local syntax check on your DNS configuration, and then deploy the changes to the live DNS server/provider.

      Firstly, move into your octodns directory:

      Double check that you're still operating within your Python virtualenv by looking for the name of it before your Bash prompt:

      (env) user@digitalocean:~/octodns$
      

      Next, use the octodns-validate command to check the syntax of your configuration file(s). You'll need to specify the path to your configuration file:

      • octodns-validate --config=./config/config.yaml

      If the YAML syntax of your DNS configuration file is correct, OctoDNS will return with no output.

      If you see an error or warning in your output, OctoDNS will provide details on what and where the error is located within your YAML file.

      Next, you can perform a dry-run push of the DNS configuration, which will output which changes will be made, without actually making them:

      • octodns-sync --config=./config/config.yaml

      This should produce an output similar to the following:

      Output

      ******************************************************************************** * your-domain. ******************************************************************************** * digitalocean (DigitalOceanProvider) * Create <ARecord A 300, mail.your-domain., ['your-server-ipv4-address']> (config) * Create <AaaaRecord AAAA 300, mail.your-domain., ['your-server-ipv6-address']> (config) * Create <TxtRecord TXT 300, your-domain., ['v=spf1 -all']> (config) * Create <AaaaRecord AAAA 300, your-domain., ['your-server-ipv6-address']> (config) * Create <ARecord A 300, your-domain., ['your-server-ipv4-address']> (config) * Create <ARecord A 300, www.your-domain., ['your-server-ipv4-address']> (config) * Create <MxRecord MX 300, your-domain., [''10 mail.your-domain.'']> (config) * Create <TxtRecord TXT 300, _dmarc.your-domain., ['v=DMARC1; p=reject; rua=mailto:abuse@your-domain; aspf=s; adkim=s;']> (config) * Create <AaaaRecord AAAA 300, www.your-domain., ['your-server-ipv6-address']> (config) * Summary: Creates=9, Updates=0, Deletes=0, Existing Records=2 ********************************************************************************

      Warning: The next command will make live changes to your DNS records and possibly other settings. Please ensure that you are prepared for this, including taking a backup of your existing DNS configuration, as well as ensuring that you have the means to roll back if needed.

      Finally, you can push out the changes to your live DNS provider:

      • octodns-sync --config=./config/config.yaml --doit

      Note: In some cases, OctoDNS will refuse to push changes if it is making a significant number of adjustments. This is an automatic protection feature to prevent accidental misconfigurations. If you encounter this refusal, you can re-run octodns-sync using the --force option, but please ensure you are ready to do so.

      You'll see an output like the dry-run earlier in this step, but with the addition of something similar to the following:

      Output

      2019-07-07T23:17:27 INFO DigitalOceanProvider[digitalocean] apply: making changes 2019-07-07T23:17:30 INFO Manager sync: 9 total changes

      Now, if you check the DNS settings for your domain in the DigitalOcean control panel, you'll see the changes.

      A screenshot of the DigitalOcean control panel, showing some of the DNS changes that OctoDNS has made.

      You can also check the record creation by running a DNS query for your domain/delegated zone. You'll see that the records have been updated accordingly:

      You'll see output showing the IP address and relevant DNS record from your zone that was deployed using OctoDNS. DNS records can take some time to propagate, so you may need to wait and run this command again.

      In this final step, you ran a local syntax check of the DNS configuration file, then deployed it to your live DNS provider, and tested that the changes were made successfully.

      Conclusion

      In this article you set up OctoDNS and deployed a DNS configuration to a live provider. Now you can manage and test your DNS configuration changes in a safe, offline environment before deploying them to production.

      If you wish to explore this subject further, OctoDNS is designed to be integrated into your CI/CD pipeline, allowing you to run in-depth tests and have more control over your deployment to production. You could also look into integrating OctoDNS into your infrastructure build/deployment processes, allowing you to deploy servers and add them to DNS completely automatically.

      If you wish to go further with OctoDNS, the following DigitalOcean articles provide some interesting next steps to help integrate OctoDNS into your change management and infrastructure deployment workflows:



      Source link