One place for hosting & domains

      Package

      How To Use the Flag Package in Go


      Introduction

      Command-line utilities are rarely useful out of the box without additional configuration. Good defaults are important, but useful utilities need to accept configuration from users. On most platforms, command-line utilities accept flags to customize the command’s execution. Flags are key-value delimited strings added after the name of the command. Go lets you craft command-line utilities that accept flags by using the flag package from the standard library.

      In this tutorial you’ll explore various ways to use the flag package to build different kinds of command-line utilities. You’ll use a flag to control program output, introduce positional arguments where you mix flags and other data, and then implement sub-commands.

      Using a Flag to Change a Program’s Behavior

      Using the flag package involves three steps: First, define variables to capture flag values, then define the flags your Go application will use, and finally, parse the flags provided to the application upon execution. Most of the functions within the flag package are concerned with defining flags and binding them to variables that you have defined. The parsing phase is handled by the Parse() function.

      To illustrate, you’ll create a program that defines a Boolean flag that changes the message that will be printed to standard output. If there’s a -color flag provided, the program will print a message in blue. If no flag is provided, the message will be printed without any color.

      Create a new file called boolean.go:

      Add the following code to the file to create the program:

      boolean.go

      package main
      
      import (
          "flag"
          "fmt"
      )
      
      type Color string
      
      const (
          ColorBlack  Color = "u001b[30m"
          ColorRed          = "u001b[31m"
          ColorGreen        = "u001b[32m"
          ColorYellow       = "u001b[33m"
          ColorBlue         = "u001b[34m"
          ColorReset        = "u001b[0m"
      )
      
      func colorize(color Color, message string) {
          fmt.Println(string(color), message, string(ColorReset))
      }
      
      func main() {
          useColor := flag.Bool("color", false, "display colorized output")
          flag.Parse()
      
          if *useColor {
              colorize(ColorBlue, "Hello, DigitalOcean!")
              return
          }
          fmt.Println("Hello, DigitalOcean!")
      }
      

      This example uses ANSI Escape Sequences to instruct the terminal to display colorized output. These are specialized sequences of characters, so it makes sense to define a new type for them. In this example, we’ve called that type Color, and defined the type as a string. We then define a palette of colors to use in the const block that follows. The colorize function defined after the const block accepts one of these Color constants and a string variable for the message to colorize. It then instructs the terminal to change color by first printing the escape sequence for the color requested, then prints the message, and finally requests that the terminal reset its color by printing the special color reset sequence.

      Within main, we use the flag.Bool function to define a Boolean flag called color. The second parameter to this function, false, sets the default value for this flag when it is not provided. Contrary to expectations you may have, setting this to true does not invert the behavior such that providing a flag will cause it to become false. Consequently, the value of this parameter is almost always false with Boolean flags.

      The final parameter is a string of documentation that can be printed as a usage message. The value returned from this function is a pointer to a bool. The flag.Parse function on the next line uses this pointer to set the bool variable based on the flags passed in by the user. We are then able to check the value of this bool pointer by dereferencing the pointer. More information about pointer variables can be found in the tutorial on pointers. Using this Boolean value, we can then call colorize when the -color flag is set, and call the fmt.Println variable when the flag is absent.

      Save the file and run the program without any flags:

      You’ll see the following output:

      Output

      Hello, DigitalOcean!

      Now run this program again with the -color flag:

      The output will be the same text, but this time in the color blue.

      Flags are not the only values passed to commands. You might also send file names or other data.

      Working with Positional Arguments

      Typically commands will take a number of arguments that act as the subject of the command’s focus. For example, the head command, which prints the first lines of a file, is often invoked as head example.txt. The file example.txt is a positional argument in the invocation of the head command.

      The Parse() function will continue to parse flags that it encounters until it detects a non-flag argument. The flag package makes these available through the Args() and Arg() functions.

      To illustrate this, you’ll build a simplified re-implementation of the head command, which displays the first several lines of a given file:

      Create a new file called head.go and add the following code:

      head.go

      package main
      
      import (
          "bufio"
          "flag"
          "fmt"
          "io"
          "os"
      )
      
      func main() {
          var count int
          flag.IntVar(&count, "n", 5, "number of lines to read from the file")
          flag.Parse()
      
          var in io.Reader
          if filename := flag.Arg(0); filename != "" {
              f, err := os.Open(filename)
              if err != nil {
                  fmt.Println("error opening file: err:", err)
                  os.Exit(1)
              }
              defer f.Close()
      
              in = f
          } else {
              in = os.Stdin
          }
      
          buf := bufio.NewScanner(in)
      
          for i := 0; i < count; i++ {
              if !buf.Scan() {
                  break
              }
              fmt.Println(buf.Text())
          }
      
          if err := buf.Err(); err != nil {
              fmt.Fprintln(os.Stderr, "error reading: err:", err)
          }
      }
      

      First, we define a count variable to hold the number of lines the program should read from the file. We then define the -n flag using flag.IntVar, mirroring the behavior of the original head program. This function allows us to pass our own pointer to a variable in contrast to the flag functions that do not have the Var suffix. Apart from this difference, the rest of the parameters to flag.IntVar follow its flag.Int counterpart: the flag name, a default value, and a description. As in the previous example, we then call flag.Parse() to process the user’s input.

      The next section reads the file. We first define an io.Reader variable that will either be set to the file requested by the user, or standard input passed to the program. Within the if statement, we use the flag.Arg function to access the first positional argument after all flags. If the user supplied a file name, this will be set. Otherwise, it will be the empty string (""). When a filename is present, we use the os.Open function to open that file and set the io.Reader we defined before to that file. Otherwise, we use os.Stdin to read from standard input.

      The final section uses a *bufio.Scanner created with bufio.NewScanner to read lines from the io.Reader variable in. We iterate up to the value of count using a for loop, calling break if scanning the line with buf.Scan produces a false value, indicating that the number of lines is less than the number requested by the user.

      Run this program and display the contents of the file you just wrote by using head.go as the file argument:

      • go run head.go -- head.go

      The -- separator is a special flag recognized by the flag package which indicates that no more flag arguments follow. When you run this command, you receive the following output:

      Output

      package main import ( "bufio" "flag"

      Use the -n flag you defined to adjust the amount of output:

      • go run head.go -n 1 head.go

      This outputs only the package statement:

      Output

      package main

      Finally, when the program detects that no positional arguments were supplied, it reads input from standard input, just like head. Try running this command:

      • echo "fishnlobstersnsharksnminnows" | go run head.go -n 3

      You’ll see the output:

      Output

      fish lobsters sharks

      The behavior of the flag functions you’ve seen so far has been limited to examining the entire command invocation. You don’t always want this behavior, especially if you’re writing a command line tool that supports sub-commands.

      Using FlagSet to Implement Sub-commands

      Modern command-line applications often implement “sub-commands” to bundle a suite of tools under a single command. The most well-known tool that uses this pattern is git. When examining a command like git init, git is the command and init is the sub-command of git. One notable feature of sub-commands is that each sub-command can have its own collection of flags.

      Go applications can support sub-commands with their own set of flags using the flag.(*FlagSet) type. To illustrate this, create a program that implements a command using two sub-commands with different flags.

      Create a new file called subcommand.go and add the following content to the file:

      package main
      
      import (
          "errors"
          "flag"
          "fmt"
          "os"
      )
      
      func NewGreetCommand() *GreetCommand {
          gc := &GreetCommand{
              fs: flag.NewFlagSet("greet", flag.ContinueOnError),
          }
      
          gc.fs.StringVar(&gc.name, "name", "World", "name of the person to be greeted")
      
          return gc
      }
      
      type GreetCommand struct {
          fs *flag.FlagSet
      
          name string
      }
      
      func (g *GreetCommand) Name() string {
          return g.fs.Name()
      }
      
      func (g *GreetCommand) Init(args []string) error {
          return g.fs.Parse(args)
      }
      
      func (g *GreetCommand) Run() error {
          fmt.Println("Hello", g.name, "!")
          return nil
      }
      
      type Runner interface {
          Init([]string) error
          Run() error
          Name() string
      }
      
      func root(args []string) error {
          if len(args) < 1 {
              return errors.New("You must pass a sub-command")
          }
      
          cmds := []Runner{
              NewGreetCommand(),
          }
      
          subcommand := os.Args[1]
      
          for _, cmd := range cmds {
              if cmd.Name() == subcommand {
                  cmd.Init(os.Args[2:])
                  return cmd.Run()
              }
          }
      
          return fmt.Errorf("Unknown subcommand: %s", subcommand)
      }
      
      func main() {
          if err := root(os.Args[1:]); err != nil {
              fmt.Println(err)
              os.Exit(1)
          }
      }
      

      This program is divided into a few parts: the main function, the root function, and the individual functions to implement the sub-command. The main function handles errors returned from commands. If any function returns an error, the if statement will catch it, print the error, and the program will exit with a status code of 1, indicating that an error occurred to the rest of the operating system. Within main, we pass all of the arguments the program was invoked with to root. We remove the first argument, which is the name of the program (in the previous examples ./subcommand) by slicing os.Args first.

      The root function defines []Runner, where all sub-commands would be defined. Runner is an interface for sub-commands that allows root to retrieve the name of the sub-command using Name() and compare it against the contents subcommand variable. Once the correct sub-command is located after iterating through the cmds variable we initialize the sub-command with the rest of the arguments and invoke that command’s Run() method.

      We only define one sub-command, though this framework would easily allow us to create others. The GreetCommand is instantiated using NewGreetCommand where we create a new *flag.FlagSet using flag.NewFlagSet. flag.NewFlagSet takes two arguments: a name for the flag set, and a strategy for reporting parsing errors. The *flag.FlagSet’s name is accessible using the flag.(*FlagSet).Name method. We use this in the (*GreetCommand).Name() method so the name of the sub-command matches the name we gave to the *flag.FlagSet. NewGreetCommand also defines a -name flag in a similar way to previous examples, but it instead calls this as a method off the *flag.FlagSet field of the *GreetCommand, gc.fs. When root calls the Init() method of the *GreetCommand, we pass the arguments provided to the Parse method of the *flag.FlagSet field.

      It will be easier to see sub-commands if you build this program and then run it. Build the program:

      Now run the program with no arguments:

      You’ll see this output:

      Output

      You must pass a sub-command

      Now run the program with the greet sub-command:

      This produces the following output:

      Output

      Hello World !

      Now use the -name flag with greet to specify a name:

      • ./subcommand greet -name Sammy

      You’ll see this output from the program:

      Output

      Hello Sammy !

      This example illustrates some principles behind how larger command line applications could be structured in Go. FlagSets are designed to give developers more control over where and how flags are processed by the flag parsing logic.

      Conclusion

      Flags make your applications more useful in more contexts because they give your users control over how the programs execute. It’s important to give users useful defaults, but you should give them the opportunity to override settings that don’t work for their situation. You’ve seen that the flag package offers flexible choices to present configuration options to your users. You can choose a few simple flags, or build an extensible suite of sub-commands. In either case, using the flag package will help you build utilities in the style of the long history of flexible and scriptable command line tools.

      To learn more about the Go programming language, check out our full How To Code in Go series.



      Source link

      Understanding Package Visibility in Go


      Introduction

      When creating a package in Go, the end goal is usually to make the package accessible for other developers to use, either in higher order packages or whole programs. By importing the package, your piece of code can serve as the building block for other, more complex tools. However, only certain packages are available for importing. This is determined by the visibility of the package.

      Visibility in this context means the file space from which a package or other construct can be referenced. For example, if we define a variable in a function, the visibility (scope) of that variable is only within the function in which it was defined. Similarly, if you define a variable in a package, you can make it visible to just that package, or allow it to be visible outside the package as well.

      Carefully controlling package visibility is important when writing ergonomic code, especially when accounting for future changes that you may want to make to your package. If you need to fix a bug, improve performance, or change functionality, you’ll want to make the change in a way that won’t break the code of anyone using your package. One way to minimize breaking changes is to allow access only to the parts of your package that are needed for it to be used properly. By limiting access, you can make changes internally to your package with less of a chance of affecting how other developers are using your package.

      In this article, you will learn how to control package visibility, as well as how to protect parts of your code that should only be used inside your package. To do this, we will create a basic logger to log and debug messages, using packages with varying degrees of item visibility.

      Prerequisites

      To follow the examples in this article, you will need:

      .
      ├── bin 
      │ 
      └── src
          └── github.com
              └── gopherguides
      

      Exported and Unexported Items

      Unlike other program languages like Java and Python that use access modifiers such as public, private, or protected to specify scope, Go determines if an item is exported and unexported through how it is declared. Exporting an item in this case makes it visible outside the current package. If it’s not exported, it is only visible and usable from within the package it was defined.

      This external visibility is controlled by capitalizing the first letter of the item declared. All declarations, such as Types, Variables, Constants, Functions, etc., that start with a capital letter are visible outside the current package.

      Let’s look at the following code, paying careful attention to capitalization:

      greet.go

      package greet
      
      import "fmt"
      
      var Greeting string
      
      func Hello(name string) string {
          return fmt.Sprintf(Greeting, name)
      }
      

      This code declares that it is in the greet package. It then declares two symbols, a variable called Greeting, and a function called Hello. Because they both start with a capital letter, they are both exported and available to any outside program. As stated earlier, crafting a package that limits access will allow for better API design and make it easier to update your package internally without breaking anyone’s code that is depending on your package.

      Defining Package Visibility

      To give a closer look at how package visibility works in a program, let’s create a logging package, keeping in mind what we want to make visible outside our package and what we won’t make visible. This logging package will be responsible for logging any of our program messages to the console. It will also look at what level we are logging at. A level describes the type of log, and is going to be one of three statuses: info, warning, or error.

      First, within your src directory, let’s create a directory called logging to put our logging files in:

      Move into that directory next:

      Then, using an editor like nano, create a file called logging.go:

      Place the following code in the logging.go file we just created:

      logging/logging.go

      package logging
      
      import (
          "fmt"
          "time"
      )
      
      var debug bool
      
      func Debug(b bool) {
          debug = b
      }
      
      func Log(statement string) {
          if !debug {
              return
          }
      
          fmt.Printf("%s %sn", time.Now().Format(time.RFC3339), statement)
      }
      

      The first line of this code declared a package called logging. In this package, there are two exported functions: Debug and Log. These functions can be called by any other package that imports the logging package. There is also a private variable called debug. This variable is only accessible from within the logging package. It is important to note that while the function Debug and the variable debug both have the same spelling, the function is capitalized and the variable is not. This makes them distinct declarations with different scopes.

      Save and quit the file.

      To use this package in other areas of our code, we can import it into a new package. We’ll create this new package, but we’ll need a new directory to store those source files in first.

      Let’s move out of the logging directory, create a new directory called cmd, and move into that new directory:

      Create a file called main.go in the cmd directory we just created:

      Now we can add the following code:

      cmd/main.go

      package main
      
      import "github.com/gopherguides/logging"
      
      func main() {
          logging.Debug(true)
      
          logging.Log("This is a debug statement...")
      }
      

      We now have our entire program written. However, before we can run this program, we’ll need to also create a couple of configuration files for our code to work properly. Go uses Go Modules to configure package dependencies for importing resources. Go modules are configuration files placed in your package directory that tell the compiler where to import packages from. While learning about modules is beyond the scope of this article, we can write just a couple lines of configuration to make this example work locally.

      Open the following go.mod file in the cmd directory:

      Then place the following contents in the file:

      go.mod

      module github.com/gopherguides/cmd
      
      replace github.com/gopherguides/logging => ../logging
      

      The first line of this file tells the compiler that the cmd package has a file path of github.com/gopherguides/cmd. The second line tells the compiler that the package github.com/gopherguides/logging can be found locally on disk in the ../logging directory.

      We’ll also need a go.mod file for our logging package. Let’s move back into the logging directory and create a go.mod file:

      • cd ../logging
      • nano go.mod

      Add the following contents to the file:

      go.mod

      module github.com/gopherguides/logging
      

      This tells the compiler that the logging package we created is actually the github.com/gopherguides/logging package. This makes it possible to import the package in our main package with the following line that we wrote earlier:

      cmd/main.go

      package main
      
      import "github.com/gopherguides/logging"
      
      func main() {
          logging.Debug(true)
      
          logging.Log("This is a debug statement...")
      }
      

      You should now have the following directory structure and file layout:

      ├── cmd
      │   ├── go.mod
      │   └── main.go
      └── logging
          ├── go.mod
          └── logging.go
      

      Now that we have all the configuration completed, we can run the main program from the cmd package with the following commands:

      You will get output similar to the following:

      Output

      2019-08-28T11:36:09-05:00 This is a debug statement...

      The program will print out the current time in RFC 3339 format followed by whatever statement we sent to the logger. RFC 3339 is a time format that was designed to represent time on the internet and is commonly used in log files.

      Because the Debug and Log functions are exported from the logging package, we can use them in our main package. However, the debug variable in the logging package is not exported. Trying to reference an unexported declaration will result in a compile-time error.

      Add the following highlighted line to main.go:

      cmd/main.go

      package main
      
      import "github.com/gopherguides/logging"
      
      func main() {
          logging.Debug(true)
      
          logging.Log("This is a debug statement...")
      
          fmt.Println(logging.debug)
      }
      

      Save and run the file. You will receive an error similar to the following:

      Output

      . . . ./main.go:10:14: cannot refer to unexported name logging.debug

      Now that we have seen how exported and unexported items in packages behave, we will next look at how fields and methods can be exported from structs.

      Visibility Within Structs

      While the visibility scheme in the logger we built in the last section may work for simple programs, it shares too much state to be useful from within multiple packages. This is because the exported variables are accessible to multiple packages that could modify the variables into contradictory states. Allowing the state of your package to be changed in this way makes it hard to predict how your program will behave. With the current design, for example, one package could set the Debug variable to true, and another could set it to false in the same instance. This would create a problem since both packages that are importing the logging package are affected.

      We can make the logger isolated by creating a struct and then hanging methods off of it. This will allow us to create an instance of a logger to be used independently in each package that consumes it.

      Change the logging package to the following to refactor the code and isolate the logger:

      logging/logging.go

      package logging
      
      import (
          "fmt"
          "time"
      )
      
      type Logger struct {
          timeFormat string
          debug      bool
      }
      
      func New(timeFormat string, debug bool) *Logger {
          return &Logger{
              timeFormat: timeFormat,
              debug:      debug,
          }
      }
      
      func (l *Logger) Log(s string) {
          if !l.debug {
              return
          }
          fmt.Printf("%s %sn", time.Now().Format(l.timeFormat), s)
      }
      

      In this code, we created a Logger struct. This struct will house our unexported state, including the time format to print out and the debug variable setting of true or false. The New function sets the initial state to create the logger with, such as the time format and debug state. It then stores the values we gave it internally to the unexported variables timeFormat and debug. We also created a method called Log on the Logger type that takes a statement we want to print out. Within the Log method is a reference to its local method variable l to get access back to its internal fields such as l.timeFormat and l.debug.

      This approach will allow us to create a Logger in many different packages and use it independently of how the other packages are using it.

      To use it in another package, let’s alter cmd/main.go to look like the following:

      cmd/main.go

      package main
      
      import (
          "time"
      
          "github.com/gopherguides/logging"
      )
      
      func main() {
          logger := logging.New(time.RFC3339, true)
      
          logger.Log("This is a debug statement...")
      }
      

      Running this program will give you the following output:

      Output

      2019-08-28T11:56:49-05:00 This is a debug statement...

      In this code, we created an instance of the logger by calling the exported function New. We stored the reference to this instance in the logger variable. We can now call logging.Log to print out statements.

      If we try to reference an unexported field from the Logger such as the timeFormat field, we will receive a compile-time error. Try adding the following highlighted line and running cmd/main.go:

      cmd/main.go

      
      package main
      
      import (
          "time"
      
          "github.com/gopherguides/logging"
      )
      
      func main() {
          logger := logging.New(time.RFC3339, true)
      
          logger.Log("This is a debug statement...")
      
          fmt.Println(logger.timeFormat)
      }
      

      This will give the following error:

      Output

      . . . cmd/main.go:14:20: logger.timeFormat undefined (cannot refer to unexported field or method timeFormat)

      The compiler recognizes that logger.timeFormat is not exported, and therefore can’t be retrieved from the logging package.

      Visibility Within Methods

      In the same way as struct fields, methods can also be exported or unexported.

      To illustrate this, let’s add leveled logging to our logger. Leveled logging is a means of categorizing your logs so that you can search your logs for specific types of events. The levels we will put into our logger are:

      • The info level, which represents information type events that inform the user of an action, such as Program started, or Email sent. These help us debug and track parts of our program to see if expected behavior is happening.

      • The warning level. These types of events identify when something unexpected is happening that is not an error, like Email failed to send, retrying. They help us see parts of our program that aren’t going as smoothly as we expected them to.

      • The error level, which means the program encountered a problem, like File not found. This will often result in the program’s operation failing.

      You may also desire to turn on and off certain levels of logging, especially if your program isn’t performing as expected and you’d like to debug the program. We’ll add this functionality by changing the program so that when debug is set to true, it will print all levels of messages. Otherwise, if it’s false, it will only print error messages.

      Add leveled logging by making the following changes to logging/logging.go:

      logging/logging.go

      
      package logging
      
      import (
          "fmt"
          "strings"
          "time"
      )
      
      type Logger struct {
          timeFormat string
          debug      bool
      }
      
      func New(timeFormat string, debug bool) *Logger {
          return &Logger{
              timeFormat: timeFormat,
              debug:      debug,
          }
      }
      
      func (l *Logger) Log(level string, s string) {
          level = strings.ToLower(level)
          switch level {
          case "info", "warning":
              if l.debug {
                  l.write(level, s)
              }
          default:
              l.write(level, s)
          }
      }
      
      func (l *Logger) write(level string, s string) {
          fmt.Printf("[%s] %s %sn", level, time.Now().Format(l.timeFormat), s)
      }
      

      In this example, we introduced a new argument to the Log method. We can now pass in the level of the log message. The Log method determines what level of message it is. If it’s an info or warning message, and the debug field is true, then it writes the message. Otherwise it ignores the message. If it is any other level, like error, it will write out the message regardless.

      Most of the logic for determining if the message is printed out exists in the Log method. We also introduced an unexported method called write. The write method is what actually outputs the log message.

      We can now use this leveled logging in our other package by changing cmd/main.go to look like the following:

      cmd/main.go

      package main
      
      import (
          "time"
      
          "github.com/gopherguides/logging"
      )
      
      func main() {
          logger := logging.New(time.RFC3339, true)
      
          logger.Log("info", "starting up service")
          logger.Log("warning", "no tasks found")
          logger.Log("error", "exiting: no work performed")
      
      }
      

      Running this will give you:

      Output

      [info] 2019-09-23T20:53:38Z starting up service [warning] 2019-09-23T20:53:38Z no tasks found [error] 2019-09-23T20:53:38Z exiting: no work performed

      In this example, cmd/main.go successfully used the exported Log method.

      We can now pass in the level of each message by switching debug to false:

      main.go

      package main
      
      import (
          "time"
      
          "github.com/gopherguides/logging"
      )
      
      func main() {
          logger := logging.New(time.RFC3339, false)
      
          logger.Log("info", "starting up service")
          logger.Log("warning", "no tasks found")
          logger.Log("error", "exiting: no work performed")
      
      }
      

      Now we will see that only the error level messages print:

      Output

      [error] 2019-08-28T13:58:52-05:00 exiting: no work performed

      If we try to call the write method from outside the logging package, we will receive a compile-time error:

      main.go

      package main
      
      import (
          "time"
      
          "github.com/gopherguides/logging"
      )
      
      func main() {
          logger := logging.New(time.RFC3339, true)
      
          logger.Log("info", "starting up service")
          logger.Log("warning", "no tasks found")
          logger.Log("error", "exiting: no work performed")
      
          logger.write("error", "log this message...")
      }
      

      Output

      cmd/main.go:16:8: logger.write undefined (cannot refer to unexported field or method logging.(*Logger).write)

      When the compiler sees that you are trying to reference something from another package that starts with a lowercase letter, it knows that it is not exported, and therefore throws a compiler error.

      The logger in this tutorial illustrates how we can write code that only exposes the parts we want other packages to consume. Because we control what parts of the package are visible outside the package, we are now able to make future changes without affecting any code that depends on our package. For example, if we wanted to only turn off info level messages when debug is false, you could make this change without affecting any other part of your API. We could also safely make changes to the log message to include more information, such as the directory the program was running from.

      Conclusion

      This article showed how to share code between packages while also protecting the implementation details of your package. This allows you to export a simple API that will seldom change for backwards compatibility, but will allow for changes privately in your package as needed to make it work better in the future. This is considered a best practice when creating packages and their corresponding APIs.

      To learn more about packages in Go, check out our Importing Packages in Go and How To Write Packages in Go articles, or explore our entire How To Code in Go series.



      Source link

      Using the context Go package


      Updated by Linode Contributed by Mihalis Tsoukalos

      Go is a compiled, statically typed programming language developed by Google. Many modern applications, including Docker, Kubernetes, and Caddy, are written in Go.

      Running a go command is as simple as:

      go run [filename]
      

      The context package provides contextual information that a goroutine may need such as how long it should run and how and when it should end. It can also pass informational key-value pairs for use down the call chain.

      In this guide you will learn:

      Before You Begin

      You will need to install a recent version of Go on your computer in order to follow the presented commands. Any Go version newer than 1.8 will do but it is considered a good practice to have the latest version of Go installed. You can check your Go version by executing go version.

      If you still need to install Go, you can follow our guide for Ubuntu installation here.

      Note

      This guide is written for a non-root user. Depending on your configuration, some commands might require the help of sudo in order to get property executed. If you are not familiar with the sudo command, see the Users and Groups guide.

      About the context package

      The context package supports both the handling of multiple concurrent operations and the passing of (typically request-scoped) contextual data in key-value pairs.

      If you take a look at the source code of the context package, you will realize that its implementation is pretty simple. The context package defines the Context type, which is a Go interface with four methods, named Deadline(), Done(), Err(), and Value():

      context.go
      1
      2
      3
      4
      5
      6
      
      type Context interface {
          Deadline() (deadline time.Time, ok bool)
          Done() <-chan struct{}
          Err() error
          Value(key interface{}) interface{}
      }
      • The developer will need to declare and modify a Context variable using functions such as context.WithCancel(), context.WithDeadline() and context.WithTimeout().

      • All three of these functions return a derived Context (the child) and a CancelFunc function. Calling the CancelFunc function removes the parent’s reference to the child and stops any associated timers. This means that the Go garbage collector is free to garbage collect the child goroutines that no longer have associated parent goroutines.

      • For garbage collection, the parent goroutine needs to keep a reference to each child goroutine. If a child goroutine ends without the parent knowing about it, then a memory leak occurs until the parent is canceled as well.

      A simple example

      This first code example is relatively simple and illustrates the use of the context.Context type with the help of simple.go.

      Explaining the Go code of the Example

      The code of simple.go is as follows:

      ./simple.go
       1
       2
       3
       4
       5
       6
       7
       8
       9
      10
      11
      12
      13
      14
      15
      16
      17
      18
      19
      20
      21
      22
      23
      24
      25
      26
      27
      28
      29
      30
      31
      32
      33
      34
      35
      36
      37
      38
      39
      40
      41
      42
      43
      44
      45
      46
      47
      48
      49
      50
      51
      52
      53
      54
      55
      56
      57
      58
      59
      60
      61
      62
      63
      64
      65
      66
      67
      68
      69
      70
      71
      72
      73
      74
      75
      76
      77
      78
      79
      80
      81
      82
      83
      84
      85
      86
      87
      88
      89
      90
      91
      92
      93
      94
      
      package main
      
      import (
              "context"
              "fmt"
              "os"
              "strconv"
              "time"
      )
      
      // The f1 function creates and executes a goroutine
      // The time.Sleep() call simulates the time it would take a real goroutine
      // to do its job - in this case it is 4 seconds. If the c1 context calls
      // the Done() function in less than 4 seconds, the goroutine will not have
      // enough time to finish.
      func f1(t int) {
              c1 := context.Background()
              // WithCancel returns a copy of parent context with a new Done channel
              c1, cancel := context.WithCancel(c1)
              defer cancel()
      
              go func() {
                      time.Sleep(4 * time.Second)
                      cancel()
              }()
      
              select {
              case <-c1.Done():
                      fmt.Println("f1() Done:", c1.Err())
                      return
              case r := <-time.After(time.Duration(t) * time.Second):
                      fmt.Println("f1():", r)
              }
              return
      }
      
      func f2(t int) {
              c2 := context.Background()
              c2, cancel := context.WithTimeout(c2, time.Duration(t)*time.Second)
              defer cancel()
      
              go func() {
                      time.Sleep(4 * time.Second)
                      cancel()
              }()
      
              select {
              case <-c2.Done():
                      fmt.Println("f2() Done:", c2.Err())
                      return
              case r := <-time.After(time.Duration(t) * time.Second):
                      fmt.Println("f2():", r)
              }
              return
      }
      
      func f3(t int) {
              c3 := context.Background()
              deadline := time.Now().Add(time.Duration(2*t) * time.Second)
              c3, cancel := context.WithDeadline(c3, deadline)
              defer cancel()
      
              go func() {
                      time.Sleep(4 * time.Second)
                      cancel()
              }()
      
              select {
              case <-c3.Done():
                      fmt.Println("f3() Done:", c3.Err())
                      return
              case r := <-time.After(time.Duration(t) * time.Second):
                      fmt.Println("f3():", r)
              }
              return
      }
      
      func main() {
              if len(os.Args) != 2 {
                      fmt.Println("Need a delay!")
                      return
              }
      
              delay, err := strconv.Atoi(os.Args[1])
              if err != nil {
                      fmt.Println(err)
                      return
              }
              fmt.Println("Delay:", delay)
      
              f1(delay)
              f2(delay)
              f3(delay)
      }
      • The program contains four functions including the main() function. Functions f1(), f2(), and f3() each require just one parameter, which is a time delay, because everything else they need is defined inside their functions.

      • In this example we call the context.Background() function to initialize an empty Context. The other function that can create an empty Context is context.TODO() which will be presented later in this guide.

      • Notice that the cancel variable, a function, in f1() is one of the return values of context.CancelFunc(). The context.WithCancel() function uses an existing Context and creates a child with cancellation. The context.WithCancel() function also returns a Done channel that can be closed, either when the cancel() function is called, as shown in the preceding code, or when the Done channel of the parent context is closed.

        Note

        One of the return values of Context.Done() is a Go channel, which means that you will have to use a select statement to work with. Although select looks like switch, select allows a goroutine to wait on multiple communications operations.

      • The cancel variable in f2() comes from context.WithTimeout(). context.WithTimeout() requires two parameters: a Context parameter and a time.Duration parameter. When the timeout period expires, the cancel() function is called automatically.

      • The cancel variable in f3() comes from context.WithDeadline(). context.WithDeadline() requires two parameters: a Context variable and a time in the future that signifies the deadline of the operation. When the deadline passes, the cancel() function is called automatically.

        Note

        Notice that contexts should not be stored in structures – they should be passed as separate parameters to functions. It is considered a good practice to pass them as the first parameter of a function.

      Using simple.go

      Execute simple.go with a delay period of 3 seconds:

      go run simple.go 3
      

      It will generate the following kind of output:

        
      go run simple.go 3
      Delay: 3
      f1(): 2019-05-31 19:29:38.664568 +0300 EEST m=+3.004314767
      f2(): 2019-05-31 19:29:41.664942 +0300 EEST m=+6.004810929
      f3(): 2019-05-31 19:29:44.668795 +0300 EEST m=+9.008786881
      
      

      The long lines of the output are the return values from the time.After() function. They denote normal operation of the program. The point here is that the operation of the program is canceled when there are delays in its execution.

      If you use a bigger delay (10 seconds), which is executed as a call to time.Sleep():

      go run simple.go 10
      

      You will get the following kind of output:

        
      Delay: 10
      f1() Done: context canceled
      f2() Done: context canceled
      f3() Done: context canceled
      
      

      The calls to time.Sleep() simulate a program that is slow or an operation that takes too much time to finish. Production code does not usually have such time.Sleep() function calls.

      Using Context for HTTP

      In this section of the guide you will learn how to timeout HTTP connections on the client side.

      Note

      This example makes a request to a local web server. A suitable, simple web server is available via Python and can be started with the following commands:

      Python 3.X

      python3 -m http.server
      

      Python 2.X

      python -m SimpleHTTPServer
      

      The presented utility, which is called http.go, requires two command line arguments, which are the URL to connect to and the allowed delay value in seconds.

      Explaining the Go Code of the Example

      The Go code of the http.go utility is the following:

      ./http.go
       1
       2
       3
       4
       5
       6
       7
       8
       9
      10
      11
      12
      13
      14
      15
      16
      17
      18
      19
      20
      21
      22
      23
      24
      25
      26
      27
      28
      29
      30
      31
      32
      33
      34
      35
      36
      37
      38
      39
      40
      41
      42
      43
      44
      45
      46
      47
      48
      49
      50
      51
      52
      53
      54
      55
      56
      57
      58
      59
      60
      61
      62
      63
      64
      65
      66
      67
      68
      69
      70
      71
      72
      73
      74
      75
      76
      77
      78
      79
      80
      81
      82
      83
      84
      85
      86
      87
      88
      89
      90
      91
      92
      93
      94
      95
      96
      97
      98
      99
      
      package main
      
      import (
              "context"
              "fmt"
              "io/ioutil"
              "net/http"
              "os"
              "strconv"
              "sync"
              "time"
      )
      
      var (
              myUrl string
              delay int = 5
              w     sync.WaitGroup
      )
      
      type myData struct {
              r   *http.Response
              err error
      }
      
      // In packages that use contexts, convention is to pass them as
      // the first argument to a function.
      func connect(c context.Context) error {
              defer w.Done()
              data := make(chan myData, 1)
              tr := &http.Transport{}
              httpClient := &http.Client{Transport: tr}
              req, _ := http.NewRequest("GET", myUrl, nil)
      
              go func() {
                      response, err := httpClient.Do(req)
                      if err != nil {
                              fmt.Println(err)
                              data <- myData{nil, err}
                              return
                      } else {
                              pack := myData{response, err}
                              data <- pack
                      }
              }()
      
              select {
              case <-c.Done():
                      tr.CancelRequest(req)
                      <-data
                      fmt.Println("The request was canceled!")
                      return c.Err()
              case ok := <-data:
                      err := ok.err
                      resp := ok.r
                      if err != nil {
                              fmt.Println("Error select:", err)
                              return err
                      }
                      defer resp.Body.Close()
      
                      realHTTPData, err := ioutil.ReadAll(resp.Body)
                      if err != nil {
                              fmt.Println("Error select:", err)
                              return err
                      }
                      // Although fmt.Printf() is used here, server processes
                      // use the log.Printf() function instead.
                      fmt.Printf("Server Response: %sn", realHTTPData)
              }
              return nil
      }
      
      func main() {
              if len(os.Args) == 1 {
                      fmt.Println("Need a URL and a delay!")
                      return
              }
      
              myUrl = os.Args[1]
              if len(os.Args) == 3 {
                      t, err := strconv.Atoi(os.Args[2])
                      if err != nil {
                              fmt.Println(err)
                              return
                      }
                      delay = t
              }
      
              fmt.Println("Delay:", delay)
              c := context.Background()
              c, cancel := context.WithTimeout(c, time.Duration(delay)*time.Second)
              defer cancel()
      
              fmt.Printf("Connecting to %s n", myUrl)
              w.Add(1)
              go connect(c)
              w.Wait()
              fmt.Println("Exiting...")
      }
      • The timeout period is defined by the context.WithTimeout() method in main().

      • The connect() function that is executed as a goroutine will either terminate normally or when the cancel() function is executed.

        Note

        It is considered a good practice to use context.Background() in the main() function, the init() function of a package or at tests.

      • The connect() function is used for connecting to the desired URL. The connect() function also starts a goroutine before the select block takes control in order to either wait for web data as returned by the goroutine or for a timeout with the help of the Context variable.

      Using http.go

      If the desired delay is too small, then http.go will timeout. One such example is when you declare that you want a delay of 0 seconds, as in the following example:

      go run http.go https://www.linode.com/ 0
      

      The output is as follows:

        
      Delay: 0
      Connecting to https://www.linode.com/
      The request was canceled!
      Exiting...
      
      

      If the timeout period is sufficient, say 10 seconds.

      go run http.go http://localhost:8000 10
      

      Then the output from http.go will be similar to the following:

      Delay: 1
      Connecting to http://localhost:8000
      Server Response: Serving: <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd">
      <html>
      <head>
      <meta http-equiv="Content-Type" content="text/html; charset=utf-8">
      <title>Directory listing for /</title>
      </head>
      <body>
      <h1>Directory listing for /</h1>
      <hr>
      <ul>
      <li><a href="http.go">http.go</a></li>
      <li><a href="more.go">more.go</a></li>
      <li><a href="simple.go">simple.go</a></li>
      </ul>
      <hr>
      </body>
      </html>
      

      Notice that http://localhost:8000 uses a custom made HTTP server that returns a small amount of data. However, nothing prohibits you from trying commands such as:

      go run http.go https://www.linode.com/ 10
      

      Using Contexts as key-value stores

      In this section of the guide you will pass values in a Context and use it as a key-value store. This is a case where we do not pass values into contexts in order to provide further information about why they where canceled.

      The more.go program illustrates the use of the context.TODO() function as well as the use of the context.WithValue() function.

      Explaining the Go Code

      The Go code of more.go is the following:

      ./more.go
       1
       2
       3
       4
       5
       6
       7
       8
       9
      10
      11
      12
      13
      14
      15
      16
      17
      18
      19
      20
      21
      22
      23
      24
      25
      26
      27
      28
      
      package main
      
      import (
              "context"
              "fmt"
      )
      
      type aKey string
      
      func searchKey(ctx context.Context, k aKey) {
              v := ctx.Value(k)
              if v != nil {
                      fmt.Println("found value:", v)
                      return
              } else {
                      fmt.Println("key not found:", k)
              }
      }
      
      func main() {
              myKey := aKey("mySecretValue")
              ctx := context.WithValue(context.Background(), myKey, "mySecretValue")
              searchKey(ctx, myKey)
      
              searchKey(ctx, aKey("notThere"))
              emptyCtx := context.TODO()
              searchKey(emptyCtx, aKey("notThere"))
      }
      • This time we create a context using context.TODO() instead of context.Background(). Although both functions return a non-nil, empty Context, their purposes differ. You should never pass a nil context –– use the context.TODO() function to create a suitable context. Use the context.TODO() function when you are not sure about the Context that you want to use.

      • The context.TODO() function signifies that we intend to use an operation context, without being sure about it yet. The good thing is that TODO() is recognized by static analysis tools, which allows them to determine whether a context.Context variable is propagated correctly in a program or not.

      • The context.WithValue() function that is used in main() offers a way to associate a value with a Context`.

      • The searchKey() function retrieves a value from a Context variable and checks whether that value exists or not.

      Using more.go

      Execute more.go with the following command:

      go run more.go
      

      It will generate the following output:

        
      found value: mySecretValue
      key not found: notThere
      key not found: notThere
      
      

      Propagation over HTTP

      In order to share a common context among multiple processes, you will need to propagate that context on your own.

      The logic of this technique is based on the Go code of more.go. First use the context.WithValue() function to add your data into a context, serialize and send over HTTP, decode the data, get the context, and finally use context.Value() to check whether the desired key and desired values are in place or not.

      Note

      The http.Request type has the Context() method that returns the context of the request and the WithContext() method that according to the Go documentation returns a shallow copy of r with its context changed to ctx. You can learn more about both methods at https://golang.org/pkg/net/http/.

      More Information

      You may wish to consult the following resources for additional information on this topic. While these are provided in the hope that they will be useful, please note that we cannot vouch for the accuracy or timeliness of externally hosted materials.

      Find answers, ask questions, and help others.

      This guide is published under a CC BY-ND 4.0 license.



      Source link