One place for hosting & domains

      January 2022

      How to Add Extra Information to Errors in Go


      The author selected the Diversity in Tech Fund to receive a donation as part of the Write for DOnations program.

      Introduction

      When a function in Go fails, the function will return a value using the error interface to allow the caller to handle that failure. In many cases, developers will use the fmt.Errorf function in the fmt package to return these values. Prior to Go 1.13, though, a downside of using this function is that you would lose information about any errors that may have caused the error to be returned. To solve this, developers would either use packages to provide a way to “wrap” errors inside other errors or create custom errors by implementing the Error() string method on one of their struct error types. Sometimes it can be tedious to create these struct types if you have a number of errors that don’t need to be handled explicitly by the callers, though, so in Go 1.13, the language added features to make it easier to handle these cases.

      One feature is the ability to wrap errors using the fmt.Errorf function with an error value that can be unwrapped later to access the wrapped errors. This builds the error-wrapping functionality into the Go standard library, so there’s no longer any need to use a third-party library.

      Additionally, the functions errors.Is and errors.As make it easier to determine if a specific error is wrapped anywhere inside a given error, and will also give you access to that specific error directly without needing to unwrap all the errors yourself.

      In this tutorial, you’ll create a program that uses these functions to include additional information in errors returned from your functions, and then create your own custom error struct that supports the wrapping and unwrapping functionality.

      Prerequisites

      Returning and Handling Errors in Go

      When an error occurs in a program, it’s good practice to handle those errors so your users never see them — but to handle the errors, you need to know about them first. In Go, you can handle errors in your program by returning information about the error from your functions using a special interface type, the error interface. Using the error interface allows any Go type to be returned as an error value as long as that type has an Error() string method defined. The Go standard library provides functionality to create errors for these return values, such as the fmt.Errorf function.

      In this section, you’ll create a program with a function that uses fmt.Errorf to return an error, and you will also add an error handler to check for the errors that the function could return. (If you’d like more information on handling errors in Go, please see the tutorial, Handling Errors in Go.)

      Many developers have a directory to keep current projects. In this tutorial, you’ll use a directory named projects.

      To begin, make the projects directory and navigate to it:

      • mkdir projects
      • cd projects

      From the projects directory, create a new errtutorial directory to keep the new program in:

      Next, navigate into the new directory with the cd command:

      Once you’re in the errtutorial directory, use the go mod init command to create a new module named errtutorial:

      After creating the Go module, open a file named main.go in the errtutorial directory using nano, or your favorite editor:

      Next, you will write a program. The program will loop over the numbers 1 through 3 and try to determine if those numbers are valid or not using a function called validateValue. If the number is determined to be invalid, the program will use the fmt.Errorf function to generate an error value that is returned from the function. The fmt.Errorf function allows you to create an error value where the error message is the message you provide to the function. It works similarly to fmt.Printf, but instead of printing the message to the screen it returns it as an error instead.

      Then, in the main function, the error value will be checked to see if it’s a nil value or not. If it is a nil value, the function succeeded and the valid! message is printed. If it’s not, the error received is printed instead.

      To begin your program, add the following code into your main.go file:

      projects/errtutorial/main.go

      package main
      
      import (
          "fmt"
      )
      
      func validateValue(number int) error {
          if number == 1 {
              return fmt.Errorf("that's odd")
          } else if number == 2 {
              return fmt.Errorf("uh oh")
          }
          return nil
      }
      
      func main() {
          for num := 1; num <= 3; num++ {
              fmt.Printf("validating %d... ", num)
              err := validateValue(num)
              if err != nil {
                  fmt.Println("there was an error:", err)
              } else {
                  fmt.Println("valid!")
              }
          }
      }
      

      The validateValue function from the program takes a number and then returns an error based on whether it was determined to be a valid value or not. In this program, the number 1 is invalid and returns the error that's odd. The number 2 is invalid and returns the error uh oh. The validateValue function uses the fmt.Errorf function to generate the error value being returned. The fmt.Errorf function is convenient for returning errors because it allows you to format an error message using formatting similar to fmt.Printf or fmt.Sprintf without needing to then pass that string to errors.New.

      In the main function, the for loop will start by iterating over each number from 1 to 3 and will store the value in the num variable. Inside the loop body, a call to fmt.Printf will print the number the program is currently validating. Then, it will call the validateValue function and pass in num, the current number being validated, and store the error result in the err variable. Lastly, if err is not nil it means an error occured during validation and the error message is printed using fmt.Println. The else clause of the error check will print "valid!" when an error wasn’t encountered.

      After saving your changes, run your program using the go run command with main.go as the argument from the errtutorial directory:

      The output from running the program will show that validation was run for each number and number 1 and number 2 returned their appropriate errors:

      Output

      validating 1... there was an error: that's odd validating 2... there was an error: uh oh validating 3... valid!

      When you look at the output from the program, you’ll see the program tried to validate all three numbers. The first time it says the validateValue function returned the that's odd error, which would be expected for the value of 1. The next value, 2, also shows it returned an error, but it was the uh oh error this time . Finally, the 3 value returns nil for the error value, meaning there wasn’t an error and the number is valid. The way the validateValue function is written, the nil error value would be returned for any values that aren’t either 1 or 2.

      In this section, you used fmt.Errorf to create error values you returned from a function. You also added an error handler to print out the error message when any error is returned from the function. At times, though, it can be useful to know what an error means, not just that an error occurred. In the next section, you’ll learn to customize error handling for specific cases.

      Handling Specific Errors Using Sentinel Errors

      When you receive an error value from a function, the most basic error handling is to check if the error value is nil or not. This will tell you if the function had an error, but sometimes you may want to customize error handling for a specific error case. For example, imagine you have code connecting to a remote server, and the only error information you get back is “you had an error”. You may wish to tell whether the error was because the server was unavailable or if your connection credentials were invalid. If you knew the error meant a user’s credentials were wrong, you might want to let the user know right away. But if the error means the server was unavailable, you may want to try reconnecting a few times before letting the user know. Determining the difference between these errors allows you to write more robust and user-friendly programs.

      One way you could check for a particular type of error might be using the Error method on an error type to get the message from the error and compare that value to the type of error you’re looking for. Imagine that in your program, you want to show a message other than there was an error: uh oh when the error value is uh oh. One approach to handling this case would be to check the value returned from the Error method, like so:

      if err.Error() == "uh oh" {
          // Handle 'uh oh' error.
          fmt.Println("oh no!")
      }
      

      Checking the string value of err.Error() to see if it’s the value uh oh, as in the code above, would work in this case. But the code would not work if the uh oh error string is slightly different elsewhere in the program. Checking errors this way can also lead to significant updates to code if the error’s message itself needs to be updated because every place the error is checked would need to be updated. Take the following code, for example:

      func giveMeError() error {
          return fmt.Errorf("uh h")
      }
      
      err := giveMeError()
      if err.Error() == "uh h" {
          // "uh h" error code
      }
      

      In this code, the error message includes a typo and is missing the o in uh oh. If this is noticed and fixed at some point, but only after adding this error checking in several places, all those places will need to have their checks updated to err.Error() == "uh oh". If one is missed, which could be easy because it’s only a single character change, the expected custom error handler will not run because it’s expecting uh h and not uh oh.

      In cases like these, where you may want to handle a specific error differently than others, it’s common to create a variable whose purpose is to hold an error value. This way, the code can check against that variable instead of a string. Typically, these variables begin with either err or Err in their names to signify they’re errors. If the error is only meant to be used within the package it’s defined in, you would want to use the err prefix. If the error is meant to be used elsewhere, you would instead use the Err prefix to make it an exported value, similar to a function or a struct.

      Now, let’s say you were using one of these error values in the typo example from before:

      var errUhOh = fmt.Errorf("uh h")
      
      func giveMeError() error {
          return errUhOh
      }
      
      err := giveMeError()
      if err == errUhOh {
          // "uh oh" error code
      }
      

      In this example, the variable errUhOh is defined as the error value for an “uh oh” error (even though it’s misspelled). The giveMeError function returns the value of errUhOh because it wants to let the caller know that an “uh oh” error happened. Then, the error handling code compares the err value returned from giveMeError against errUhOh to see if an “uh oh” error is the one that happened. Even if the typo is found and fixed, all the code would still be working because the error check is checking against the value of errUhOh, and the value of errUhOh is the fixed version of the error value that giveMeError is returning.

      An error value that is intended to be checked and compared in this way is known as a sentinel error. A sentinel error is an error that’s designed to be a unique value that can always be compared against for a specific meaning. The errUhOh value above will always have the same meaning, that an “uh oh” error occurred, so a program can rely on comparing an error to errUhOh to determine whether that error occurred.

      The Go standard library also defines a number of sentinel errors that are available when developing Go programs. One example would be the sql.ErrNoRows error. The sql.ErrNoRows error is returned when a database query doesn’t return any results, so that error can be handled differently from a connection error. Since it’s a sentinel error, it can be compared against in error-checking code to know when a query doesn’t return any rows, and the program can handle that differently than other errors.

      Generally, when creating a sentinel error value, the errors.New function from the errors package is used instead of the fmt.Errorf function you’ve been using thus far. Using errors.New instead of fmt.Errorf does not make any foundational changes to how the error works, though, and both functions could be used interchangeably most of the time. The biggest difference between the two is the errors.New function will only create an error with a static message and the fmt.Errorf function allows formatting the string with values, similar to fmt.Printf or fmt.Sprintf. Since sentinel errors are fundamental errors with values that don’t change, it’s common to use errors.New to create them.

      Now, update your program to use a sentinel error for the “uh oh” error instead of fmt.Errorf.

      First, open the main.go file to add the new errUhOh sentinel error and update the program to use it. The validateValue function is updated to return the sentinel error instead of using fmt.Errorf. The main function is updated to check for the errUhOh sentinel error and print oh no! when it encounters it instead of the there was an error: message it shows for other errors.

      projects/errtutorial/main.go

      package main
      
      import (
          "errors"
          "fmt"
      )
      
      var (
          errUhOh = errors.New("uh oh")
      )
      
      func validateValue(number int) error {
          if number == 1 {
              return fmt.Errorf("that's odd")
          } else if number == 2 {
              return errUhOh
          }
          return nil
      }
      
      func main() {
          for num := 1; num <= 3; num++ {
              fmt.Printf("validating %d... ", num)
              err := validateValue(num)
              if err == errUhOh {
                  fmt.Println("oh no!")
              } else if err != nil {
                  fmt.Println("there was an error:", err)
              } else {
                  fmt.Println("valid!")
              }
          }
      }
      

      Now, save your code and use go run to run your program again:

      This time the output will show the generic error output for the 1 value, but it uses the custom oh no! message when it sees the errUhOh error returned from validateValue for 2:

      Output

      validating 1... there was an error: that's odd validating 2... oh no! validating 3... valid!

      Using sentinel errors inside your error checking makes it easier to handle special error cases. For example, they can help determine whether the file you’re reading is failing because you’ve reached the end of the file, which is signified by the io.EOF sentinel error, or if it’s failing for some other reason.

      In this section, you created a Go program that uses a sentinel error using errors.New to signify when a specific type of error occurred. Over time as your program grows, though, you may get to the point where you’d like more information included in your error than just the uh oh error value. This error value doesn’t give any context on where the error happened or why it happened, and it can be hard to track down specifics of the error in larger programs. To aid in troubleshooting and to cut down the time for debugging, you can make use of error wrapping to include the specifics you need.

      Wrapping and Unwrapping Errors

      Wrapping errors means taking one error value and putting another error value inside it, like a wrapped gift. Similar to a wrapped gift, though, you need to unwrap it to know what’s inside. Wrapping an error allows you to include additional information about where the error came from or how it happened without losing the original error value, since it’s inside the wrapper.

      Before Go 1.13, it was possible to wrap errors since you could create custom error values that included the original error. But you would either have to create your own wrappers or use a library that already did the work for you. In Go 1.13, though, Go added support for wrapping and unwrapping errors as part of the standard library by adding the errors.Unwrap function and the %w verb for the fmt.Errorf function. In this section, you’ll update your program to use the %w verb to wrap errors with more information, and you’ll then use errors.Unwrap to retrieve the wrapped information.

      Wrapping Errors with fmt.Errorf

      The first feature to examine when wrapping and unwrapping errors is an addition to the existing fmt.Errorf function. In the past, fmt.Errorf was used to create formatted error messages with additional information using verbs such as %s for strings and %v for generic values. Go 1.13 added a new verb with a special case, the %w verb. When the %w verb is included in a format string and an error is provided for the value, the error returned from fmt.Errorf will include the value of the error wrapped in the error being created.

      Now, open the main.go file and update it to include a new function called runValidation. This function will take the number currently being validated and run any validation needed on that number. In this case, it only needs to run the validateValue function. If it encounters an error validating the value it will wrap the error using fmt.Errorf and the %w verb to show there was a run error that occurred, then return that new error. You should also update the main function so instead of calling validateValue directly it calls runValidation instead:

      projects/errtutorial/main.go

      
      ...
      
      var (
          errUhOh = errors.New("uh oh")
      )
      
      func runValidation(number int) error {
          err := validateValue(number)
          if err != nil {
              return fmt.Errorf("run error: %w", err)
          }
          return nil
      }
      
      ...
      
      func main() {
          for num := 1; num <= 3; num++ {
              fmt.Printf("validating %d... ", num)
              err := runValidation(num)
              if err == errUhOh {
                  fmt.Println("oh no!")
              } else if err != nil {
                  fmt.Println("there was an error:", err)
              } else {
                  fmt.Println("valid!")
              }
          }
      }
      

      Once you’ve saved your updates, run the updated program using go run:

      The output will look similar to this:

      Output

      validating 1... there was an error: run error: that's odd validating 2... there was an error: run error: uh oh validating 3... valid!

      There are a few things to look at in this output. First, you’ll see the error message being printed for the value 1 now includes run error: that's odd in the error message. This shows the error was wrapped by runValidation’s fmt.Errorf and that the value of the error being wrapped, that's odd, is included in the error message.

      Next, though, there’s a problem. The special error handling that was added for the errUhOh error isn’t running. If you look at the line validating the 2 input, you’ll see it shows the default error message of there was an error: run error: uh oh instead of the expected oh no! message. You know the validateValue function is still returning the uh oh error because you can see it at the end of the wrapped error, but the error detection of errUhOh is no longer working. This happens because the error being returned by runValidation is no longer errUhOh, it’s the wrapped error created by fmt.Errorf. When the if statement tries to compare the err variable to errUhOh, it returns false because err isn’t equal to errUhOh any more, it’s equal to the error that’s wrapping errUhOh. To fix the errUhOh error checking, you’ll need to retrieve the error from inside the wrapper, using the errors.Unwrap function.

      Unwrapping Errors with errors.Unwrap

      In addition to the %w verb being added in Go 1.13, a few new functions were added to the Go errors package. One of these, the errors.Unwrap function, takes an error as a parameter and, if the error passed in is an error wrapper, it will return the wrapped error. If the error provided isn’t a wrapper, the function will return nil.

      Now, open the main.go file again and, using errors.Unwrap, update the errUhOh error check to handle the case where errUhOh is wrapped inside an error wrapper:

      projects/errtutorial/main.go

      func main() {
          for num := 1; num <= 3; num++ {
              fmt.Printf("validating %d... ", num)
              err := runValidation(num)
              if err == errUhOh || errors.Unwrap(err) == errUhOh {
                  fmt.Println("oh no!")
              } else if err != nil {
                  fmt.Println("there was an error:", err)
              } else {
                  fmt.Println("valid!")
              }
          }
      }
      

      After saving the edits, run the program again:

      The output will look similar to this:

      Output

      validating 1... there was an error: run error: that's odd validating 2... oh no! validating 3... valid!

      Now, in the output, you’ll see the oh no! error handling for the 2 input value is back. The additional errors.Unwrap function call you added to the if statement allows it to detect errUhOh both when err itself is an errUhOh value as well as if err is an error that is directly wrapping errUhOh.

      In this section, you used the %w verb added to fmt.Errorf to wrap the errUhOh error inside another error and give it additional information. Then, you used errors.Unwrap to access the errorUhOh error that is wrapped inside another error. Including errors inside other errors as string values is OK for humans reading error messages, but sometimes you’ll want to include additional information with the error wrapper to aid the program in handling the error, such as the status code in an HTTP request error. When this happens, you can create a new custom error to return.

      Custom Wrapped Errors

      Since Go’s only rule for the error interface is that it includes an Error method, it’s possible to turn many Go types into a custom error. One way is by defining a struct type with extra information about the error, and then also including an Error method.

      For a validation error, it may be useful to know which value actually caused the error. Next, let’s create a new ValueError struct that contains a field for the Value that caused the error and an Err field that contains the actual validation error. Custom error types commonly use the Error suffix on the end of the type name to signify it’s a type that conforms to the error interface.

      Open your main.go file and add the new ValueError error struct, as well as a newValueError function to create instances of the error. You will also need to create a method called Error for ValueError so the struct will be considered an error. This Error method should return the value you want to be displayed whenever the error is converted to a string. In this case, it will use fmt.Sprintf to return a string that shows value error: and then the wrapped error. Also, update the validateValue function so instead of returning just the basic error, it uses the newValueError function to return a custom error:

      projects/errtutorial/main.go

      
      ...
      
      var (
          errUhOh = fmt.Errorf("uh oh")
      )
      
      type ValueError struct {
          Value int
          Err   error
      }
      
      func newValueError(value int, err error) *ValueError {
          return &ValueError{
              Value: value,
              Err:   err,
          }
      }
      
      func (ve *ValueError) Error() string {
          return fmt.Sprintf("value error: %s", ve.Err)
      }
      
      ...
      
      func validateValue(number int) error {
          if number == 1 {
              return newValueError(number, fmt.Errorf("that's odd"))
          } else if number == 2 {
              return newValueError(number, errUhOh)
          }
          return nil
      }
      
      ...
      

      Once your updates are saved, run the program again with go run:

      The output will look similar to this:

      Output

      validating 1... there was an error: run error: value error: that's odd validating 2... there was an error: run error: value error: uh oh validating 3... valid!

      You’ll see that the output now shows the errors are wrapped inside of ValueError by the value error: before them in the output. However, the uh oh error detection is broken again because errUhOh is now inside two layers of wrappers, ValueError and the fmt.Errorf wrapper from runValidation. The code code only uses errors.Unwrap once on the error, so this results in the first errors.Unwrap(err) now only returning a *ValueError and not errUhOh.

      One way to fix this would be to update the errUhOh check to add an additional error check that calls errors.Unwrap() twice to unwrap both layers. To add this, open your main.go file and update your main function to include this change:

      projects/errtutorial/main.go

      
      ...
      
      func main() {
          for num := 1; num <= 3; num++ {
              fmt.Printf("validating %d... ", num)
              err := runValidation(num)
              if err == errUhOh ||
                  errors.Unwrap(err) == errUhOh ||
                  errors.Unwrap(errors.Unwrap(err)) == errUhOh {
                  fmt.Println("oh no!")
              } else if err != nil {
                  fmt.Println("there was an error:", err)
              } else {
                  fmt.Println("valid!")
              }
          }
      }
      

      Now, save your main.go file and use go run to run your program again:

      The output will look similar to this:

      Output

      validating 1... there was an error: run error: value error: that's odd validating 2... there was an error: run error: value error: uh oh validating 3... valid!

      You’ll see that, uh oh, the errUhOh special error handling is still not working. The line validating the 2 input where we’d expect to see the special error handling oh no! output still shows the default there was an error: run error: ... error output. This happens because the errors.Unwrap function doesn’t know how to unwrap the ValueError custom error type. In order for a custom error to be unwrapped, it needs to have its own Unwrap method that returns the inner error as an error value. When creating errors using fmt.Errorf with the %w verb earlier, Go was actually creating an error for you that already has an Unwrap method added, so you didn’t need to do it yourself. Now that you’re using your own custom function, though, you need to add your own.

      To finally fix the errUhOh error case, open main.go and add an Unwrap method to ValueError that returns Err, the field the inner wrapped error is stored in:

      projects/errtutorial/main.go

      
      ...
      
      func (ve *ValueError) Error() string {
          return fmt.Sprintf("value error: %s", ve.Err)
      }
      
      func (ve *ValueError) Unwrap() error {
          return ve.Err
      }
      
      ...
      

      Then, once you’ve saved the new Unwrap method, run your program:

      The output will look similar to this:

      Output

      validating 1... there was an error: run error: value error: that's odd validating 2... oh no! validating 3... valid!

      The output shows the oh no! error handling for the errUhOh error is working again because errors.Unwrap is now able to also unwrap ValueError.

      In this section you created a new, custom ValueError error to provide yourself or your users with information about the validation process as part of the error message. You also added support for error unwrapping to your ValueError so errors.Unwrap can be used to access the wrapped error.

      The error handling is getting a bit clunky and hard to maintain, though. Every time there’s a new layer of wrapping you’ve had to add another errors.Unwrap to the error checking to handle it. Thankfully, the errors.Is and errors.As functions in the errors package are available to make working with wrapped errors easier.

      Working with Wrapped Errors

      If you needed to add a new errors.Unwrap function call for every potential layer of error wrapping your program had, it would get very long and difficult to maintain. For this reason, two additional functions were also added to the errors package in the Go 1.13 release. Both of these functions make it easier to work with errors by allowing you to interact with errors no matter how deeply they’re wrapped inside other errors. The errors.Is function allows you to check if a specific sentinel error value is anywhere inside a wrapped error. The errors.As function allows you to get a reference to a certain type of error anywhere inside a wrapped error.

      Checking an Error Value with errors.Is

      Using errors.Is to check for a specific error makes the errUhOh special error handling much shorter because it handles all the nested error unwrapping you were doing manually. The function takes two error parameters, the first being the error you actually received and the second parameter being the error you want to check against.

      To clean up the errUhOh error handling, open your main.go file and update the errUhOh check in the main function to use errors.Is instead:

      projects/errtutorial/main.go

      
      ...
      
      func main() {
          for num := 1; num <= 3; num++ {
              fmt.Printf("validating %d... ", num)
              err := runValidation(num)
              if errors.Is(err, errUhOh) {
                  fmt.Println("oh no!")
              } else if err != nil {
                  fmt.Println("there was an error:", err)
              } else {
                  fmt.Println("valid!")
              }
          }
      }
      

      Then, save your code and run the program again using go run:

      The output will look similar to this:

      Output

      validating 1... there was an error: run error: value error: that's odd validating 2... oh no! validating 3... valid!

      The output shows the oh no! error message, which means that even though there’s only one error check for errUhOh, it will still be found in the error chain. errors.Is takes advantage of an error type’s Unwrap method to keep digging deeper into a chain of errors until it either finds the error value you’re looking for, a sentinel error, or encounters an Unwrap method that returns a nil value.

      Using errors.Is is the recommended way to check for specific errors now that error wrapping exists as a feature in Go. Not only can it be used for your own error values, but it can also be used for other error values such as the sql.ErrNoRows error mentioned earlier in this tutorial.

      Retrieving an Error Type with errors.As

      The last function added to the errors package in Go 1.13 is the errors.As function. This function is used when you want to get a reference to a certain type of error to interact with it in more detail. For example, the ValueError custom error you added earlier gives access to the actual value being validated in the Value field of the error, but you can only access it if you have a reference to that error first. This is where errors.As comes in. You can give errors.As an error, similar to errors.Is, and a variable for a type of error. It will then go through the error chain to see if any of the wrapped errors match the type provided. If one matches, the variable passed in for the error type will be set with the error errors.As found, and the function will return true. If no error types match, it will return false.

      Using errors.As you can now take advantage of the ValueError type to show additional error information in your error handler. Open your main.go file one last time and update the main function to add a new error handling case for ValueError-type errors that prints out value error, the invalid number, and the validation error:

      projects/errtutorial/main.go

      
      ...
      
      func main() {
          for num := 1; num <= 3; num++ {
              fmt.Printf("validating %d... ", num)
              err := runValidation(num)
      
              var valueErr *ValueError
              if errors.Is(err, errUhOh) {
                  fmt.Println("oh no!")
              } else if errors.As(err, &valueErr) {
                  fmt.Printf("value error (%d): %vn", valueErr.Value, valueErr.Err)
              } else if err != nil {
                  fmt.Println("there was an error:", err)
              } else {
                  fmt.Println("valid!")
              }
          }
      }
      

      In the code above, you declared a new valueErr variable and used errors.As to get a reference to the ValueError if it’s wrapped inside the err value. By getting access to the error as a ValueError, you’re then able to access any additional fields the type provides, such as the actual value that failed validation. This could be helpful if the validation logic happens deeper inside the program and you don’t normally have access to the values to give users hints on where something might have gone wrong. Another example of where this could be helpful is if you’re doing network programming and run into a net.DNSError. By getting a reference to the error, you are able to see if the error was the result of not being able to connect, or if the error was caused by being able to connect, but your resource was not found. Once you know this, you can handle the error in different ways.

      To see errors.As in action, save your file and run the program using go run:

      The output will look similar to this:

      Output

      validating 1... value error (1): that's odd validating 2... oh no! validating 3... valid!

      This time in the output you won’t see the default there was an error: ... message, because all the errors are being handled by other error handlers. The output for validating 1 shows that the errors.As error check returned true because the value error ... error message is being displayed. Since the errors.As function returned true, the valueErr variable is set to be a ValueError and can be used to print out the value that failed validation by accessing valueErr.Value.

      For the 2 value, the output also shows that even though the errUhOh is also wrapped inside a ValueError wrapper, the oh no! special error handler is still executed. This is because the special error handler using errors.Is for errUhOh comes first in the collection of if statements handling the errors. Since this handler returns true before the errors.As even runs, the special oh no! handler is the one executed. If the errors.As in your code came before the errors.Is, the oh no! error message would become the same value error ... as the 1 value, except in this case it would print value error (2): uh oh.

      In this section, you updated your program to use the errors.Is function to remove a lot of additional calls to errors.Unwrap and make your error handling code more robust and future-proof. You also used the errors.As function to check if any of the wrapped errors is a ValueError, and then used fields on the value if one was found.

      Conclusion

      In this tutorial, you wrapped an error using the %w format verb and unwrapped an error using errors.Unwrap. You also created a custom error type that supports errors.Unwrap in your own code. Finally, you used your custom error type to explore the new helper functions errors.Is and errors.As.

      Using these new error functions makes it easier to include deeper information about the errors you create or work with. It also future proofs your code to ensure your error checking continues to work even if errors become deeply nested going forward.

      If you’d like to find more details about how to use the new error features, the Go blog has a post about Working with Errors in Go 1.13. The documentation for the errors package package also includes more information.

      This tutorial is also part of the DigitalOcean How to Code in Go series. The series covers a number of Go topics, from installing Go for the first time to how to use the language itself.



      Source link

      How To Install Matomo Web Analytics on Ubuntu 20.04


      Introduction

      Matomo is an open-source, self-hosted web analytics application written in PHP.

      In this tutorial you will install Matomo and a MariaDB database using Docker Compose, then install Nginx to act as a reverse proxy for the Matomo app. Finally, you will enable secure HTTPS connections by using Certbot to download and configure SSL certificates from the Let’s Encrypt Certificate Authority.

      Prerequisites

      In order to complete this tutorial, you’ll first need the following:

      Note: These prerequisite steps can be skipped if you’re using DigitalOcean’s One-Click Docker Image. This image will have Docker, Docker Compose, and UFW already installed and configured.

      Launch a new Docker image in the region of your choice, then log in as the root user and proceed with the tutorial. Because you’ll be using the root user, you could leave off the sudo parts of all the commands that follow, but it’s not necessary.

      Finally, to enable SSL you’ll need a domain name pointed at your server’s public IP address. This should be something like example.com or matomo.example.com, for instance. If you’re using DigitalOcean, please see our DNS Quickstart for information on creating domain resources in our control panel.

      When you’ve satisfied all the prerequisites, proceed to Step 1, where you’ll download and launch the Matomo software.

      Step 1 — Running Matomo and MariaDB with Docker Compose

      Your first step will be to create the Docker Compose configuration that will launch containers for both the Matomo app and a MariaDB database.

      This tutorial will put your configuration inside a matomo directory in your home directory. You could also choose to work in an /opt/matomo directory or some other directory of your choosing.

      First ensure you’re in your home directory:

      Then create the matomo directory and cd into it:

      Now open a new blank YAML file called docker-compose.yml:

      This is the configuration file that the docker-compose software will read when bringing up your containers. Paste the following into the file:

      docker-compose.yml

      version: "3"
      
      services:
        db:
          image: mariadb
          command: --max-allowed-packet=64MB
          restart: always
          environment:
            - MARIADB_DATABASE=matomo
            - MARIADB_USER
            - MARIADB_PASSWORD
            - MARIADB_ROOT_PASSWORD
          volumes:
            - ./db:/var/lib/mysql
      
        app:
          image: matomo
          restart: always
          volumes:
            - ./matomo:/var/www/html
          ports:
            - 127.0.0.1:8080:80
      

      The file defines two services, one db service which is the MariaDB container, and an app service which runs the Matomo software. Both services also reference a named volume where they store some data, and the app service also opens up port 8080 on the loopback (127.0.0.1) interface, which we’ll connect to via localhost.

      Save the file and exit your text editor to continue. In nano, press CTRL+O then ENTER to save, then CTRL+X to exit.

      The MariaDB container needs some configuration to be passed to it through environment variables in order to function. The docker-compose.yml file lists these environment variables, but not all of them have associated values. That’s because it’s good practice to keep passwords out of your docker-compose.yml file, especially if you’ll be committing it to a Git repository or other source control system.

      Instead, we’ll put the necessary information in a .env file in the same directory, which the docker-compose command will automatically load when we start our containers.

      Open a new .env file with nano:

      You’ll need to fill in a user name and password, as well as a strong password for the MariaDB root superuser account:

      .env

      MARIADB_USER=matomo
      MARIADB_PASSWORD=a_strong_password_for_user
      MARIADB_ROOT_PASSWORD=a_strong_password_for_root
      

      One way of generating a strong password is to use the openssl command, which should be available on most any operating system. The following command will print out a random 30 character hash that you can use as a password:

      • openssl rand 30 | base64 -w 0 ; echo

      When you’re done filling out the information in your .env file, save it and exit your text editor.

      You’re now ready to bring up the two containers with docker-compose:

      • sudo docker-compose up -d

      The up subcommand tells docker-compose to start the containers (and volumes and networks) defined in the docker-compose.yml file, and the -d flag tells it to do so in the background (“daemonize”) so the command doesn’t take over your terminal. docker-compose will print some brief output as it starts the containers:

      Output

      Creating matomo_db_1 ... done Creating matomo_app_1 ... done

      When that’s done, Matomo should be running. You can test that a webserver is running at localhost:8080 by fetching the homepage using the curl command:

      • curl --head http://localhost:8080

      This will print out only the HTTP headers from the response:

      Output

      HTTP/1.1 200 OK Date: Tue, 25 Jan 2022 19:56:16 GMT Server: Apache/2.4.51 (Debian) X-Powered-By: PHP/8.0.14 X-Matomo-Request-Id: 1e953 Cache-Control: no-store, must-revalidate Referrer-Policy: same-origin Content-Security-Policy: default-src 'self' 'unsafe-inline' 'unsafe-eval'; img-src 'self' 'unsafe-inline' 'unsafe-eval' data:; Set-Cookie: MATOMO_SESSID=dde7d477b0822e166ed90448964ec1e7; path=/; HttpOnly; SameSite=Lax Content-Type: text/html; charset=utf-8

      The 200 OK response means the Matomo server is up and running, but it’s only available on localhost. The highlighted X-Matomo-Request-Id header indicates that the server is Matomo and not something else that might be configured to listen on port 8080. Next we’ll set up Nginx to proxy public traffic to the Matomo container.

      Step 2 — Installing and Configuring Nginx

      Putting a web server such as Nginx in front of your Matomo server can improve performance by offloading caching, compression, and static file serving to a more efficient process. We’re going to install Nginx and configure it to reverse proxy requests to Matomo, meaning it will take care of handing requests from your users to Matomo and back again. Using a non-containerized Nginx will also make it easier to add Let’s Encrypt SSL certificates in the next step.

      First, refresh your package list, then install Nginx using apt:

      • sudo apt update
      • sudo apt install nginx

      Allow public traffic to ports 80 and 443 (HTTP and HTTPS) using the “Nginx Full” UFW application profile:

      • sudo ufw allow "Nginx Full"

      Output

      Rule added Rule added (v6)

      Next, open up a new Nginx configuration file in the /etc/nginx/sites-available directory. We’ll call ours matomo.conf but you could use a different name:

      • sudo nano /etc/nginx/sites-available/matomo.conf

      Paste the following into the new configuration file, being sure to replace your_domain_here with the domain that you’ve configured to point to your Matomo server. This should be something like matomo.example.com, for instance:

      /etc/nginx/sites-available/matomo.conf

      server {
          listen       80;
          listen       [::]:80;
          server_name  your_domain_here;
      
          access_log  /var/log/nginx/matomo.access.log;
          error_log   /var/log/nginx/matomo.error.log;
      
          location / {
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-Host $host;
            proxy_set_header X-Forwarded-Proto https;
            proxy_pass http://localhost:8080;
        }
      }
      

      This configuration is HTTP-only for now, as we’ll let Certbot take care of configuring SSL in the next step. The rest of the config sets up logging locations and then passes all traffic, as well as some important proxy headers, along to http://localhost:8080, the Matomo container we started up in the previous step.

      Save and close the file, then enable the configuration by linking it into /etc/nginx/sites-enabled/:

      • sudo ln -s /etc/nginx/sites-available/matomo.conf /etc/nginx/sites-enabled/

      Use nginx -t to verify that the configuration file syntax is correct:

      Output

      nginx: the configuration file /etc/nginx/nginx.conf syntax is ok nginx: configuration file /etc/nginx/nginx.conf test is successful

      And finally, reload the nginx service to pick up the new configuration:

      • sudo systemctl reload nginx

      Your Matomo site should now be available on plain HTTP. Load http://your_domain_here (you may have to click through a security warning) and it will look like this:

      Screenshot of the first page of the Matomo web installation process, with a

      Now that you have your site up and running over HTTP, it’s time to secure the connection with Certbot and Let’s Encrypt certificates. You should do this before going through Matomo’s web-based setup procedure.

      Step 3 — Installing Certbot and Setting Up SSL Certificates

      Thanks to Certbot and the Let’s Encrypt free certificate authority, adding SSL encryption to our Matomo app will take only two commands.

      First, install Certbot and its Nginx plugin:

      • sudo apt install certbot python3-certbot-nginx

      Next, run certbot in --nginx mode, and specify the same domain you used in the Nginx server_name config:

      • sudo certbot --nginx -d your_domain_here

      You’ll be prompted to agree to the Let’s Encrypt terms of service, and to enter an email address.

      Afterwards, you’ll be asked if you want to redirect all HTTP traffic to HTTPS. It’s up to you, but this is generally recommended and safe to do.

      After that, Let’s Encrypt will confirm your request and Certbot will download your certificate:

      Output

      Congratulations! You have successfully enabled https://matomo.example.com You should test your configuration at: https://www.ssllabs.com/ssltest/analyze.html?d=matomo.example.com - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - IMPORTANT NOTES: - Congratulations! Your certificate and chain have been saved at: /etc/letsencrypt/live/matomo.example.com/fullchain.pem Your key file has been saved at: /etc/letsencrypt/live/matomo.example.com/privkey.pem Your cert will expire on 2021-12-06. To obtain a new or tweaked version of this certificate in the future, simply run certbot again with the "certonly" option. To non-interactively renew *all* of your certificates, run "certbot renew" - Your account credentials have been saved in your Certbot configuration directory at /etc/letsencrypt. You should make a secure backup of this folder now. This configuration directory will also contain certificates and private keys obtained by Certbot so making regular backups of this folder is ideal. - If you like Certbot, please consider supporting our work by: Donating to ISRG / Let's Encrypt: https://letsencrypt.org/donate Donating to EFF: https://eff.org/donate-le

      Certbot will automatically reload Nginx to pick up the new configuration and certificates. Reload your site and it should switch you over to HTTPS automatically if you chose the redirect option.

      Your site is now secure and it’s safe to continue with the web-based setup steps.

      Step 4 — Setting Up Matomo

      Back in your web browser you should now have Matomo’s Welcome! page open via a secure https:// connection. Now you can enter usernames and passwords safely to complete the installation process.

      Click the Next button. You’ll be taken to the System Check step:

      Screenshot of Matomo's "System Check" page with a list of system properties with green checkmarks next to them

      This is a summary of the system Matomo is running on, and everything should be green checkmarks indicating there are no problems. Scroll all the way to the bottom and click the Next button.

      Now you’ll be on the Database Setup page:

      Screenshot of Matomo's "Database Setup" page, with a form for inputting database connection details

      The information you fill in on this page will tell the Matomo application how to connect to the MariaDB database. You’ll need the MARIADB_USER and MARIADB_PASSWORD that you chose in Step 1. You can copy them out of your .env file if you need to.

      Fill out the first four fields:

      • Database Server: db
      • Login: the username you set in the MARIADB_USER environment variable
      • Password: the password you set in the MARIADB_PASSWORD environment variable
      • Database Name: matomo

      The defaults are fine for the remaining two fields.

      Click Next once more. You’ll get a confirmation that the database was set up correctly. Click Next again. You’ll then need to set up an admin user, and finally you’ll set up information about the first website you want to collect analytics for.

      After all that, you should end up on step 8, a Congratulations page. You’re almost all done. Scroll down to the bottom and click the Continue to Matomo button, and you’ll be taken to the homepage:

      Screenshot of the Matomo homepage with a large orange

      There will be a large warning at the top of the page. There’s a small update you’ll need to do to Matomo’s configuration file to finish up this process.

      Back on the command line, open up the configuration file with a text editor:

      • sudo nano matomo/config/config.ini.php

      Near the top you should have a [General] section. Add the last three lines, highlighted below, to the end of that section:

      config.ini.php

      [General]
      proxy_client_headers[] = "HTTP_X_FORWARDED_FOR"
      proxy_host_headers[] = "HTTP_X_FORWARDED_HOST"
      salt = "e0a81d6e54d6d2200efd0f0ef6ef8563"
      trusted_hosts[] = "localhost"
      trusted_hosts[] = "example.com"
      trusted_hosts[] = "localhost:8080"
      assume_secure_protocol = 1
      force_ssl = 1
      

      These options let Matomo know that it’s safe to use port 8080, and that it should assume it’s always being accessed over a secure connection.

      Save and close the configuration file, then switch back to your browser and reload the page. The error should be gone, and you’ll be presented with a login prompt:

      Screenshot of Matomo's "Sign in" screen with a form for username and password

      Log in with the admin account you created during setup, and you should be taken to the dashboard:

      Screenshot of Matomo's homepage dashboard with a placeholder indicating "No data has been recorded yet" and instructions on how to set up the tracking code

      Because you’ve probably not set up your tracking code yet, the dashboard will indicate that no data has been recorded. Follow the instructions to finish setting up the JavaScript code on your website to start receiving analytics data.

      Conclusion

      In this tutorial, you launched the Matamo analytics app and a MariaDB database using Docker Compose, then set up an Nginx reverse proxy and secured it using Let’s Encrypt SSL certificates.

      You’re now ready to set up your website and add the Matomo analytics tracking script. For more information about operating the Matomo software, please see the official Matomo documentation.



      Source link

      Building Interactive Live Streaming Apps for Millions of Viewers


      How to Join

      This Tech Talk is free and open to everyone. Register below to get a link to join the live stream or receive the video recording after it airs.

      Date Time RSVP
      February 15, 2022 11:00 a.m.–12:00 p.m. ET / 4:00–5:00 p.m. GMT

      About the Talk

      Today’s CDNs aren’t optimized for real-time, scalable, multi-way live streaming. See how an XDN (Experience Delivery Network) helps deliver seamless interactive experiences for massive audiences by managing signal acquisition/ingest, latency, interactivity, and scaling in the cloud.

      What You’ll Learn

      • When to use cloud services, edge compute, and traditional CDNs
      • Protocols that reduce live streaming latency
      • Monitoring strategies that deliver seamless viewer experience* Avoiding vendor lock-in by controlling your stack
      • Avoiding vendor lock-in by controlling your stack

      This Talk Is Designed For

      • Software developers
      • Anyone that wants to build applications that offer live, interactive streaming
      • Broadcasters and businesses that use OTT (over-the-top communications)

      Prerequisites

      Basic knowledge of how streaming is applied to cloud-based distribution and delivery to end users

      Resources

      Introducing XDN
      (Experience Delivery Network)

      About the Presenters

      Chris Allen, CEO, Red5 Pro
      Mark Pace, CTO, Red5 Pro
      Monica Dunphy, Developer Evangelist, Red5 Pro

      Moderated by
      Darian Wilkin, Senior Solutions Engineer, DigitalOcean

      Need help with large deployments, migration planning, or a proof of concept? Fill out this form and we’ll be in touch!



      Source link