One place for hosting & domains

      Scripting

      An Intermediate Guide to Bash Scripting


      Updated by Linode Contributed by Mihalis Tsoukalos

      In the previous guide of this series, Getting Started with Bash Scripting, you learned Bash basics, like creating and using variables, getting user input, using environment variables, and more. In this guide, you will build off what you have already learned and put together more complex Bash scripts for common operations used by Linux system administrators like creating interactive Bash scripts with menu options, scripts that generate formatted output of your data, and scripts that work with files and directories. Each section will provide a brief introduction to each concept and commands with a few examples that you can run to better understand its function.

      In this guide, you will learn about:

      Before You Begin

      1. All example scripts in this guide are run from the bin directory in the user’s home directory, i.e. /home/username/bin/. If you do not have a bin directory in your home directory, create one and move into the directory:

        cd ~ && mkdir bin && cd bin
        
      2. Verify that the bin directory is in your system PATH (i.e. /home/username/bin):

        echo $PATH
        
      3. If it is not, add the new bin directory to your system’s PATH:

        PATH=$PATH:$HOME/bin/
        

      Note

      Ensure all scripts throughout this guide are executable. To add execute permissions to a file, issue the following command:

      chmod +x my-script.sh
      

      Standard Streams

      A standard stream is a communication mechanism used between a computer program and its environment. Every UNIX operating system contains three types of standard streams, standard input (stdin), standard output (stdout), and standard error (stderr). These three streams are represented by three files, /dev/stdin, /dev/stdout and /dev/stderr. Since these three files are always open, you can redirect their stream to another location. Redirection is when you use the output from one source, a file, program, script, or command and feed it as input to another source. In the context of Bash scripting, you can access stdin, stdout, and stderr using file descriptors 0, 1, and 2, respectively.

      Reading from Standard Input

      Bash scripts very often make use of standard input. The example script input.sh gets its input from a file, but if the file is not available in the expected location, it tries to read standard input (/dev/stdin):

      input.sh
       1
       2
       3
       4
       5
       6
       7
       8
       9
      10
      11
      12
      13
      14
      
      #!/bin/bash
      
      file=$1
      
      if [[ "$file" == "" || (! -f "$file") ]]
      then
          echo Using standard input!
          file="/dev/stdin"
      fi
      
      while read -r line
      do
          echo "$line"
      done < "${file}"
      • The script reads the first value passed as a command line argument, represented by $1. If a text file is passed, the script will read and output each line of text.
      • If a no command line argument is passed or if the file does not exist, standard input (/dev/stdin) is used instead. This will prompt you to enter text and will output to the terminal screen what is received as input. To signal the end of your stdin input type CTRL+D
      1. Using your preferred text editor create an example file for the input.sh script to read:

        echo -e 'Ultimately, literature is nothing but carpentry. nWith both you are working with reality, a material just as hard as wood.' > marquez.txt
        
      2. Run the script and pass marquez.txt as a command line argument:

        ./input.sh marquez.txt
        
          
        Ultimately, literature is nothing but carpentry.
        With both you are working with reality, a material just as hard as wood.
            
        
      3. Run the script without a command line argument:

        ./input.sh
        

        Enter some text after the prompt followed by enter and you will see it echoed back to you in the terminal. Use CTRL+D to end the script.

      You can use the select statement to create menu systems in your bash scripts that users can interact with. When you combine select with the case statement you can create more sophisticated menu options. This section will provide three examples that use select to create menus. If you are not familiar with the case statement, you can refer to our Getting Started with Bash Shell Scripting guide.

      The general format for the select statement is the following:

      bash
      1
      2
      3
      
      select WORD [in LIST];
      do COMMANDS;
      done

      The simple-menu.sh script expands on the skeleton example to create a basic menu that will prompt the user for their favorite color, print out the value of any valid menu selection, and then break out of the select statement:

      simple-menu.sh
      1
      2
      3
      4
      5
      6
      7
      8
      9
      
      #!/bin/bash
      
      echo "Enter the number corresponding to your favorite color:"
      
      select COLOR in blue yellow red green
      do
          echo "Your selection is: $COLOR"
          break
      done
      1. Copy and paste the contents of simple-menu.sh into a new file and save it.

      2. Run the script:

        ./simple-menu.sh
        

        Your output will resemble the following, but may vary depending on the menu selection you make:

          
        Enter the number corresponding to your favorite color:
        1) blue
        2) yellow
        3) red
        4) green
        #? 2
        Your selection is: yellow
            
        

      The second example script, computing-terms.sh, improves on the previous example script by using the case statement and by explicitly providing a way for the user to exit the script. By adding a case for each selection, the script can execute separate tasks based on what the user selects. The reserved Bash variable PS3 is reserved for use with select statements to provide a custom prompt to the user. This script will prompt you to select one of a series of cloud related terms and return its corresponding definition when selected.

      computing-terms.sh
       1
       2
       3
       4
       5
       6
       7
       8
       9
      10
      11
      12
      13
      14
      15
      16
      17
      18
      19
      20
      21
      22
      23
      24
      25
      26
      
      #!/bin/bash
      
      echo "This script shows you how to create select menus in your Bash scripts"
      echo "Enter a number corresponding to the term whose definition you'd like to view"
      PS3="My selection is:"
      
      select TERM in cloud-computing virtual-machine object-storage exit;
      do
          case $TERM in
              cloud-computing)
                  echo "Cloud Computing: A combined system of remote servers, hosted on the internet, to store, manage, and process data."
                  ;;
              virtual-machine)
                  echo "Virtual Machine: The emulating of a computer system, such that a single piece of hardware can deploy and manage a number of host environments, by providing the functionality of physical hardware."
                  ;;
              object-storage)
                  echo "Object Storage: stores data, called objects, in containers, called buckets, and each object is given a unique identifier with which it is accessed."
                  ;;
              exit)
                  echo "You are now exiting this script."
                      break
                      ;;
              *)
                  echo "Please make a selection from the provided options."
          esac
      done
      1. Copy and paste the contents of computing-terms.sh into a new file and save it.

      2. Run the script:

        ./computing-terms.sh
        

        Your output will resemble the following, but may vary depending on the menu selection you make:

          
        This script shows you how to create select menus in your Bash scripts
        Enter a number corresponding to the term whose definition you'd like to view
        1) cloud-computing
        2) virtual-machine
        3) object-storage
        4) exit
        My selection is:3
        Object Storage: stores data, called objects, in containers, called buckets, and each object is given a unique identifier with which it is accessed.
        My selection is:4
        You are now exiting this script.
            
        

      The third example, submenu.sh, uses all the previously covered concepts and enhances them by adding a submenu with a new series of options for the user to select. The script will read all files in the current working directory and display them to the user as selectable options. Once the user selects a file, a submenu will appear prompting the user to select an action to perform on the previously selected file. The submenu allows a user to delete a file, to display the file’s contents, or to simply exit the script.

      submenu.sh
       1
       2
       3
       4
       5
       6
       7
       8
       9
      10
      11
      12
      13
      14
      15
      16
      17
      18
      19
      20
      21
      22
      23
      24
      25
      26
      27
      28
      29
      30
      31
      32
      33
      34
      35
      36
      37
      38
      39
      40
      41
      42
      
      #!/bin/bash
      
      echo "Use this script to manipulate files in your current working directory:"
      echo "----------------------------------------------------------------------"
      echo "Here is a list of all your files. Select a file to access all"
      echo "available file actions:"
      
      select FILE in * exit;
      do
          case $FILE in
          exit)
              echo "Exiting script ..."
              break
              ;;
          *)
              select ACTION in delete view exit;
              do
                  case $ACTION in
                  delete)
                      echo "You've chose to delete your file" "$FILE"
                      rm -i "$FILE"
                      echo "File ""$FILE" "has been deleted"
                      echo "Exiting script ..."
                      break
                      ;;
                  view)
                      echo "Your selected file's contents will be printed to the terminal:"
                      cat "$FILE"
                      echo "------------------------"
                      echo "Exiting script ..."
                      break
                      ;;
                  exit)
                      echo "Exiting script ..."
                      break
                      ;;
                  esac
              done
              break
              ;;
          esac
      done
      1. Copy and paste the contents of submenu.sh into a new file and save it.

      2. Run the script:

        ./submenu.sh
        

        Note

        Ensure that the directory you are executing your script from contains at least one file in order to run through the full demo of the submenu.sh script.

        Your output will resemble the following, but may vary depending on the menu selection you make:

          
        Use this script to manipulate files in your current working directory:
        ----------------------------------------------------------------------
        Here is a list of all your files. Select a file to access all
        available file actions:
        1) example-file-1.txt
        2) example-file-2.txt
        3) exit
        #? 2
        1) delete
        2) view
        3) exit
        #? 2
        Your selected file's contents will be printed to the terminal:
        Lorem ipsum lorem ipsum
        ------------------------
        Script is exiting ...
            
        

      Introduction to the printf Command

      The bash scripting language supports the printf command, which allows you to customize the terminal output of your scripts. Its roots are in the C programming language. You can read about C’s printf function by accessing your operating system’s manual pages with the following command: man 3 printf.

      The general syntax for printf is a format string followed by arguments that will be first modified by the defined format and then inserted into the final output. A format string specifies where and how data will be output by printf.

      printf FORMAT [ARGUMENT]...
      

      You can use variables as arguments to your printf commands. This is a powerful way to write dynamic scripts that will display varied output based on variable values. For example, the following printf command will format your output by adding line breaks, defining an output color, and replacing part of the format string with the argument’s variable value. $PWD is an environment variable that stores your current working directory.

      printf "Your current working directory is: x1b[32mn %sn" $PWD
      
        
      Your current working directory is:
       /home/user
        
      

      Format Strings

      Format strings accept regular characters, which are unchanged in their output and format specifiers, which define where and how a string will be presented in the output.

      Below is a list of common format specifiers:

      • %s: formatting an argument as a string

        printf "%sn" $OSTYPE
        
          
        linux-gnu
            
        
      • %d: printing a value as an integer

        printf "%dn" "0xF9"
        
          
        249
            
        
      • %x: printing a value as a hexadecimal number with lower case a-f. You could similarly use an upper case X to print the hexadecimal value with upper case A-F

        printf "%xn" "2000000"
        
          
        1e8480
            
        
      • %f: printing floating point values

        printf "%fn" "0.01" "0.99"
        
          
        0.010000
        0.990000
            
        

      Note

      The -v var option causes the output of printf to be assigned to a variable instead of being printed to the standard output. In the example below, the result of the printf format specifier and argument will be stored in a variable named myvar. To view the result, the example echoes the value of $myvar.

      printf -v myvar "%dn" "0xF9"
      echo $myvar
      
        
      249
        
      

      Use printf in a Script

      The example script below makes use of printf to create a readable and nicely formatted report of various sequences of numbers. A for loop is used with the seq command to generate the number sequence, while each printf statement uses different format specifiers to provide slightly varying information from each number sequence. Below is a list of the format specifiers used in the script that have not yet been covered:

      1. Copy and paste the contents of printf.sh into a new file and save it.

      2. Run the script:

        ./printf.sh
        

        The output of printf.sh will resemble the following:

          
        0001	0002	0003	0004	0005	0006	0007	0008	0009	0010
        1	2	3	4	5	6	7	8	9	a
        1	2	3	4	5	6	7	8	9	A
        0010	 is A	 in HEX.
        0011	 is B	 in HEX.
        0012	 is C	 in HEX.
        0013	 is D	 in HEX.
        0014	 is E	 in HEX.
        0015	 is F	 in HEX.
        5......... is 5 in HEX.
        6......... is 6 in HEX.
        7......... is 7 in HEX.
        8......... is 8 in HEX.
        9......... is 9 in HEX.
        10........ is A in HEX.
            
        

      File and Directory Test Operators

      Bash offers file and directory test operators that return a boolean value based on each operator’s specific test criteria. These operators can be used in your Bash scripts to present different behaviors depending on the state of a file or directory. A list of all test operators is included in the expandable note, “File and Directory Test Operators” below.

      The general format for file and directory test operators is the following:

      test -[OPERATOR] [FILE]
      

      The example below tests if your /etc/passwd file exists. If the file exists, you will see "Yes, it exists!" printed as output. If the file does not exist, the first part of the command, test -a /etc/passwd, will return an exit status of 1 (the exit value will not print as output) and the second part of the command, echo "Yes, it exists!", will not execute.

      test -a /etc/passwd && echo "Yes, it exists!"
      

      File and Directory Test Operators

      OperatorDescription
      -aFile exists.
      -bFile exists and is a block special file.
      -cFile exists and is a character special file.
      -dFile exists and is a directory.
      -eFile exists and is a file of any type (node, directory, socket, etc.).
      -fFile exists and is a regular file (not a directory or a device file).
      -GFile exists and has the same group as the active user running the bash script.
      -hFiles exists and is a symbolic link.
      -gFiles exists and has the set group ID flag set.
      -kFile exists and has a sticky bit flag set.
      -LFile exists and is a symbolic link.
      -NFile exists and has been modified since it was last read.
      -OFile exists and is owned by the effective user id.
      -pFile exists and is a pipe.
      -rFile exists and is readable.
      -SFile exists and is socket.
      -sFile exists and has a nonzero size.
      -uFile exists and its set user ID flag is set.
      -wFile exists and is writable by the current user.
      -xFile exists and is executable by the current user.

      Use File and Directory Test Operators in a Script

      The example script, file-operator.sh, takes file or directory locations as arguments and returns information about each type of file that is passed to it. The script makes use of file and directory test operators to generate this information. The first if statement tests to ensure you have passed the script arguments. The for loop then goes on to test if the arguments are files that actually exist and then continues through a series of statements to test the file or directory for other criteria.

      Note

      You can use [] and [[]] commands instead of using the if conditional statement to create file conditions. The script makes use of this format on lines 26 – 40.

      file-operator.sh
       1
       2
       3
       4
       5
       6
       7
       8
       9
      10
      11
      12
      13
      14
      15
      16
      17
      18
      19
      20
      21
      22
      23
      24
      25
      26
      27
      28
      29
      30
      31
      32
      33
      34
      35
      36
      37
      38
      39
      40
      41
      42
      
      #!/bin/bash
      
      if [[ $# -le 0 ]]
      then
          echo "You did not pass any files as arguments to the script."
          echo "Usage:" "$0" "my-file-1 my-file-2"
          exit
      fi
      
      for arg in "[email protected]"
      do
          # Does it actually exist?
          if [[ ! -e "$arg" ]]
          then
              echo "* Skipping ${arg}"
              continue
          fi
      
          # Is it a regular file?
          if [ -f "$arg" ]
          then
              echo "* $arg is a regular file!"
          else
              echo "* $arg is not a regular file!"
          fi
      
          [ -b "$arg" ] && echo "* $arg is a block device."
          [ -d "$arg" ] && echo "* $arg is a directory."
          [ ! -d "$arg" ] && echo "* $arg is not a directory."
      
          [ -x "$arg" ] && echo "* $arg is executable."
          [ ! -x "$arg" ] && echo "* $arg is not executable."
      
          [[ -h "$arg" ]] && echo "* $arg is a symbolic link."
          [ ! -h "$arg" ] && echo "* $arg is not a symbolic link."
      
          [[ -s "$arg" ]] && echo "* $arg has nonzero size."
          [ ! -s "$arg" ] && echo "* $arg has zero size."
      
          [[ -r "$arg" && -d "$arg" ]] && echo "* $arg is a readable directory."
          [[ -r "$arg" && -f "$arg" ]] && echo "* $arg is a readable regular file."
      done
      1. Copy and paste the contents of file-operator.sh into a new file and save it.

      2. Run the script and pass it a file location as an argument:

        ./file-operator.sh /dev/fd/2
        

        Your output will resemble the following:

          
        * /dev/fd/2 is not a regular file!
        * /dev/fd/2 is not a directory.
        * /dev/fd/2 is not executable.
        * /dev/fd/2 is not a symbolic link.
        * /dev/fd/2 has zero size.
            
        
      3. Run the script and pass it a directory location as an argument:

        ./file-operator.sh /var/log
        

        Your output will resemble the following:

          
        * /var/log is not a regular file!
        * /var/log is a directory.
        * /var/log is executable.
        * /var/log is not a symbolic link.
        * /var/log has nonzero size.
        * /var/log is a readable directory.
            
        

      Read Files and Searching Directories

      This section will present a few utility scripts that can be adopted and expanded on to perform common operations on files and directories, like reading the contents of a text file by line, word, or character. These scripts make use of several of the concepts and techniques covered in this guide and in the Getting Started with Bash Shell Scripting guide.

      Read a File Line by Line

      The example file, line-by-line.sh, expects a file passed to it as an argument. It will then read the contents of the file line by line. The IFS variable (internal field separator) is a built-in Bash variable that defines how Bash recognizes word boundaries when splitting words. The script sets IFS to the null string to preserve leading and trailing white space within your text file.

      line-by-line.sh
       1
       2
       3
       4
       5
       6
       7
       8
       9
      10
      11
      12
      13
      14
      15
      16
      17
      18
      19
      
      #!/bin/bash
      
      if [[ $# -le 0 ]]
      then
          echo "You did not pass any files as arguments to the script."
          echo "Usage:" "$0" "my-file"
          exit
      fi
      
      file=$1
      
      if [ ! -f "$file" ]
      then
          echo "File does not exist!"
      fi
      
      while IFS='' read -r line || [[ -n "$line" ]]; do
          echo "$line"
      done < "${file}"
      1. Copy and paste the contents of line-by-line.sh into a new file and save it.

      2. Create an example file for the line-by-line.sh script to read. The leading whitespace in the example file is intentional.

        echo -e '     Ultimately, literature is nothing but carpentry. With both you are working with reality, a material just as hard as wood.' > marquez.txt
        
      3. Run the script and pass it a file location as an argument:

        ./line-by-line.sh marquez.txt
        

        Your output will resemble the following:

          
             Ultimately, literature is nothing but carpentry. With both you are working with reality, a material just as hard as wood.
            
        

      Read a File Word by Word

      The example bash script, word-by-word.sh expects a file to be passed as an argument. It will run checks to ensure an argument has been passed to the script and that it is a file. It then uses a for loop with the cat command to echo each word in the file to your output. The default value of the IFS variable separates a line into words, so in this case there is no need to change its value.

      word-by-word.sh
       1
       2
       3
       4
       5
       6
       7
       8
       9
      10
      11
      12
      13
      14
      15
      16
      17
      18
      19
      20
      
      #!/bin/bash
      
      if [[ $# -le 0 ]]
      then
          echo "You did not pass any files as arguments to the script."
          echo "Usage:" "$0" "my-file"
          exit
      fi
      
      file=$1
      
      if [ ! -f "$file" ]
      then
          echo "File does not exist!"
      fi
      
      for word in $(cat "${file}")
      do
          echo "$word"
      done
      1. Copy and paste the contents of word-by-word.sh into a new file and save it.

      2. Create an example file for the word-by-word.sh script to read.

        echo -e 'Ultimately, literature is nothing but carpentry. With both you are working with reality, a material just as hard as wood.' > marquez.txt
        
      3. Run the script and pass it a file location as an argument:

        ./word-by-word.sh marquez.txt
        

        Your output will resemble the following:

          
        Ultimately,
        literature
        is
        nothing
        but
        carpentry.
        With
        both
        you
        are
        working
        with
        reality,
        a
        material
        just
        as
        hard
        as
        wood.
            
        

      Read a File Character by Character

      The example bash script, char-by-char.sh expects a file to be passed as an argument. It will run checks to ensure an argument has been passed to the script and that it is a file. It then uses a while loop with the read command to echo each character in the file to your shell’s output. The -n1 flag is added to the standard read command in order to specify the number of characters to read at a time, which in this case is 1.

      char-by-char.sh
       1
       2
       3
       4
       5
       6
       7
       8
       9
      10
      11
      12
      13
      14
      15
      16
      17
      18
      19
      
      #!/bin/bash
      
      if [[ $# -le 0 ]]
      then
          echo "You did not pass any files as arguments to the script."
          echo "Usage:" "$0" "my-file"
          exit
      fi
      
      file=$1
      
      if [ ! -f "$file" ]
      then
          echo "File does not exist!"
      fi
      
      while read -r -n1 char; do
          echo "$char"
      done < "${file}"
      1. Copy and paste the contents of char-by-char.sh into a new file and save it.

      2. Create an example file for the char-by-char.sh script to read. The leading whitespace in the example file is intentional.

        echo -e 'Linode' > linode.txt
        
      3. Run the script and pass it a file location as an argument:

        ./char-by-char.sh linode.txt
        

        Your output will resemble the following:

          
        L
        i
        n
        o
        d
        e
            
        

      Search Directories

      The bash script, search.sh will search a directory for files and directories that begin with the string passed as a command line argument. All matching regular files and directories will be presented as output. The script expects the search string as the first argument and a directory location as the second argument. The script uses the find UNIX command for searching a directory and looks for everything that begins with matched regular expression, $string*.

      search.sh
       1
       2
       3
       4
       5
       6
       7
       8
       9
      10
      11
      12
      13
      14
      15
      16
      17
      18
      19
      20
      21
      22
      23
      24
      25
      26
      27
      28
      
      #!/bin/bash
      
      if [[ $# -le 1 ]]
      then
          echo "You did not pass any files as arguments to the script."
          echo "Usage:" "$1" "my-file"
          exit
      fi
      
      dir=$2
      string=$1
      
      if [ ! -d "$dir" ]
      then
          echo "Directory" "$dir" "does not exist!"
          exit
      fi
      
      for i in $(find "$dir" -name "$string*");
      do
          if [ -d "$i" ]
          then
             echo "$i" "[Directory]"
          elif [ -f "$i" ]
          then
              echo "$i" "[File]"
          fi
      done
      1. Copy and paste the contents of search.sh into a new file and save it.

      2. Move to your home directory and create an example file and directory for the search.sh script to find.

        cd ~ && echo -e 'Ultimately, literature is nothing but carpentry.' > marquez.txt && mkdir marketing
        
      3. Run the script and pass it a string and directory location as arguments. Ensure you replace the value of /home/user/ with your own home directory.

        ./bin/search.sh mar /home/user/
        

        Your output will resemble the following:

          
        /home/user/marketing [Directory]
        /home/user/marquez.txt [File]
            
        

      Bash Exit Codes

      An exit code is the code returned to a parent process after executing a command or a program. Using exit codes in your Bash scripts allows the script to modify its behavior based on the success or failure of your script’s commands. Exit codes range between 0 - 255. An exit code of 0 indicates success, while any non-zero value indicates failure. This section will provide an introduction to Bash exit codes and a few examples on how to use them in your scripts.

      Exit Codes

      The table lists and describes reserved exit codes. You should not use any of the reserved exit codes in your Bash scripts.

      CodeDescription
      0Successful execution
      1General failure
      2Incorrect usage of Bash built-in commands, by using invalid options, or missing arguments
      126Command found, but is not executable
      127Command not found
      128+nCommand terminated on a fatal signal n. The final exit code will be 128 plus the corresponding termination signal number. For example, a script that is terminated using the kill signal will have an exit code of 137 (128+9).
      130Execution terminated by CTRL-C
      255Exit status out of range

      Learning the Exit Code of a Shell Command

      You can understand whether a bash command was executed successfully or not by accessing the exit code of the command. The built-in Bash variable $? stores the exit (return) status of the previously executed command. The example below issues the long format list files (ls -l) command against your /tmp directory and redirects standard output and standard error to /dev/null in order to suppress any output. Without any direct output there is no way of knowing if the command executed successfully or failed. To circumvent this scenario you can echo the value of the $? variable to view the command’s exit status.

      1. Execute the following example command. You should not see any output, however, the command should have executed successfully.

        ls -l /tmp 2>/dev/null 1>/dev/null
        
      2. Find the value of $? to determine if your command executed successfully or not.

        echo "$?"
        

        The exit code status should output 0 if the command was successful:

          
        0
              
        
      3. Issue the long form list files command against a directory that does not exist

        ls -l /doesNotExist 2>/dev/null 1>/dev/null
        
      4. Find the value of $? to determine if your command executed successfully or not.

        echo "$?"
        

        The exit code status should output 1 if the command failed:

          
        1
              
        

        Note

        After you execute echo $?, the value of $? will always be 0 because echo $? was successfully executed.

      Using set -e

      The set command is used to set or unset different shell options or positional parameters. A very useful option that can be set with this command is the -e option, which causes a bash script to exit if any command or statement generates a non-zero exit code. This option is useful, because it works globally on all commands contained in a script, so you don’t have to test the return status of each command that is executed.

      The example script, set-example.sh, tries to create a file at the specified path. If the file cannot be written to and created, the script will immediately exit and none of the remaining commands will be executed. In the case of a non-zero exit code, you should not expect to see the last line execute the echo "Script is exiting" command.

      set-example.sh
       1
       2
       3
       4
       5
       6
       7
       8
       9
      10
      11
      12
      13
      14
      15
      16
      17
      18
      19
      20
      21
      22
      23
      24
      25
      
      #!/bin/bash
      
      set -e
      
      if [[ $# -le 0 ]]
      then
          echo "You did not pass any file paths as arguments to the script."
          echo "Usage:" "$0" "my-new-file-path"
          exit
      fi
      
      fpath=$1
      
      echo "About to create file: " "$fpath"
      
      if [ -e "$fpath" ]
      then
          echo "${fpath}" "already exists!"
          exit
      fi
      
      echo "Creating and writing to the file: " "$fpath"
      echo "Test" >> "$fpath"
      
      echo "Script is exiting"
      1. Copy and paste the contents of set-example.sh into a new file and save it.

      2. Run the script and pass it a file location as an argument.

        ./set-example.sh /tmp/new-file
        

        Creating a file in this location should be successful and your output will resemble the following:

          
        About to create file:  /tmp/new-file
        About to create and write to the file:  /tmp/new-file
        Script is exiting
            
        
      3. Now, run the script and pass it a file location that you likely do not have elevated enough permissions to write to.

        ./set-example.sh /dev/new-file
        

        Creating a file in this location should not be successful and your script will exit prior to executing the final echo command:

          
        About to create file:  /dev/new-file
        About to create and write to the file:  /dev/new-file
        ./set-e.sh: line 23: /dev/new-file: Permission denied
            
        

      Using set -x

      Another handy way to use the set command is by enabling the -x option. This option displays commands and arguments before they’re executed, which makes this a great option for debugging scripts.

      Note

      Any output generated by the set -x execution trace will be preceded by a + character. This value is stored in the built-in variable, PS4.

      The example script below, debug-set-example.sh, contains identical code to the example in the previous section, however, it makes use of set -x in order to print out all commands before they’re executed.

      debug-set-example.sh
       1
       2
       3
       4
       5
       6
       7
       8
       9
      10
      11
      12
      13
      14
      15
      16
      17
      18
      19
      20
      21
      22
      23
      24
      25
      
      #!/bin/bash
      
      set -xe
      
      if [[ $# -le 0 ]]
      then
          echo "You did not pass any file paths as arguments to the script."
          echo "Usage:" "$0" "my-new-file-path"
          exit
      fi
      
      fpath=$1
      
      echo "About to create file: " "$fpath"
      
      if [ -e "$fpath" ]
      then
          echo "${fpath}" "already exists!"
          exit
      fi
      
      echo "Creating and writing to the file: " "$fpath"
      echo "Test" >> "$fpath"
      
      echo "Script is exiting"
      1. Copy and paste the contents of debug-set-example.sh into a new file and save it.

      2. Run the script and pass it a file location as an argument.

        ./debug-set-example.sh /dev/new-file
        

        Creating a file in this location should not be successful and your script will exit prior to executing the final echo command. However, since you also have the set -x option enabled, you will be able to clearly see on which command the script exited.

          
        + [[ 1 -le 0 ]]
        + fpath=/dev/new-file
        + echo 'About to create file: ' /dev/new-file
        About to create file:  /dev/new-file
        + '[' -e /dev/new-file ']'
        + echo 'About to create and write to the file: ' /dev/new-file
        About to create and write to the file:  /dev/new-file
        + echo Test
        ./set-e.sh: line 23: /dev/new-file: Permission denied
            
        

      More Information

      You may wish to consult the following resources for additional information on this topic. While these are provided in the hope that they will be useful, please note that we cannot vouch for the accuracy or timeliness of externally hosted materials.

      Find answers, ask questions, and help others.

      This guide is published under a CC BY-ND 4.0 license.



      Source link

      Introduction to Bash Shell Scripting


      Updated by Linode Contributed by Mihalis Tsoukalos

      Introduction

      This guide is an introduction to bash shell programming. Bash shell programming empowers Linux users to take programmatic control over the Linux operating system. The bash shell provides a number of concepts common to many other programming languages, so if you know another language then you should be able to pick up bash with relative ease.

      In This Guide

      Among other things, you will learn about:

      Note

      This guide is written for a non-root user. Depending on your configuration, some commands might require the help of sudo in order to properly execute. If you are not familiar with the sudo command, see the Users and Groups guide.

      Bash Basics

      The bash Executable

      The bash shell is an executable file. The executable file for the bash shell can be usually found inside /bin – its full path being /bin/bash. Please keep this file path in mind as it will be used in all bash scripts.

      Bash Scripts

      The following code is the “Hello World” program written in bash(1):

      hello_world.sh
      1
      2
      3
      
      #!/bin/bash
      
      echo "Hello World!"

      The first line is required for the bash script to become autonomous and executable as a command. The #! characters are called a shebang, and instruct Linux to use following path as the file interpreter. The .sh file extension is not required but it is good to have it in order to inform people that this is a shell script and not a binary file.

      Notice that echo is a bash shell built-in command that outputs text. All shells have their own built-in commands.

      In order for a bash script to be executable, it needs to have the appropriate file permissions. To give a file permissions that allow it to be executable, use the chmod +x file.sh command, substituting file.sh for the name of the file. After that you can execute it as ./file.sh. Alternatively you can use chmod 755 file.sh.

      For the hello_world.sh script to become executable, you will need to run one of the following two commands:

      chmod +x hello_world.sh
      chmod 755 hello_world.sh
      

      After that the file permissions of hello_world.sh will be similar to the following:

      ls -l hello_world.sh
      
        
      -rwxr-xr-x 1 mtsouk  staff  32 Aug  1 20:09 hello_world.sh
      
      

      Note

      You will need to give all bash scripts of this guide the execute file permission in order to be able to execute them as regular UNIX commands. For more information on file permissions, see our Linux Users and Groups Guide.

      Executing hello_world.sh will generate the following output:

      ./hello_world.sh
      
        
      Hello World!
      
      

      The ./ in front of the script name tells bash that the file you want to execute is in the current directory. This is necessary for executing any file that is not located in the PATH. The PATH environment variable contains a list of directories that bash will search through for executable commands. You can execute echo $PATH to find its current value.

      Note

      The # character is used for adding single line comments in bash scripts. The bash shell also supports multi line comments, but they are not used as often, so it would be better to use multiple single line comments when you want to write bigger comment blocks.

      Defining and Using Variables

      The programming language of the bash shell has support for variables. Variables, like in math, have values that can be declared in a program and later changed or passed around to different functions. Variables are illustrated in vars.sh, which is as follows:

      vars.sh
       1
       2
       3
       4
       5
       6
       7
       8
       9
      10
      11
      
      #!/bin/bash
      
      VAR1="Mihalis"
      myVar="$(pwd)"
      
      echo "My name is ${VAR1}"
      echo -n "and I work from "
      echo $myVar
      
      myVar=`pwd`
      echo $myVar

      There are two variables defined in this example: the first one is called VAR1 and the second one is called myVar. Although both variables are defined inside the program, the first variable is defined with a direct assignment whereas the second variable is defined as the output of an external program, the pwd(1) command, which outputs the current working directory. The value of myVar depends on your place in the filesystem.

      The two variables are read as ${VAR1} and $myVar, respectively – both notations work. Notice that in order to prevent echo from printing a newline character, you will have to call it as echo -n. Lastly, notice that "$(pwd)" and `pwd` (note the use of backticks instead of quotation marks) are equivalent.

      Executing vars.sh will generate the following kind of output:

      ./vars.sh
      
        
      My name is Mihalis
      and I work from /home/mtsouk/
      /home/mtsouk/
      
      

      Getting User Input

      The bash shell offers the read command for getting user input. However, this is rarely used because it makes bash shell scripts less autonomous as it depends on user interaction. Nevertheless, the read.sh script illustrates the use of read:

      read.sh
       1
       2
       3
       4
       5
       6
       7
       8
       9
      10
      
      #!/bin/bash
      
      echo -n "What is your name? "
      read name
      
      echo "Hello" "$name!"
      
      echo -n "Please state your name and your surname: "
      read name surname
      echo "Hello" "$name $surname!"

      Executing read.sh will generate the following kind of output:

      ./read.sh
      
        
      What is your name? Mihalis
      Hello Mihalis!
      Please state your name and your surname: Mihalis Tsoukalos
      Hello Mihalis Tsoukalos!
      
      

      The second read is different than the first, because it accepts two variable values. The read command looks for a space or tab separator in the input text in order to split the text into multiple values. If more than one space is provided, then all remaining values are combined. This means that if the user’s surname is a compound of two or more additional words they will all become the value of $surname.

      The sections on Environment Variables and Command Line Arguments show alternative ways of retrieving user input that is more common in the UNIX world than the read command.

      The if statement

      The bash shell supports if statements using a unique syntax, which is illustrated in whatIf.sh:

      whatIf.sh
       1
       2
       3
       4
       5
       6
       7
       8
       9
      10
      11
      12
      13
      14
      15
      16
      17
      18
      19
      20
      21
      22
      23
      24
      25
      26
      
      #!/bin/bash
      
      VAR1="4"
      VAR2="4"
      
      if [ $VAR1 == 4 ]
      then
          echo Equal!
      fi
      
      if [ "$VAR1" == 4 ]
      then
          echo Equal!
      else
          echo Not equal!
      fi
      
      if [ "$VAR1" == $VAR2 ]
      then
          echo Equal!
      elif [ "$VAR1" == $VAR1 ]
      then
          echo Tricky Equal!
      else
          echo Not equal!
      fi

      if statements allow for logic to be applied to a block of code. If the statement is true, the code is executed. if statements in bash script use square brackets for the logical condition and also have support for else and elif (else if) branches. Bash supports standard programming language conditional operators such as equals (==), not equals (!=), less than and greater than (<, >), and a number of other file specific operators.

      All if statements contain a conditional express, and a then statement, and all statements are ended with fi.

      As filenames and paths may contain space characters, it is good to embed them in double quotes – this has nothing to do with the if statement per se. Notice that this unofficial rule applies to other variables that might contain space characters in them. This is illustrated with the use of the VAR1 variable. Lastly, the == operator is for checking string values for equality.

      ./whatIf.sh
      
        
      Equal!
      Not equal!
      Not equal!
      
      

      The if statement is used extensively in bash scripts, which means that you are going to see it many times in this guide.

      Loops

      The bash shell has support for loops, which are illustrated in this section of the guide using the code of loops.sh:

      loops.sh
       1
       2
       3
       4
       5
       6
       7
       8
       9
      10
      11
      12
      13
      14
      15
      16
      17
      18
      19
      20
      21
      22
      23
      24
      25
      26
      27
      28
      29
      30
      31
      32
      33
      34
      35
      36
      37
      38
      
      #!/bin/bash
      
      # For loop
      numbers="One Two Three List"
      for n in $numbers
      do
          echo $n
      done
      
      for x in {1..4}
      do
          echo -n "$x "
      done
      echo
      
      for x in ./c*.sh
      do
          echo -n "$x "
      done
      echo
      
      # While loop
      c=0
      while [ $c -le 5 ]
      do
          echo -n "$c "
          ((c++))
      done
      echo
      
      # Until loop
      c=0
      until [ $c -gt 5 ]
      do
          echo -n "$c "
          ((c++))
      done
      echo

      Note

      The bash scripting language offers support for the break statement for exiting a loop, and the continue statement for skipping the current iteration.

      The loops.sh example begins with three for loops. These loops will iterate over values in a series, here represented by the numbers list variable or a range like {1..4}, and complete the block of code after the do command for each value. In a set of four values a loop will iterate four times. Notice that the third for loop processes the output of ./c*.sh, which is equivalent to the output of the ls ./c*.sh command – this is a pretty handy way of selecting and processing files from the Linux filesystem.

      Similarly, the while statement and the until statement will continually loop so long as the conditional statement is true (while), or until the statement becomes true (until). The -le and -gt operators used in the while and until loops are used strictly to compare numbers, and mean “less than or equal to” and “greater than,” respectively.

      Executing loops.sh will create the following output:

      ./loops.sh
      
        
      One
      Two
      Three
      List
      1 2 3 4
      ./case.sh ./cla.sh
      0 1 2 3 4 5
      0 1 2 3 4 5
      
      

      Note

      The ./case.sh and ./cla.sh scripts will be created in later sections of this guide.

      Using UNIX Environment Variables

      In this section of the guide you will learn how to read a UNIX environment variable, change its value, delete it, and create a new one. This is a very popular way of getting user input or reading the setup of the current user.

      The related bash shell script is called env.sh and is as follows:

      env.sh
       1
       2
       3
       4
       5
       6
       7
       8
       9
      10
      11
      12
      13
      14
      15
      16
      17
      18
      19
      20
      21
      22
      23
      24
      
      #!/bin/bash
      
      # Read
      if [[ -z "${PATH}" ]]; then
          echo "PATH is empty!"
      else
          echo "PATH: $PATH"
      fi
      
      # Change
      PATH=${PATH}:/tmp
      echo "PATH: $PATH"
      
      # Delete
      export PATH=""
      if [[ -z "${PATH}" ]]; then
          echo "PATH is empty!"
      else
          echo "PATH: $PATH"
      fi
      
      # Create
      MYPATH="/bin:/sbin:/usr/bin"
      echo "MYPATH: ${MYPATH}"

      Notice that the PATH environment variable is automatically available to the bash script. You can view it’s current value in the output of the first if statement. The -z operator tests whether a variable has a length of zero or not and can be pretty handy when checking if an environment variable is set or not.

      Executing env.sh will create the following output:

      ./env.sh
      
        
      PATH: /usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/texbin:/opt/X11/bin
      PATH: /usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/texbin:/opt/X11/bin:/tmp
      PATH is empty!
      MYPATH: /bin:/sbin:/usr/bin
      
      

      Note

      Notice that all changes to environment variables that take place inside a bash script will be lost when that bash script ends because they have a local scope.

      Bash and Command Line Arguments

      Command Line Arguments

      The easiest and most common way to pass your own data to scripts is the use of command line arguments. For instance, review the following command:

      ./cla.sh 1 2 3
      

      This example command is executing the cla.sh command, and supplying a number of arguments, in this case the numbers 1, 2, and 3. Those numbers could be any type of information the bash script needs to execute.

      The cla.sh bash script demonstrates how to work with command line arguments:

      cla.sh
       1
       2
       3
       4
       5
       6
       7
       8
       9
      10
      11
      12
      13
      14
      15
      16
      17
      
      #!/bin/bash
      
      echo "Arguments: [email protected]"
      echo "Number of arguments: $#"
      
      for arg in "[email protected]"
      do
          echo "$arg"
      done
      
      echo "The name of the script is: $0"
      echo "The first argument is: $1"
      
      if [ -x $0 ]
      then
          echo "$0" file exists!
      fi

      The full list of arguments is stored as [email protected] and the number of arguments is stored as $#. A for loop can be used for iterating over the list of command line arguments. Lastly, the name of the program is always $0 and the first command line argument, if it exists, is always $1.

      Executing cla.sh will generate the following output:

      ./cla.sh 1 2 3
      
        
      Arguments: 1 2 3
      Number of arguments: 3
      1
      2
      3
      The name of the script is: ./cla.sh
      The first argument is: 1
      ./cla.sh file exists!
      
      

      Checking the Number of Command Line Arguments

      In the previous section you learned how to pass command line arguments to a bash script. The following bash script, which is named nCla.sh, requires that you pass at least two command line arguments to it:

      nCla.sh
      1
      2
      3
      4
      5
      6
      7
      8
      
      #!/bin/bash
      
      if [ "$#" -lt 2 ]
      then
          echo Need more arguments than $#!
      else
          echo "Thanks for the $# arguments!"
      fi

      Notice that numeric comparisons require the use of -lt in an if statement, which is an alias for less than.

      Executing nCla.sh with the right number of arguments will generate the following output:

      ./nCla.sh 1 2
      
        
      Thanks for the 2 arguments!
      
      

      Executing nCla.sh with an insufficient number of arguments will generate the following output:

      ./nCla.sh
      
        
      Need more arguments than 0!
      
      

      Combining Commands in bash Scripts

      Bash has the additional capability of executing a combination of commands. This capability is illustrated in combine.sh:

      combine.sh
       1
       2
       3
       4
       5
       6
       7
       8
       9
      10
      11
      12
      13
      14
      
      #!/bin/bash
      
      total=0
      for n in ./*.sh
      do
          ti=`grep while ${n} | wc -l`
          ti=$(($ti + 0))
          if [[ $ti -gt 0 ]]
          then
              total=$(($total+$ti))
          fi
      done
      
      echo "Total:" $total

      The for loop in this example iterates over every bash script file in the current working directory. The initial value of ti is taken from the output of a command with two parts. The first part uses grep to look for the while word in the file that is being processed and the second part counts the number of times that the while word was found in the current file. The total=$(($total+$ti)) statement is needed for adding the value of ti to the value of total. Additionally, the ti=$(($ti + 0)) statement is used for converting the value of ti from string to integer. Last, before exiting, combine.sh prints the total number of times the while word was found in all processed files.

      ./combine.sh
      
        
      Total: 2
      
      

      The Case Statement

      The bash scripting language supports the case statement. A case statement provides a number of possible values for a variable and maps code blocks to those values. For example, the case statement included in case.sh script below defines a number of possible outputs depending on the number provided as a command line argument:

      case.sh
       1
       2
       3
       4
       5
       6
       7
       8
       9
      10
      11
      12
      13
      14
      15
      16
      17
      18
      19
      20
      21
      22
      23
      24
      25
      26
      27
      28
      29
      30
      31
      32
      33
      34
      35
      36
      37
      38
      39
      40
      41
      42
      43
      44
      45
      46
      47
      48
      49
      50
      51
      52
      
      #!/bin/bash
      
      if [ $# -lt 1 ]
      then
          echo "Usage : $0 integer"
          exit
      fi
      
      NUM=$1
      echo "Testing ${NUM}"
      
      if [[ ! $NUM =~ ^[0-9]+$ ]] ; then
          echo "Not an integer"
          exit
      fi
      
      case $NUM in
          0)
              echo "Zero!"
              ;;
          1)
              echo "One!"
              ;;
          ([2-9]|[1-7][0-9]|80) echo "From 2 to 80"
              ;;
          (8[1-9]|9[0-9]|100) echo "From 81 to 100"
              ;;
          *)
              echo "Too big!"
              ;;
      esac
      
      case  1:${NUM:--} in
      (1:*[!0-9]*|1:0*[89]*)
          ! echo NAN
      ;;
      ($((NUM<81))*)
          echo "$NUM smaller than 80"
      ;;
      ($((NUM<101))*)
          echo "$NUM between 81 and 100"
      ;;
      ($((NUM<121))*)
          echo "$NUM between 101 and 120"
      ;;
      ($((NUM<301))*)
          echo "$NUM between 121 and 300"
      ;;
      ($((NUM>301))*)
          echo "$NUM greater than 301"
      ;;
      esac

      The script requires a command line argument, which is an integer value. A regular expression verifies that the input is a valid positive integer number with the help of an if statement.

      From the presented code you can understand that branches in case statements do not offer direct support for numeric ranges, which makes the code more complex. If you find the code difficult to understand, you may also use multiple if statements instead of a case block.

      case.sh illustrates two ways of supporting ranges in a case statement. The first one uses regular expressions (which are divided by the OR operator, a pipe |), whereas the second offers ranges through a different approach. Each expression such as (NUM<81) is evaluated and if it is true, the code in the respective branch is executed. Notice that the order of the branches is significant because only the code from the first match will be executed.

      Executing case.sh will generate the following output:

      ./case.sh 12
      
        
      Testing 12
      From 2 to 80
      12 smaller than 80
      
      

      If you give case.sh a different input, you will get the following output:

      ./case.sh 0
      
        
      Testing 0
      Zero!
      0 smaller than 80
      
      

      File and Directory Basics

      Working with Files and Directories

      One often important operation that you may find yourself needing to perform is specifying whether a given file or directory actually exists. This is idea is illustrated in files.sh:

      files.sh
       1
       2
       3
       4
       5
       6
       7
       8
       9
      10
      11
      12
      13
      14
      15
      16
      17
      18
      19
      20
      21
      22
      23
      24
      25
      26
      
      #!/bin/bash
      
      if [[ $# -le 0 ]]
      then
          echo Not enough arguments!
      fi
      
      for arg in "[email protected]"
      do
          # Does it actually exist?
          if [[ -e "$arg" ]]
          then
              echo -n "$arg exists "
          fi
      
          # Is it a file or Is it a directory?
          if [ -f "$arg" ]
          then
              echo "and is a regular file!"
          elif [ -d "$arg" ]
          then
              echo "and is a regular directory!"
          else
              echo "and is neither a regular file nor a regular directory!"
          fi
      done

      The -e operator will check whether a file exists regardless of its type – this is the first test that we are performing. The other two tests use -f and -d for specifying whether we are dealing with a regular file or a directory, respectively. Last, the -le operator stands for less than or equal and is used for comparing numeric values.

      Executing files.sh will generate the following output:

      ./files.sh /tmp aFile /dev/stdin
      
        
      /tmp exists and is a regular directory!
      aFile exists and is a regular file!
      /dev/stdin exists and is neither a regular file nor a regular directory!
      
      

      The following bash script will accept one command line argument, which is a string you’d like to find, and then a list of files that will be searched for that given string. If there is a match, then the filename of the file will appear on the screen.

      match.sh
       1
       2
       3
       4
       5
       6
       7
       8
       9
      10
      11
      12
      13
      14
      15
      16
      17
      18
      19
      20
      21
      22
      23
      24
      25
      26
      27
      28
      29
      
      #!/bin/bash
      
      if [[ $# -le 1 ]]
      then
          echo Usage: $0 string files!
      fi
      
      string=$1
      for arg in "${@:2}"
      do
          # Does it actually exist?
          if [[ ! -e "$arg" ]]
          then
              echo "* Skipping ${arg}"
              continue
          fi
          # Is it a regular file?
          if [ -f "$arg" ]
          then
              ti=`grep ${string} ${arg} | wc -l`
              ti=$(($ti + 0))
              if [[ $ti -gt 0 ]]
              then
                  echo ${arg}
              fi
          else
              echo "* $arg is not a regular file!"
          fi
      done

      The "${@:2}" notation allows you to skip the first element from the list of command line arguments because this is the string that we will be looking for in the list of files that follow.

      Executing match.sh will generate the following output:

      ./match.sh while *.sh /tmp /var/log ./combine.sh doesNotExist
      
        
      combine.sh
      loops.sh
      * /tmp is not a regular file!
      * /var/log is not a regular file!
      ./combine.sh
      * Skipping doesNotExist
      
      

      In part 2 of the bash scripting guides you will learn more about working with files and directories with the bash scripting language.

      Summary

      The scripting language of bash can do many more things than the ones presented in this guide. The next part of this guide will present more interesting bash shell scripts and shed more light into topics such as working with files and directories, the printf command, the select statement and reading files.

      More Information

      You may wish to consult the following resources for additional information on this topic. While these are provided in the hope that they will be useful, please note that we cannot vouch for the accuracy or timeliness of externally hosted materials.

      Find answers, ask questions, and help others.

      This guide is published under a CC BY-ND 4.0 license.



      Source link

      Introduction to Bash Shell Scripting


      Updated by Linode Contributed by Mihalis Tsoukalos

      Introduction

      This guide is an introduction to bash shell programming. Bash shell programming empowers Linux users to take programmatic control over the Linux operating system. The bash shell provides a number of concepts common to many other programming languages, so if you know another language then you should be able to pick up bash with relative ease.

      In This Guide

      Among other things, you will learn about:

      Note

      This guide is written for a non-root user. Depending on your configuration, some commands might require the help of sudo in order to properly execute. If you are not familiar with the sudo command, see the Users and Groups guide.

      Bash Basics

      The bash Executable

      The bash shell is an executable file. The executable file for the bash shell can be usually found inside /bin – its full path being /bin/bash. Please keep this file path in mind as it will be used in all bash scripts.

      Bash Scripts

      The following code is the “Hello World” program written in bash(1):

      hello_world.sh
      1
      2
      3
      
      #!/bin/bash
      
      echo "Hello World!"

      The first line is required for the bash script to become autonomous and executable as a command. The #! characters are called a shebang, and instruct Linux to use following path as the file interpreter. The .sh file extension is not required but it is good to have it in order to inform people that this is a shell script and not a binary file.

      Notice that echo is a bash shell built-in command that outputs text. All shells have their own built-in commands.

      In order for a bash script to be executable, it needs to have the appropriate file permissions. To give a file permissions that allow it to be executable, use the chmod +x file.sh command, substituting file.sh for the name of the file. After that you can execute it as ./file.sh. Alternatively you can use chmod 755 file.sh.

      For the hello_world.sh script to become executable, you will need to run one of the following two commands:

      chmod +x hello_world.sh
      chmod 755 hello_world.sh
      

      After that the file permissions of hello_world.sh will be similar to the following:

      ls -l hello_world.sh
      
        
      -rwxr-xr-x 1 mtsouk  staff  32 Aug  1 20:09 hello_world.sh
      
      

      Note

      You will need to give all bash scripts of this guide the execute file permission in order to be able to execute them as regular UNIX commands. For more information on file permissions, see our Linux Users and Groups Guide.

      Executing hello_world.sh will generate the following output:

      ./hello_world.sh
      
        
      Hello World!
      
      

      The ./ in front of the script name tells bash that the file you want to execute is in the current directory. This is necessary for executing any file that is not located in the PATH. The PATH environment variable contains a list of directories that bash will search through for executable commands. You can execute echo $PATH to find its current value.

      Note

      The # character is used for adding single line comments in bash scripts. The bash shell also supports multi line comments, but they are not used as often, so it would be better to use multiple single line comments when you want to write bigger comment blocks.

      Defining and Using Variables

      The programming language of the bash shell has support for variables. Variables, like in math, have values that can be declared in a program and later changed or passed around to different functions. Variables are illustrated in vars.sh, which is as follows:

      vars.sh
       1
       2
       3
       4
       5
       6
       7
       8
       9
      10
      11
      
      #!/bin/bash
      
      VAR1="Mihalis"
      myVar="$(pwd)"
      
      echo "My name is ${VAR1}"
      echo -n "and I work from "
      echo $myVar
      
      myVar=`pwd`
      echo $myVar

      There are two variables defined in this example: the first one is called VAR1 and the second one is called myVar. Although both variables are defined inside the program, the first variable is defined with a direct assignment whereas the second variable is defined as the output of an external program, the pwd(1) command, which outputs the current working directory. The value of myVar depends on your place in the filesystem.

      The two variables are read as ${VAR1} and $myVar, respectively – both notations work. Notice that in order to prevent echo from printing a newline character, you will have to call it as echo -n. Lastly, notice that "$(pwd)" and `pwd` (note the use of backticks instead of quotation marks) are equivalent.

      Executing vars.sh will generate the following kind of output:

      ./vars.sh
      
        
      My name is Mihalis
      and I work from /home/mtsouk/
      /home/mtsouk/
      
      

      Getting User Input

      The bash shell offers the read command for getting user input. However, this is rarely used because it makes bash shell scripts less autonomous as it depends on user interaction. Nevertheless, the read.sh script illustrates the use of read:

      read.sh
       1
       2
       3
       4
       5
       6
       7
       8
       9
      10
      
      #!/bin/bash
      
      echo -n "What is your name? "
      read name
      
      echo "Hello" "$name!"
      
      echo -n "Please state your name and your surname: "
      read name surname
      echo "Hello" "$name $surname!"

      Executing read.sh will generate the following kind of output:

      ./read.sh
      
        
      What is your name? Mihalis
      Hello Mihalis!
      Please state your name and your surname: Mihalis Tsoukalos
      Hello Mihalis Tsoukalos!
      
      

      The second read is different than the first, because it accepts two variable values. The read command looks for a space or tab separator in the input text in order to split the text into multiple values. If more than one space is provided, then all remaining values are combined. This means that if the user’s surname is a compound of two or more additional words they will all become the value of $surname.

      The sections on Environment Variables and Command Line Arguments show alternative ways of retrieving user input that is more common in the UNIX world than the read command.

      The if statement

      The bash shell supports if statements using a unique syntax, which is illustrated in whatIf.sh:

      whatIf.sh
       1
       2
       3
       4
       5
       6
       7
       8
       9
      10
      11
      12
      13
      14
      15
      16
      17
      18
      19
      20
      21
      22
      23
      24
      25
      26
      
      #!/bin/bash
      
      VAR1="4"
      VAR2="4"
      
      if [ $VAR1 == 4 ]
      then
          echo Equal!
      fi
      
      if [ "$VAR1" == 4 ]
      then
          echo Equal!
      else
          echo Not equal!
      fi
      
      if [ "$VAR1" == $VAR2 ]
      then
          echo Equal!
      elif [ "$VAR1" == $VAR1 ]
      then
          echo Tricky Equal!
      else
          echo Not equal!
      fi

      if statements allow for logic to be applied to a block of code. If the statement is true, the code is executed. if statements in bash script use square brackets for the logical condition and also have support for else and elif (else if) branches. Bash supports standard programming language conditional operators such as equals (==), not equals (!=), less than and greater than (<, >), and a number of other file specific operators.

      All if statements contain a conditional express, and a then statement, and all statements are ended with fi.

      As filenames and paths may contain space characters, it is good to embed them in double quotes – this has nothing to do with the if statement per se. Notice that this unofficial rule applies to other variables that might contain space characters in them. This is illustrated with the use of the VAR1 variable. Lastly, the == operator is for checking string values for equality.

      ./whatIf.sh
      
        
      Equal!
      Not equal!
      Not equal!
      
      

      The if statement is used extensively in bash scripts, which means that you are going to see it many times in this guide.

      Loops

      The bash shell has support for loops, which are illustrated in this section of the guide using the code of loops.sh:

      loops.sh
       1
       2
       3
       4
       5
       6
       7
       8
       9
      10
      11
      12
      13
      14
      15
      16
      17
      18
      19
      20
      21
      22
      23
      24
      25
      26
      27
      28
      29
      30
      31
      32
      33
      34
      35
      36
      37
      38
      
      #!/bin/bash
      
      # For loop
      numbers="One Two Three List"
      for n in $numbers
      do
          echo $n
      done
      
      for x in {1..4}
      do
          echo -n "$x "
      done
      echo
      
      for x in ./c*.sh
      do
          echo -n "$x "
      done
      echo
      
      # While loop
      c=0
      while [ $c -le 5 ]
      do
          echo -n "$c "
          ((c++))
      done
      echo
      
      # Until loop
      c=0
      until [ $c -gt 5 ]
      do
          echo -n "$c "
          ((c++))
      done
      echo

      Note

      The bash scripting language offers support for the break statement for exiting a loop, and the continue statement for skipping the current iteration.

      The loops.sh example begins with three for loops. These loops will iterate over values in a series, here represented by the numbers list variable or a range like {1..4}, and complete the block of code after the do command for each value. In a set of four values a loop will iterate four times. Notice that the third for loop processes the output of ./c*.sh, which is equivalent to the output of the ls ./c*.sh command – this is a pretty handy way of selecting and processing files from the Linux filesystem.

      Similarly, the while statement and the until statement will continually loop so long as the conditional statement is true (while), or until the statement becomes true (until). The -le and -gt operators used in the while and until loops are used strictly to compare numbers, and mean “less than or equal to” and “greater than,” respectively.

      Executing loops.sh will create the following output:

      ./loops.sh
      
        
      One
      Two
      Three
      List
      1 2 3 4
      ./case.sh ./cla.sh
      0 1 2 3 4 5
      0 1 2 3 4 5
      
      

      Note

      The ./case.sh and ./cla.sh scripts will be created in later sections of this guide.

      Using UNIX Environment Variables

      In this section of the guide you will learn how to read a UNIX environment variable, change its value, delete it, and create a new one. This is a very popular way of getting user input or reading the setup of the current user.

      The related bash shell script is called env.sh and is as follows:

      env.sh
       1
       2
       3
       4
       5
       6
       7
       8
       9
      10
      11
      12
      13
      14
      15
      16
      17
      18
      19
      20
      21
      22
      23
      24
      
      #!/bin/bash
      
      # Read
      if [[ -z "${PATH}" ]]; then
          echo "PATH is empty!"
      else
          echo "PATH: $PATH"
      fi
      
      # Change
      PATH=${PATH}:/tmp
      echo "PATH: $PATH"
      
      # Delete
      export PATH=""
      if [[ -z "${PATH}" ]]; then
          echo "PATH is empty!"
      else
          echo "PATH: $PATH"
      fi
      
      # Create
      MYPATH="/bin:/sbin:/usr/bin"
      echo "MYPATH: ${MYPATH}"

      Notice that the PATH environment variable is automatically available to the bash script. You can view it’s current value in the output of the first if statement. The -z operator tests whether a variable has a length of zero or not and can be pretty handy when checking if an environment variable is set or not.

      Executing env.sh will create the following output:

      ./env.sh
      
        
      PATH: /usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/texbin:/opt/X11/bin
      PATH: /usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/texbin:/opt/X11/bin:/tmp
      PATH is empty!
      MYPATH: /bin:/sbin:/usr/bin
      
      

      Note

      Notice that all changes to environment variables that take place inside a bash script will be lost when that bash script ends because they have a local scope.

      Bash and Command Line Arguments

      Command Line Arguments

      The easiest and most common way to pass your own data to scripts is the use of command line arguments. For instance, review the following command:

      ./cla.sh 1 2 3
      

      This example command is executing the cla.sh command, and supplying a number of arguments, in this case the numbers 1, 2, and 3. Those numbers could be any type of information the bash script needs to execute.

      The cla.sh bash script demonstrates how to work with command line arguments:

      cla.sh
       1
       2
       3
       4
       5
       6
       7
       8
       9
      10
      11
      12
      13
      14
      15
      16
      17
      
      #!/bin/bash
      
      echo "Arguments: [email protected]"
      echo "Number of arguments: $#"
      
      for arg in "[email protected]"
      do
          echo "$arg"
      done
      
      echo "The name of the script is: $0"
      echo "The first argument is: $1"
      
      if [ -x $0 ]
      then
          echo "$0" file exists!
      fi

      The full list of arguments is stored as [email protected] and the number of arguments is stored as $#. A for loop can be used for iterating over the list of command line arguments. Lastly, the name of the program is always $0 and the first command line argument, if it exists, is always $1.

      Executing cla.sh will generate the following output:

      ./cla.sh 1 2 3
      
        
      Arguments: 1 2 3
      Number of arguments: 3
      1
      2
      3
      The name of the script is: ./cla.sh
      The first argument is: 1
      ./cla.sh file exists!
      
      

      Checking the Number of Command Line Arguments

      In the previous section you learned how to pass command line arguments to a bash script. The following bash script, which is named nCla.sh, requires that you pass at least two command line arguments to it:

      nCla.sh
      1
      2
      3
      4
      5
      6
      7
      8
      
      #!/bin/bash
      
      if [ "$#" -lt 2 ]
      then
          echo Need more arguments than $#!
      else
          echo "Thanks for the $# arguments!"
      fi

      Notice that numeric comparisons require the use of -lt in an if statement, which is an alias for less than.

      Executing nCla.sh with the right number of arguments will generate the following output:

      ./nCla.sh 1 2
      
        
      Thanks for the 2 arguments!
      
      

      Executing nCla.sh with an insufficient number of arguments will generate the following output:

      ./nCla.sh
      
        
      Need more arguments than 0!
      
      

      Combining Commands in bash Scripts

      Bash has the additional capability of executing a combination of commands. This capability is illustrated in combine.sh:

      combine.sh
       1
       2
       3
       4
       5
       6
       7
       8
       9
      10
      11
      12
      13
      14
      
      #!/bin/bash
      
      total=0
      for n in ./*.sh
      do
          ti=`grep while ${n} | wc -l`
          ti=$(($ti + 0))
          if [[ $ti -gt 0 ]]
          then
              total=$(($total+$ti))
          fi
      done
      
      echo "Total:" $total

      The for loop in this example iterates over every bash script file in the current working directory. The initial value of ti is taken from the output of a command with two parts. The first part uses grep to look for the while word in the file that is being processed and the second part counts the number of times that the while word was found in the current file. The total=$(($total+$ti)) statement is needed for adding the value of ti to the value of total. Additionally, the ti=$(($ti + 0)) statement is used for converting the value of ti from string to integer. Last, before exiting, combine.sh prints the total number of times the while word was found in all processed files.

      ./combine.sh
      
        
      Total: 2
      
      

      The Case Statement

      The bash scripting language supports the case statement. A case statement provides a number of possible values for a variable and maps code blocks to those values. For example, the case statement included in case.sh script below defines a number of possible outputs depending on the number provided as a command line argument:

      case.sh
       1
       2
       3
       4
       5
       6
       7
       8
       9
      10
      11
      12
      13
      14
      15
      16
      17
      18
      19
      20
      21
      22
      23
      24
      25
      26
      27
      28
      29
      30
      31
      32
      33
      34
      35
      36
      37
      38
      39
      40
      41
      42
      43
      44
      45
      46
      47
      48
      49
      50
      51
      52
      
      #!/bin/bash
      
      if [ $# -lt 1 ]
      then
          echo "Usage : $0 integer"
          exit
      fi
      
      NUM=$1
      echo "Testing ${NUM}"
      
      if [[ ! $NUM =~ ^[0-9]+$ ]] ; then
          echo "Not an integer"
          exit
      fi
      
      case $NUM in
          0)
              echo "Zero!"
              ;;
          1)
              echo "One!"
              ;;
          ([2-9]|[1-7][0-9]|80) echo "From 2 to 80"
              ;;
          (8[1-9]|9[0-9]|100) echo "From 81 to 100"
              ;;
          *)
              echo "Too big!"
              ;;
      esac
      
      case  1:${NUM:--} in
      (1:*[!0-9]*|1:0*[89]*)
          ! echo NAN
      ;;
      ($((NUM<81))*)
          echo "$NUM smaller than 80"
      ;;
      ($((NUM<101))*)
          echo "$NUM between 81 and 100"
      ;;
      ($((NUM<121))*)
          echo "$NUM between 101 and 120"
      ;;
      ($((NUM<301))*)
          echo "$NUM between 121 and 300"
      ;;
      ($((NUM>301))*)
          echo "$NUM greater than 301"
      ;;
      esac

      The script requires a command line argument, which is an integer value. A regular expression verifies that the input is a valid positive integer number with the help of an if statement.

      From the presented code you can understand that branches in case statements do not offer direct support for numeric ranges, which makes the code more complex. If you find the code difficult to understand, you may also use multiple if statements instead of a case block.

      case.sh illustrates two ways of supporting ranges in a case statement. The first one uses regular expressions (which are divided by the OR operator, a pipe |), whereas the second offers ranges through a different approach. Each expression such as (NUM<81) is evaluated and if it is true, the code in the respective branch is executed. Notice that the order of the branches is significant because only the code from the first match will be executed.

      Executing case.sh will generate the following output:

      ./case.sh 12
      
        
      Testing 12
      From 2 to 80
      12 smaller than 80
      
      

      If you give case.sh a different input, you will get the following output:

      ./case.sh 0
      
        
      Testing 0
      Zero!
      0 smaller than 80
      
      

      File and Directory Basics

      Working with Files and Directories

      One often important operation that you may find yourself needing to perform is specifying whether a given file or directory actually exists. This is idea is illustrated in files.sh:

      files.sh
       1
       2
       3
       4
       5
       6
       7
       8
       9
      10
      11
      12
      13
      14
      15
      16
      17
      18
      19
      20
      21
      22
      23
      24
      25
      26
      
      #!/bin/bash
      
      if [[ $# -le 0 ]]
      then
          echo Not enough arguments!
      fi
      
      for arg in "[email protected]"
      do
          # Does it actually exist?
          if [[ -e "$arg" ]]
          then
              echo -n "$arg exists "
          fi
      
          # Is it a file or Is it a directory?
          if [ -f "$arg" ]
          then
              echo "and is a regular file!"
          elif [ -d "$arg" ]
          then
              echo "and is a regular directory!"
          else
              echo "and is neither a regular file nor a regular directory!"
          fi
      done

      The -e operator will check whether a file exists regardless of its type – this is the first test that we are performing. The other two tests use -f and -d for specifying whether we are dealing with a regular file or a directory, respectively. Last, the -le operator stands for less than or equal and is used for comparing numeric values.

      Executing files.sh will generate the following output:

      ./files.sh /tmp aFile /dev/stdin
      
        
      /tmp exists and is a regular directory!
      aFile exists and is a regular file!
      /dev/stdin exists and is neither a regular file nor a regular directory!
      
      

      The following bash script will accept one command line argument, which is a string you’d like to find, and then a list of files that will be searched for that given string. If there is a match, then the filename of the file will appear on the screen.

      match.sh
       1
       2
       3
       4
       5
       6
       7
       8
       9
      10
      11
      12
      13
      14
      15
      16
      17
      18
      19
      20
      21
      22
      23
      24
      25
      26
      27
      28
      29
      
      #!/bin/bash
      
      if [[ $# -le 1 ]]
      then
          echo Usage: $0 string files!
      fi
      
      string=$1
      for arg in "${@:2}"
      do
          # Does it actually exist?
          if [[ ! -e "$arg" ]]
          then
              echo "* Skipping ${arg}"
              continue
          fi
          # Is it a regular file?
          if [ -f "$arg" ]
          then
              ti=`grep ${string} ${arg} | wc -l`
              ti=$(($ti + 0))
              if [[ $ti -gt 0 ]]
              then
                  echo ${arg}
              fi
          else
              echo "* $arg is not a regular file!"
          fi
      done

      The "${@:2}" notation allows you to skip the first element from the list of command line arguments because this is the string that we will be looking for in the list of files that follow.

      Executing match.sh will generate the following output:

      ./match.sh while *.sh /tmp /var/log ./combine.sh doesNotExist
      
        
      combine.sh
      loops.sh
      * /tmp is not a regular file!
      * /var/log is not a regular file!
      ./combine.sh
      * Skipping doesNotExist
      
      

      In part 2 of the bash scripting guides you will learn more about working with files and directories with the bash scripting language.

      Summary

      The scripting language of bash can do many more things than the ones presented in this guide. The next part of this guide will present more interesting bash shell scripts and shed more light into topics such as working with files and directories, the printf command, the select statement and reading files.

      More Information

      You may wish to consult the following resources for additional information on this topic. While these are provided in the hope that they will be useful, please note that we cannot vouch for the accuracy or timeliness of externally hosted materials.

      Find answers, ask questions, and help others.

      This guide is published under a CC BY-ND 4.0 license.



      Source link