One place for hosting & domains

      Scripts

      Writing Scripts for Use with Linode StackScripts – A Tutorial


      Updated by Linode

      Contributed by
      Linode

      What are StackScripts?

      StackScripts provide Linode users with the ability to automate the deployment of custom systems on top of Linode’s default Linux distribution images. For example, every time you deploy a new Linode you might execute the same tasks, like updating your system’s software, installing your favorite Linux tools, and adding a limited user account. These tasks can be automated using a StackScript that will perform these actions for you as part of your Linode’s first boot process.

      All StackScripts are stored in the Linode Cloud Manager and can be accessed whenever you deploy a Linode. A StackScript authored by you is an Account StackScript. While a Community StackScript is a StackScript created by a Linode community member that has made their StackScript publicly available in the Linode Cloud Manager.

      In this Guide

      Writing a script for use in a StackScript will generally be the same as writing a script that will be executed from the command line or another program. This guide includes information about the StackScript system, including the following:

      The StackScript System

      StackScript Requirements

      • The primary requirement for your scripts is that the interpreter needed to execute your script should exist in the Linode base image you are deploying. While Bash is an obvious choice for a script, you may choose any scripting language.

        Note

        Linode images are created using “vanilla” versions of its given distribution. Consult our Choosing a Linux Distribution guide to see list of all distributions Linode provides and to access each distribution’s corresponding websites. You can find more information on the interpreters available for each distribution on their official websites.
      • When writing a script, you must use a shebang as the first line of your script. This indicates to your Linux system which interpreter to use when running the script. For example, if you are writing a Bash script, the beginning of your script should include the following line:

        Or, if you are writing a Python script, the beginning of your script should include the following line:

        1
        2
        
        #!/usr/bin/env python
              

      Import a StackScript

      Your scripts can import any Account StackScript that you own or any Community StackScript. This allows you to reuse code minimizing what you need to write in your own scripts.

      • The example below shows the syntax to import another StackScript. As a result of including this line in a script, the imported StackScript will be downloaded as ssinclude-[NUMBER] to your Linode. However, it must be run in order to execute its contents.

        1
        2
        
        <ssinclude StackScriptID="[NUMBER]">
            

        In Bash, you can download and run the script in the following way:

        1
        2
        
        source <ssinclude StackScriptID="[NUMBER]">
            

        If you’re scripting in another language, import the StackScript, then execute it on a second line:

        1
        2
        3
        
        <ssinclude StackScriptID="[NUMBER]">
        ./ssinclude-[NUMBER]
            
      • Linode provides a StackScript Bash Library that includes a set of functions that perform various common tasks users might wish to execute on their Linodes. This script creates the functions, but does not run them. A new StackScript can import the Bash Library and then execute functions from it.

        Note

      Access a StackScript’s ID Number

      Follow the steps in this section to find the ID number of a StackScript.

      1. Log into the Linode Cloud Manager.

      2. Click on the StackScripts link in the left-hand navigation menu. You will be brought to the StackScripts page.

        Click on the StackScripts link in the left-hand navigation menu.

      3. Click on the Account StackScripts tab or the Community StackScripts tab, depending on the type of StackScript whose ID you’d like to find

      4. Click on the StackScript whose ID you’d like to access. This will bring you to its StackScript detail page.

        View the details and contents of an Account StackScript.

      5. The StackScript detail page’s URL will display the StackScript’s ID number. You can now use this number to import the StackScript into your own script.

        Access a StackScript's ID number.

      User Defined Fields (UDFs)

      The StackScript system provides a basic markup specification that interfaces with the Linode deployment process so that users can customize the behavior of a StackScript on a per-deployment basis. When a StackScript contains a user defined field (UDF), the Linode Cloud Manager will present the UDF as a form field, so a user can insert a corresponding custom value. The values and their related variables are inserted into the script’s environment when used to deploy a new Linode.

      Default Environment Variables

      Linode StackScripts provide a set of default environment variables that you can use to provide your script with information about the Linode it has deployed.

      Environment Variable Description
      LINODE_ID The deployed Linode’s ID number
      LINODE_LISHUSERNAME The deployed Linode’s full Linode Shell (LISH) accessible name.
      LINODE_RAM The RAM available on this Linode’s plan.
      LINODE_DATACENTERID The ID number of the data center containing the Linode. You can use the Linode API to see a list of all data center IDs.



      Set your Environment Variables Using an External File

      It is possible to set your script’s environment variables using externally hosted files. The example Bash script uses the wget utility to download two files named base.env and $IPADDR.env from the external site http://example.com/. The source command will load the downloaded files into the script.

      StackScript
      1
      2
      3
      4
      5
      6
      7
      8
      9
      
      # [...]
      IPADDR=$(/sbin/ifconfig eth0 | awk '/inet / { print $2 }' | sed 's/addr://')
      
      wget http://example.com/base.env --output-document=/tmp/base.env
      wget http://example.com/$IPADDR.env --output-document=/tmp/system.env
      
      source /tmp/base.env
      source /tmp/system.env
      # [...]

      Note

      The files you reference within your script must exist and be accessible via HTTP. Also, ensure that the files you host externally do not contain any sensitive information.

      StackScript Examples

      Using an External Script

      • If you have an existing deployment script, you can use a StackScript to deploy Linode instances with it. The following example StackScript installs PHP on the Linode, downloads an external PHP script from the URL http://example.com/deployment-script.php, makes it executable, and then runs the downloaded script.

        StackScript
         1
         2
         3
         4
         5
         6
         7
         8
         9
        10
        11
        12
        13
        14
        15
        16
        17
        18
        19
        20
        
        #!/bin/bash
        if [ -f /etc/apt/sources.list ]; then
           apt-get upgrade
           apt-get -y install php
        elif [-f /etc/yum.conf ]; then
           yum -y install php
        elif [-f /etc/pacman.conf ]; then
           pacman -Sy
           pacman -S --noconfirm pacman
           pacman -S --noconfirm php
        else
           echo "Your distribution is not supported by this StackScript"
           exit
        fi
        
        wget http://example.com/deployment-script.php --output-document=/opt/deployment-script.php
        chmod +x /opt/deployment-script.php
        
        ./opt/deployment-script.php
            
      • If you do not want to rely on an existing external server to host your scripts for download, you can embed the bootstrapped script into the StackScript.

        StackScript
         1
         2
         3
         4
         5
         6
         7
         8
         9
        10
        11
        12
        13
        14
        15
        16
        17
        18
        19
        20
        21
        22
        23
        24
        25
        26
        
        #!/bin/bash
        
        if [ -f /etc/apt/sources.list ]; then
           apt-get upgrade
           apt-get -y install php5
        elif [-f /etc/yum.conf ]; then
           yum -y install php
        elif [-f /etc/pacman.conf ]; then
           pacman -Sy
           pacman -S --noconfirm pacman
           pacman -S --noconfirm php
        else
           echo "Your distribution is not supported by this StackScript"
           exit
        fi
        
        cat >/opt/deployment-script.php <<EOF
        #!/usr/bin/php
        <?php print('Hello World!'); ?>
        EOF
        
        chmod +x /opt/deployment-script.php
        
        ./opt/deployment-script.php
        
            

      Next Steps

      This guide is published under a CC BY-ND 4.0 license.



      Source link

      Solving Real World Problems With Bash Scripts – A Tutorial


      Updated by Linode Contributed by Mihalis Tsoukalos

      Introduction

      This guide presents some of the advanced capabilities of the bash shell by showing practical and fully functional bash scripts. It also illustrates how you can work with dates and times in bash scripts and how to write and use functions in bash.

      In This Guide

      In this guide, you will find the following information about bash scripts:

      Note

      This guide is written for a non-root user. Depending on your configuration, some commands might require the help of sudo in order to properly execute. If you are not familiar with the sudo command, see the Users and Groups guide.

      Functions in bash shell

      The bash scripting language has support for functions. The parameters of a function can be accessed as $1, $2, etc. and you can have as many parameters as you want. If you are interested in finding out the name of the function, you can use the FUNCNAME variable. Functions are illustrated in functions.sh, which is as follows:

      functions.sh
       1
       2
       3
       4
       5
       6
       7
       8
       9
      10
      11
      12
      13
      14
      15
      16
      17
      18
      19
      20
      21
      22
      
      #!/bin/bash
      
      function f1 {
          echo Hello from $FUNCNAME!
          VAR="123"
      }
      
      f2() {
          p1=$1
          p2=$2
          sum=$((${p1} + ${p2}))
          echo "${sum}"
      }
      
      f1
      echo ${VAR}
      
      mySum="$(f2 1 2)"
      echo mySum = $mySum
      
      mySum="$(f2 10 -2)"
      echo mySum = $mySum

      Run the script with the following command:

      ./functions.sh
      

      The output will look like this:

        
      Hello from f1!
      123
      mySum = 3
      mySum = 8
      
      

      Note

      If you want to check whether a function parameter exists or not, you can use the statement:

      if [ -z "$1" ]
      

      Using bash Functions as Shell Commands

      This is a trick that allows you to use bash functions as shell commands. You can execute the above code as

      . ./functions.sh
      

      Notice the dot in front of the text file. After that you can use f1 as a regular command in the terminal where you executed . ./my_function.sh. You will also be able to use the f2 command with two integers of your choice to quickly calculate a sum. If you want that function to be globally available, you can put its implementation to a bash configuration file that is automatically executed by bash each time a new bash session begins. A good place to put that function implementation would be ~/.bash_profile.

      Working with Dates and Times

      Bash allows you to work with dates and times using traditional UNIX utilities such as date(1). The main difficulty many programmers run into when working with dates and times is getting or using the correct format. This is a matter of using date(1) with the correct parameters and has nothing to do with bash scripting per se. Using date(1) as date +[something] means that we want to use a custom format – this is signified by the use of + in the command line argument of date(1).

      A good way to create unique filenames is to use UNIX epoch time or, if you want your filename to be more descriptive, a date-time combination. The unique nature of the filename is derived from a focus on a higher level of detail in defining your output. If done correctly, you will never have the exact same time value even if you execute the script multiple times on the same UNIX machine.

      The example that follows will shed some light on the use of date(1).

      Using Dates and Times in bash scripts

      The code of dateTime.sh is the following:

      dateTime.sh
       1
       2
       3
       4
       5
       6
       7
       8
       9
      10
      11
      12
      13
      14
      15
      16
      17
      18
      19
      20
      21
      22
      23
      24
      25
      26
      27
      28
      29
      30
      31
      32
      33
      34
      35
      36
      37
      38
      39
      40
      41
      42
      43
      44
      45
      46
      47
      48
      
      #!/bin/bash
      
      # Print default output
      echo `date`
      
      # Print current date without the time
      echo `date +"%m-%d-%y"`
      
      # Use 4 digits for year
      echo `date +"%m-%d-%Y"`
      
      # Display time only
      echo `date +"%T"`
      
      # Display 12 hour time
      echo `date +"%r"`
      
      # Time without seconds
      echo `date +"%H:%M"`
      
      # Print full date
      echo `date +"%A %d %b %Y %H:%M:%S"`
      
      # Nanoseconds
      echo Nanoseconds: `date +"%s-%N"`
      
      # Different timezone by name
      echo Timezone: `TZ=":US/Eastern" date +"%T"`
      echo Timezone: `TZ=":Europe/UK" date +"%T"`
      
      # Print epoch time - convenient for filenames
      echo `date +"%s"`
      
      # Print week number
      echo Week number: `date +"%V"`
      
      # Create unique filename
      f=`date +"%s"`
      touch $f
      ls -l $f
      rm $f
      
      # Add epoch time to existing file
      f="/tmp/test"
      touch $f
      mv $f $f.`date +"%s"`
      ls -l "$f".*
      rm "$f".*

      If you want an even more unique filename, you can also use nanoseconds when defining the behaviour of your script.

      Run the dateTime script:

      ./dateTime.sh
      

      The output of dateTime.sh will resemble the following:

        
      Fri Aug 30 13:05:09 EST 2019
      08-30-19
      08-30-2019
      13:05:09
      01:05:09 PM
      13:05
      Friday 30 Aug 2019 13:05:09
      Nanoseconds: 1567159562-373152585
      Timezone: 06:05:09
      Timezone: 10:05:09
      1567159509
      Week number: 35
      -rw-r--r--  1 mtsouk  staff  0 Aug 30 13:05 1567159509
      -rw-r--r--  1 mtsouk  wheel  0 Aug 30 13:05 /tmp/test.1567159509
      
      

      Bash scripts for Administrators

      This section will present some bash scripts that are generally helpful for UNIX system administrators and power users.

      Watching Free Disk Space

      The bash script that follows watches the free space of your hard disks and warns you when that free space drops below a given threshold – the value of the threshold is given by the user as a command line argument. Notice that if the program gets no command line argument, a default value is used as the threshold.

      freeDisk.sh
       1
       2
       3
       4
       5
       6
       7
       8
       9
      10
      11
      12
      13
      14
      15
      16
      17
      18
      19
      20
      21
      22
      23
      24
      25
      26
      27
      28
      29
      30
      
      #!/bin/bash
      
      # default value to use if none specified
      PERCENT=30
      
      # test for command line arguement is present
      if [[ $# -le 0 ]]
      then
          printf "Using default value for threshold!n"
      # test if argument is an integer
      # if it is, use that as percent, if not use default
      else
          if [[ $1 =~ ^-?[0-9]+([0-9]+)?$ ]]
          then
              PERCENT=$1
          fi
      fi
      
      let "PERCENT += 0"
      printf "Threshold = %dn" $PERCENT
      
      df -Ph | grep -vE '^Filesystem|tmpfs|cdrom' | awk '{ print $5,$1 }' | while read data;
      do
          used=$(echo $data | awk '{print $1}' | sed s/%//g)
          p=$(echo $data | awk '{print $2}')
          if [ $used -ge $PERCENT ]
          then
              echo "WARNING: The partition "$p" has used $used% of total available space - Date: $(date)"
          fi
      done
      • The sed s/%//g command is used for omitting the percent sign from the output of df -Ph.
      • df is the command to report file system disk space usage, while the options -Ph specify POSIX output and human-readable, meaning, print sizes in powers of 1024.
      • awk(1) is used for extracting the desired fields from output of the df(1) command.

      Run ./freeDisk.sh with this command:

      ./freeDisk.sh
      

      The output of freeDisk.sh will resemble the following:

        
      Using default value for threshold!
      Threshold = 30
      WARNING: The partition "/dev/root" has used 61% of total available space - Date: Wed Aug 28 21:14:51 EEST 2019
      
      

      Note

      This script and others like it can be easily executed as cron jobs and automate tasks the UNIX way.

      Notice that the code of freeDisk.sh looks relatively complex. This is because bash is not good at the conversion between strings and numeric values – more than half of the code is for initializing the PERCENT variable correctly.

      Rotating Log Files

      The presented bash script will help you to rotate a log file after exceeding a defined file size. If the log file is connected to a server process, you might need to stop the process before the rotation and start it again after the log rotation is complete – this is not the case with rotate.sh.

      rotate.sh
       1
       2
       3
       4
       5
       6
       7
       8
       9
      10
      11
      12
      13
      14
      15
      16
      17
      18
      19
      20
      21
      
      #!/bin/bash
      
      f="/home/mtsouk/connections.data"
      
      if [ ! -f $f ]
      then
        echo $f does not exist!
        exit
      fi
      
      touch ${f}
      MAXSIZE=$((4096*1024))
      
      size=`du -b ${f} | tr -s 't' ' ' | cut -d' ' -f1`
      if [ ${size} -gt ${MAXSIZE} ]
      then
          echo Rotating!
          timestamp=`date +%s`
          mv ${f} ${f}.$timestamp
          touch ${f}
      fi
      • Note that the path to the log file /home/mtsouk/connections.data will not exist by default. You’ll need to either use a log file that already exists like kern.log on some Linux systems, or replace it with a new one.

      • Additionally, the value of MAXSIZE can be a value of your choice, and the script can be edited to suit the needs of your own configuration – you can even make changes to the existing code and provide the MAXSIZE value as a command line argument to the program.

      • The du command is used to estimate the file space usage. It’s use to track the files and directories that are consuming excessive space on the hard disk. The -b option tells this command to print the size in bytes.

      Run the rotate script with the following command:

      ./rotate.sh
      

      The output of rotate.sh when it has reached the threshold defined by MAXSIZE will resemble the following:

        
      Rotating!
      
      

      After running, two files will be created on the system. You can see them with this command:

      ls -l connections.data*
      
        
      -rw-r--r-- 1 mtsouk mtsouk       0 Aug 28 20:18 connections.data
      -rw-r--r-- 1 mtsouk mtsouk 2118655 Aug 28 20:18 connections.data.1567012710
      
      

      If you want to make rotate.sh more generic, you can provide the name of the log file as a command line argument to the bash script.

      Monitoring the Number of TCP Connections

      The presented bash script calculates the number of TCP connections on the current machine and prints that on the screen along with date and time related information.

      tcpConnect.sh
      1
      2
      3
      4
      5
      6
      
      #!/bin/bash
      
      C=$(/bin/netstat -nt | tail -n +3 | grep ESTABLISHED | wc -l)
      D=$(date +"%m %d")
      T=$(date +"%H %M")
      printf "%s %s %sn" "$C" "$D" "$T"
      • The main reason for using the full path of netstat(1) when calling it is to make the script as secure as possible.
      • If you do not provide the full path then the script will search all the directories of the PATH variable to find that executable file.
      • Apart from the number of established connections (defined by the C variable), the script prints the month, day of the month, hour of the day, and minutes of the hour. If you want, you can also print the year and seconds.

      Execute the tcpConnect script with the following command:

      ./tcpConnect.sh
      

      The output will be similar to the following:

        
      8 08 28 16 22
      
      

      tcpConnect.sh can be easily executed as a cron(8) by adding the following to your cron file:

      */4 * * * * /home/mtsouk/bin/tcpConnect.sh >> ~/connections.data
      

      The previous cron(8) job executes tcpConnect.sh every 4 minutes, every hour of each day and appends the results to ~/connections.data in order to be able to watch or visualize them at any time.

      Additional Examples

      Sorting in bash

      The presented example will show how you can sort integer values in bash using the sort(1) utility:

      sort.sh
       1
       2
       3
       4
       5
       6
       7
       8
       9
      10
      11
      12
      13
      14
      15
      16
      17
      18
      19
      20
      21
      22
      23
      
      #!/bin/bash
      
      # test that at least one argument was passed
      if [[ $# -le 0 ]]
      then
          printf "Not enough arguments!n"
          exit
      fi
      
      count=1
      
      for arg in "[email protected]"
      do
          if [[ $arg =~ ^-?[0-9]+([0-9]+)?$ ]]
          then
              n[$count]=${arg}
              let "count += 1"
          else
              echo "$arg is not a valid integer!"
          fi
      done
      
      sort -n <(printf "%sn" "${n[@]}")
      • The presented technique uses an array to store all integer values before sorting them.
      • All numeric values are given as command line arguments to the script.
      • The script tests whether each command line argument is a valid integer before adding it to the n array.
      • The sorting part is done using sort -n, which sorts the array numerically. If you want to deal with strings, then you should omit the -n option.
      • The printf command, after sort -n, prints every element of the array in a separate line whereas the < character tells sort -n to use the output of printf as input.

      Run the sort script with the following command:

      ./sort.sh 100 a 1.1 1 2 3 -1
      

      The output of sort.sh will resemble the following:

        
      a is not a valid integer!
      1.1 is not a valid integer!
      -1
      1
      2
      3
      100
      
      

      A Game Written in bash

      This section will present a simple guessing game written in bash(1). The logic of the game is based on a random number generator that produces random numbers between 1 and 20 and expects from the user to guess them.

      guess.sh
       1
       2
       3
       4
       5
       6
       7
       8
       9
      10
      11
      12
      13
      14
      15
      16
      17
      18
      19
      20
      
      #!/bin/bash
      NUMGUESS=0
      
      echo "$0 - Guess a number between 1 and 20"
      
      (( secret = RANDOM % 20 + 1 ))
      
      while [[ guess -ne secret ]]
      do
          (( NUMGUESS = NUMGUESS + 1 ))
          read -p "Enter guess: " guess
      
          if (( guess < $secret )); then
              echo "Try higher..."
          elif (( $guess > $secret )); then
              echo "Try lower..."
          fi
      done
      
      printf "Yes! You guessed it in $NUMGUESS guesses.n"

      Run the guess script:

      ./guess.sh
      

      The output of guess.sh will resemble the following:

        
      ./guess.sh - Guess a number between 1 and 20
      Enter guess: 1
      Try higher...
      Enter guess: 5
      Try higher...
      Enter guess: 7
      Try lower...
      Enter guess: 6
      Yes! You guessed it in 4 guesses.
      
      

      Calculating Letter Frequencies

      The following bash script will calculate the number of times each letter appears on a file.

      freqL.sh
       1
       2
       3
       4
       5
       6
       7
       8
       9
      10
      11
      12
      13
      
      #!/bin/bash
      
      if [ -z "$1" ]; then
          echo "Usage: $0 filename."
          exit 1
      fi
      
      filename=$1
      
      while read -n 1 c
      do
          echo "$c"
      done < "$filename" | grep '[[:alpha:]]' | sort | uniq -c | sort -nr
      • The script reads the input file character by character, prints each character, and processes the output using the grep, sort, and uniq commands to count the frequency of each character.
      • The [:alpha:] pattern used by grep(1) matches all alphabetic characters and is equivalent to A-Za-z.
      • If you also want to include numeric characters in the output, you should use [:alnum:] instead.
      • Additionally, if you want the output to be sorted alphabetically instead of numerically, you can execute freqL.sh and then process its output using the sort -k2,2 command.

      Run the freqL script:

      ./freqL.sh text.txt
      

      The output of freqL.sh will resemble the following:

        
         2 b
         1 s
         1 n
         1 i
         1 h
         1 a
      
      

      Note

      The file text.txt will not exist by default. You can use a pre-existing text file to test this script, or you can create the text.txt file using a text editor of your choice.

      Timing Out read Operations

      The read builtin command supports the -t timeout option that allows you to time out a read operation after a given time, which can be very convenient when you are expecting user input that takes too long. The technique is illustrated in timeOut.sh.

      timeOut.sh
       1
       2
       3
       4
       5
       6
       7
       8
       9
      10
      11
      12
      13
      14
      15
      16
      17
      18
      19
      20
      21
      22
      23
      24
      25
      26
      27
      28
      29
      30
      31
      32
      
      #!/bin/bash
      
      if [[ $# -le 0 ]]
      then
          printf "Not enough arguments!n"
          exit
      fi
      
      TIMEOUT=$1
      VARIABLE=0
      
      while :
      do
        ((VARIABLE = VARIABLE + 1))
        read -t $TIMEOUT -p "Do you want to Quit(Y/N): "
        if [ $VARIABLE -gt $TIMEOUT ]; then
          echo "Timing out - user response took too long!"
          break
        fi
      
        case $REPLY in
        [yY]*)
          echo "Quitting!"
          break
          ;;
        [nN]*)
          echo "Do not quit!"
          ;;
        *) echo "Please choose Y or N!"
           ;;
        esac
      done
      • The timeout of the read operation is given as a command line argument to the script, an integer representing the number of seconds that will pass before the script will “time out” and exit.
      • The case block is what handles the available options.
      • Notice that what you are going to do in each case is up to you – the presented code uses simple commands to illustrate the technique.

      Run the timeOut script:

      ./timeOut.sh 10
      

      The output of timeOut.sh will resemble the following:

        
      Do you want to Quit(Y/N): Please choose Y or N!
      Do you want to Quit(Y/N): Y
      Quitting!
      
      

      Alternatively, you can wait the full ten seconds for your script to time out:

        
      Do you want to Quit(Y/N):
      Timing out - user response took too long!
      
      

      Converting tabs to spaces

      The presented utility, which is named t2s.sh, will read a text file and convert each tab to the specified number of space characters. Notice that the presented script replaces each tab character with 4 spaces but you can change that value in the code or even get it as command line argument.

      tabs2spaces.sh
       1
       2
       3
       4
       5
       6
       7
       8
       9
      10
      11
      12
      13
      
      #!/bin/bash
      
      for f in "[email protected]"
      do
          if [ ! -f $f ]
          then
            echo $f does not exist!
            continue
          fi
          echo "Converting $f.";
          newFile=$(expand -t 4 "$f");
          echo "$newFile" > "$f";
      done
      • The script uses the expand(1) utility that does the job of converting tabs to spaces for us.
      • expand(1) writes its results to standard output – the script saves that output and replaces the current file with the new output, which means that the original file will change.
      • Although tabs2spaces.sh does not use any fancy techniques or code, it does the job pretty well.

      Run the tabs2spaces script:

      ./tabs2spaces.sh textfile.txt
      

      The output of tabs2spaces.sh will resemble the following:

        
      Converting textfile.txt.
      
      

      Note

      The file textfile.txt will not exist by default. You can use a pre-existing text file to test this script, or you can create the textfile.txt file using a text editor of your choice.

      Counting files

      The following script will look into a predefined list of directories and count the number of files that exist in each directory and its subdirectories. If that number is above a threshold, then the script will generate a warning message.

      ./countFiles.sh
       1
       2
       3
       4
       5
       6
       7
       8
       9
      10
      11
      12
      13
      14
      15
      16
      17
      18
      19
      20
      21
      22
      23
      24
      25
      26
      27
      28
      29
      
      #!/bin/bash
      
      DIRECTORIES="/bin:/home/mtsouk/code:/srv/www/www.mtsoukalos.eu/logs:/notThere"
      
      # Count the number of arguments passed in
      if [[ $# -le 0 ]]
      then
          echo "Using default value for COUNT!"
      else
          if [[ $1 =~ ^-?[0-9]+([0-9]+)?$ ]]
          then
              COUNT=$1
          fi
      fi
      
      while read -d ':' dir; do
          if [ ! -d "$dir" ]
          then
              echo "**" Skipping $dir
              continue
          fi
          files=`find $dir -type f | wc -l`
          if [ $files -lt $COUNT ]
          then
              echo "Everything is fine in $dir: $files"
          else
              echo "WARNING: Large number of files in $dir: $files!"
          fi
      done <<< "$DIRECTORIES:"

      The counting of the files is done with the find $dir -type f | wc -l command. You can read more about the find command in our guide.

      Run the countFiles script:

      ./countFiles.sh 100
      

      The output of countFiles.sh will resemble the following:

        
      WARNING: Large number of files in /bin: 118!
      Everything is fine in /home/mtsouk/code: 81
      WARNING: Large number of files in /srv/www/www.mtsoukalos.eu/logs: 106!
      ** Skipping /notThere
      
      

      Summary

      The bash scripting language is a powerful programming language that can save you time and energy when applied effectively. If you have a lot of useful bash scripts, then you can automate things by creating cron jobs that execute your bash scripts. It is up to the developer to decide whether they prefer to use bash or a different scripting language such as perl, ruby, or python.

      More Information

      You may wish to consult the following resources for additional information on this topic. While these are provided in the hope that they will be useful, please note that we cannot vouch for the accuracy or timeliness of externally hosted materials.

      Find answers, ask questions, and help others.

      This guide is published under a CC BY-ND 4.0 license.



      Source link