Log File monitor Script




by Ross Moffatt, October 2009

Overview

Here is a script that can be used to monitor a log file and run a command (or perform a number of actions if the script itself is modified) when a certain pattern is matched a set number of times. The script copes with the log file being monitored, being rolled, etc.

The Script

#!/usr/bin/bash
# Log File monitor
#

# Ok, so here we set the log we are monitoring and how often in seconds
LOGFOLDER="/home/ross/"
LOGFILE="ross.log"
LOGFILECHECKDELAY=10

# Now I need to know what to look for, and what to do if I find
# how many occurrences
LOGMESSAGE="ross not found"
LOGMESSAGESNUMBER=1
LOGACTIONCOMMAND="/home/ross/runross.sh"

# If you want me to use a lock file, give me the info here
STOPONLOCKFILE=0
MYLOCKFILEFOLDER="/var/tmp/"
MYLOCKFILE="Log_File_Monitor.lck"

# Debug message; I can do, just set DEBUGMESS to 1 and give me some info
DEBUGMESS=0
MYDEBUGFOLDER="/home/ross/"
DEBUGMESSLOG="Log_File_Monitor.log"

# Now I really start to do something; let's initialize counters
ERRORCOUNT=0
CURRENTERRORCOUNT=0

# Let's check if I'm already running so we can use cron to run me to check
MYTEMPORARYTEMPFILE=/tmp/Log_File_Monitor_$$
ps -ef > $MYTEMPORARYTEMPFILE
if [ "`grep "$(basename "$0")" $MYTEMPORARYTEMPFILE | grep -v $$ | grep -v sh -c`" != "" ]
then
 rm $MYTEMPORARYTEMPFILE
 exit 2
fi
rm $MYTEMPORARYTEMPFILE

# Let's hang around when the shell exits, trap SIGHUP, 1, and get picked up by init
trap "" 1

# Let's clean up if I get a SIGTERM, 15
trap "exit 0" 15

# Test if log file exists
if [ -f $LOGFOLDER$LOGFILE ]
  then
    if [ $DEBUGMESS -eq "1" ]; then echo "Log file been used = "$LOGFOLDER$LOGFILE >> $MYDEBUGFOLDER$DEBUGMESSLOG; fi
  else
    if [ $DEBUGMESS -eq "1" ]; then echo "Log file "$LOGFOLDER$LOGFILE" not found" >> $MYDEBUGFOLDER$DEBUGMESSLOG; fi
    exit 1
fi

if [ $DEBUGMESS -eq "1" ]; then echo "LOG Pattern = "$LOGMESSAGE >> $MYDEBUGFOLDER$DEBUGMESSLOG; fi

# Get the size of the log file and save it

# option 1, start monitoring from the beginning of the file I am monitoring
# LINECOUNT=0

# option 2, start monitoring from the end of the file
LINECOUNT=`awk 'END

Unknown macro: { print NR }

' $LOGFOLDER$LOGFILE`
# Ok lets loop from here
while :
do
  # So here is a delay for how often the file is checked
  sleep $LOGFILECHECKDELAY
 
  # Let's get the current log file size
  NEWLINECOUNT=`awk 'END

' $LOGFOLDER$LOGFILE`

  if [ $DEBUGMESS -eq "1" ]; then echo "NEWLINECOUNT = "$NEWLINECOUNT >> $MYDEBUGFOLDER$DEBUGMESSLOG; fi

  # Compare current log size with when we last looped
  if [ $NEWLINECOUNT -lt $LINECOUNT ]
    then
      SWITCH="less"
    else
      SWITCH="more"
  fi

  if [ $NEWLINECOUNT -eq $LINECOUNT ]
    then
      SWITCH="equal"
  fi

  case $SWITCH in

    "equal") if [ $DEBUGMESS -eq "1" ]; then echo "Switch is - equal" >> $MYDEBUGFOLDER$DEBUGMESSLOG; fi
  # if the current size is the same, assume nothing has changed
  # nothing to do
    ;;

    "less") if [ $DEBUGMESS -eq "1" ]; then echo "Switch is - less" >> $MYDEBUGFOLDER$DEBUGMESSLOG; fi
  # if the current size is smaller, assume log has rolled;
  # check from the beginning of the log; reset the NEWLINECOUNT size to 0
  NEWLINECOUNT=0
    ;;

    "more") if [ $DEBUGMESS -eq "1" ]; then echo "Switch is - more" >> $MYDEBUGFOLDER$DEBUGMESSLOG; fi
  # Get how much the log has grown
  DIFFLINECOUNT=`expr $NEWLINECOUNT - $LINECOUNT`
  if [ $DEBUGMESS -eq "1" ]; then echo "DIFFLINECOUNT = "$DIFFLINECOUNT >> $MYDEBUGFOLDER$DEBUGMESSLOG; fi

  # option 1
  # Get the current log using head; then grab the new lines using tail of the
  # difference between old and new log sizes; oh, and grep a count of the pattern
  # we are after
  # CURRENTERRORCOUNT=`head -$NEWLINECOUNT $LOGFOLDER$LOGFILE | tail -$DIFFLINECOUNT | egrep -c '$LOGMESSAGE'`

  # option 2
  # Tail just the different in sizes; assume no growth between earlier line count and now
  # oh, and grep a count of the pattern
  CURRENTERRORCOUNT=`tail -$DIFFLINECOUNT $LOGFOLDER$LOGFILE | grep -c "$LOGMESSAGE"`

  # Keep track of the number of errors from this check with previous checks
  ERRORCOUNT=`expr $ERRORCOUNT + $CURRENTERRORCOUNT`
  if [ $DEBUGMESS -eq "1" ]; then echo "ERRORCOUNT = "$ERRORCOUNT >> $MYDEBUGFOLDER$DEBUGMESSLOG; fi
    ;;

  esac

  # if the count is > threshold, do something
  if [ $ERRORCOUNT -ge $LOGMESSAGESNUMBER ]
    then
    # Test if lock file exists
    if [ -f $MYLOCKFILEFOLDER$MYLOCKFILE ]
    then
      if [ $DEBUGMESS -eq "1" ]; then echo "Lock File exists, no action" >> $MYDEBUGFOLDER$DEBUGMESSLOG; fi
    else
      # Here is where we actually do something
      if [ $DEBUGMESS -eq "1" ]; then echo "Taking Action" >> $MYDEBUGFOLDER$DEBUGMESSLOG; fi
      $LOGACTIONCOMMAND
      if [ $DEBUGMESS -eq "1" ]; then echo "Action Taken" >> $MYDEBUGFOLDER$DEBUGMESSLOG; fi

      # create a lock file if we are asked to do so
      if [ $STOPONLOCKFILE -eq "1" ]
      then
        touch $MYLOCKFILEFOLDER$MYLOCKFILE
        if [ $DEBUGMESS -eq "1" ]; then echo "Lock File created" >> $MYDEBUGFOLDER$DEBUGMESSLOG; fi
      fi
    fi
    # Clear the error counter
    ERRORCOUNT=0
  fi

  # set the current position where we are in the file
  LINECOUNT=$NEWLINECOUNT
  if [ $DEBUGMESS -eq "1" ]; then echo "LINECOUNT is now = "$LINECOUNT >> $MYDEBUGFOLDER$DEBUGMESSLOG; fi

  # if there is a lock file, don't keep track of the errors
    if [ -f $MYLOCKFILEFOLDER$MYLOCKFILE ]; then ERRORCOUNT=0; fi

# ok loop
done

About The Author

Ross Moffatt has been a UNIX System Administrator for more than 10 years, and can be contacted at ross.stuff@telstra.com.

Labels

bigadmin bigadmin Delete
sysadmin sysadmin Delete
script script Delete
monitor monitor Delete
log log Delete
Enter labels to add to this page:
Please wait 
Looking for a label? Just start typing.

Sign up or Log in to add a comment or watch this page.


The individuals who post here are part of the extended Sun Microsystems community and they might not be employed or in any way formally affiliated with Sun Microsystems. The opinions expressed here are their own, are not necessarily reviewed in advance by anyone but the individual authors, and neither Sun nor any other party necessarily agrees with them.

Copyright 1994-2009 Sun Microsystems, Inc.
Powered by Atlassian Confluence
Sun Guidelines on Public Discourse Privacy Policy Terms of Use Trademarks Site Map Employment Investor Relations Contact