_ _ _
|_) _ |_ / |_ \ | _ _
| \(_)(_, \_ (_, _/ (_,(_)(_)
By Stefano Falsetto ,_)
This is Rot[t]Log
archive, rotates, compresses, and mails system logs.
This is a replacement to Red Hat's logrotate. It have similar syntax, but more powerful features to cut and store logs. This README is heavily bases on Red Hat's logrotate man page. It's all written in BASH (2.x compatible).
This script require GNU fileutils, GNU grep, GNU awk , GNU shell-utils and util-linux.
USE WITH CARE
At this moment this software is still in alpha stage, so... handle with care
your logs...
Main configuration file : /etc/rottlog/rc Other configuration files : /etc/rottlog/[daily|weekly|monthly|custom]
rottlog first read main configuration file, then read (in order) daily, weekly and monthly configuration files.
::::: RC FILE SYNTAX :::::
This is the main configuration file. It defines default behavoir of
Rot[t]Log.
Following keywords are valid:
compress=<options>
createdir
default_storefile=<storefile>
dir_grp=<group>
dir_own=<owner>
dir_perm=<permissions>
extension=<ext>
follow_symlinks
fromuser=<sender address>
ifempty
mail=<mailer prg>
maxdepth=<depth>
missingok
nocompress
nocreate
nomail
nomissingok
notifempty
packdir=<path-to-archive-dir>
packer=<absolute path>
pager=<pager-program>
remove_missing
SunMon=<sun|mon>
touser=<receiver address>
uncompress=<options>
unpacker=<complete path>
compress=<options>
options to be used with packer program to compress data
IMPORTANT: Packer program must output data to stdout.
(i.e. for gzip/bzip2 use compress="bzip2 -c")
createdir
Create dirs specified in storedir if needed. It's useful to use
with meta-vars. See section USE OF META-VARIABLES for more info.
default_storefile=<storefile>
Default filename to use with archived/rotated logfiles. It's useful
to use with meta-vars. See section USE OF META-VARIABLES for more info.
WARNING:
>>> Time-related meta-vars are NOT compatibles with rotate parameter
dir_grp=<group>
Default group of newly created directory. Must exists
in /etc/group file.
dir_own=<owner>
Default owner of newly created directory. Must exists
in /etc/passwd file.
dir_perm=<permissions>
Default permissions of newly created directory. Only numerical
notation is permitted.
fil_grp=<group>
Default group of newly created logfile. Must exists
in /etc/group file.
fil_own=<owner>
Default owner of newly created logfile. Must exists
in /etc/passwd file.
fil_perm=<permissions>
Default permissions of newly created logfile. Only numerical
notation is permitted.
extension=<ext>
Extension normally used when compressing data with packer.
(i.e. for gzip is gz, for bzip2 is bz2, ...)
This variable is used to expand meta-variable @COMP_EXT.
follow_symlinks
Follow symlinks to logfiles. Default behavoir is to not to follow
symlinks for security reasons.
fromuser=<sender address>
Sender address used in mail messages.
ifempty
Rotate the log file even if it is empty.
mail=<mailer prg>
Mail program to send messages. Must be used option to read header
recipient addresses.
- Example
-
- for sendmail use: mail="sendmail -t"
- for qmail use: mail="qmail-inject -h"
maxdepth=<depth>
Default maximum depth to search for files when used * metachar in log
filename definition.
missingok
If the log file is missing, go on to the next one without issuing an
error message. Lastaction (if defined) will be performed. Postrotate
script (if defined) will NOT be performed.
nocompress
Don't compress logfiles to be archived/rotated.
nocreate
Don't create a new filename. Can't be used with logpart option.
nomail
Don't mail rottlog's activity report.
nomissingok
rottlog will return an error message and stop execution
if the log file is missing.
notifempty
Do not rotate a logfile if it's empty.
packdir=<path-to-archive-dir>
default basedir to store rotated/archived files.
This variable is used to expand @DEF_PATH meta-variable.
packer=<absolute path>
complete path to packer program to be used to compress logs.
pager=<pager-program>
Path to program to use with --showlog parameter. It's used in a pipe,
so must be a program able to read input from stdin.
remove_missing
rottlog will stop execution if a delayed file is no longer existant
or readable.
SunMon=<sun|mon>
Use sunday or monday as first day of week.
This option is useful only in conjunction with @WEEK meta-variable.
touser=<receiver address>
Receiver of mail messages.
uncompress=<options>
options to be used with unpacker program to uncompress data
IMPORTANT: Unpacker program must output data to stdout.
(i.e. for gunzip/bunzip2 use -c)
unpacker=<complete path>
complete path to program to be used to uncompress logs.
(i.e. gunzip, bunzip2...)
::::: DAILY|WEEKLY|MONTHLY|CUSTOM FILE SYNTAX :::::
Period config files are checked once per month, week and day, and all logfiles are handled at the same time. Custom config file is read every time rottlog is called. Example:
- rottlog is run on 1st of a month
- Logfiles in 'monthly' will all be handled on 1st of each following month
- Logfiles in 'weekly' will all be handled on 1st, 7th, 14th, 21th, 28th day and so on
- Logfiles in 'daily' will all be handled each day in the month
- Logfiles in 'custom' will all be handled every time rottlog is called.
So if a logfile must be rotated/archived only one day of each month it must be inserted in custom config file and must be used "period" parameter.
Global structure of a period-config file is:
out-of-block-definitions
configuration-block-1-start {
block-contents
}
configuration-block-2-start {
block-contents
}
....
Comments can be inserted in rows beginning with '#'. Comments on same row of valid items are not allowed.
- Example
This is a valid comment
logpart "1 day" # This is an INVALID comment
There is only one out-of-block-definition (at the moment):
include <path-to-conffiles>[,<path-to-conffiles>,...]
This option permits to include other files at the end of configuration
file. Nested includes are not supported: only one level of inclusion
is supported.
This keyword must be inserted at beginning of configuration file before
and out of all configuration blocks. If used in other positions will
be treated as error.
<path-to-conffiles> must be an absolute path to a file or a group of
files (using * wildcard). To include all files in a directory must be
used path-to-dir/* (i.e. /etc/rottlog/conf-monthly/*)
A configuration block for logfiles has following structure:
<logfile1>,<logfile2>,... {
<append-only>
<collate> <logfile>
<create> <mode> <owner> <group>
<createdir> [<perms>] [<own> <grp>]
<delaycompress>
<dateoffset>
<firstaction> ... <endaction>
<ifempty>
<lastaction> ... <endaction>
<logpart> <qdldl>
<maxdepth> <depth>
<missingok>
<nocompress>
<nocreate>
<nomail>
<nostoredir>
<notifempty>
<period> <period-definitions>
<postrotate> ... <endscript>
<prerotate> ... <endscript>
<rotate> <number>
<sharedscripts>
<size> <num[b,k,M]>
<storedir> <path-to-dir>
<storefile> <filename>
<tarcollate> <logfile>
<touser> <email address>
}
- Where
logfile1, logfile2, ...
Is a comma separated list of complete absolute path to logfiles to
archive/rotate. From 0.10alpha+ version it's possible to use ONLY
metachar asterisk. (i.e. /var/adm/log.* means that definitions between
curly brackets are applied to all files in /var/adm dir beginning by
'log.'). If logfile is a filename containing special chars like
']' or '{' you must protect it from expansion with DOUBLE slash
(i.e. apache\\[1\\].log). If same logfile is used more than one time
in same configuration file, will be used only the first.
append-only
To be used only on ext[23] fs.
Set append-only attribute to a log (More precisely: remove this
attribute before handle log, and reset it after performed all
operations).
collate
Store more than one logfile in only one file compressed.
This option makes possible to weekly rotate a logfile (i.e. in a little
volume) and monthly store a compressed file containing previously rotated
files (i.e. in a bigger backup volume).
If used in monthly configuration file, <logfile> will be searched in weekly
configuration file.
If used in weekly configuration file, <logfile> will be searched in daily
configuration file.
Can't be used in daily and custom configuration files.
Logfiles referenced in collate option must use rotate option with correct
- periods
-
- collate monthly a weekly rotated file : rotate 4
- collate weekly a daily rotated file : rotate 7
- Example
- Collect monthly a weekly rotated log.daemon
In weekly configuration file:
/var/adm/log.daemon {
storedir /mnt/tiny/@YEAR/@BASENAME
createdir 4700 stefano root
create 600 stefano root
prerotate
/sbin/killall -HUP syslogd
endscript
# Important!
rotate 4
delaycompress
}
In monthly configuration file:
/var/adm/log.daemon {
storedir /mnt/BigVolume/Collections/@YEAR/@BASENAME
createdir
collate
}
In this example in /mnt/BigVolume/Collections/2002/log.daemon will be stored a file called log.daemon.<number> containing log.daemon.1, log.daemon.2, log.daemon.3 and log.daemon.4 weekly created in /mnt/tiny/2002/log.daemon dir.
This option is compatible only with following options:
- [no]storedir
- storefile
- createdir
- [no]missingok
- firstaction/lastaction
- prerotate/postrotate
- nocompress
- touser
- nomail Options ifempty/notifempty are silently ignored.
create [[<mode>] [<owner> <group>]]
Immediately after rotation (before the postrotate script is run)
the log file is created (with the same name as the log file
just rotated). mode specifies the mode for the log file in octal (the
same as chmod(2)), owner specifies the user name who will own the
log file, and group specifies the group the log file will belong to.
If none of three parameters is specified new file will be created
with permissions and owner defined in default variables fil_perm,
fil_own, and fil_grp. You can specify mode only, or mode and owner,
but you can't to specify mode and group (second parameter is always
interpreted as owner).
createdir [[<mode>] [<owner> <group>]] Create (if needed) dir to store archived/rotated logs with specified permissions and owner:group. Directory will be created with permissions and owner defined in default variables dir_perm, dir_own, and dir_grp if none of three parameters is used. You can specify mode only, or mode and owner, but you can't to specify owner and group (second parameter is always interpreted as owner). perms specifies the mode for the log file in octal (the same as chmod(2)).
- IMPORTANT
- If path to be created is deeper than one dir, only last dir will have owner, group and mode specified.
- Example
storedir 2002/04/apache_logs
createdir 4300 rottlog
If dir 2002 (and following) are not already present, on your HD after rottlog run you will have (rottlog is run by user stefano): $ ls -ld 2002 2002/04 2002/04/apache_logs
drwxr-xr-x 2 stefano users 1024 Apr 11 18:35 2002/ drwxr-xr-x 2 stefano users 1024 Apr 11 18:35 2002/04 d-ws------ 2 rottlog users 1024 Apr 11 18:35 2002/04/apache_logs
delaycompress
Postpone compression of the previous log file to the next rotation
cycle.
It can be used when some program can not be told to close its
logfile and thus might continue writing to the previous log file for
some time.
dateoffset <+/-day>
Force time related metavariables to be calculed with specified days of offset.
- Example
- if today is Aug 5 and is used "dateoffset -1", @DAY metavariable will be 04 and not 05.
firstaction <EXIT_CODE>... endaction
Unique line between firstaction and endaction (both of which must
appear on lines by themselves) are executed before ANY operation. It
will be performed during parameter parsing, before filename and metavars
expansion. Optional parameter EXIT_CODE is default exit code expected
by action. If not specified will expect 0.
If used in multiple files definition action will be run only one
time, before processing first file.
ifempty
Rotate the log file even if it is empty.
lastaction <EXIT_CODE>... endaction
Unique line between lastaction and endaction (both of which must
appear on lines by themselves) are executed after ALL operations. It
will be performed just before handle next logfile. Optional parameter
EXIT_CODE is default exit code expected by action. If not specified
will expect 0.
If used in multiple files definition action will be run only one
time, after processed all files.
logpart <qdldl>
Leave a part of log to archive/rotate in new logfile.
qdldl can be:
<number>l
<number>b
"<regexp>"
"<#1 day>"
Where l and b means lines and bytes respectively.
"regexp" is a regular expression used by grep to find first row to
leave in log.
"#1 day" is a shortcut to leave in log all events recorded from first
day of current month.
maxdepth <depth>
Default maximum depth to search for files when used * metachar in log
filename definition.
missingok
If the log file is missing, go on to the next one without issuing an
error message. Default behavoir is to stop executing and return an
error exit code.
nocompress
Old versions of log files are not compressed with packer defined in
rc file.
nocreate
Don't create a new filename. Can't be used with logpart.
nomail
Don't send notification e-mail message.
nostoredir
Logs are rotated/archived in the same directory the log normally
resides in.
notifempty
Do not rotate the log if it is empty.
period <period-definitions>
This is a very powerful option usable only in custom config file.
It force a rotation only in a specified period or specific day of
month.
<period-definitions> is a comma-separated list of items defining
a day, or a set of days, a month and so on. Usable items can be:
D -> Each Dth day of the month DDd -> Each DD days WWw -> Each WW weeks mmM -> Each mm months <weekday> -> Each <weekday> (abbreviated or not) of the week <monthname> -> Each <monthname> (abbreviated or not) of the year HH:MM -> At specified hour and minutes (*) 0 -> Each time rottlog is called (*)
(*) It depends on rottlog's entries in crontab
In definition of period parameters following characters have particular
- signification
, (comma) -> OR " " (white space) -> AND ! (exclamation mark) -> NOT - (minus) -> Week or month range
Minus carachter can be used to define a list of days or months.
Comma separed tokens are analized in given order. When one of it
defines a true condition, the check is stopped without continue to
parse following conditions.
WARNING: Correct behavoir of rottlog and period parameter strictly
depends from correct configuration of rottlog's crontab.
Some examples:
- period mon-fri 21:00,sat 10:00
Is equivalent to:
period mon 21:00,tue 21:00,wed 21:00,thu 21:00,fri 21:00,sat 10:00
And means that rotation must be performed if rottlog is run and: today is monday AND it is 21:00 hour OR today is tuesday AND at it is 21:00 hour, and so on just to friday, OR today is saturday and at is 10:00 hour.
2) period jan-jun 23:00,jul-sep 10:00,oct-dec 23:00
Is equivalent to:
period !jul-sep 23:00,jul-sep 10:00
And means that rotation must be performet if rottlog is run and: today is NOT in a month between (including) july and september AND it is 23:00 hour, OR it is 10:00.
postrotate ... endscript
Lines between postrotate and endscript (both of which must
appear on lines by themselves) are executed after the log file is
archived/rotated. Can be used metavariables. See section USE OF
META-VARIABLES for more info.
prerotate ... endscript
Lines between prerotate and endscript (both of which must appear
on lines by themselves) are executed before the log file is
archived/rotated. Can be used metavariables. See section USE OF
META-VARIABLES for more info.
rotate <number>
Log files are rotated <number> times before being overwrited.
sharedscripts
Let postrotate script to be run only one time, after handled last file in
a multiple file definition block.
- Example
- /var/adm/log.* { sharedscripts postrotate killall -HUP syslogd endscript } The killall command will be run only after handled all log files beginning by "log" in /var/adm dir.
size <num[b,k,M]>
Rotate a logfile only if it grows specified dimension.
Parameter must be a number followed by one of following chars:
b - means bytes
k - means kbytes (num * 1024)
M - means megabytes (num * 1000000)
If none of these three chars is used, b is default.
storedir <path-to-dir>
If it's a relative path it will be appended to packdir defined in rc
file, if it's an absolute path it will be used instead of packdir. Can
be used metavariables. See section USE OF META-VARIABLES for more info.
storefile <filename>
Name of compressed logfile to be used (whitout any path: filename
only) Can be used metavariables. See section USE OF META-VARIABLES
for more info.
tarcollate
Same of collect, but store files in a compressed tar file. In addition this
option is compatible with ifempty option too. If set, a void file will be
archived in tar file. If not set it will be skipped.
Tar archive will contain relative path if possible (if archived files are in
subdirs of packdir defined in main rc file) or absolute path.
touser <email address>
Override global value of touser variable defined in rc file.
::::: USE OF META-VARIABLES :::::
In monthly/weekly/daily logs can be used metavariables. Metavariables are identifiers of a runtime defined value. Handled metavariables:
@DIRNAME : Dir containing logfile to be archived/rotated @FILENAME : filename with numerical extension (BASENAME+NEXT_EXT) @BASENAME : filename without extension @DEF_PATH : default path to archive dir @COMP_EXT : extension of compressed format file (bz2, gz, ...) @NEXT_EXT : next available numbered extension
@YEAR : actual year(*) @DAY : actual day(*) @MONTH : actual month(*) @WEEK : number of week in year (0-53) accordingly to SunMon variable (*)
@TEMPDIR : a temporary directory
@1..@n : string corresponding to first, second and so on directory
in path of file to archive/rotate
(*) Can't be used in storefile if using rotate parameter too
Parameters handling metavariables:
storedir
storefile
prerotate...endscript
postrotate...endscript
Values of metavariables are defined while script is running, so a definition like this:
/var/adm/log.* {
storedir /var/adm/archive-log/@BASENAME
rotate 5
...
}
Means that I wants to rotate 5 times all files in /var/adm beginning by "log." and store each file in a dir named with logname. (i.e. /var/adm/archive-log/log.kern, /var/adm/archive-log/log.daemon, ...)
- WARNING
- Metavariables substitution is merely textual, so in a definition like this:
/var/http/www.mysite.com/log/access.log {
storefile @3-@13-@FILENAME
rotate 4
....
}
Will create logfiles called somethink like:
www.mysite.com-var3-access.log.1
@13 is not defined (path in first line of block definition contains only 5 tokens) so @1 will be substituted and following text will not be affected.
::::: COMMAND LINE PARAMETERS :::::
Valid command line parameters allowed are:
--help, -h
Display help page and exit. Does nothing else.
--version, -V
Display version, disclamier and exit. Does nothing else.
--defaults
Display defaults values and exit. Useful to know default behavoir of
rottlog
--showlog <logfile>, -s <logfile>
Show contents of archived <logfile> with default pager. <logfile> can be
only a filename without any path or an absolute or relative path. If
<logfile> is a relative path to a file, will be searched in $packdir
dir.
--pager <program>
Use <program> for paging output of showlog. Useful only with --showlog
--summary <logfile>
Display a summary of some useful information about <logfile>. Does not
exec any rotate/archive action. It's something like a --checkrc
but more verbose.
--checkrc <what>,<what>,...
Check for syntax of specified configuration file(s)
<what> must be one of: daily, weekly, monthly, custom, all
--force [<daily>] [<weekly>] [<monthly>] [<custom>] --forceonly [<daily>] [<weekly>] [<monthly>] [<custom>]
Force rotation/archiviation of logfiles , even if it doesn't think
this is necessary. Sometimes this is useful after adding new entries
to some config file, or if old log files have been removed by hand,
as the new files will be created, and logging will continue
correctly. If none of daily,weekly and monthly is used, will be
forced rotation/archiviation of all entries in all config files;
If used --force option will be forced rotation/archiviation of specified
config file and will be checked for rotation/archiviation remaining config
files.
If used --forceonly option will be forced rotation/archiviation of specified
config file without check for rotation/archiviation remaining config files.
(For custom config file this option is useful only if used forceonly
parameter.)