$Id: README,v 1.43 2005/03/15 11:35:57 bw Exp $
fwlogwatch is a security tool written in C by Boris Wesslowski originally for RUS-CERT. It is a packet filter/firewall/IDS log analyzer with support for a lot of log formats and has many analysis options. It also features incident report and realtime response capabilities, an interactive web interface and internationalization.
It is available at http://fwlogwatch.inside-security.de/
FEATURES
- General features:
- Can detect and process log entries in the following formats:
- Linux ipchains
- Linux netfilter/iptables
- Solaris/BSD/Irix/HP-UX ipfilter
- BSD ipfw
- Cisco IOS
- Cisco PIX / FWSM
- NetScreen
- Windows XP firewall
- Elsa Lancom router
- Snort IDS
- Entries can be parsed from single, multiple and combined log files, the parsers to be used can be selected.
- Gzip-compressed logs are supported transparently.
- Can separate recent from old entries and detects timewarps in log files.
- Can recognize 'last message repeated' entries concerning the firewall.
- Integrated resolver for protocols, services and host names.
- Can do lookups in the whois database.
- Own DNS and whois information cache and GNU adns support for faster lookups.
- Hosts, networks, ports, chains and branches (targets) can be selected or excluded as needed.
- Support for internationalization (available in english, german, portuguese, simplified and traditional chinese, swedish and japanese).
- Can detect and process log entries in the following formats:
- Log summary mode:
- A lot of options to find and display relevant patterns in connection attempts.
- Intelligent selection of certain fields (e.g. the host name column is omitted and the host mentioned in the header of the summary if the log is from a single host, the same happens with chains, targets and interfaces).
- Output as plain text or HTML (W3C XHTML 1.1 with inline or linked CSS level 2) with limit and sort options.
- Can send summaries by email.
- Interactive report mode:
- The integrated report generator fills in and presents a report that can be sent to abuse contacts of attacking sites or computer emergency response teams (CERTs).
- Supports templates and incident number generation.
- All fields can be adjusted as needed interactively.
- Realtime response mode:
- The program detaches and stays in background as a daemon.
- For ipchains setups detection of necessary rules with logging turned on can be configured.
- Can catch up reading existing entries to provide up-to-date state information from program start on.
- Response can be a notification (in form of a log file entry, an email, a remote winpopup message or whatever you can put into a shell script), or a customizable firewall modification.
- The included response script adds a new chain for fwlogwatch to ipchains or netfilter setups and attackers are blocked with new firewall rules.
- Supports trusted hosts (anti-spoofing).
- The current status of the program can be followed and controlled through a web interface (supports IPv6).
The commented configuration file supports and explains all options and will get you started quickly. Please read the man page for details on the command line options.
PARSER NOTES
- Cisco PIX support focuses on denied packets and ignores all other kinds of PIX log entries. Basic PIX OS version 6 support is available.
- The ipfilter parser does not support logs with resolved service names and tcp/udp entries without ports.
- The Snort parser does not analyze portscan entries yet.
- The NetScreen parser does not recognize packet-filter-unrelated entries and some icmp types which the NetScreen does not seem to recognize itself...
INSTALLATION
- General
The Makefile assumes you use Linux, to compile on Solaris, OpenBSD,
FreeBSD or IRIX look for the corresponding lines at the top of the
Makefile. fwlogwatch may also be compiled and run on Windows with help
of cygwin. You may want to have a look at main.h if you want to change
some default values.
A simple 'make' should be enough to obtain a working binary. If your make (like the one on OpenBSD) thinks flex can only produce files called lex.yy.c type make several times, after all parsers are generated linking will work.
If you use the realtime response mode you will need to install fwlogwatch with superuser permissions for certain configurations. If all you need is read access to the system's standard log file (e.g. /var/log/messages) you can use group permissions. You can also bind a nonprivileged port for the status server. 'make install' will install the binary (in /usr/local/sbin) and the man page, 'make install-config' will also install a sample configuration file and template in /etc.
You can define the TMPDIR environment variable if you don't want fwlogwatch to create it's temporary files (only needed in report mode) in /tmp.
- Zlib/Gettext/IPv6/GNU adns
If you define HAVE_ZLIB, HAVE_GETTEXT, HAVE_IPV6 and HAVE_ADNS fwlogwatch
will be compiled with zlib, gettext, IPv6 and adns support. The output of
fwlogwatch -V will contain an overview of the compiler options used.
You will need the zlib compression library version 1.0.9 or newer and it's header files to be able to use zlib support.
To enable the internationalization support you will need to have the gettext package (sometimes with it's surroundings libtool, GNU m4, autoconf and automake) installed.
You need an operating system with a dual IPv4/IPv6 stack and headers with IPv6 support to be able to use this protocol.
The GNU adns library and header files must be installed to build fwlogwatch with support for faster, asynchronous DNS lookups.
- Linux For ipchains you will need at least kernel 2.2.10 (which you should have updated for security reasons anyway), kernel versions before 2.2.10 don't log enough.
BASICS (with examples for ipchains)
- Create a packet filter with ACCEPT rules for all connections you need, as
tight as possible.
- Your default policies should be DENY:
ipchains -P input DENY
ipchains -P forward DENY
although with ipchains they won't be needed since all prohibited traffic will be intercepted by the following logging rules:
ipchains -A input -l -j DENY
ipchains -A forward -l -j DENY
so the last line in your input, forward and custom chains should be
target prot opt source destination ports
DENY all ----l- anywhere anywhere n/a
- You can also specify rules for the output chain but you should use the target REJECT (you/your users will notice faster what the problem is when trying something the policy forbids).
- If you have a persistent log spammer drop his packets by inserting a DENY rule before the logging rule above (or use the block response mode).
CONFIGURATION AND EXAMPLES
- You should make a configuration file for each function you want, look at the included sample file, it should be easy to adapt to your needs.
- Example (the options are in the order they are mentioned in the text):
If you want very verbose generation of a HTML summary in the file 'log.html'
of all packet filter entries at most one day old representing at least two
connection attempts with output including start and end timestamps, time
intervals, resolved IP addresses and service names and with connections
separated by protocol, source and destination ports and TCP options using
the files that match the expression 'messages*.gz' as input you would use
the command
fwlogwatch -v -v -w -o log.html -l 1d -m 2 -t -e -z -n -N -p -s -d -y messages*.gz
- If you want to use fwlogwatch as a CGI, e.g. to have a quick look at what happened in the last hour when you get a notification: Copy the file fwlogsummary_small.cgi from the contrib directory to a place where your web server can execute it (fwlogwatch must be reachable and have enough permissions to read the log file).
- A script for generation of 8 general summaries (fwlogsummary.cgi) is also included. You can use it as above or if you comment a few lines also on the command line or from cron. It will use the output directory /var/www/html/fwlogwatch as default. Look at the index.html file to select the level of detail you want.
- Contrib also contains a web frontend written in PHP (fwlogwatch.php) to apply fwlogwatch to selected files and test different options. Change the header of the script to adapt it to your system (e.g. prefix of the log file names, location of fwlogwatch).
- Sample init files (for Red Hat and SuSE Linux systems) to start fwlogwatch in realtime response mode at system start are also included.
- You might want to replace your /etc/services file by the one supplied with nmap (http://www.insecure.org/nmap/), a lot more services will be recognized, you can also add the ICMP types (don't assign one to type 0 since it is the ipchains default). The same applies to /etc/protocols, you can use RFC 1700 (Assigned Numbers) to extend your version.
FEEDBACK
If you find a bug or have an idea for a new feature please send an email
to Boris Wesslowski <bw@inside-security.de>.
If you want to report a parser problem please specify which log format you are trying to parse, the version of the log generator and fwlogwatch you are using and provide a sample incuding log lines for which parsing fails. You also can submit unrecognized entries at http://fwlogwatch.inside-security.de/unrecognized.php