> Question:
> If you have multiple security devices reporting to a remote syslog server,
> what is the best way to parse or separate the logs? Would you set up
> different daemons on different listing ports and have devices send to a
> specific port and thus be logged to different files?

It's not trivial to set up daemons listening on different ports, and
many syslog _sending_ processes will only send to the standard port.

It is easier to set up different daemons listening on different IP
addresses, by adding alias IPs to the log host. To run multiple listeners
on BSD, I had to hack up the syslogd source code to add command-line
options to bind to a specific IP and/or port.

For example, 'syslog-ng' will log data to different files based on
any number of criteria, including the source IP. Last time I tried the
package (about 6 months ago) I ran into serious bugs that kept me from
deploying. These may be fixed now?

        http://www.balabit.hu/en/downloads/syslog-ng/

> Or would you use a script to separate out the different log info?
> Or would you use physically separate syslog servers?

For multiple devices of the same "class", I log everything to one file and
have a parsing script separate it out. For example, a pool of many
firewalls at a single physical site, in parallel with similar rules and data
from all of them being equally 'sensitive', I might add a "trusted"
interface to each firewall, connected to a standalone switch that only
serves that group of hosts and their dedicated log server.

On a 24-port 100Mb switch, you can have many hosts sending logs, and
more than one loghost receiving logs. With a bit of magic, you can even
have the same log packet go to more than one loghost.

it's important to note that standard syslog is UPD, making it easy to
lose packets and never notice, and trivial for somebody to forge fake
packets, either to insert log events, or to flood the server.

Several of the popular syslogd replacements (including syslog-ng) can
use TCP for reliable delivery, and some will add a checksum and/or
cryptographic signature to each log event.

> Example:
> Assume a syslog server is secured and listening for router logs, firewall
> logs, IDS logs, etc.
>
> Our current syslog config separates out locally logged info into various
> files and simply dumps remotely logged info into a single file.

At a minimum, most people would split these three different type of logs
into at least three different files. This can be as simple as setting
a different 'facility' tag for the IDS versus the routers, at the source.

Personally, I would prefer to deploy a log server on a private
(hardwired) LAN _solely_ to collect events from the firewalls
and IDS, then have another log host on the common internal network
that receives events from the routers and also from any internal servers.
 

> I've looked through quite a few references over the past week and still
> can't find the exact answer I'm looking for, either in configuration
> references for syslog or OpenSource software log parsers.

Log parsing is tough to do in an efficient, scalable manner.
I've tried a number of commercial products that claim to parse various
formats and store the info in a database backend, none of them could
deal well with a large scale network producing gigabytes of log data
each day. So far, nothing I can buy approaches the reliability and
speed of our collection of home-brew Perl scripts running on OpenBSD.

There are numerous "Open Source" log parsing projects, just check out
Freshmeat, SourceForge, Google, etc.

Kevin Kadow

• Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]