Centralized LogServer in SuSE Linux

LogServer: ServerA [IP address: 192.168.1.5]
NetworkServers: ServerB, ServerC... and more

ServerB-------send log files-----> ServerA[LogServer] <-----------send log files---- ServerC

Here we want ServerB, Server C to send it's log file to ServerA for centralized access of log files.

Daemon: syslog-ng
Files:
/etc/sysconfig/syslog
/etc/syslog-ng/syslog-ng.conf

Commands:
/etc/init.d/syslog start|restart|stop

ps aux|grep syslog --> to see if syslog-ng is running or not

SuSEconfig --module syslog-ng --> to reload the change done on /etc/syslog-ng/syslog-ng.conf


Configure LogServer i.e ServerA to accept the log files from NetworkServers

Edit /etc/syslog-ng/syslog-ng.conf on ServerA(Log Server)

source src {
#
# include internal syslog-ng messages
# note: the internal() soure is required!
#
internal();

#
# the default log socket for local logging:
#
unix-dgram("/dev/log");

#
# uncomment to process log messages from network:
#
udp(ip("0.0.0.0") port(514));
#I uncommented above line telling ServerA to accept the log files from network
};


At the bottom of this file, I defined the destination and log

#
#Added by DShah 05/25/11
#
destination std { file("/var/log/HOSTS/$YEAR-$MONTH/$HOST/$FACILITY-$YEAR-$MONTH-$DAY" owner(root) group(root) perm(0600) dir_perm(0700) create_dirs(yes));
};

log { source(src);
destination(std);
};

Over here I am telling ServerA to process the log files coming source src to destination std.
Destination std tells ServerA to save log messages from each host in a separate directory called /var/log/HOSTS/YEAR-MONTH/hostname/.

Now run the command
#SuSEconfig --module syslog-ng --> to reload the config changes done

#/etc/init.d/syslog restart OR
#syslog-ng restart

#ps aux|grep syslog --> to check if syslog-ng is running

If you need to kill syslog-ng process for any reason, you can simply run the command

#killall syslog-ng
or
#kill -9 [PID-of-syslog-ng]

Configure NetworkServers (Server B, ServerC... ) to send log files to LogServer(ServerA):

Edit /etc/syslog-ng/syslog-ng.conf or /etc/syslog-ng/syslog-ng.conf.in (preffered) on ServerB, ServerC

#
#Added by DShah 05/25/2011
#
destination logserver {
udp("192.168.1.5" port(514));
#Note: here 192.168.1.5 is an IP add of LogServer i.e ServerA
};

log {
source(src);
destination(logserver);
};

Now run the command
#SuSEconfig --module syslog-ng --> to reload the config changes done

#/etc/init.d/syslog restart OR
#syslog-ng restart

#ps aux|grep syslog --> to check if syslog-ng is running


ServerA should be already collecting the log files. You can go to /var/log/HOSTS directory on ServerA to see the log files from different Network Servers.

Illustration by Additional applications:
Let's say I want remote asterisk server to dump it's log file /var/log/asterisk/full in the centralized log server
Edit /etc/syslog-ng/syslog-ng.conf or /etc/syslog-ng/syslog-ng.conf.in (preffered) on remote Asterisk Server

#
# Added by DShah
#
source asterisklog { pipe("/var/log/asterisk/full");
};

destination logserver { udp("192.168.1.5" port(514));
};

log { source(asterisklog); destination(logserver); };


and run the command

#syslog-ng restart

Now please check /var/log/HOSTS , you should see log file from asterisk server.


If you need any help on Linux/Unix systems, you can email me at erdevendra@gmail.com with subject title rapidtechguide.

For more info: http://www.novell.com/coolsolutions/feature/18044.html
20 minutes video on syslog-ns : http://www.balabit.com/network-security/syslog-ng/opensource-logging-system/overview#
Syslog-ns to collect apache logs: http://peter.blogs.balabit.com/2010/02/how-to-collect-apache-logs-by-syslog-ng/