Centralized LogServer in SuSE Linux

LogServer: ServerA [IP address: 192.168.1.5]
NetworkServers: ServerB, ServerC... and more

ServerB-------send log files-----> ServerA[LogServer] <-----------send log files---- ServerC

Here we want ServerB, Server C to send it's log file to ServerA for centralized access of log files.

Daemon: syslog-ng
Files:
/etc/sysconfig/syslog
/etc/syslog-ng/syslog-ng.conf

Commands:
/etc/init.d/syslog start|restart|stop

ps aux|grep syslog --> to see if syslog-ng is running or not

SuSEconfig --module syslog-ng --> to reload the change done on /etc/syslog-ng/syslog-ng.conf


Configure LogServer i.e ServerA to accept the log files from NetworkServers

Edit /etc/syslog-ng/syslog-ng.conf on ServerA(Log Server)

source src {
#
# include internal syslog-ng messages
# note: the internal() soure is required!
#
internal();

#
# the default log socket for local logging:
#
unix-dgram("/dev/log");

#
# uncomment to process log messages from network:
#
udp(ip("0.0.0.0") port(514));
#I uncommented above line telling ServerA to accept the log files from network
};


At the bottom of this file, I defined the destination and log

#
#Added by DShah 05/25/11
#
destination std { file("/var/log/HOSTS/$YEAR-$MONTH/$HOST/$FACILITY-$YEAR-$MONTH-$DAY" owner(root) group(root) perm(0600) dir_perm(0700) create_dirs(yes));
};

log { source(src);
destination(std);
};

Over here I am telling ServerA to process the log files coming source src to destination std.
Destination std tells ServerA to save log messages from each host in a separate directory called /var/log/HOSTS/YEAR-MONTH/hostname/.

Now run the command
#SuSEconfig --module syslog-ng --> to reload the config changes done

#/etc/init.d/syslog restart OR
#syslog-ng restart

#ps aux|grep syslog --> to check if syslog-ng is running

If you need to kill syslog-ng process for any reason, you can simply run the command

#killall syslog-ng
or
#kill -9 [PID-of-syslog-ng]

Configure NetworkServers (Server B, ServerC... ) to send log files to LogServer(ServerA):

Edit /etc/syslog-ng/syslog-ng.conf or /etc/syslog-ng/syslog-ng.conf.in (preffered) on ServerB, ServerC

#
#Added by DShah 05/25/2011
#
destination logserver {
udp("192.168.1.5" port(514));
#Note: here 192.168.1.5 is an IP add of LogServer i.e ServerA
};

log {
source(src);
destination(logserver);
};

Now run the command
#SuSEconfig --module syslog-ng --> to reload the config changes done

#/etc/init.d/syslog restart OR
#syslog-ng restart

#ps aux|grep syslog --> to check if syslog-ng is running


ServerA should be already collecting the log files. You can go to /var/log/HOSTS directory on ServerA to see the log files from different Network Servers.

Illustration by Additional applications:
Let's say I want remote asterisk server to dump it's log file /var/log/asterisk/full in the centralized log server
Edit /etc/syslog-ng/syslog-ng.conf or /etc/syslog-ng/syslog-ng.conf.in (preffered) on remote Asterisk Server

#
# Added by DShah
#
source asterisklog { pipe("/var/log/asterisk/full");
};

destination logserver { udp("192.168.1.5" port(514));
};

log { source(asterisklog); destination(logserver); };


and run the command

#syslog-ng restart

Now please check /var/log/HOSTS , you should see log file from asterisk server.


If you need any help on Linux/Unix systems, you can email me at erdevendra@gmail.com with subject title rapidtechguide.

For more info: http://www.novell.com/coolsolutions/feature/18044.html
20 minutes video on syslog-ns : http://www.balabit.com/network-security/syslog-ng/opensource-logging-system/overview#
Syslog-ns to collect apache logs: http://peter.blogs.balabit.com/2010/02/how-to-collect-apache-logs-by-syslog-ng/

Basic guide for Logrotate in Linux

Log files in Linux usually reside at /var/log... It keeps on growing so log management is essential. Log management is usually achieved using logrotate. Logrotate is managed by cronjobs in Linux.

For logrotate, you can configure /etc/logrotate.conf or create the individual configuration files for each application or each log file in /etc/logrotate.d

step 1:

Let's say, I have VOIP application 'asterisk' running on my system. Asterisk generates various log files under /var/log/asterisk directory. I would create astlog under /etc/logrotate.d to manage the log files.

#cd /etc/logrotate.d
#vi astlog
/var/log/asterisk/full /var/log/asterisk/messages /var/log/asterisk/debug /var/log/asterisk/*.log {
nocompress
daily
rotate 5
missingok
copytruncate
}

Here we listed all the log files to be managed and provided the attributes of the log management. Don't compress the log file, rotate the log file daily, max number of log rotation 5 ( i.e logfilexxx.1, logfilexxx.2, .... , logfilexxx.5). It only keeps 5 log files. With copytruncate option, the original log file is truncated in place after creating a copy, instead of moving the old log file and optionally creating a new one. It is useful when some program cannot be told its logfile and thus might continue writing(apending) to the previous log file.

[you can use #stat < filename > or # ls -l < filename > to check the inode number
copytruncate helps the log file to preserver it's inode(unique file number) ]

If you don't want to use copytruncate option, then you have to tell the program that log file has been recreated (with new INODE number). For example, in my case I could have done

#cd /etc/logrotate.d
#vi astlog
/var/log/asterisk/full /var/log/asterisk/messages /var/log/asterisk/debug /var/log/asterisk/*.log {
nocompress
daily
rotate 5
missingok
create

        postrotate
                /usr/sbin/asterisk -rx 'logger reload' > /dev/null 2> /dev/null
    endscript

}

Here, we are telling our program 'asterisk' to reload logger as new log file has been created after log rotation.

step 2:

By default, Logrotate is scheduled daily. You can find 'logrotate' under /etc/cron.daily

Let's look at /etc/crontab

# less /etc/crontab
SHELL=/bin/sh
PATH=/usr/bin:/usr/sbin:/sbin:/bin:/usr/lib/news/bin
MAILTO=root
#
# check scripts in cron.hourly, cron.daily, cron.weekly, and cron.monthly
#
-*/15 * * * * root test -x /usr/lib/cron/run-crons && /usr/lib/cron/run-crons >/dev/null 2>&1


The time to execute the scripts is managed by crontab. /usr/lib/cron/run-crons script controls the cron.hourly, cron.daily, cron.weekly and cron.monthly. run_crons runs every 15 minutes and ensures that cron jobs are taken care of.

In SLES, if you need to change the default daily time , you can go to YAST --> System --> /etc/sysconfig editor --> System --> Cron --> DAILY_TIME and change the time.

Let's say, I want logrotation to be done at OFF hours (10:30 pm) to avoid the possible load on the server, then, I can change DAILY_TIME to 22:30

For more information:

http://www.linuxtopia.org/online_books/suse_linux_guides/SLES10/suse_enterprise_linux_server_installation_admin/sec_suse_pakete.html