How to filter SYSLOG data to collect meaningful information only?

Login to Reply

Thread Tools Search this Thread
# 1  
Old 04-03-2015
How to filter SYSLOG data to collect meaningful information only?

Dear users,

SUSE 10 sp3 and SUSE 11.

I made configuration changes in '/etc/syslog-ng/syslog-ng.conf'
to move SYSLOG content to LogRhythm.

This is what I changed in the file.
Un-comment out the following lines
#Enable this and admop IP to send log messages to a log server
Destination logserver Enter in syslog server ip address
log {source(src)
destination allmessages

Now, logs are being generated and transferred to LogRhythm. But the problem is too much log information is being generated and it's filling up our Logrhythm quickly, 30% in a week. We'd like to reduce log generated in SYSLOG by collecting only useful information which describe below.

The log information we'd like to capture is below.

what file should I edit and what changes should I make in that file? Is there any document or procedure posted on the web?

Always appreciate your support.

- Successful login and logoff attempts
- Unsuccessful login and authorization attempts
- All identification and authentication attempts
- All actions, connections and requests performed by privileged users
- All changes to logical access control authorities (e.g., rights, permissions
- System changes with the potential to compromise the integrity of audit -policy configurations, security policy configurations and audit record generation services.
- Creation, modification and deletion of objects including files, directories and user accounts
- Creation, modification and deletion of user accounts and group accounts
- Creation, modification and deletion of user account and group account privileges
- The date of the system event; ii)the time of the system event; iii) the type of system event initiated; and iv) the user account, system account, service or process responsible for initiating the system event.

- System start-up and shutdown functions.
- Modifications to administrator account(s) and administrator group
account(s)including: i) escalation of user account privileges
commensurate with administrator-equivalent account(s); and ii) adding or
deleting users from the administrator group account(s).

enabling or disabling of audit report generation services

-command line changes, batch file changes and queries made to the system (e.g., operating system, application, and database).
# 2  
Old 04-03-2015
I would continue to send all logging data to your logging server and use a tool like Solarwinds LEM (Log and Event Manager) to produce the reports and eventing output you looking for.
# 3  
Old 04-06-2015
Flitering SYSLOG

is there a way to control producing or generating SYSLOG data?

I'd like SYSLOG to generate only certain data such as update, delete, logon etc.

We have LogRhythm and it doesn't want to store all SYSLOG data generated.

# 4  
Old 04-10-2015
Filter unnessary log through syslog-ng.conf file

The SYSLOG file '/var/log/messages/' generates so many lines of Mcafee scan results.

I'd like to get rid of these lines.

The log lines start with 'nails-ddds' and end with 'cmd=update'.

What command do I add in '/etc/syslog-ng/syslog-ng.conf' file to get rid of these log information?

Thank you,
# 5  
Old 04-10-2015
You need to consult your McAfee manual and see if the "facility" is configurable. Usually, you'd choose one of the "user" facility types (local0 - local7) and configure it to use that facility... then you can adjust your syslog conf to take messages for that facility and output to a separate log area.

Alternatively, McAfee may have support for outputting to a local log file outside of using syslog messages, in which case you can configure that.

Syslog really doesn't support the idea of fine grained log selection (which is why SolarWinds and Splunk and such exist).

With that said, as systemd (the borg) takes over everything, logging is going to change in some pretty radical ways... so whatever you do with syslog today, don't get used to it. I promise you it will change if your distribution switches to systemd.
# 6  
Old 04-10-2015
You never know what you'll want from your logfiles until you do, so I wouldn't try being too incredibly specific except for separating mcaffee results from everything else interesting.
# 7  
Old 04-13-2015
I would go for collecting everything and then extracting what you actually need when you need it. It leaves masses of records never read, but there is no way to say to the server, "Hey, just go back in time, replay a process that I don't know quite when it happened and show me the logging now."

We have load and load & LOADS of output that is never looked at unless there is a problem and we rotate the logs out and away at sensible intervals to keep on top of it. if we don't need most of it in a week (production batch messages) then they are sent to tape and never seen again.

Login to Reply

Thread Tools Search this Thread
Search this Thread:
Advanced Search

Similar Threads More UNIX and Linux Forum Topics You Might Find Helpful
Thread Thread Starter Forum Replies Last Post
Collect last 2 days data from /var/log/messages makauser Shell Programming and Scripting 4 04-14-2014 04:50 PM
Use PYTHON to collect data weather and print in new file ragaga123 Shell Programming and Scripting 0 07-16-2013 12:24 AM
Korn script to access multiple host and collect data blinkingdan Shell Programming and Scripting 2 07-25-2012 01:30 AM
shell script to collect information from current and remote unix boxes amir07 Shell Programming and Scripting 12 02-07-2012 11:54 AM
collect data from files kanikasharma UNIX Desktop Questions & Answers 5 11-11-2011 04:13 AM
grep PID and collect data ? sabercats Shell Programming and Scripting 5 05-11-2011 06:40 PM
script to collect all db information mvsramarao Shell Programming and Scripting 3 10-30-2009 01:46 AM
How to filter data Muhammad Rahiz UNIX for Dummies Questions & Answers 6 10-24-2009 02:35 PM
extract data from a data matrix with filter criteria ssshen Shell Programming and Scripting 7 04-16-2009 11:38 AM
Collect information from switches chinni-script Shell Programming and Scripting 1 04-03-2009 09:20 PM
Enable a shell to collect data from unix, return to windows rossalyn_maxter UNIX for Dummies Questions & Answers 1 12-19-2007 01:55 AM
collect data from another file bikas_jena Shell Programming and Scripting 3 11-22-2007 03:27 AM
ccall database and collect data from one table rinku Shell Programming and Scripting 0 05-28-2007 02:16 AM
Planning for DR, I have to collect information adel8483 Solaris 1 03-26-2007 08:45 AM
filter data deep.singh UNIX for Dummies Questions & Answers 3 08-15-2005 12:24 AM
All times are GMT -4. The time now is 12:53 PM.

Unix & Linux Forums Content Copyright 1993-2018. All Rights Reserved.