How to filter SYSLOG data to collect meaningful information only?

 
Thread Tools Search this Thread
Operating Systems Linux SuSE How to filter SYSLOG data to collect meaningful information only?
# 1  
Old 04-03-2015
How to filter SYSLOG data to collect meaningful information only?

Dear users,

SUSE 10 sp3 and SUSE 11.

I made configuration changes in '/etc/syslog-ng/syslog-ng.conf'
to move SYSLOG content to LogRhythm.

This is what I changed in the file.
Un-comment out the following lines
#Enable this and admop IP to send log messages to a log server
Destination logserver - Enter in syslog server ip address
log {source(src)
destination allmessages


Now, logs are being generated and transferred to LogRhythm. But the problem is too much log information is being generated and it's filling up our Logrhythm quickly, 30% in a week. We'd like to reduce log generated in SYSLOG by collecting only useful information which describe below.

The log information we'd like to capture is below.

what file should I edit and what changes should I make in that file? Is there any document or procedure posted on the web?

Always appreciate your support.

- Successful login and logoff attempts
- Unsuccessful login and authorization attempts
- All identification and authentication attempts
- All actions, connections and requests performed by privileged users
- All changes to logical access control authorities (e.g., rights, permissions
- System changes with the potential to compromise the integrity of audit -policy configurations, security policy configurations and audit record generation services.
- Creation, modification and deletion of objects including files, directories and user accounts
- Creation, modification and deletion of user accounts and group accounts
- Creation, modification and deletion of user account and group account privileges
- The date of the system event; ii)the time of the system event; iii) the type of system event initiated; and iv) the user account, system account, service or process responsible for initiating the system event.

- System start-up and shutdown functions.
- Modifications to administrator account(s) and administrator group
account(s)including: i) escalation of user account privileges
commensurate with administrator-equivalent account(s); and ii) adding or
deleting users from the administrator group account(s).

enabling or disabling of audit report generation services

-command line changes, batch file changes and queries made to the system (e.g., operating system, application, and database).
# 2  
Old 04-03-2015
I would continue to send all logging data to your logging server and use a tool like Solarwinds LEM (Log and Event Manager) to produce the reports and eventing output you looking for.
# 3  
Old 04-06-2015
Flitering SYSLOG

is there a way to control producing or generating SYSLOG data?

I'd like SYSLOG to generate only certain data such as update, delete, logon etc.

We have LogRhythm and it doesn't want to store all SYSLOG data generated.

Thanks,
# 4  
Old 04-10-2015
Filter unnessary log through syslog-ng.conf file

The SYSLOG file '/var/log/messages/' generates so many lines of Mcafee scan results.

I'd like to get rid of these lines.

The log lines start with 'nails-ddds' and end with 'cmd=update'.

What command do I add in '/etc/syslog-ng/syslog-ng.conf' file to get rid of these log information?

Thank you,
# 5  
Old 04-10-2015
You need to consult your McAfee manual and see if the "facility" is configurable. Usually, you'd choose one of the "user" facility types (local0 - local7) and configure it to use that facility... then you can adjust your syslog conf to take messages for that facility and output to a separate log area.

Alternatively, McAfee may have support for outputting to a local log file outside of using syslog messages, in which case you can configure that.

Syslog really doesn't support the idea of fine grained log selection (which is why SolarWinds and Splunk and such exist).

With that said, as systemd (the borg) takes over everything, logging is going to change in some pretty radical ways... so whatever you do with syslog today, don't get used to it. I promise you it will change if your distribution switches to systemd.
# 6  
Old 04-10-2015
You never know what you'll want from your logfiles until you do, so I wouldn't try being too incredibly specific except for separating mcaffee results from everything else interesting.
# 7  
Old 04-13-2015
I would go for collecting everything and then extracting what you actually need when you need it. It leaves masses of records never read, but there is no way to say to the server, "Hey, just go back in time, replay a process that I don't know quite when it happened and show me the logging now."

We have load and load & LOADS of output that is never looked at unless there is a problem and we rotate the logs out and away at sensible intervals to keep on top of it. if we don't need most of it in a week (production batch messages) then they are sent to tape and never seen again.



Robin
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Collect last 2 days data from /var/log/messages

I need to collect last 2 days data from /var/log/messages into a separate file (file format: flmessagetimedaymonth). I have collect today's month, date, time information in separate variable. Please help me in this issue (Probably need awk and grep function). month=$(date|awk '{print $2}')... (4 Replies)
Discussion started by: makauser
4 Replies

2. Shell Programming and Scripting

Use PYTHON to collect data weather and print in new file

Hi all, i have a task to do with Python and because i'm a beginner on it i would like your help on this. Create a python script that: Every hour collect the Temperature (e.g. 29C) and Current Condition (e.g. Clear) from this website wunderground.com/global/stations/54511.html Create a CSV... (0 Replies)
Discussion started by: ragaga123
0 Replies

3. Shell Programming and Scripting

shell script to collect information from current and remote unix boxes

I am using AIX 5.3.0.0 ; I need a script to find out each remote AIX boxes Hostname, Model name and Serial number in following format Hostname Modelname SerialID AIXMC01 IBM,7026-B80 IBM,0110BBA1F AIXMC02 IBM,7026-H50 IBM,0110BBA56 AIXMC03 IBM,7026-H50 IBM,0110BBARR... (12 Replies)
Discussion started by: amir07
12 Replies

4. UNIX Desktop Questions & Answers

collect data from files

there are 200 files named file1_0.pdb,file1_60.pdb etc....it looks like: ATOM 1 N VAL 1 8.897 -21.545 -7.276 1.00 0.00 ATOM 2 H1 VAL 1 9.692 -22.015 -6.868 1.00 0.00 ATOM 3 H2 VAL 1 9.228 -20.766 -7.827 1.00 0.00 ATOM 4 H3 ... (5 Replies)
Discussion started by: kanikasharma
5 Replies

5. Shell Programming and Scripting

grep PID and collect data ?

I have a file current.csv file 2011/05/06 11:12:20 | 16:guest pid=014782 opened Boards 0, 1, 2, 3 2011/05/06 12:23:13 | 16:guest pid=014782 closed 2011/05/06 12:26:05 | 17:guest pid=022383 opened Boards 0, 1, 2, 3 2011/05/06 20:09:04 | 17:guest pid=022383 closed 2011/05/06... (5 Replies)
Discussion started by: sabercats
5 Replies

6. Shell Programming and Scripting

script to collect all db information

hi all I am looking for shell script ,by which i need to gather all the DB information in Sybase server . Thanks in advance :) (3 Replies)
Discussion started by: mvsramarao
3 Replies

7. Shell Programming and Scripting

Collect information from switches

Hi All , In my environment we have 12 SAN switches .Culd u pls help me for below queries . would like to telnet to all switches and collect information in daily basis with "switchstatus" command and store the output under /tmp/ folder on systemA . To keep passwords of 12switches in a... (1 Reply)
Discussion started by: chinni-script
1 Replies

8. Shell Programming and Scripting

collect data from another file

Hi Friends , I am urgently needed your help. could you pleas help me in creating the shell script. Requirement : collect the data from no_file.txt no_file.txt ============ 1 11 265 290 300 313 326 351 Then create another shele script like my_shell.csh (3 Replies)
Discussion started by: bikas_jena
3 Replies

9. Shell Programming and Scripting

ccall database and collect data from one table

I want to connect to one database and collect data from any table using shell script. (0 Replies)
Discussion started by: rinku
0 Replies

10. Solaris

Planning for DR, I have to collect information

Dear All, We are going for Disaster Recovery project, the vendor asked for more details about how much is the daily data changes only. using sar / iostat can any one help me to collect this ?! Note: only I need the changed data size not the daily increasing data. this is to know how much... (1 Reply)
Discussion started by: adel8483
1 Replies
Login or Register to Ask a Question