linux operating commands and unix operating commands

Event Correlation


 
Thread Tools Search this Thread
# 1  
Old 09-02-2008
Event Correlation

<!-- @page { size: 8.5in 11in; margin: 0.79in } P { margin-bottom: 0.08in } --> I recently had an opportunity to work on some existing event correlation infrastructure in a large environment. I found there were a lot of considerations that came into play to get the event correlation aspects functioning optimally. I found understanding the quality of the data that came into the device from its sources, intelligent design of incident creation, and assuring a proper grasp of the configuration options and limitations within the event correlation software were all particularly important.
As I have said, the event correlation infrastructure already existed. Unfortunately, it appeared that whoever had worked on it had not had much time available to learn the way to properly configure the software. Rather than let this dishearten me, I decided it was a great opportunity to learn more about event correlation. Most of the existing rules designed to create "Incidents" which needed additional action were not triggering in an expected fashion. They were either generating a large number of false positives, or there were a lot of false negatives that on the surface were invisible. As this software was being used primarily for antivirus software, this was obviously not the best outcome. With some time spent researching what the rules were supposed to do, reading the manual, and contacting the technical support team for the product, I was able to get the existing rules working properly.
That warm up armed me with the knowledge needed for planning additional incident triggers to benefit the Incident Response Team at this company. Based on my background in antivirus software and internal customer feedback, I realized that it was most crucial to have incidents trigger for particularly concerning events such as root kits being detected or viruses not able to be deleted by the antivirus software. I was able to generate these rules with a little trial and error.
Unfortunately, some of the desired granularity was lacking. For example, Temporary Internet Files often has detections of adware and spyware. These files are "locked" by Internet Explorer and are often unable to be deleted by antivirus software at the time of the detection. If you manually view the file that is left, however, you can see that it's a 0 byte file. The antivirus software did not differentiate between "can't delete file" and "can't delete 0 byte file" so there was no easy way to trigger an incident only when the file was an actual threat. This was obviously not due so much to the event correlation software as the antivirus software in question; but it required a reasonably in depth knowledge of the antivirus software. Due to this I was unable to create a rule that only triggered when there was real concern. Any detection in Temporary Internet Files would require a manual perusal to determine whether there was an issue.

After completing this project, the Incident Response Team has stated they feel much more capable of identifying issues which require their attention in a much shorter time frame. I feel that taking the time out of my already busy schedule to get this software functioning was well worth it from a cost/benefit perspective. Event correlation software is something I would recommend to anyone who would like a better tactical and strategic view of their environment. Obviously any vendor's event correlation software can be used for much, much more than just antivirus events.



Image
Image

More...
Login or Register to Ask a Question

Previous Thread | Next Thread

7 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Correlation Between 3 Different Loops using Bash

I have 3 loops that I use to determine the permission level of AWS user accounts. This array lists the AWS policy ARN (Amazon Resource Name): for ((policy_index=0;policy_index<${#aws_managed_policies};++policy_index)); do aws_policy_arn="${aws_managed_policies}" ... (1 Reply)
Discussion started by: bluethundr
1 Replies

2. Shell Programming and Scripting

3 column .csv --> correlation matrix; awk, perl?

Greetings, salutations. I have a 3 column csv file with ~13 million rows and I would like to generate a correlation matrix. Interestingly, you all previously provided a solution to the inverse of this problem. Thread title: "awk? adjacency matrix to adjacency list / correlation matrix to list"... (6 Replies)
Discussion started by: R3353
6 Replies

3. Shell Programming and Scripting

Pearson correlation between two files

Hi, I want a quick way to determine the pearson correlation between two files. The two files have the same format with only the 3rd column varying. E.g. of file 1 chr1 0 62 chr1 1 260 chr1 2 474 chr1 3 562 chr1 4 633 chr1 5 870 chr1 6 931 chr1 7 978 chr1 8 1058 chr1 9 1151 E.g.... (1 Reply)
Discussion started by: kylle345
1 Replies

4. Shell Programming and Scripting

AWK - calculating simple correlation of rows

Is there any way to calculate a simple correlation of few selected rows with all the rows in input ? In the below example I selected Row01,02,03 and correlated with all the rows. I was trying to run in R. But the this big data matrix is too much to handle for R and eventually my system is... (3 Replies)
Discussion started by: quincyjones
3 Replies

5. Shell Programming and Scripting

Calculate Correlation between two fields !

Hello, I request your help with a shell script (awk) that ask for two inputs in order to calculate the correlation of the last rows between two fields ( 3 and 4). Data: EC-GLD,1/25/2011,41.270000,129.070000 EC-GLD,1/26/2011,41.550000,129.280000 EC-GLD,1/27/2011,42.260000,127.800000... (1 Reply)
Discussion started by: csierra
1 Replies

6. Shell Programming and Scripting

correlation coefficient - Awk

Hi guys I have an input file with multiple columns and and rows. Is it possible to calculate correlation of certain value of certain No (For example x of S1 = 112) with all other values (for example start with x 112 corr a 3 of S1 = x-a 0.2 ) INPUT ******* No S1 S2 S3 S4 Sn a 3 ... (2 Replies)
Discussion started by: quincyjones
2 Replies

7. UNIX for Dummies Questions & Answers

chmod and cgi correlation

How much do chmod settings affect cgi scripts?? I have a "webmaster" at my work that says I cannot change the permissions on the cgi scripts, and that they work with only certain permissions. They are set for 644, I want to change them to 775 and put her in her own group, like she should be, not... (6 Replies)
Discussion started by: bigmacc
6 Replies
Login or Register to Ask a Question