Logfile parsing with variable, multiple criterias among multiple lines
Hi all
I've been working on a bash script parsing through debug/trace files and extracting all lines that relate to some search string. So far, it works pretty well. However, I am challenged by one requirement that is still open.
What I want to do:
1) parse through a file and identify all packet numbers (PXXX:) that match my search, hereafter called "interesting packets"
2) parse again through the same file, searching this time now for packets that relate to the packets identified in step 1)
See note around P3712451 in the example below!
3) what I would also like to get are related log messages that may appear just underneath a interesting packet. Any other log message should be ignored.
4) output all log file lines that somehow relate to the searched string into another file.
Example trace file (simplified):
Step 1), 2) and 4) are already working using egrep.
Step 1)
The above fills $PACKETS with interesting packets matching the $QUERYSTRING (e.g. testuser) in the form "(P3712446|P3712451|P3712460)"
Step 2)
Step 4)
Finally, I write the interesting packets into a new file using the following
I've got 2 questions now:
Q1) How can I catch Log lines like...
...if it follows an interesting packet and ignore any other Log line?
I've been looking at multiple line matching examples... but I am not able to apply what I've seen in combination with the sometimes huge list of interesting packets I've got.
Q2) Any obvious and easy way to simplify what I've done already?
I started with parsing each line... but that was far too time consuming (1h+). The above still takes 2-3 minutes for a 130MB file, which is ok. But maybe someone has even something faster on his mind.
I've been told that sed is a "Turing complete" language, whatever the bleep that means... so programmatically speaking, you could do the whole thing in sed.
Many thanks for this.
Problem is that all log messages would follow an interesting packet with the initial egrep in your example. Hence too many irrelevant log messages would be displayed.
Or do I miss something?
Any other idea how the 4 steps and step 3 in particular could be realized?
Seems I need to refine my problem. I've probably provided too much background information.
Assuming a file as follows
Assuming I've got a list of interesting content, in my example above "(content1|content3)", how can I extract all lines with interesting content as well as their related log lines just underneath?
Hello Gurus,
I have a multiple pipe separated files which have records going over multiple Lines. End of line separator is \n and records going over multiple lines have <CR> as separator. below is example from one file.
1|ABC DEF|100|10
2|PQ
RS
T|200|20
3| UVWXYZ|300|30
4| GHIJKL|400|40... (7 Replies)
from the CLI on a Mac, if you type networksetup -listallnetworkservices then you get results in a multi-line paragraph that look something like this:
networksetup -listallnetworkservices
An asterisk (*) denotes that a network service is disabled.
Wi-Fi
Display Ethernet
Bluetooth DUN... (7 Replies)
GM,
I have an issue at work, which requires a simple solution. But, after multiple attempts, I have not been able to hit on the code needed.
I am assuming that sed, awk or even perl could do what I need.
I have an application that adds extra blank page feeds, for multiple reports, when... (7 Replies)
Hello,
I would like to ask for help with csh script.
An example of an input in .txt file is below, the number of lines varies from file to file and I have 2 or 3 columns with values. I would like to read all the values (probably one by one) and set them to independent unique variables that... (7 Replies)
Good Morning/Afternoon All,
I am having some trouble creating a variable called "total" to display the sum of the values in a specific field, $6 for example.
The data I am working on is in the following form:
John Doe:(555) 555-5555:1:2:3
Jane Doe:(544) 444-5556:4:5:6
Moe Doe:(654)... (2 Replies)
I have a file that contains the following lines
the brown quick fox
jumped over
the white laze dog
0123456789
I wanted to put the contents of this file into a variable so I used this code:
VAR_LIST=`cat $2`
where $2 is the file name passed as an argument to the script
If I... (3 Replies)
I have a file with a set of insert statements some of which have a single column value that crosses multiple lines causing the statement to fail in sql*plue. Can someone help me with a sed script to replace the new lines with chr(10)?
here is an example:
insert into mytable(id, field1, field2)... (3 Replies)
Good morning,
Novice scripter in Unix here, and I've run into and sed task I can't quite wrap my head around. I'm pulling my hair out fast enough as it is and thought I would go to the knowledge bank.
I have a sorted file that I'm trying to trim down by deleting any line whose first few... (2 Replies)