06-22-2007
Sed/awk gods, I need your Help! Fancy log extraction
Hi! I'm trying to find a way to extract a certain amount of lines from a log file. This would allow me to "follow" a web user through our log files.
Here is a sample fake log file to explain what i want to accomplish :
[2007-06-22 09:33:15,843][thread-1][BEG_]BEGIN REQUEST sessionID=123456
[2007-06-22 09:33:15,844][thread-1][DEB_]boatload of lines for thread-1 detailing the whereabouts of the customer
[2007-06-22 09:33:15,844][thread-2][DEB_]Here is activity from another customer - we don't need that
[2007-06-22 09:33:15,844][thread-1][DEB_]boatload of lines for thread-1 detailing the whereabouts of the customer
[2007-06-22 09:33:15,844][thread-3][DEB_]more activity from yet another customer- we don't need that
[2007-06-22 09:33:15,844][thread-1][DEB_]boatload of lines for thread-1 detailing the whereabouts of the customer
[2007-06-22 09:33:15,843][thread-1][BEG_]END REQUEST
[2007-06-22 09:33:15,843][thread-34][BEG_]BEGIN REQUEST sessionID=123456
[2007-06-22 09:33:15,844][thread-1][DEB_]Another customer took thread-1! We don't want that log entry either
[2007-06-22 09:33:15,844][thread-34][DEB_]yet more activity from the customer but under a different thread!
[2007-06-22 09:33:15,843][thread-34][BEG_]END REQUEST
What i need is a request that, using sessionID=123456, will identify the appropriate thread ID and extract the lines containing the thread ID between the BEGIN REQUEST and END REQUEST tags.
So basically, the result would be :
[2007-06-22 09:33:15,843][thread-1][BEG_]BEGIN REQUEST sessionID=123456
[2007-06-22 09:33:15,844][thread-1][DEB_]boatload of lines for thread-1 detailing the whereabouts of the customer
[2007-06-22 09:33:15,844][thread-1][DEB_]boatload of lines for thread-1 detailing the whereabouts of the customer
[2007-06-22 09:33:15,844][thread-1][DEB_]boatload of lines for thread-1 detailing the whereabouts of the customer
[2007-06-22 09:33:15,843][thread-1][BEG_]END REQUEST
[2007-06-22 09:33:15,843][thread-34][BEG_]BEGIN REQUEST sessionID=123456
[2007-06-22 09:33:15,844][thread-34][DEB_]yet more activity from the customer but under a different thread!
[2007-06-22 09:33:15,843][thread-34][BEG_]END REQUEST
what the expression would need to do :
1 - locate sessionID=123456
2 - grab threadID from the same line
3 - dump all threadID lines up to threadID.*END REQUEST
4 - rinse and repeat
Unfortunately, i'm only a neophyte in using sed or awk so i have no idea how to proceed...
Not even sure this can be done. If not i'll use perl, but having a nice expression that could do that (and understanding it) would be a big help for me.
If someone can lend me a hand or at least give me pointers, that'd be very appreciated. Hope my question is clear enough!
Thanks
Last edited by gnagus; 06-22-2007 at 04:09 PM..
Reason: Edit : removed references to UniqueID, replaced by sessionID
7 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
Forgive me if this is a dumb question...I'm a Windows sys admin with little programming knowledge.
I have files containing anywhere from 3 to 200 lines. Using SED, I want to extract only lines containing a specific instance of "ISS." It is possible that "ISS" will occur several times in a... (10 Replies)
Discussion started by: thuston22
10 Replies
2. Shell Programming and Scripting
Hi there,
My samba configuration file looks like that :
...
...
path = /home/samba/profiles/
...
path = /home/samba/shares/family
valid users = family
path = /home/samba/shares/admins
valid users = admins
path = /home/samba/shares/publicI want to extract the list of standard... (3 Replies)
Discussion started by: chebarbudo
3 Replies
3. Shell Programming and Scripting
Hi All,
I need to extract 543 from the command below :
# pvscan
PV /dev/sdb1 VG vg0 lvm2
Total: 1 543.88 GB] / in use: 1 / in no VG: 0
I have the following command which does the job, but I think this could be achieved in a more simple way using sed or awk. Any help is... (7 Replies)
Discussion started by: nua7
7 Replies
4. Shell Programming and Scripting
Hi,
Could anyone clearly explain me the below sed construct in detail to get to know what it actually does?
sed 's/\(* *\)//4'
echo 'test;10;20' | sed 's/*;\(*\)/\1/' (1 Reply)
Discussion started by: royalibrahim
1 Replies
5. UNIX for Dummies Questions & Answers
Hi,
I have a file like this-
aa
12
23
34
aa
21
34
56
aa
78
45
56
I want to print out only the lines after the last aa. How do I do this? I tried using grep -A and sed -n, but both didnt work as I wanted to.
Could someone help me out please.. (3 Replies)
Discussion started by: jamie_123
3 Replies
6. Shell Programming and Scripting
Hi everyone!
I'm writting a function in .bashrc to extract some text from a file. The file looks like this:
" random text
Begin CG step 1
random text
Begin CG step 2
...
Begin CG step 100
random text"
For a given number, let's say 70, I want all the text between "Begin CG... (4 Replies)
Discussion started by: radudownload
4 Replies
7. Shell Programming and Scripting
Hi All
I am trying to remove the line having specific pattern from a file by using sed command
I have the file named ODS_REP_SRCE_File.txt with content as:
... (1 Reply)
Discussion started by: Shilpi Gupta
1 Replies
LEARN ABOUT HPUX
pthread_join
pthread_join(3T) pthread_join(3T)
NAME
pthread_join() - wait for the termination of a specified thread
SYNOPSIS
PARAMETERS
thread Thread whose termination is awaited by the caller.
value_ptr Pointer to the location where the exit status of thread is returned.
DESCRIPTION
The function waits for the termination of the target thread. If the target thread has already terminated, this function returns immedi-
ately. Only threads created with a detachstate attribute value of may be specified in the target thread parameter. On successful return
from the value_ptr argument, if it is not a null pointer, will contain the value passed to by the terminating thread.
When a call returns successfully, the caller is guaranteed the target thread has terminated. If more than one thread calls for the same
target thread, one thread is guaranteed to return successfully. Undefined behavior results for other callers specifying the same thread.
If the thread calling is canceled, the target thread shall not be joined. The exit status of the target thread will remain available for
another thread to call
If the target thread was canceled, its exit status is
It is unspecified whether a thread that has exited, but remains unjoined, counts against the limit.
RETURN VALUE
Upon successful completion, returns zero. Otherwise, an error number is returned to indicate the error (the variable is not set).
ERRORS
If any of the following occur, the function returns the corresponding error number:
[EINVAL] The value specified by thread does not refer to a joinable thread.
[ESRCH] No thread could be found corresponding to thread.
For each of the following conditions, if the condition is detected, the function returns the corresponding error number:
[EDEADLK] This operation would result in process deadlock or thread specifies the calling thread.
AUTHOR
was derived from the IEEE POSIX P1003.1c standard.
SEE ALSO
pthread_create(3T), wait(2).
STANDARDS CONFORMANCE
Pthread Library pthread_join(3T)