fast way to retreive a list of lines


 
Thread Tools Search this Thread
Top Forums UNIX for Advanced & Expert Users fast way to retreive a list of lines
# 1  
Old 12-03-2008
fast way to retreive a list of lines

hi there,

i have to read lines [ time-stamps ] from the file, where the line is just above the pattern am looking for
typically this looks like this

<time-stamp>|-----
<pattern am searching >......
<time-stamp>|.....
<some garbage >
....


the log file is big [2M lines ]
wc -l ~/log/ompe.log.20081203
2808766

after getting the list of line numbers here the pattern is
am trying to read time-stamps from the perticualr line using

for line_no in `cat /tmp/jms1`
do
prv_line_no=$(expr $line_no - 1 )
sed -n "${prv_line_no}p" $1 >> /tmp/jms_times
# head -${prv_line_no} $1 | tail -1 >> /tmp/jms_times1
done

sed option is very slow , on the other hand using head/tail option is fast inatially but as the line number is increases the retreval slows down

is there any fast way to retreive a perticular line number
# 2  
Old 12-03-2008
try going by the line number with
grep -n "pattern" file | cut -d ":" -f1

your line no will be echo `grep -n "pattern" file | cut -d ":" -f1` - 1 |bc

then you can get the line printed with sed without having extracted the two lines and the head|tailing them ..
# 3  
Old 12-03-2008
Hi,

a sed one liner is enough, you don't need the for-loop and all
the other processes. This should speed up your search.

Code:
sed -n '/pattern/{g;1!p;};h' file

It will search for pattern but print the line immediately before
the line containg pattern, but not the line with pattern itself.

HTH Chris
# 4  
Old 12-04-2008
thanks you very much every one , for the help
# 5  
Old 12-04-2008
hi chris,
i tried your command , its not giving the output

can you explain me the command " {g:1!p;};h "

$ sed -n '/JMS message rec/{g;1!p;};h' ompeJMSServerB.log.20081202004203 | head -2


$
can i execute this command in any shell ?
# 6  
Old 12-06-2008
Sorry,

i have overlooked your answer:

-n -- print only lines when asked
'/pattern/ -- if you match pattern
{g;1!p} -- then execute the following group of command
g -- overwrite the pattern space with the hold space
1!p -- print the pattern space if it is not the first lien
;h' -- this is executed on all lines and means save every line in the
hold space.

file -- the file to proceed.

What it does: a) it saves every line in the hold space, b) if your pattern
is matched, the pattern space -- the line with the pattern you searched --
is replaced by the line previously saved in the hold space -- that is the
line preceeding your pattern. Thus the pattern you searched is replaced by
the line preceeding it.

This sed-command should be the usable on all unixes.

If you have problems, please give concrete examples of what you are
searching for and the data in your file.
# 7  
Old 12-08-2008
hi chris,

am trying to use your command but getting the following error !

tcenh253!omsrec:~/infraBin/logs/tmp [8]> sed -n '/JMS message rec/{g;1!p;};h' ompeJMSServerB.log.20081207004201 | wc
p: Event not found.

when i try to remove shell-interpretation of "!" like below all am getting is blank/empty lines

tcenh253!omsrec:~/infraBin/logs/tmp [11]> sed -n '/JMS message rec/{g;1\!p;};h' ompeJMSServerB.log.20081207004201


tcenh253!omsrec:~/infraBin/logs/tmp [12]>grep "JMS message rec" ompeJMSServerB.log.20081207004201 |wc
552 2760 28704


typically the logfile looks like below

-------------------------------------------
at amdocs.epi.appserver.ClientConnectionCls.doAs(ClientConnectionCls.java:175)
at amdocs.epi.jms.JmsQueueConsumerCls$ListenerThread.run(JmsQueueConsumerCls.java:711)
at java.lang.Thread.run(Thread.java:534)

|D |02:00:48 |13 |AL_0 |
------------ JMS message received -----------------
Message Object: weblogic.jms.common.TextMessageImpl
Replay Queue: OMS_Ics_iq@EAIJMSServer_mserver2
Message Text:
------------------------------------
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

How To Retreive Files With a Special Condition?

Everyday I have to get a list of files in a directory with a special condition and feed this list to a for loop to be processed. Since I do not use Unix all the time, it is tricky for me to get that list of files. So, the question is whether there are commands that will give me the file names... (12 Replies)
Discussion started by: april
12 Replies

2. Shell Programming and Scripting

Retreive data with arrangement

Hi all I have following part of a big file TTDS00002 Synonyms M1 receptor TTDS00002 Disease Alzheimer's disease TTDS00002 Disease Bronchospasm (histamine induced) TTDS00002 Disease Cognitive deficits TTDS00002 Disease Schizophrenia TTDS00002 Function The muscarinic acetylcholine... (2 Replies)
Discussion started by: kareena
2 Replies

3. Shell Programming and Scripting

Fast algorithm to compare an IP address against a list of IP sections?

I have two files: file1: 41.138.128.0 41.138.159.255 location 41.138.160.0 41.138.191.255 location 41.138.192.0 41.138.207.255 location 41.138.208.0 41.138.223.255 location 41.138.224.0 41.138.239.255 location 41.138.240.0 41.138.255.255 location 41.138.32.0 ... (7 Replies)
Discussion started by: kevintse
7 Replies

4. Shell Programming and Scripting

how to retreive certain section of the line

Hi I am using "grep" command to get certain pattern out of the file: PNUM=34 $ grep -w "#${PNUM}" myfile #34 * 2297 * 410 * 964 * * 4352 $ Is there a way to retrieve the section of the above output without #34 so the output would look like this:... (3 Replies)
Discussion started by: aoussenko
3 Replies

5. UNIX for Dummies Questions & Answers

Cannot retreive correct $PATH using PLINK

Hi, I'm using plink to execute shell script on UNIX machines. It works pretty well excepted with some machines where I don't have the same $PATH than with putty. I'm using the command PLINK.EXE -ssh machinename -l user -pw password echo $PATHAnd for putty nothing special set, I use ssh as... (2 Replies)
Discussion started by: Peuj
2 Replies

6. Shell Programming and Scripting

how to retreive a block of data from the file

Hi I am running a 'grep' command to retrieve a line from the file. The problem is that I also need 21 lines which go right after the line I just 'grep'(ed) Is there a way to retrieve this block of data? Thanks -A (4 Replies)
Discussion started by: aoussenko
4 Replies

7. Shell Programming and Scripting

Retreive content between specific lines ina file

Hi I have a text file which has two sets of lines repeating for "n" number of times.Some data is printed between the two lines.I want to retrieve all the data thats there in between those two set of lines.I have the string value of those two set of lines. To be much more clearer ... (4 Replies)
Discussion started by: chennaitomcruis
4 Replies

8. Shell Programming and Scripting

Retreive string between two chars

I want to write a shell script in order to retreive some data from a log file that i have written into. The string that i want to get is the number 2849 (that is located between | | ). To explain further, this is the result i get after running "grep LOGIN filename.log" but i need to get the... (25 Replies)
Discussion started by: danland
25 Replies

9. UNIX for Dummies Questions & Answers

Trying to retreive and compare value

Hi All, I'm have a file test.txt that looks like this 1939399393 03094994949 948383 I have to check whether the first character in the first line is 1 or not. I have tried the following option but it seems to fail head -1 test.txt | grep ^1 but it seems to display the entire... (3 Replies)
Discussion started by: omahablues
3 Replies

10. UNIX for Advanced & Expert Users

Retreive process output

Hi, I had a process that was producing a standard output (no log of it eing produced), unfortunalty the xterm it was running in died and I lost the output. I have logged back in and can see that the process didn't die. How can I bring this process to the foreground so that I can see the output?... (2 Replies)
Discussion started by: nhatch
2 Replies
Login or Register to Ask a Question