![]() |
|
|
|
|
|||||||
| Forums | Portal | Register | Forum Rules | FAQ | Contribute | Members List | Arcade | Search | Today's Posts | Mark Forums Read |
| Shell Programming and Scripting Post questions about KSH, CSH, SH, BASH, PERL, PHP, SED, AWK and OTHER shell scripts here. |
|
|
||||
| Thread | Thread Starter | Forum | Replies | Last Post |
| Extract a line from a file using the line number | zambo | Shell Programming and Scripting | 1 | 05-01-2008 10:39 AM |
| Appending the line number and a seperator to each line of a file ? | pjcwhite | Shell Programming and Scripting | 4 | 03-20-2007 10:29 PM |
| How to select lines in unix matches a pattern at a particular position | cs_banda | UNIX for Dummies Questions & Answers | 2 | 10-06-2006 12:28 PM |
| Get line form file by line number | corny | Shell Programming and Scripting | 3 | 08-25-2006 09:18 AM |
| awk to select a column from particular line number | mab_arif16 | Shell Programming and Scripting | 4 | 05-08-2006 02:26 AM |
|
|
Submit Tools | LinkBack | Thread Tools | Search this Thread | Display Modes |
|
#1
|
|||
|
|||
|
Select matches between line number and end of file?
Hi Guys/Gals,
I have a log file that is updated once every few seconds and I am looking for a way to speed up one of my scripts. Basically what I am trying to do is grep through a text file from start to finish once. Then each subsequent grep starts at the last line of the previous grep to the end of the file. The log file can get as large as 200MB and rather then going through the file start to finish every time, I want to be able to have the second grep start off where the last grep finished, cutting down on processing time. I tried to use tail to grab the last x amount of lines of the file which is much faster, but the issue is the file keeps being updated constantly so the last 100 lines is only valid for about 3 seconds, until a new line gets appended to the log file. Is there a command to tail line 100101 to end_of_file? I figured I could determine the last line number by using something like this cat -n logfile.txt | tail -n1 | gawk "{print $1}" > lastline.txt If anyone has a suggestion on how to accomplish this, please share your thoughts. Thanks Jerrad |
| Forum Sponsor | ||
|
|
|
#2
|
||||
|
||||
|
Wow. 200MB?? That's a huge text file. Personally, I would first rotate that log to a more manageable size first.
... you could use the tail command with the + operand.. ...so you'd do something like this ( not complete code, but you'll get the idea): Code:
# cat logfile | wc -l >> marker.txt ## where marker has the number of lines in
## your logilfe at the end of your script
Code:
# COUNT=$(cat marker)
let MARKER=${COUNT}+1
# tail +$(MARKER} logfile
I'd still strongly suggest you rotate that log file more often first. It's easier and faster to work with closed, smaller sized files. ... quick example of the difference between tail -n and tail +n : ..say you have a file: Code:
# cat file 111111111111 2222222222222 33333333333333 44444444444444 555555555555555 666666666666666 777777777777777 88888888888888 999999999999999 aaaaaaaaaaaaa bbbbbbbbbbbbb ccccccccccccc #tail -5 file 88888888888888 999999999999999 aaaaaaaaaaaaa bbbbbbbbbbbbb ccccccccccccc # tail +5 file 555555555555555 666666666666666 777777777777777 88888888888888 999999999999999 aaaaaaaaaaaaa bbbbbbbbbbbbb ccccccccccccc Last edited by System Shock; 05-23-2006 at 04:25 PM. |
|
#3
|
||||
|
||||
|
Code:
sed -n '100101,$p' |
|
#4
|
||||
|
||||
|
what if
i want to print from xth line to second last line?
that is $p in sed will be short by 1. so how is the syntax? |
|
#5
|
|||
|
|||
|
why cannot you use some thing like this.
tail -f name_of_your_file | grep 'text you want to search' |
|||
| Google The UNIX and Linux Forums |
| Thread Tools | Search this Thread |
| Display Modes | |
|
|