05-23-2006
Select matches between line number and end of file?
Hi Guys/Gals,
I have a log file that is updated once every few seconds and I am looking for a way to speed up one of my scripts.
Basically what I am trying to do is grep through a text file from start to finish once. Then each subsequent grep starts at the last line of the previous grep to the end of the file.
The log file can get as large as 200MB and rather then going through the file start to finish every time, I want to be able to have the second grep start off where the last grep finished, cutting down on processing time.
I tried to use tail to grab the last x amount of lines of the file which is much faster, but the issue is the file keeps being updated constantly so the last 100 lines is only valid for about 3 seconds, until a new line gets appended to the log file.
Is there a command to tail line 100101 to end_of_file?
I figured I could determine the last line number by using something like this
cat -n logfile.txt | tail -n1 | gawk "{print $1}" > lastline.txt
If anyone has a suggestion on how to accomplish this, please share your thoughts.
Thanks
Jerrad
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
The awk command awk -F: '{print $1}' test1 gives the first columns of all the lines in file ,is there some command to get a particular column from particular line .
Any help is appreciated.
thanks arif (4 Replies)
Discussion started by: mab_arif16
4 Replies
2. Shell Programming and Scripting
Hi All
I have a script which has produced a list, I have used 'sed' to number my list, but i want to list at end of line with the first line starting at zero (0) and brackets round it
ie
My List i want
Hello (0)
this (1)
day (2)
can (3)
be (4)
sed '/./=' filename | sed '/./N; s/\n/) /'... (5 Replies)
Discussion started by: chassis
5 Replies
3. Shell Programming and Scripting
Hi:
I have few rows in file..Like suppose...
9063C0 44 00051363603253033253347 3333 070248 06
9063C0 5G PAN00013
9063C0 44 00061030305040404250724 0506 100248 08
9063C0 43 01 00000089
I need to cut the row starting after... (5 Replies)
Discussion started by: grajesh_955
5 Replies
4. Shell Programming and Scripting
Hello, Would someone guide me on how to write a shell script the would search for a phone no using at the end text file using sed or awk and store it in a varaible or print it.
The text file is in this form
text or numbers in first line
text or numbers in second line
.
.
.
Firsname... (6 Replies)
Discussion started by: amuthiga
6 Replies
5. Shell Programming and Scripting
Hi Folks,
Using the Vi, how can I add a numbers at the end of the line.
For eg: I have the numbers in the file as:
58.125.33
22.58.68
25.144.225
114.25.38
I need to add .0/8 at the end of all the line. So, it should be like
58.125.33.0/8
22.58.68.0/8
25.144.225.0/8
114.25.38.0/8 (6 Replies)
Discussion started by: gsiva
6 Replies
6. Shell Programming and Scripting
If I have a file something like as shown below,
ARM*187878*hjhj
BAG*88778*jjjj
COD*7777*kkkk
BAG*87878*kjjhjk
DEF*65656*89989*khjkk
I need the line numbers to be added with a colon when it matches the string "BAG". Here in my case, I need something like
ARM*187878*hjhj... (4 Replies)
Discussion started by: Muthuraj K
4 Replies
7. Shell Programming and Scripting
Does anyone know how to use awk to act like grep from a particular line number to the end of file? I am using Solaris 10 and I don't have any GNU products installed.
Say I want to print all occurrences of red starting at line 3 to the end of file.
EXAMPLE FILE:
red
green
red
red... (1 Reply)
Discussion started by: thibodc
1 Replies
8. Shell Programming and Scripting
Given a file like this:
abc
def
ghi
I need to get to
somestandardtext abc1 morestandardtext
somestandardtext def2 morestandardtext
somestandardtext ghi3 morestandardtext
Notice that in addition to the standard text there is the line number added in as well. What I conceived is... (4 Replies)
Discussion started by: edstevens
4 Replies
9. Shell Programming and Scripting
I have a directory of files, each with a variable (though small) number of lines. I would like to go through each line in each file, and print the:
-file name
-line number
-number of matches to the pattern /comp/ for each line.
Two example files:
cat... (4 Replies)
Discussion started by: pathunkathunk
4 Replies
10. Shell Programming and Scripting
datafile:
2017-03-24 10:26:22.098566|5|'No Route for Sndr:RETEK RMS 00040 /ZZ Appl:PF Func:PD Txn:832 Group Cntr:None ISA CntlNr:None Ver:003050 '|'2'|'PFI'|'-'|'EAI_ED_DeleteAll'|'EAI_ED'|NULL|NULL|NULL|139050594|ActivityLog|
2017-03-27 02:50:02.028706|5|'No Route for... (7 Replies)
Discussion started by: SkySmart
7 Replies
LEARN ABOUT SUSE
startpar
STARTPAR(8) System Manager's Manual STARTPAR(8)
NAME
startpar - start runlevel scripts in parallel
SYNOPSIS
startpar [-p par] [-i iorate] [-t timeout] [-T global_timeout] [-a arg] prg1 prg2 ...
startpar [-p par] [-i iorate] [-t timeout] [-T global_timeout] -M [ boot|start|stop]
DESCRIPTION
startpar is used to run multiple run-level scripts in parallel. The degree of parallelism on one CPU can be set with the -p option, the
default is full parallelism. An argument to all of the scripts can be provided with the -a option. Processes block by pending I/O will
weighting by the factor 800. To change this factor the option -i can be used to specify an other value.
The output of each script is buffered and written when the script exits, so output lines of different scripts won't mix. You can modify
this behaviour by setting a timeout.
The timeout set with the -t option is used as buffer timeout. If the output buffer of a script is not empty and the last output was timeout
seconds ago, startpar will flush the buffer.
The -T option timeout works more globally. If no output is printed for more than global_timeout seconds, startpar will flush the buffer of
the script with the oldest output. Afterwards it will only print output of this script until it is finished.
The -M option switches startpar into a make(1) like behaviour. This option takes three different arguments: boot, start, and stop for
reading .depend.boot or .depend.start or .depend.stop respectively in the directory /etc/init.d/. By scanning the boot and runlevel direc-
tories in /etc/init.d/ it then executes the appropriate scripts in parallel.
FILES
/etc/init.d/.depend.boot
/etc/init.d/.depend.start
/etc/init.d/.depend.stop
SEE ALSO
init.d(7), insserv(8), startproc(8).
COPYRIGHT
2003,2004 SuSE Linux AG, Nuernberg, Germany.
2007 SuSE LINUX Products GmbH, Nuernberg, Germany.
AUTHOR
Michael Schroeder <mls@suse.de>
Takashi Iwai <tiwai@suse.de>
Werner Fink <werner@suse.de>
Jun 2003 STARTPAR(8)