Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

How to process from middle of a file?

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting How to process from middle of a file?
# 8  
Old 01-02-2008
Ok - lets take a file with 2 million lines with "Summary" wedged in between:

Code:
$ wc -l bigfile.txt
 2000001 bigfile.txt
$ grep -n Summary bigfile.txt
1000001:Summary
$ time nawk '/^Summary/,/\*/' bigfile.txt > /dev/null

real    0m1.51s
user    0m1.40s
sys     0m0.12s
$ time sed -ne '/Summary/,$p' bigfile.txt > /dev/null

real    0m0.76s
user    0m0.65s
sys     0m0.10s

sed shows almost double the speed Smilie

HTH
# 9  
Old 01-02-2008
sweet!!!Smilie
# 10  
Old 01-03-2008
For newer version of awk, such as GNUawk,
Code:
# wc -l file2
3703262 file2
# time awk 'BEGIN{RS="^.*Summary"}1' file2 >  awktest

real    0m0.696s
user    0m0.208s
sys     0m0.320s
# time sed -ne '/Summary/,$p' file2 > sedtest

real    0m6.960s
user    0m6.600s
sys     0m0.336s

# 11  
Old 01-03-2008
Interesting - I'll have to run this on my Gutsy laptop soon Smilie
# 12  
Old 01-03-2008
Computer
Indeed, GNU awk is faster:

Code:
$ uname -a
Linux loki 2.6.22-14-generic #1 SMP Tue Dec 18 08:02:57 UTC 2007 i686 GNU/Linux
$ time awk 'BEGIN{RS="^.*Summary"}1' bigfile.txt > /dev/null

real    0m2.235s
user    0m2.204s
sys     0m0.020s
$ time sed -ne '/Summary/,$p' bigfile.txt > /dev/null

real    0m3.095s
user    0m3.080s
sys     0m0.008s

# 13  
Old 01-04-2008
Thanks Guys.

it is working fine for me.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Subsituting contents of entire file to middle of another file using awk

Hi I have a data file 'File2' consisting of 105670 lines. I want to copy and paste 17928 lines from 'File1' to 'File2' but I want to place it in between lines 21 and 17950 of 'File2'. How do I do it in awk? For example- File A has 5 lines X Y 1 2 3 4 5 6 7 8 9 and File B has A b... (1 Reply)
Discussion started by: ananyob
1 Replies

2. Shell Programming and Scripting

Subsituting contents of entire file to middle of another file using awk

Hi I have a data file 'File2' consisting of 105670 lines. I want to copy and paste 17928 lines from 'File1' to 'File2' but I want to place it in between lines 21 and 17950 of 'File2'. How do I do it in awk? For example- File A has 5 lines X Y 1 2 3 4 5 6 7 8 9 and File B has A b... (1 Reply)
Discussion started by: ananyob
1 Replies

3. Shell Programming and Scripting

How to read from middle of a file for a particular text

Hi, Below is my issue which I desperately need and I want a shell script which can do this job. 1. There are 10 log files in a particular location. 2. open each log file. Goto to the end of the file. From the end go up to find a particular text. From this particular text till the end of... (3 Replies)
Discussion started by: kashriram
3 Replies

4. UNIX for Dummies Questions & Answers

Add string to middle of a file

Hi, I want to write a script that takes a file and a string as params and adds the string to the middle line of the file. Also, I want to output the results back to the original file passed without using temp files. I am very much new to UNIX so this is all a little like black magic to me at... (15 Replies)
Discussion started by: Chiefos
15 Replies

5. UNIX for Dummies Questions & Answers

How to insert text in the middle of a file

Hey guys, how do we take a line of text as an argument from a user and then insert it in the middle of a file irrespective of the number of lines in the file. I am trying to do this without SED or AWK. Inserting it in the beginning and at the end is easy, but i am trying to accomplish inserting... (6 Replies)
Discussion started by: kartikkumar84@g
6 Replies

6. Shell Programming and Scripting

add text in the middle of file

Can anyone help me pls? I want to add a text into the middle of file. I've writtenthe following script text to add="$1" file="$2" lines=$(wc -l $2) half_lines=$(expr $lines / 2) head -$half_lines $2 > temp echo "text to add" >> temp ((half_lines=$half_lines + 1)) tail -$half_lines $2... (6 Replies)
Discussion started by: relle
6 Replies

7. Shell Programming and Scripting

removing whitespace from middle of file -help

I have a file in which I clean out a bunch of nonsense text as well as path information. What I end up with is something like the following: johnson.........................................................933 Where the periods represent the whitespace The file comes out originally with... (2 Replies)
Discussion started by: roninuta
2 Replies

8. Shell Programming and Scripting

add a string in the middle of the file

i want to add a string in a very top of a file without using VI or SED or AWK this is what ive done: (echo '0a'; echo 'LINE OF TEXT'; echo '.'; echo 'wq') | ed -s myfile to add astrng right in the middle i could have count the lines of the file and just chenge the address. ... (6 Replies)
Discussion started by: ciroredz
6 Replies

9. Shell Programming and Scripting

How to insert text into first line of the file and middle of the file?

Script 1 Pre-requisites Create a file with x amount of lines in it, the content of your choice. Write a script that takes two arguments. The first being a line of text, the second being your newly created file. The script should take the first argument and insert it into the very top (the... (3 Replies)
Discussion started by: ali hussain
3 Replies

10. Shell Programming and Scripting

appending to sed output of one file into the middle of file

hi, i have a file file1 file2 ----------- ----------------- aa bbb ccc 111 1111 1111 ddd eee fff 222 3333 4444 ggg hhh... (5 Replies)
Discussion started by: go4desperado
5 Replies
Login or Register to Ask a Question