Editing the end of the file without loading the entire file


 
Thread Tools Search this Thread
Top Forums UNIX for Advanced & Expert Users Editing the end of the file without loading the entire file
# 1  
Old 06-21-2004
Editing the end of the file without loading the entire file

hi! I am a newbee. I would really appreciate if you can answer the following question:

I have a huge data file, 214MB with several coloumns. I need to delete the very last line of the file. Everything I know takes a lot of time to do it ( because I have to open the file in an editor or run a script in awk or something).

Does anybody know of a really quick way to accomplish this?

I am using Solaris 9.

Thanks.
# 2  
Old 06-21-2004
If the file has carriage control - ie., it's a text file this may help:
Code:
#!/bin/ksh -v
# parameters old file = $1
#            new file = $2
# I know this first line is slow... but you need the total
lines=`cat $1 | wc -l`
let lines=lines-1
head -n $lines  $1 > $2

# 3  
Old 06-21-2004
The whole point is to avoid reading the entire file even once. So even something like
sed '$d' <old > new
is too much.
# 4  
Old 06-22-2004
Thanks

I just ran a small awk script to accomplish what I needed to do, but I wanted to find out if there was a much better way to do it for next time.

Driver, thanks for the program. I was looking for someway to access the file from the end ( essentially what your last for loop does), but just from the shell ( instead of writing a program).

I like Jim's solution, but yes wc -l takes some time. I dont know sed yet, but I am learning it.

Thanks everybody.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Replacing entire fields with specific text at end or beginning of field

Greetings. I've got a csv file with data along these lines: Spumoni's Pizza Place, Placemats n Things, Just Lamps Counterfeit Dollars by Vinnie, Just Shades, Dollar StoreI want to replace the entire comma-delimited field if it matches something ending in "Place" or beginning with "Dollar",... (2 Replies)
Discussion started by: palmfrond
2 Replies

2. Shell Programming and Scripting

Grabbing entire content of file except spaces at the end

so i have several files that look somewhat like this: { "afafa": "afaf", "afafaa" : "" } <newline> <newline> i want to grab everything in the file except the empty new lines at the end. note, there may be newlines within the content itself. ( "afafa": "afaf",... (3 Replies)
Discussion started by: SkySmart
3 Replies

3. Shell Programming and Scripting

Copy entire file to a new file in awk

Hi, How do we write the contents of multiple files created in awk body to a new file in awk (END block) ? when I used "getline", it is consuming more time. Do we have any other way other than getline? (1 Reply)
Discussion started by: Cool
1 Replies

4. Shell Programming and Scripting

editing line in text file adding number to value in file

I have a text file that has data like: Data "12345#22" Fred ID 12345 Age 45 Wilma Dino Data "123#22" Tarzan ID 123 Age 33 Jane I need to figure out a way of adding 1,000,000 to the specific lines (always same format) in the file, so it becomes: Data "1012345#22" Fred ID... (16 Replies)
Discussion started by: say170
16 Replies

5. Shell Programming and Scripting

Subsituting contents of entire file to middle of another file using awk

Hi I have a data file 'File2' consisting of 105670 lines. I want to copy and paste 17928 lines from 'File1' to 'File2' but I want to place it in between lines 21 and 17950 of 'File2'. How do I do it in awk? For example- File A has 5 lines X Y 1 2 3 4 5 6 7 8 9 and File B has A b... (1 Reply)
Discussion started by: ananyob
1 Replies

6. Shell Programming and Scripting

Subsituting contents of entire file to middle of another file using awk

Hi I have a data file 'File2' consisting of 105670 lines. I want to copy and paste 17928 lines from 'File1' to 'File2' but I want to place it in between lines 21 and 17950 of 'File2'. How do I do it in awk? For example- File A has 5 lines X Y 1 2 3 4 5 6 7 8 9 and File B has A b... (1 Reply)
Discussion started by: ananyob
1 Replies

7. Shell Programming and Scripting

Copy an entire file to specific position to another file

Hi , I need your kind help for my below requirement I need to copy and entire txt file to a certain position to the target file . Source file has 3 lines and it has to be copied to the target file in position from line 10 to 12. Thanks for your support (1 Reply)
Discussion started by: Pratik4891
1 Replies

8. Shell Programming and Scripting

Help with file editing while keeping file format intact

Hi, I am having a file which is fix length and comma seperated. And I want to replace values for one column. I am reading file line by line in variable $LINE and then replacing the string. Problem is after changing value and writing new file temp5.txt, formating of original file is getting... (8 Replies)
Discussion started by: Mruda
8 Replies

9. UNIX for Dummies Questions & Answers

How to copy entire file content into another file being in last line mode of vi ?

How to copy entire file content into another file being in last line mode of vi ? ---------- Post updated at 10:07 AM ---------- Previous update was at 09:56 AM ---------- Got it : :1,30w file.txt (1 Reply)
Discussion started by: presul
1 Replies

10. Shell Programming and Scripting

SED - editing file names (End of line problem?)

For lists in sed, to say what to replace, is this correct: I am hoping that this would recognise that either a "." is present, or that the substitution happens at the end of the line. For files with extensions , my script works perfectly. My problem is, files without extentions, i.e. . ... (1 Reply)
Discussion started by: busillis
1 Replies
Login or Register to Ask a Question