deleting lines from multiple text files


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting deleting lines from multiple text files
# 1  
Old 04-25-2008
deleting lines from multiple text files

I have a directory full of text data files.

Unfortunately I need to get rid of the 7th and 8th line from them all so that I can input them into a GIS application.

I've used an awk script to do one at a time but due to the sheer number of files I need some kind of loop mechanism to automate it.

The awk script used: -
awk 'BEGIN{getline f;getline t}FNR==f,FNR==t{next}1' numbers.txt inputfile > outputfile

where numbers.txt is merely a document with the numbers 7 and 8

My guess is that I need a way of piping the output from ls (of the directory) into where the inputfile is situated and a counter to loop through till the end.

Any suggestions will be welcome (an awk suggestion would be preferable to pearl)

Thanks all
# 2  
Old 04-25-2008
sed or awk are good choices - here is sed:
Code:
for file in  `ls *.txtdatafile`
do
      sed '7,8d' $file > tmp.tmp
      mv tmp.tmp $file
done

# 3  
Old 04-25-2008
Quote:
Originally Posted by jim mcnamara
sed or awk are good choices - here is sed:
Code:
for file in  `ls *.txtdatafile`
do
      sed '7,8d' $file > tmp.tmp
      mv tmp.tmp $file
done

Thanks for the Reply Jim

Unfortunately this sends them all into one big tmp.file. I need to create a new text file for every single text file that is in the directory.
# 4  
Old 04-25-2008
Gack, please don't post the same question multiple times. I already replied once.
https://www.unix.com/unix-dummies-que...#post302189269

The second line in Jim's script (and mine, too) moves the tmp file back over the original file. You can't easily redirect back onto a file because the redirection happens before awk gets to read it, so the file ends up being empty. There are various ways to avoid using temporary files but in this particular case it's probably not worth the hassle.

The ls in backticks is an antipattern; see An example of dealing with file names with spaces in them for a discussion of the drawbacks. Simply use "for file in *.txtdatafile" instead.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Deleting all the N lines after the line in which text is found.

Hi! I want to delete N (say 10) lines after the line which text is found in a file "A".Also to delete the line in which the text is found. Only one occurrence of the search string in the file "A" The text to be deleted is in another text file "B". All the search texts in the file "B" are in... (3 Replies)
Discussion started by: shahid1632
3 Replies

2. Shell Programming and Scripting

Deleting specific lines from text file via scripting

Hi, I'm trying to search for some number and from that line, i need to delete the 5th line exactly. Eg: Consider below as text file data: 10000 a b c d e . . . 10000 w q t (8 Replies)
Discussion started by: Gautham
8 Replies

3. Shell Programming and Scripting

Deleting Multiple Lines in a File1 using critera found from File 2

Hi Everyone! I would like ask if there's a better way I can delete multiple lines in a file1 by collecting all criteria from file2. file1: a b c d e f file2: a e f The expected value will be: b (3 Replies)
Discussion started by: zzavilz
3 Replies

4. UNIX for Dummies Questions & Answers

Deleting lines that contain a specific string from a space delimited text file?

Hi, I have a space delimited text file that looks like the following: 250 rs10000056 0.04 0.0888 4 189321617 250 rs10000062 0.05 0.0435 4 5254744 250 rs10000064 0.02 0.2403 4 127809621 250 rs10000068 0.01 NA 250 rs1000007 0.00 0.9531 2 237752054 250 rs10000081 0.03 0.1400 4 17348363... (5 Replies)
Discussion started by: evelibertine
5 Replies

5. UNIX for Dummies Questions & Answers

Deleting Multiple Lines in Hosts File

Hello all, I'm using the Bash shell on Solaris 8. Please can someone tell me how I can delete multiple lines in the hosts file? I have a list of hosts that I want to quickly delete in the hosts file, but I'm looking for a quicker way than using VI to delete the lines one by one. Regards,... (4 Replies)
Discussion started by: wthomas
4 Replies

6. Shell Programming and Scripting

[bash help]Adding multiple lines of text into a specific spot into a text file

I am attempting to insert multiple lines of text into a specific place in a text file based on the lines above or below it. For example, Here is a portion of a zone file. IN NS ns1.domain.tld. IN NS ns2.domain.tld. IN ... (2 Replies)
Discussion started by: cdn_humbucker
2 Replies

7. Shell Programming and Scripting

pattern matching over multiple lines and deleting the first

I've got a longish log file with content such as Uplink traffic: Downlink traffic: I want to parse the log file and remove any line that contains the string "Uplink traffic:" at the beginning of the line, but only if the line following it beginnings with the string "Downlink traffic:" (in... (7 Replies)
Discussion started by: Yorkie99
7 Replies

8. Shell Programming and Scripting

Deleting multiple lines from file

Hi, I have 10 different strings. I have to delete the whole line matching with any one string. I can use sed like below sed '/$keyword1/d' fileList.txt > temp_fileList.txt sed '/$keyword2/d' temp_fileList.txt > temp_fileList1.txt . . . Here is the problem i do not have fixed number... (9 Replies)
Discussion started by: shekhar_v4
9 Replies

9. UNIX for Dummies Questions & Answers

Deleting lines in text file

Hi everyone, I have text files that I want to delete lines from. I have searched through this forum for quite some time and found examples of both awk and sed. Unfortunately, I was not able to successfully do what I want. Well to some extent. I did manage to delete the first 15 lines from each... (5 Replies)
Discussion started by: hern14
5 Replies

10. Shell Programming and Scripting

Deleting Multiple Lines with sed

I am trying to use sed to delete multiple lines in a file. The problem is that I need to search for a certain line and then once found delete it plus the next 4 lines. For instance if I had a file that consisted of the following lines: #Data1.start ( (Database= data1) (Name = IPC)... (1 Reply)
Discussion started by: rambo15
1 Replies
Login or Register to Ask a Question