deleting a varying amount of lines from a list of files


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting deleting a varying amount of lines from a list of files
# 1  
Old 04-14-2006
deleting a varying amount of lines from a list of files

I did search the posts for info on this and while there were some in the ballpark, none addressed this specifically. (also I tried to post this once it said I was logged out, so hopefully I'm not sending a duplicate here).

I have a set of files (250 +/-) where I need to delete the first "$x" number of lines and output what remains to a new file (or cat all the results into one file)

so far, I was able to piece together this much:

cat -n index20a6.html index3e43.html index6a84.html (...and so on) | grep
'<TITLE>Section 440.' | awk '{print $1}'

it results in this:

973
1722
2219
...

'<TITLE>Section 440.' is on a different line in each file so the amount of line to delete will vary.

The missing piece is to delete '1,973d' '1,1722d' '1,2219d' as it goes through and create a new file.

(also if I can go through this list recursively rather than list all the files in my command, that would be a bonus!)

Thanks for any assitance on this!
# 2  
Old 04-14-2006
Try:
Code:
#! /usr/bin/ksh
cd /the/directory
exec > onebigoutputfile
find . -name index\*.html | while read fname ; do
           sed '1,/<TITLE> Section 440./d' < $fname
done
exit 0

# 3  
Old 04-14-2006
First, Thanks for the reply!

That results in just a list of the filenames showing in "onebigoutputfile"

./index018e.html
./index0271.html
./index031c.html
./index03bb.html
./index03fe.html
./index04b3.html
...

I'll continue to work with it.
# 4  
Old 04-14-2006
Retested....
Code:
$ cat script
#! /usr/bin/ksh
#cd /the/directory
#exec > onebigoutputfile
find . -name index\*.html | while read fname ; do
           sed '1,/<TITLE> Section 440./d' < $fname
done
exit 0
$
$
$
$ cat index1.html
hhhh
kkkkk
hhhh <TITLE> Section 440.
jjjjj 5
jjjjjj8
jjjj6
iiii9
$
$
$
$
$
$ ./script
jjjjj 5
jjjjjj8
jjjj6
iiii9
$

I commented out the cd and the exec for easy testing. The only way your result makes sense is if each of your files contains a single line after the "seection 440" line and that single line is only the file name. I assume you can cut and paste accurately, so then it beats me... Smilie
# 5  
Old 04-14-2006
I rechecked and the original cut and paste was directly from what I was running earlier. Of course, I'm no pro when it comes to working in the Shell, so I'll just have to keep working with what you posted and see where I'm going astray.

Thank you for taking the time to help with this.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Merge lines with varying characters

Hi, I have a large set of data (firewall logs) that I'm trying to summarize. I've been able to write a script to consolidate the ports, now am looking to conslidate even further, based on IP. Source Destination Type Port 192.168.5.108 192.168.11.12 TCP 1, 2, 3, 4, 5, 15 192.168.5.109... (6 Replies)
Discussion started by: umang2382
6 Replies

2. Shell Programming and Scripting

Mainain the files after deleting lines with awk

hello I want to delete the lines of many files (e.g. file1, file2, ..., file1111) in my directory (e.g. dir1) that they have the 2nd column grater than a value (e.g. 40) placing them to a different directory (e.g. newdir) and keeping the originals. I wrote the code which is functional awk '{... (3 Replies)
Discussion started by: phaethon
3 Replies

3. Shell Programming and Scripting

Paste files of varying lengths

I have three files of varying lengths and different number of columns. How can I paste all three with all columns aligned? File1 ---- 123 File2 ---- 234 345 678 File3 ---- 456 789 Output should look like: 123 234 456 345 789 (6 Replies)
Discussion started by: Un1xNewb1e
6 Replies

4. UNIX for Advanced & Expert Users

Limit on amount of lines less can handle

Is there a limit on the amount of lines less can handle? I grepped for something that is pretty common in my files so I tried to filter it through less so it was easier to look at. So I then used a redirect ">" to create a file. It created a file that was 103K long. So that makes me think there is... (6 Replies)
Discussion started by: cokedude
6 Replies

5. Shell Programming and Scripting

Deleting lines based on a condition for a group of files

hi i have a set of similar files. i want to delete lines until certain pattern appears in those files. for a single file the following command can be used but i want to do it for all the files at a time since the number is in thousands. awk '/PATTERN/{i++}i' file (6 Replies)
Discussion started by: anurupa777
6 Replies

6. UNIX for Dummies Questions & Answers

deleting lines from a delimited files based on a 2nd file

I need a little help... I have a file with 3 fields, Time/Date, IP address, UniqueID I have a 2nd file of UniqueIDs. I want to either (which ever is easier): 1. delete entries in file 1 that have a UniqueID in file 2 2. create a new file with the fields from File 1, excluding the... (4 Replies)
Discussion started by: goldie363
4 Replies

7. UNIX for Dummies Questions & Answers

deleting specific lines from all files in a directory

I have a directory full of text data files. Unfortunately I need to get rid of the 7th and 8th line from them all so that I can input them into a GIS application. I've used an awk script to do one at a time but due to the sheer number of files I need some kind of loop mechanism to automate... (3 Replies)
Discussion started by: vrms
3 Replies

8. Shell Programming and Scripting

deleting lines from multiple text files

I have a directory full of text data files. Unfortunately I need to get rid of the 7th and 8th line from them all so that I can input them into a GIS application. I've used an awk script to do one at a time but due to the sheer number of files I need some kind of loop mechanism to automate... (3 Replies)
Discussion started by: vrms
3 Replies

9. Shell Programming and Scripting

Deleting Lines from .CSV Files

I have written an script which will excluded some records from .csv file and put it on another excluded file from primary file.This is working very fine.Now the problem is that I want to delete those excluded lines from Primary file but not able to delete it. I have stored the line number in... (1 Reply)
Discussion started by: 009satya
1 Replies

10. Shell Programming and Scripting

PHP Outputting finite amount of lines from a file

Hi, I have the code below which outputs all the lines of a CSV file but I'd like it to only output 15 lines then save the current line as a variable which could be used (as a link) to display the next 15 lines. I can't get my head around the logic! Please help :D while ($data = fgetcsv($fp,... (1 Reply)
Discussion started by: pondlife
1 Replies
Login or Register to Ask a Question