Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Delete multiple lines from a file

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Delete multiple lines from a file
# 1  
Old 10-25-2010
Delete multiple lines from a file
Hi,

I'm trying to delete some entry's, the source is a file1, from file2

what I have until now is this

file1 :
Code:
68255706,234200801053269,447916926187,8944200006353029289F
73495477,234200101579319,447861769299,8944200006852033303F

file2:
Code:
353851164675 NEW : 272050001241889 -ok 
447846740467 NEW : 234207300638869 -ok 
447588786746 NEW : 234207200443474 -ok 
447861769299 NEW : 234207200529330  not updated (234200101579319)
353851164675 NEW : 272050001241889 -ok 
447846740467 NEW : 234207300638869 -ok 
447588786746 NEW : 234207200443474 -ok 
447916926187 NEW : 234207200432928  not updated (234200801053269)

So I want to delete 447916926187 and 447861769299 from the file2.

I've try this command :

Code:
cat file1 | cut -d ',' -f 3 | while read i
do sed /$i/d file2 >> file 3 # need the output into a 3rd file 
done

But doing this I got duplicate lines ....

Any ideas ?
Last edited by Franklin52; 10-29-2010 at 09:02 AM.. Reason: Please use code tags, looks much better, no?
# 2  
Old 10-25-2010
Duplicates lines comes from every "do" execution in that the first file gets read again and again. Try fgrep or grep -v -f filepattern commands. get the dups in you for loop and put in a file (a file contains the file pattern you will use in the grep -v -f). Good luck.



---------- Post updated at 07:46 PM ---------- Previous update was at 07:42 PM ----------




In the sample below, i pass to file names when I execute the script "./lookup fileone filetwo. It will create a results file that no longer has the bad records. dup.txt contains the records you want removed and the file will be used in fgrep or grep.

Code:
#!/usr/bin/ksh
for r in `cut -c 42-51 $1`   
do                          
  grep $r $2 >> dup.txt       
done
fgrep -v -f dup.txt $2 > results

Last edited by Franklin52; 10-29-2010 at 09:03 AM.. Reason: Please use code tags
Harleyrci
# 3  
Old 10-25-2010
Or if your grep allows it:
Code:
cut -d, -f3 file1 | grep -Fvf - file2

fgrep or grep -F is not really needed here, but it may speed things up.
# 4  
Old 10-25-2010
Thx for the quick answer !!! I've made it Smilie

Here is code that was implemented in the script.
Code:
# Remove any bad line from the log file
cut -d ',' -f 3 cc_IMSI_SWAP-0"$BATCH"_BAD_LINES |while read i
do grep $i $FILE1 >> $FILE2
done

fgrep -v -f $FILE2 $FILE1 >> $FILE3

Thx again !!
Last edited by Franklin52; 10-29-2010 at 09:03 AM.. Reason: Please use code tags
# 5  
Old 10-25-2010
Couldn't you just do:
Code:
cut -d, -f 3 cc_IMSI_SWAP-0"$BATCH"_BAD_LINES > "$FILE2"
fgrep -v -f "$FILE2" "$FILE1" >> "$FILE3"

# 6  
Old 10-25-2010
Thx for the optimization Smilie
# 7  
Old 10-25-2010
No need to generate temp file.

Code:
awk -F , '
NR==FNR{a[$3];next} 
{s=0;split($0,b," ");for (i in a) {if (i==b[1]) s++}; if (s==0) print} 
' file1 file2

And from your sample, seems you just grep -v "not updated" file2 can get the answer directly.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Delete multiple lines between blank lines containing two patterns

Hi all, I'm looking for a way (sed or awk) to delete multiple lines between blank lines containing two patterns ex: user: alpha parameter_1 = 15 parameter_2 = 1 parameter_3 = 0 user: alpha parameter_1 = 15 parameter_2 = 1 parameter_3 = 0 user: alpha parameter_1 = 16... (3 Replies)
Discussion started by: ce9888
3 Replies

2. Shell Programming and Scripting

Removing multiple lines from input file, if multiple lines match a pattern.

GM, I have an issue at work, which requires a simple solution. But, after multiple attempts, I have not been able to hit on the code needed. I am assuming that sed, awk or even perl could do what I need. I have an application that adds extra blank page feeds, for multiple reports, when... (7 Replies)
Discussion started by: jxfish2
7 Replies

3. Shell Programming and Scripting

search and replace, when found, delete multiple lines, add new set of lines?

hey guys, I tried searching but most 'search and replace' questions are related to one liners. Say I have a file to be replaced that has the following: $ cat testing.txt TESTING AAA BBB CCC DDD EEE FFF GGG HHH ENDTESTING This is the input file: (3 Replies)
Discussion started by: DeuceLee
3 Replies

4. Shell Programming and Scripting

delete multiple lines by line number

I have file with 10000 records and i need to delete the lines in single shot based on line number range say from 10 to 51 , 53 to 59 , 105 to 107, 311 to 592 etc... between range works fine for me but how to achive for above case? please help sed '10,51 {d}' infile > outfile (5 Replies)
Discussion started by: zooby
5 Replies

5. Shell Programming and Scripting

Delete multiple lines in a file

The high level requirement is as follows: I have a file which has multiple line starting with pattern (which is fixed say "Hello" and i need to search for one more pattern in that line which starts with "Hello" and if the pattern matches, i need to delete lines from that line to the next line... (5 Replies)
Discussion started by: KeerthiReddy
5 Replies

6. UNIX for Dummies Questions & Answers

delete multiple lines by line number

I have been googling, but cannot find that works for me. I have a text file tmp.out with contents: sadfsdf sdfosuidhfousdhof soduhf osdfu osudfhosudhfd sdfgsdfg asdfiojhsdf asdoludhflsdjfhskldjfhsdjdlfsjdhnlj h sdja ouahsdjdafkljsa oljhljh I have another file... (11 Replies)
Discussion started by: ChicagoBlues
11 Replies

7. Shell Programming and Scripting

regex to delete multiple blank lines in a file?

can't figure out a way to delete multiple empty lines but keep single empty lines in a file, file is like this #cat file 1 2 3 4 5 6 - What I want is 1 2 (6 Replies)
Discussion started by: fedora
6 Replies

8. Shell Programming and Scripting

Need to delete multiple lines in a file.

Hi, I'm new to this forum, and searched through the previous posts, but didn't see anything close enough to what i'm looking for. I have a radius file like this: testone Password = "11111" Service-Type = "Framed-User", Session-Timeout =... (6 Replies)
Discussion started by: kangdom
6 Replies

9. Shell Programming and Scripting

delete multiple empty lines

Hi, I need to delete the lines with empty name. What is the best way to do it? Thanks a lot for your help! EMID MMDDYY HOURS JOB EMNAME 0241 051605 11.40 52062 someone 0520 051605 10.63 52062 0520 051605 10.66 52062 0520 051605 10.65 52062 (3 Replies)
Discussion started by: whatisthis
3 Replies

10. Shell Programming and Scripting

Delete multiple lines w/ sed

Hi all, I am trying to figure out the syntx to delete multiple lines w/ sed. I know the following syntax will delete lines 1 THROUGH 5 from filex: sed 1,5d filex But I wan to delete lines 1 AND 5 (keeping lines 2,3, and 4). Does anyone know how to do this in a single sed statement? ... (2 Replies)
Discussion started by: bookoo
2 Replies
Login or Register to Ask a Question