Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Remove regularly repeated lines

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Remove regularly repeated lines
# 1  
Old 01-03-2013
Remove regularly repeated lines
How can i delete some regular repeated lines in a file?
example:
in_file

Code:
EDGE 1 2 12
EDGE 2 3 23
EDGE 3 4 34
EDGE 5 6 56
EDGE 6 7 67
EDGE 7 8 78
EDGE 9 10 910
EDGE 10 11 1011
EDGE 11 12 1112
EDGE 13 14 1314
EDGE 14 15 1415
EDGE 15 16 1516
EDGE 17 18 1718
EDGE 18 19 1819
EDGE 19 20 1920
EDGE 21 22 2122
EDGE 22 23 2223
EDGE 23 24 2324
EDGE 25 26 2526
...

i want remove each 3 lines together after 5 line passed, so the result of that action in this example would be :
Code:
EDGE 1 2 12
EDGE 2 3 23
EDGE 3 4 34
EDGE 5 6 56
EDGE 6 7 67

EDGE 11 12 1112
EDGE 13 14 1314
EDGE 14 15 1415
EDGE 15 16 1516
EDGE 17 18 1718

EDGE 22 23 2223
EDGE 23 24 2324
EDGE 25 26 2526
...

Last edited by Scrutinizer; 01-04-2013 at 12:45 AM.. Reason: code tags
# 2  
Old 01-03-2013
Not clear. Your output's first and third block have four lines, and I can't see a pattern on which to decide which lines to keep and which to remove. Pls be more specific.
# 3  
Old 01-03-2013
Your requirement doesn't seem to match the sample data:

Code:
EDGE 1 2 12 
EDGE 2 3 23 
EDGE 3 4 34 
EDGE 5 6 56

These are four, not five lines.
# 4  
Old 01-03-2013
try:
Code:
awk '{if (++c % 6 && !d) {e=d=0; print } else {if (++e % 4) {d=1; c=0} else {print "\n"$0; d=e=0}}}' input

Last edited by rdrtx1; 01-03-2013 at 02:53 PM.. Reason: added blocks of 5 spacing
This User Gave Thanks to rdrtx1 For This Post:
saeed.soltani
# 5  
Old 01-03-2013
Code:
awk 'BEGIN{L=1}{if(!(L>5&&L<9)) print; L++; if(L==9) {print " "; L=1;} } ' filename

This User Gave Thanks to Yoda For This Post:
saeed.soltani
# 6  
Old 01-03-2013
perfectly worked! thanks a lot, but could you please axplain berifly how it works
Quote:
Originally Posted by rdrtx1
try:
Code:
awk '{if (++c % 6 && !d) {e=d=0; print } else {if (++e % 4) {d=1; c=0} else {print "\n"$0; d=e=0}}}' input

# 7  
Old 01-03-2013
roughly:
Code:
awk '
# c print line counter
# d skip line flag
# e skipped line counter
{if (++c % 6 && !d) {    # mod test for c line count (not 0) and skip line flag set to 0
  e=d=0; print           # print line reset e set d to 0
} else {
  if (++e % 4) {         # mod test for skipped line counter (not 0)
    d=1; c=0             # set skip line to 1, reset print line counter
  } else {
    print "\n"$0; d=e=0  # print first line of block with separator, reset print line counter,
                         # and set skip line to 0
  }
}
}' input

This User Gave Thanks to rdrtx1 For This Post:
saeed.soltani
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Remove duplicate lines which has been repeated 4 times

Remove duplicate lines which has been repeated 4 times attached test.txt below command tried and not getting expect output. for i in `cat test.txt | uniq` do num=`cat test.txt | grep $i | wc -l` echo $i $num done test.txt ... (17 Replies)
Discussion started by: Kalia
17 Replies

2. Shell Programming and Scripting

Remove repeated letter words

Hi, I have this text file with these words and I need help with removing words with repeated letter from these lines. 1 ama 5 bib 29 bob 2 bub 5 civic 2 dad 10 deed 1 denned 335 did 1 eeee 1 eeeee 2 eke 8... (4 Replies)
Discussion started by: crepe6
4 Replies

3. Shell Programming and Scripting

Repeated lines-case sensitive

Hi, users file contains below names i have a requirement to keep only one case sensitive user. For e.g if user name is "aaa" then only aaa should be there in the file and other matching users(AAA,aaA) should be deleted. Tried multiple options but no luck can you please help. aaa abc AAA... (2 Replies)
Discussion started by: Satyak
2 Replies

4. Shell Programming and Scripting

Compare two files with repeated lines

Hi all, I've been trying to write a script to compare two files. This is what I want: file 1: a 1 2 b 5 9 c 4 7 file 2: a a c a b Output: a 1 2 a 1 2 (2 Replies)
Discussion started by: ernesto561
2 Replies

5. Shell Programming and Scripting

Getting most repeated 3 lines

Hi all if i want to get the 3 lines of the most repeated lines here tony,1,x tony,1,x tony,2,x tony,2,x tony,3,x tony,4,x tony,5,x adam,1,y to get output tony,1,x tony,2,x tony3,x adam,1,y (3 Replies)
Discussion started by: teefa
3 Replies

6. Shell Programming and Scripting

ksh Remove and replace repeated in file

Hi, i need to read a line from a file and count the number of times it appear in, then continuous to the second line with the same. So when i count a line i have to remove all duplicates in the file to not count it another time. while read line do n=$(grep -c $line File) print "$line... (5 Replies)
Discussion started by: ToniX
5 Replies

7. Shell Programming and Scripting

need to print lines between repeated pattern

Hi all, I have a file that looks like this: uid=bessemsj version: 1 dn: cn=Desk SpecialAdminDesk, ou=Desks, dc=DSS,c=nl,o=Vodafone dn: cn=DSS Advisors, ou=Groups, dc=DSS,c=nl,o=Vodafone dn: cn=DSS Dispatcher,ou=Groups,dc=DSS,c=nl,o=Vodafone dn: cn=Desk Retention Desk,ou=Desks,... (13 Replies)
Discussion started by: Eman_in_forum
13 Replies

8. Shell Programming and Scripting

remove brackets and put it in a column and remove repeated entry

Hi all, I want to remove the remove bracket sign ( ) and put in the separate column I also want to remove the repeated entry like in first row in below input (PA156) is repeated ESR1 (PA156) leflunomide (PA450192) (PA156) leflunomide (PA450192) CHST3 (PA26503) docetaxel... (2 Replies)
Discussion started by: manigrover
2 Replies

9. Shell Programming and Scripting

remove anything after repeated string pattern found

HI, Can anyone help me with a script. i/p calc 1 2 3 4 5 6 7 8 calc 4 5 6 calc 7 8 9 o/p calc 1 2 3 4 5 6 7 8 calc 4 5 6 i.e remove anything after where two times the string calc is found. thanks (3 Replies)
Discussion started by: Indra2011
3 Replies

10. Shell Programming and Scripting

Remove repeated line using Perl

I am new to Perl and in text file of around 1000 lines having around 500 repeated line which I felt is no use and want to remove these line.so can somebody help in same for providing sample code how can i remove these repeated line in a file. (11 Replies)
Discussion started by: dinesh.4126
11 Replies
Login or Register to Ask a Question