Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Removing lines with condition

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Removing lines with condition
# 1  
Old 06-30-2010
Question Removing lines with condition
Hello guys,

I need help with a script for removing lines that does not satisfy a condition.

For example if a file has these lines:

Code:
aaaa bbbb cccc
aaaa bbbb cccc
dddd eeee ffff
gggg hhhh iiii
jjjj kkkk llll
aaaa bbbb cccc
jjjj kkkk lllll
dddd eeee ffff
dddd eeee ffff

Then I want to remove the lines with less than 3 occurrences such that the output looks like(may be in a different order):

Code:
aaaa bbbb cccc
aaaa bbbb cccc
dddd eeee ffff
aaaa bbbb cccc
dddd eeee ffff
dddd eeee ffff

I guess doing uniq will get me the files with single occurrences and then I can remove them but how to remove them with this condition(3 occurrences or may be 4, 5,..)?
# 2  
Old 06-30-2010
Code:
awk '{a[$0]++}END{for (i in a) if (a[i]>=3) for (j=1;j<=a[i];j++) print i}' file

This User Gave Thanks to bartus11 For This Post:
jaysean
# 3  
Old 06-30-2010
Code:
awk 'END {
  for (i = 0; ++i <= NR;)
    if (count[pos[i]] >= min)
      print pos[i]
  }
{ 
  pos[NR] = $0; count[$0]++  
  }' min=3 infile

If your awk implementation has no access to NR in the END block, you should save its value in a variable.
Last edited by radoulov; 06-30-2010 at 06:16 AM.. Reason: max -> min :)
This User Gave Thanks to radoulov For This Post:
jaysean
# 4  
Old 06-30-2010
Another way is to read the file twice:
Code:
awk 'NR==FNR{a[$1]++;next} a[$1]>=min' min=3 file file

This User Gave Thanks to Franklin52 For This Post:
jaysean
# 5  
Old 07-01-2010
Thanks to all. I tried the first two and both works fine.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Add the values of the lines according to a condition

hi everybody :) I am a beginner in bash and I want to convert the result I have here I want it to be grouped by IP address so iwanna get for each ip adsress the addition of all bandwidth where ip is 100.1.1.15 in other words for 100.1.1.15 it groups me all the values whose ip address is... (11 Replies)
Discussion started by: aynar
11 Replies

2. Shell Programming and Scripting

Removing multiple lines from input file, if multiple lines match a pattern.

GM, I have an issue at work, which requires a simple solution. But, after multiple attempts, I have not been able to hit on the code needed. I am assuming that sed, awk or even perl could do what I need. I have an application that adds extra blank page feeds, for multiple reports, when... (7 Replies)
Discussion started by: jxfish2
7 Replies

3. Shell Programming and Scripting

Joining broken lines and removing empty lines

Hi - I have req to join broken lines and remove empty lines but should NOT be in one line. It has to be as is line by line. The challenge here is there is no end of line/start of line char. thanks in advance Source:- 2003-04-34024|04-10-2003|Claims|Claim|01-13-2003|Air Bag:Driver;... (7 Replies)
Discussion started by: Jackceasar123
7 Replies

4. UNIX for Dummies Questions & Answers

Removing PATTERN from txt without removing lines and general text formatting

Hi Everybody! First post! Totally noobie. I'm using the terminal to read a poorly formatted book. The text file contains, in the middle of paragraphs, hyphenation to split words that are supposed to be on multiple pages. It looks ve -- ry much like this. I was hoping to use grep -v " -- "... (5 Replies)
Discussion started by: AxeHandle
5 Replies

5. Shell Programming and Scripting

Print certain lines based on condition

Hi All, I have following listing Filesystem GB blocks Free Used Iused Iused Mounted on /dev/hd2 4.00 0.31 93 63080 43 /usr Filesystem GB blocks Free Used Iused Iused Mounted on Filesystem GB blocks Free Used Iused Iused... (11 Replies)
Discussion started by: ckwan
11 Replies

6. UNIX for Dummies Questions & Answers

Grep certain lines with condition

file input aaaa,52C aaaa,50C bbbb,50C bbbb,58C aaaa,52C bbbb,50C aaaa,30C bbbb,58C cccc,60C i want to print uniq lines with its max value of column2 expected output aaaa,52C bbbb,58C cccc,60C tks (4 Replies)
Discussion started by: radius
4 Replies

7. Shell Programming and Scripting

remove duplicate lines with condition

hi to all Does anyone know if there's a way to remove duplicate lines which we consider the same only if they have the first and the second column the same? For example I have : us2333 bbb 5 us2333 bbb 3 us2333 bbb 2 and I want to get us2333 bbb 10 The thing is I cannot... (2 Replies)
Discussion started by: vlm
2 Replies

8. Shell Programming and Scripting

Perl XML, find matching condition and grep lines and put the lines somewhere else

Hi, my xml files looks something like this <Instance Name="New York"> <Description></Description> <Instance Name="A"> <Description></Description> <PropertyValue Key="false" Name="Building A" /> </Instance> <Instance Name="B"> ... (4 Replies)
Discussion started by: tententen
4 Replies

9. Shell Programming and Scripting

Removing empty lines(space) between two lines containing strings

Hi, Please provide shell script to Remove empty lines(space) between two lines containing strings in a file. Input File : A1/EXT "BAP_BSC6/07B/00" 844 090602 1605 RXOCF-465 PDTR11 1 SITE ON BATTERY A2/EXT... (3 Replies)
Discussion started by: sudhakaryadav
3 Replies

10. Shell Programming and Scripting

numbering lines according to a condition

hello again guys, I tried to make a script but due to array's limitations I didn't succeed...so I'm asking you :) I need numbering the lines according to date (everyday I need to restart the counter) for example: ABCBD 20080101 XXX 1 FSDFD 20080101 BBB 2 FSDFD 20080102 HHH 1 and so... (3 Replies)
Discussion started by: elionba82
3 Replies
Login or Register to Ask a Question