Print lines in which value in specified Col is NOT unique


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Print lines in which value in specified Col is NOT unique
# 1  
Old 01-29-2014
Print lines in which value in specified Col is NOT unique

Hi everyone,
I have the following file, which is a 3 column tab-delineated.

Code:
cat big 24
cat small   13
cat red 63
dog big 34
chicken plays   39
fish    red 294

I would like to print only those lines, in which the value in Col2 is repeated. Thus, given the above input file, the desired output is the following:

Code:
cat big 24
dog big 34
cat red 63
fish    red 294

Is there an
Code:
awk

that would permit this operation?
Thank you
# 2  
Old 01-29-2014
Code:
awk 'NR==FNR{A[$2]++;next}A[$2]>1' file file

This User Gave Thanks to Yoda For This Post:
# 3  
Old 01-29-2014
Code:
$ awk 'FNR==NR{A[$2]++;next}A[$2]>1' file file
cat big 24
cat red 63
dog big 34
fish    red 294

edit *I just see yoda's approach and mine are same
This User Gave Thanks to Akshay Hegde For This Post:
# 4  
Old 01-29-2014
Hello,

Following may help also.

Code:
awk 'NR==FNR{a[$2]++;next} ($2 in a){if(a[$2] > 1) print $0}' file_name file_name

Code:
cat big 24
cat red 63
dog big 34
fish    red 294


Thanks,
R. Singh
This User Gave Thanks to RavinderSingh13 For This Post:
# 5  
Old 01-29-2014
Hello,
Another awk approach with only one file reading:
Code:
$ cat file
cat big 24
cat small   13
cat red 63
dog big 34
chicken plays   39
fish    red 294

Code:
$ awk '{if(A[$2]) {printf "%s\n%s\n",A[$2],$0;A[$2]="";next};A[$2]=$0}' file
cat big 24
dog big 34
cat red 63
fish    red 294

Regards.
# 6  
Old 01-29-2014
Quote:
Originally Posted by disedorgue
Code:
$ awk '{if(A[$2]) {printf "%s\n%s\n",A[$2],$0;A[$2]="";next};A[$2]=$0}' file

What of the contents of array A at EOF? This code won't work correctly whenever there are an odd number of matches.

Regards,
Alister
This User Gave Thanks to alister For This Post:
# 7  
Old 01-30-2014
Perhaps this will do with a single read:
Code:
awk '!($2 in A){A[$2]=$0; next} A[$2]{print A[$2]; A[$2]=x}1' file

But then the printed order may be different from the one in the input file...

Last edited by Scrutinizer; 01-30-2014 at 03:46 AM..
This User Gave Thanks to Scrutinizer For This Post:
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Print number of lines for files in directory, also print number of unique lines

I have a directory of files, I can show the number of lines in each file and order them from lowest to highest with: wc -l *|sort 15263 Image.txt 16401 reference.txt 40459 richtexteditor.txt How can I also print the number of unique lines in each file? 15263 1401 Image.txt 16401... (15 Replies)
Discussion started by: spacegoose
15 Replies

2. UNIX for Beginners Questions & Answers

Print lines based upon unique values in Nth field

For some reason I am having difficulty performing what should be a fairly easy task. I would like to print lines of a file that have a unique value in the first field. For example, I have a large data-set with the following excerpt: PS003,001 MZMWR/ L-DWD// * PS003,001... (4 Replies)
Discussion started by: jvoot
4 Replies

3. Emergency UNIX and Linux Support

Read values in each col starting 3rd row.Print occurrence value.

Hello Friends, Hope all are doing fine. Here is a tricky issue. my input file is like this 07 10 14 20 21 03 15 27 30 32 01 10 11 19 30 02 06 14 15 17 01 06 20 25 29 Logic: 1. Please print another column as "0-0-0-0-0" for the first and second rows. 2. Read the first column... (4 Replies)
Discussion started by: jacobs.smith
4 Replies

4. Shell Programming and Scripting

Print lines that contain a value in a specific column shared by more than 1 entity in another col

I want to expand on a question that I just asked here: I want to extract only those values in Column 2 that are shared by at least 2 unique values in Column 2. Using the same input (in this case 3- tab-separated columns): waterline-n below-sheath-v 14.8097 dock-n below-sheath-v ... (2 Replies)
Discussion started by: owwow14
2 Replies

5. UNIX for Dummies Questions & Answers

Print unique lines without sort or unique

I would like to print unique lines without sort or unique. Unfortunately the server I am working on does not have sort or unique. I have not been able to contact the administrator of the server to ask him to add it for several weeks. (7 Replies)
Discussion started by: cokedude
7 Replies

6. Shell Programming and Scripting

Modifying col values based on another col

Hi, Please help with this. I have several excel files (with and .xlsx format) with 10-15 columns each. They all have the same type of data but the columns are not ordered in the same way. Here is a 3 column example. What I want to do add the alphabet from column 2 to column 3, provided... (9 Replies)
Discussion started by: newbie83
9 Replies

7. Shell Programming and Scripting

Compare multiple files and print unique lines

Hi friends, I have multiple files. For now, let's say I have two of the following style cat 1.txt cat 2.txt output.txt Please note that my files are not sorted and in the output file I need another extra column that says the file from which it is coming. I have more than 100... (19 Replies)
Discussion started by: jacobs.smith
19 Replies

8. UNIX for Advanced & Expert Users

Print line based on highest value of col (B) and repetion of values in col (A)

Hello everyone, I am writing a script to process data from the ATP world tour. I have a file which contains: t=540 y=2011 r=1 p=N409 t=540 y=2011 r=2 p=N409 t=540 y=2011 r=3 p=N409 t=540 y=2011 r=4 p=N409 t=520 y=2011 r=1 p=N409 t=520 y=2011 r=2 p=N409 t=520 y=2011 r=3 p=N409 The... (4 Replies)
Discussion started by: imahmoud
4 Replies

9. Shell Programming and Scripting

Compare Tab Separated Field with AWK to all and print lines of unique fields.

Hi. I have a tab separated file that has a couple nearly identical lines. When doing: sort file | uniq > file.new It passes through the nearly identical lines because, well, they still are unique. a) I want to look only at field x for uniqueness and if the content in field x is the... (1 Reply)
Discussion started by: rocket_dog
1 Replies

10. Shell Programming and Scripting

Awk to print distinct col values

Hi Guys... I am newbie to awk and would like a solution to probably one of the simple practical questions. I have a test file that goes as: 1,2,3,4,5,6 7,2,3,8,7,6 9,3,5,6,7,3 8,3,1,1,1,1 4,4,2,2,2,2 I would like to know how AWK can get me the distinct values say for eg: on col2... (22 Replies)
Discussion started by: anduzzi
22 Replies
Login or Register to Ask a Question