Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

duplicate elimination with column print

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting duplicate elimination with column print
# 1  
Old 06-04-2012
duplicate elimination with column print
hi,

my input

Code:
a b c d e f g 
k i m g i k l 
b i n g o h m
a b c d e
b i n g o

my output

Code:
a b c d e f g
k i m g i k l
b i n g o h m

what I would like to do is compare the first five columns of each row to the entire records in the file and if there is a match, eliminate that record and print the matched five columns with the additional two columns.
# 2  
Old 06-04-2012
Try:
Code:
awk '!A[$1,$2,$3,$4,$5]++' infile

These 2 Users Gave Thanks to Scrutinizer For This Post:
Corona688 jacobs.smith
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Need awk or Shell script to compare Column-1 of two different CSV files and print if column-1 matche

Example: I have files in below format file 1: zxc,133,joe@example.com cst,222,xyz@example1.com File 2 Contains: hxd hcd jws zxc cst File 1 has 50000 lines and file 2 has around 30000 lines : Expected Output has to be : hxd hcd jws (5 Replies)
Discussion started by: TestPractice
5 Replies

2. Shell Programming and Scripting

Awk: duplicate column and print remaining as is

Hello there I'd like to make a copy of 2nd column and have it printed in place of column 1. Remaining columns are needed as it. test data: ProbeSet GeneSymbol X22565285 X22566285 ILMN_1050008 MYOCD 6.577 7.395 ILMN_1050014 GPRC6A 6.595 6.668 ILMN_1050017 ... (2 Replies)
Discussion started by: genome
2 Replies

3. Shell Programming and Scripting

Find duplicate values in specific column and delete all the duplicate values

Dear folks I have a map file of around 54K lines and some of the values in the second column have the same value and I want to find them and delete all of the same values. I looked over duplicate commands but my case is not to keep one of the duplicate values. I want to remove all of the same... (4 Replies)
Discussion started by: sajmar
4 Replies

4. Shell Programming and Scripting

awk to sum a column based on duplicate strings in another column and show split totals

Hi, I have a similar input format- A_1 2 B_0 4 A_1 1 B_2 5 A_4 1 and looking to print in this output format with headers. can you suggest in awk?awk because i am doing some pattern matching from parent file to print column 1 of my input using awk already.Thanks! letter number_of_letters... (5 Replies)
Discussion started by: prashob123
5 Replies

5. Shell Programming and Scripting

awk Print New Column For Every Two Lines and Match On Multiple Column Values to print another column

Hi, My input files is like this axis1 0 1 10 axis2 0 1 5 axis1 1 2 -4 axis2 2 3 -3 axis1 3 4 5 axis2 3 4 -1 axis1 4 5 -6 axis2 4 5 1 Now, these are my following tasks 1. Print a first column for every two rows that has the same value followed by a string. 2. Match on the... (3 Replies)
Discussion started by: jacobs.smith
3 Replies

6. Shell Programming and Scripting

[ask]elimination with diskspace

dear all, i want elimination file with disk space for example: $ df -h Filesystem Size Used Avail Use% Mounted on /dev/mapper/VolGroup00-LogVol00 62G 19G 41G 31% / /dev/sda1 99M 13M 82M 13% /boot tmpfs 1014M 0... (2 Replies)
Discussion started by: zvtral
2 Replies

7. Shell Programming and Scripting

comparing column of two different files and print the column from in order of 2nd file

Hi friends, My file is like: Second file is : I need to print the rows present in file one, but in order present in second file....I used while read gh;do awk ' $1=="' $gh'" {print >> FILENAME"output"} ' cat listoffirstfile done < secondfile but the output I am... (14 Replies)
Discussion started by: CAch
14 Replies

8. Shell Programming and Scripting

print first few lines, then apply regex on a specific column to print results.

abc.dat tty cpu tin tout us sy wt id 0 0 7 3 19 71 extended device statistics r/s w/s kr/s kw/s wait actv wsvc_t asvc_t %w %b device 0.0 133.2 0.0 682.9 0.0 1.0 0.0 7.2 0 79 c1t0d0 0.2 180.4 0.1 5471.2 3.0 2.8 16.4 15.6 15 52 aaaaaa1-xx I want to skip first 5 line... (4 Replies)
Discussion started by: kchinnam
4 Replies

9. Shell Programming and Scripting

Date elimination

I have a file with the following data. I want to start writing lines out of this file to another file only after the time is greater than 20:00:00 hours and the first job is U1KC000. I would like to be able to run this out of a script and pass the time and job(U1KC000) as an argument. ... (14 Replies)
Discussion started by: wawa
14 Replies

10. Shell Programming and Scripting

4 x fibre cards elimination

Hi, I have variables like mentioned below and want to separate as mentioned in output. var1="4 x fibre cards" var2="2 fibre cards" var3="6 - fibre cards" var4="4 x dual-port" I have variables like this.I want to separate numbers from this and fibre cards from this. output has to be... (4 Replies)
Discussion started by: ajilesh
4 Replies
Login or Register to Ask a Question