Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Print duplicate only lines as normal output - Awk

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Print duplicate only lines as normal output - Awk
# 1  
Old 09-30-2010
Print duplicate only lines as normal output - Awk
input

Quote:
a1 100 200 XYZ_X
a1 100 200 XYZ_X
a1 99 199 ABC
a1 98 188 ABC
a1 98 188 ABC
a2 100 200 XYZ_X
output

Code:
a1	100	200	XYZ_X
a1	98	188	ABC

# 2  
Old 09-30-2010
Something like this,
Code:
awk '{if (a[$0]) { print $0} else {a[$0]=++i;}}' input.txt

OR
Code:
uniq -d input.txt

# 3  
Old 09-30-2010
Is it possible modify some thing like this
Just print the middle value that has
one small value set and [a1 100 200 XYZ_X]
high value set [a1 600 800 XYZ_X]

input

Code:
a1	100	200	XYZ_X
a1	300	400	XYZ_X
a1	600	800	XYZ_X
a1	9	79	ABC
a1	98	108	ABC
a1	108	188	ABC
a2	100	200	XYZ_X

output1- middle value

Code:
a1	300	400	XYZ_X
a1	98	108	ABC

output2 - small value of middle value
Code:
a1	100	200	XYZ_X
a1	9	79	ABC

output3 - High value of middle value
Code:
a1	600	800	XYZ_X
a1	108	188	ABC

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Awk: duplicate column and print remaining as is

Hello there I'd like to make a copy of 2nd column and have it printed in place of column 1. Remaining columns are needed as it. test data: ProbeSet GeneSymbol X22565285 X22566285 ILMN_1050008 MYOCD 6.577 7.395 ILMN_1050014 GPRC6A 6.595 6.668 ILMN_1050017 ... (2 Replies)
Discussion started by: genome
2 Replies

2. Shell Programming and Scripting

(n)awk: print regex search output lines in one line

Hello. I have been looking high and low for the solution for this. I seems there should be a simple answer, but alas. I have a big xml file, and I need to extract certain information from specific items. The information I need can be found between a specific set of tags. let's call them... (2 Replies)
Discussion started by: Tobias-Reiper
2 Replies

3. Shell Programming and Scripting

Use sed to print first n lines and last n lines of an output.

Use sed to print first n lines and last n lines of an output. For example: n=10 Can it be done? Thanks. (7 Replies)
Discussion started by: carloszhang
7 Replies

4. Shell Programming and Scripting

Awk and duplicate lines - little complicated

So I've got problem which continues on my previous one (from few months ago: unix.com/shell-programming-scripting/171764-delete-duplicate-lines-twist.html ). Good, proven, working solutions for that old problem are those: awk '{cur=$0; gsub(/]/, "", cur); if (!a++) print}'and awk... (2 Replies)
Discussion started by: shadowww
2 Replies

5. Shell Programming and Scripting

Print duplicate lines

I have a file where some of the lines are duplicates. How do I use bash to print all the lines that have duplicates? (2 Replies)
Discussion started by: locoroco
2 Replies

6. Shell Programming and Scripting

remove duplicate lines using awk

Hi, I came to know that using awk '!x++' removes the duplicate lines. Can anyone please explain the above syntax. I want to understand how the above awk syntax removes the duplicates. Thanks in advance, sudvishw :confused: (7 Replies)
Discussion started by: sudvishw
7 Replies

7. Shell Programming and Scripting

Awk: How to merge duplicate lines and print in a single

The input file: >cat module1 200611051053 95 200523457498 35 200617890187 57 200726098123 66 200645676712 71 200744556590 68 >cat module2 200645676712 ... (10 Replies)
Discussion started by: winter9
10 Replies

8. UNIX for Dummies Questions & Answers

Which command can help me output lines with duplicate numbers?

i have a file, let's call it file. march 2008 january 2008 march 1920 march 2002 i want to output the first line, not the second as you can see the second line has different numbers. (8 Replies)
Discussion started by: hobiwhenuknowme
8 Replies

9. UNIX for Dummies Questions & Answers

Delete duplicate lines and print to file

OK, I have read several things on how to do this, but can't make it work. I am writing this to a vi file then calling it as an awk script. So I need to search a file for duplicate lines, delete duplicate lines, then write the result to another file, say /home/accountant/files/docs/nodup ... (2 Replies)
Discussion started by: bfurlong
2 Replies

10. Shell Programming and Scripting

awk to compare lines of two files and print output on screen

hey guys, I have two files both with two columns, I have already created an awk code to ignore certain lines (e.g lines that start with 963) as they wou ld begin with a certain string, however, the rest I have added together and calculated the average. At the moment the code also displays... (3 Replies)
Discussion started by: chlfc
3 Replies
Login or Register to Ask a Question