The UNIX and Linux Forums  


Go Back   The UNIX and Linux Forums > Top Forums > UNIX for Dummies Questions & Answers
.
grep unix.com with google



UNIX for Dummies Questions & Answers If you're not sure where to post a UNIX or Linux question, post it here. All UNIX and Linux newbies welcome !!

Reply
English Japanese Spanish French German Portuguese Italian Powered by Powered by Google
 
Thread Tools Search this Thread Rate Thread Display Modes
  #1 (permalink)  
Old 11-23-2009
Registered User
 

Join Date: Aug 2008
Posts: 32
Duplicate columns and lines

Hi all,

I have a tab-delimited file and want to remove identical lines, i.e. all of line 1,2,4 because the columns are the same as the columns in other lines. Any input is appreciated.

abc gi4597 9997 cgcgtgcg $%^&*()()*
abc gi4597 9997 cgcgtgcg $%^&*()()*
ttt gi9865 8879 tgcgtgtt *(())^#@!!
abc gi4597 9997 cgcgtgcg $%^&*()()*
fgy gi9876 0975 cgaggcgc @#$%^*&*((
abc gi4597 9997 ttgttgttc $%^&*()()*

---------- Post updated at 09:29 AM ---------- Previous update was at 09:19 AM ----------

It just clicked:
awk 'x[$1,$2,$3,$4,$5,$6]++' filename

Any other methods would be helpful
  #2 (permalink)  
Old 11-24-2009
Registered User
 

Join Date: Nov 2009
Posts: 12
If you want a single line among several identical lines you make a | sort -u. In fact I am not sure I understood your request.
Sponsored Links
Reply

Bookmarks

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


More UNIX and Linux Forum Topics You Might Find Helpful
Thread Thread Starter Forum Replies Last Post
how to identify duplicate columns in a row suresh3566 Shell Programming and Scripting 3 11-16-2009 02:02 AM
how to remove duplicate lines fredao Shell Programming and Scripting 3 12-13-2006 12:51 PM
Duplicate Lines x 4 serm Shell Programming and Scripting 6 08-14-2006 10:09 AM
Duplicate lines in the file guptan UNIX for Advanced & Expert Users 3 05-18-2006 06:28 AM
Filtering duplicate lines AreaMan UNIX for Advanced & Expert Users 9 02-21-2002 07:51 PM



All times are GMT -4. The time now is 10:03 PM.


Powered by: vBulletin, Copyright ©2000 - 2006, Jelsoft Enterprises Limited. Language Translations Powered by .
vBCredits v1.4 Copyright ©2007 - 2008, PixelFX Studios
The UNIX and Linux Forums Content Copyright ©1993-2009. All Rights Reserved.Ad Management by RedTyger

Content Relevant URLs by vBSEO 3.2.0