File operations

monks · #1 (**permalink**) 04-25-2006

Hi
I have a tab delimited file with 3 fields. I need to sort this file on the first field and remove all the records where the first field has dulplicates. For eg my file is

133|arrfdfdg|sdfdsg
234|asfsdgfs|aasdfs
133|affbfsde|dgfg

When this file gets sorted I need the result to be
234|asfsdgfs|aasdfs

So if there are duplicate entries in the first column, all those records should be removed. How can I do this in unix? I am able to sort it to get single records based on unique first field using

sort -u -k 1,1 filename

but this is not what I am looking for. Any help will be appreciated!

jim mcnamara · #2 (**permalink**) 04-25-2006

try:

Code:

sort -u -t=| -k1.1,1.8 filename

thestevew · #3 (**permalink**) 04-26-2006

To just print non-repeated lines in the file turns out to be a bit of a pain. The uniq command would do it but only if the key is on the right hand side of the line, so I've put it there.

The code sorts the file on the first column (delimited by pipes) then appends the key to the end of the line uses and uses uniq to remove line with non-repeated keys before stripping off the added key!

Looks a bit klunky - I'm sure that someone could do something more elegant

Code:

Code
sort -t'|' -k1,1 < yourfile.txt | while read x
do
  print $x ${x%%'|'*}
done | uniq -f1 -u | cut -d' ' -f1

cheers

More UNIX and Linux Forum Topics You Might Find Helpful
Thread	Thread Starter	Forum	Replies	Last Post
Unix File operations	nivas	Shell Programming and Scripting	14	02-11-2008 05:27 AM
Unix file operations(shell script)	nivas	Shell Programming and Scripting	6	02-07-2008 07:11 AM
command for a series of operations	phiber_optik	Shell Programming and Scripting	1	12-21-2007 06:57 AM
String Operations	Rohini Vijay	Shell Programming and Scripting	9	04-21-2006 11:32 AM
File operations	chiragmistry21	Shell Programming and Scripting	2	03-27-2006 05:00 PM

Thread Tools	Search this Thread
Show Printable Version Email this Page	Search this Thread: Advanced Search
Display Modes	Rate This Thread
Switch to Linear Mode Hybrid Mode Switch to Threaded Mode	Rate This Thread:


	Powered by