Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Sort & Uniq -u

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Sort & Uniq -u
# 1  
Old 05-21-2015
Sort & Uniq -u
Hi All,

Below the actual file which i like to sort and Uniq -u

Code:
/opt/oracle/work/Antony/Shell_Script> cat emp.1st
2233|a.k. shukula               |g.m.           |sales          |12/12/52       |6000
1006|chanchal singhvi           |director       |sales          |03/09/38       |6700
1265|s.n. dasguptha             |manager        |sales          |12/09/63       |5600
2476|anil aggarwal              |manager        |sales          |01/05/59       |5000
9876|jai sharma                 |director       |production     |12/03/50       |7000
5678|sumit chakrobarty          |d.g.m          |marketing      |19/04/43       |6000
6521|lalit chowdury             |director       |marketing      |09/26/45       |8200
2365|barun sengupta             |director       |personnel      |11/05/47       |7800
5423|n.k gupta                  |chairman       |admin          |30/08/56       |5400
6213|karuna ganguly             |g.m.           |accounts       |05/06/62       |6300
4290|jayant choudhury           |executive      |production     |07/09/50       |8200
3213|shyam saksena              |d.g.m.         |accounts       |12/12/55       |6000
3564|sudhir Ahagwal             |executive      |personnel      |06/07/47       |7500
2345|j.b. saxena                |g.m.           |marketing      |12/03/45       |8000
0110|v.k agarwal                |g.m.           |marketing      |31/12/40       |9000


When i issue below command it is giving me output

Code:
/opt/oracle/work/Antony/Shell_Script> cut -d'|' -f4 emp.1st | sort | uniq -u
admin
marketing
sales


Where i am suppose to get
Code:
admin
marketing
sales
production     
marketing 
personnel      
Accounts



Where i have done mistake ? please advice
Last edited by Scrutinizer; 05-21-2015 at 03:25 PM.. Reason: code tags
# 2  
Old 05-21-2015
what is your objective? Do you want a unique list of 4th field?

If you go step by step..

Code:
1. cut -d'|' -f4 file

Then depending on what you want,

2. If you want sorted unique list

Code:
cut -d'|' -f4 file | sort -u

3. If you want to maintain order and get unique list of 4th field.

Code:
awk -F"|" '!a[$4]++{print $4}' file

4. If you want to collapse clustered values and get a list (may have repeats)

Code:
cut -d'|' -f4 file | uniq

# 3  
Old 05-21-2015
Thanks for your suggestion

Even if i sort with sort -u, why am i getting sales and Marketing twice ?

Code:
/opt/oracle/work/Antony/Shell_Script> cut -d'|' -f4 emp.1st | sort -u
accounts
admin
marketing
marketing
personnel
production
sales
sales

# 4  
Old 05-21-2015
Quote:
Originally Posted by ANTONY ANKROSE
Thanks for your suggestion

Even if i sort with sort -u, why am i getting sales and Marketing twice ?

Code:
/opt/oracle/work/Antony/Shell_Script> cut -d'|' -f4 emp.1st | sort -u
accounts
admin
marketing
marketing
personnel
production
sales
sales

I`m getting the desired result, not sure of whats with your dataset.
Do you have \r carriage returns?

Code:
 cut -d'|' -f4 tmp | sort -u
accounts
admin
marketing
personnel
production
sales

# 5  
Old 05-21-2015
There may be extra space or hidden characters in your input file...
# 6  
Old 05-21-2015
Can you do

Code:
cut -d'|' -f4 emp.1st | sort -u | od -c

and let us know what you get?
# 7  
Old 05-21-2015
Code:
/opt/oracle/work/Antony/Shell_Script> cut -d'|' -f4 emp.1st | sort -u | od -c
0000000    a   c   c   o   u   n   t   s  \t  \n   a   d   m   i   n  \t
0000020   \t  \n   m   a   r   k   e   t   i   n   g  \t  \n   m   a   r
0000040    k   e   t   i   n   g      \t  \n   p   e   r   s   o   n   n
0000060    e   l  \t  \n   p   r   o   d   u   c   t   i   o   n  \t  \n
0000100    s   a   l   e   s  \t  \t  \n   s   a   l   e   s      \t  \t
0000120   \n
0000121

Last edited by Scrutinizer; 05-21-2015 at 04:04 PM..
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Uniq and sort -u

Hello all, Need to pick your brains, I have a 10Gb file where each row is a name, I am expecting about 50 names in total. So there are a lot of repetitions in clusters. So I want to do a sort -u file Will it be considerably faster or slower to use a uniq before piping it to sort... (3 Replies)
Discussion started by: senhia83
3 Replies

2. Shell Programming and Scripting

Uniq or sort -u or similar only between { }

Hi ! I am trying to remove doubbled entrys in a textfile only between delimiters. Like that example but i dont know how to do that with sort or similar. input: { aaa aaa } { aaa aaa } output: { aaa } { (8 Replies)
Discussion started by: fugitivus
8 Replies

3. Shell Programming and Scripting

Sort field and uniq

I have a flatfile A.txt 2012/12/04 14:06:07 |trees|Boards 2, 3|denver|mekong|mekong12 2012/12/04 17:07:22 |trees|Boards 2, 3|denver|mekong|mekong12 2012/12/04 17:13:27 |trees|Boards 2, 3|denver|mekong|mekong12 2012/12/04 14:07:39 |rain|Boards 1|tampa|merced|merced11 How do i sort and get... (3 Replies)
Discussion started by: sabercats
3 Replies

4. Shell Programming and Scripting

Sort and uniq after comparision

Hi All, I have a text file with the format shown below. Some of the records are duplicated with the only exception being date (Field 15). I want to compare all duplicate records using subscriber number (field 7) and keep only those records with greater date. ... (1 Reply)
Discussion started by: nua7
1 Replies

5. Shell Programming and Scripting

sort & uniq on specific fields problem

Hello; I have the output data set from: egrep -i 'warning| error| fail' /var/adm/syslog/syslog.log Jan 31 12:02:18 fidsrv vmunix: LVM: WARNING: VG 128 0x001000: LV 5: Some I/O requests to this LV are waiting Jan 31 12:02:23 fidsrv vmunix: Asynchronous write failed on LUN (dev=0x100000f)... (3 Replies)
Discussion started by: delphys
3 Replies

6. Shell Programming and Scripting

Help with Uniq and sort

The key is first field i want only uniq record for the first field in file. I want the output as or output as Appreciate help on this (4 Replies)
Discussion started by: pinnacle
4 Replies

7. Shell Programming and Scripting

Sort, Uniq, Duplicates

Input File is : ------------- 25060008,0040,03, 25136437,0030,03, 25069457,0040,02, 80303438,0014,03,1st 80321837,0009,03,1st 80321977,0009,03,1st 80341345,0007,03,1st 84176527,0047,03,1st 84176527,0047,03, 20000735,0018,03,1st 25060008,0040,03, I am using the following in the script... (5 Replies)
Discussion started by: Amruta Pitkar
5 Replies

8. Shell Programming and Scripting

sort and uniq in perl

Does anyone have a quick and dirty way of performing a sort and uniq in perl? How an array with data like: this is bkupArr BOLADVICE_VN this is bkupArr MLT6800PROD2A this is bkupArr MLT6800PROD2A this is bkupArr BOLADVICE_VN_7YR this is bkupArr MLT6800PROD2A I want to sort it... (4 Replies)
Discussion started by: reggiej
4 Replies

9. UNIX for Dummies Questions & Answers

Help with Last,uniq, sort and cut

Using the last, uniq, sort and cut commands, determine how many times the different users have logged in. I know how to use the last command and cut command... i came up with last | cut -f1 -d" " | uniq i dont know if this is right, can someone please help me... thanks (1 Reply)
Discussion started by: jay1228
1 Replies

10. UNIX for Dummies Questions & Answers

sort/uniq

I have a file: Fred Fred Fred Jim Fred Jim Jim If sort is executed on the listed file, shouldn't the output be?: Fred Fred Fred Fred Jim Jim Jim (3 Replies)
Discussion started by: jimmyflip
3 Replies
Login or Register to Ask a Question