cat and uniq command


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting cat and uniq command
# 1  
Old 01-24-2012
PHP cat and uniq command

Hello

i have tried to retrieve a distinct list of data from a file with a command combination but the uniq command line is not filtering the fields repeated.
Can someone help me ?

here is the file input
Code:
3837734|LAUNCH TEST01
3837735|LAUNCH TEST01
3837736|LAUNCH TEST01
3837737|LAUNCH TEST01
3837738|LAUNCH TEST01
3837739|LAUNCH TEST01
3837740|LAUNCH TEST01
3837741|LAUNCH TEST01
3837742|LAUNCH TEST01
3837743|LAUNCH TEST01
3837744|LAUNCH TEST01
3837745|LAUNCH TEST01
3837746|LAUNCH TEST01
3837747|LAUNCH TEST01
3837748|LAUNCH TEST01
3837749|LAUNCH TEST01
3837750|LAUNCH TEST01
3837751|LAUNCH TEST01
3837752|WELCOME CALLS
3837753|WELCOME CALLS
3838040|WELCOME CALLS
3838041|WELCOME CALLS
3838042|WELCOME CALLS
3838043|WELCOME CALLS
3838044|WELCOME CALLS
3838045|WELCOME CALLS
3838046|WELCOME CALLS
3838047|WELCOME CALLS
3838048|WELCOME CALLS
3838049|WELCOME CALLS
3838050|WELCOME CALLS
3838051|WELCOME CALLS
3838052|WELCOME CALLS
3856416|WELCOME CALLS
3856417|WELCOME CALLS
3856418|WELCOME CALLS
3856419|WELCOME CALLS
3856420|WELCOME CALLS
3856421|WELCOME CALLS
3856422|WELCOME CALLS
3856423|WELCOME CALLS
3856424|WELCOME CALLS
3856425|WELCOME CALLS
3856426|WELCOME CALLS
3856427|WELCOME CALLS
3856428|WELCOME CALLS
3856429|WELCOME CALLS
3856430|WELCOME CALLS
3856431|WELCOME CALLS
3856524|RELAUNCH 
3856525|RELAUNCH 
3856526|RELAUNCH 
3856527|RELAUNCH 
3856528|RELAUNCH 
3856529|RELAUNCH 
3856530|RELAUNCH 

command line used :
Code:
cat toto.txt  | sort | cut -d"|" -f2 | more
LAUNCH TEST01
LAUNCH TEST01
LAUNCH TEST01
LAUNCH TEST01
LAUNCH TEST01
LAUNCH TEST01
LAUNCH TEST01
WELCOME CALLS
WELCOME CALLS
WELCOME CALLS
WELCOME CALLS
WELCOME CALLS
WELCOME CALLS
WELCOME CALLS
WELCOME CALLS
WELCOME CALLS
WELCOME CALLS
WELCOME CALLS
WELCOME CALLS
WELCOME CALLS
WELCOME CALLS
WELCOME CALLS
WELCOME CALLS
WELCOME CALLS
WELCOME CALLS
WELCOME CALLS
WELCOME CALLS
WELCOME CALLS
WELCOME CALLS
WELCOME CALLS
WELCOME CALLS
WELCOME CALLS
WELCOME CALLS
WELCOME CALLS
WELCOME CALLS
WELCOME CALLS
WELCOME CALLS
WELCOME CALLS
RELAUNCH 
RELAUNCH 
RELAUNCH 
RELAUNCH 
RELAUNCH 
RELAUNCH 
RELAUNCH 

and finally the command that is not working fine (that is not what i expected)
Code:
cat toto.txt  | sort | cut -d"|" -f2 | uniq | more
LAUNCH TEST01
WELCOME CALLS
RELAUNCH 
LAUNCH TEST01
WELCOME CALLS
LAUNCH TEST01
WELCOME CALLS
LAUNCH TEST01
WELCOME CALLS
LAUNCH TEST01
WELCOME CALLS
LAUNCH TEST01
WELCOME CALLS
LAUNCH TEST01
WELCOME CALLS
LAUNCH TEST01
WELCOME CALLS
LAUNCH TEST01
WELCOME CALLS
TEST PHASE 01
LAUNCH TEST01
TEST PHASE 01
WELCOME CALLS
TEST PHASE 99
WELCOME CALLS
JEUNES ADULTES2_2010_C
WELCOME CALLS
LAUNCH TEST01
PROSPECTION CALL CENTER
WELCOME CALLS
WELCOME CALLS

what is wrong ?
thanks

Moderator's Comments:
Mod Comment How to use code tags

Last edited by Franklin52; 01-24-2012 at 07:09 AM.. Reason: Please use code tags for code and data samples, thank you
# 2  
Old 01-24-2012
At first, the initial cat is not needed, as all of the other commands can take a file name as argument by themselves.
The third snippet has strings that are not seen in the initial input file. To make things more clear, can you please post, using code tags, what the expected out put is? Thanks.

Anyway maybe you wanted this:
Code:
zaxxon@saburac ~
$ cut -d "|" -f2 toto.txt| sort -u
LAUNCH TEST01
RELAUNCH
RELAUNCH
WELCOME CALLS

sort has -u switch, which can save you the single uniq maybe.
# 3  
Old 01-24-2012
thanks it works fine
# 4  
Old 01-24-2012
Code:
cat testfile | cut -d "|" -f2 | uniq


Last edited by zaxxon; 01-24-2012 at 08:04 AM..
# 5  
Old 01-24-2012
Try this
Code:
awk -F '|' '{print $2}' toto.txt | sort -u

Below is your expected output correct? The above command gives it
Code:
LAUNCH TEST01
RELAUNCH
WELCOME CALLS


Last edited by machomaddy; 01-24-2012 at 09:35 AM.. Reason: Please use code tags for code and data samples, thank you
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Trying to find the distinct lines using uniq command

Platform :Oracle Linux 6.4 Shell : bash The below file has 7 lines , some of them are duplicates. There are only 3 distinct lines. But why is the uniq command still showing 7 ? I just want the distinct lines to be returned. $ cat test.txt SELECT FC.COORD_SET_ID FROM OM_ORDER_FLOW F, -... (2 Replies)
Discussion started by: kraljic
2 Replies

2. Shell Programming and Scripting

Addition as part of uniq command

Dear Friends, I want to know if it is possible to perform addition as part of uniq command. Input: 213.64.56.208 1 213.64.56.208 2 213.64.56.208 3 213.46.27.204 10 213.46.27.204 20 213.46.27.204 30 The above input should be converted as follows 213.64.56.208 6 213.46.27.204 60 ... (4 Replies)
Discussion started by: tamil.pamaran
4 Replies

3. UNIX for Advanced & Expert Users

uniq command

at which situation uniq command fails. How to delete/remove duplicate lines in an unix file (1 Reply)
Discussion started by: tp2115
1 Replies

4. Shell Programming and Scripting

Uniq Command?

Hello, I have a file with a list of car makes and specific information for each make. An example is: @Audi:Warranty @Audi:Pricing @Audi:Colors @Acura:Warranty @Acura:Pricing @Acura:Colors and so on through a bunch of makes. I need to make a list in a selection box of just one name of... (12 Replies)
Discussion started by: macbb1117
12 Replies

5. Shell Programming and Scripting

replace cat huge_multiplefiles |sort |uniq -c

Does any one know any command runs faster than this to give duplicate numbers cat file1 file2 ....file 100 | sort |uniq -c thanx (3 Replies)
Discussion started by: quincyjones
3 Replies

6. UNIX for Advanced & Expert Users

$cat file|sort|uniq

I have a file with list of redundant server names and want only unique ones of those. I have used following command but still redudant ones are listing $cat file|sort|uniq where could be the problem. Thanks, Srinivas (3 Replies)
Discussion started by: srinivas Juluri
3 Replies

7. UNIX for Dummies Questions & Answers

Uniq command behaving odd

My file has k s j v l k a s f l k s a d f j l a s (3 Replies)
Discussion started by: phoenix_nebula
3 Replies

8. Shell Programming and Scripting

Help with uniq command

I call.... cat 1.txt | uniq -c Sample of whats in 1.txt vmstat cd exit w cd cd cd newgrp xinit f cd cd cd rlogin (2 Replies)
Discussion started by: Bandit390
2 Replies

9. Shell Programming and Scripting

cat in the command line doesn't match cat in the script

Hello, So I sorted my file as I was supposed to: sort -n -r -k 2 -k 1 file1 | uniq > file2 and when I wrote > cat file2 in the command line, I got what I was expecting, but in the script itself ... sort -n -r -k 2 -k 1 averages | uniq > temp cat file2 It wrote a whole... (21 Replies)
Discussion started by: shira
21 Replies

10. UNIX for Dummies Questions & Answers

uniq command???

HI, Please tell the usage of uniq command in UNIX with example Thanks (2 Replies)
Discussion started by: skyineyes
2 Replies
Login or Register to Ask a Question