Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Getting Distinct values from second field in a file....

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Getting Distinct values from second field in a file....
# 1  
Old 07-13-2009
Getting Distinct values from second field in a file....
Hi

I have a pipe delimited file. I am trying to grab the DISTINCT value from the second field. The file is something like:

1233|apple|ron
1234|apple|elephant
1235|egg|man

the output I am trying to get from second field is apple,egg (apple coming only once)

Thanks
simi
# 2  
Old 07-13-2009
Tools What about...
Code:
cat myfile | cut -d"|" -f2 | sort | uniq -u

display file
cut the 2nd field
sort
give unique "only" values
# 3  
Old 07-13-2009
Code:
nawk -F'|' '!($2 in a) {key[++key[0]]=$2; a[$2]} END {for(i=1;i<=key[0];i++) printf("%s%c", key[i], (i==key[0])?ORS:OFS)}' OFS=, myFile

# 4  
Old 07-14-2009
Another one :

Code:
awk -F"|" ' { arr[$2]; } END { for (i in arr) print i; }'  filename

# 5  
Old 07-14-2009
Sort can do also unique
Code:
cut -d"|" -f2 myfile | sort -u



---------- Post updated at 04:35 PM ---------- Previous update was at 04:17 PM ----------

Ksh93-version using associative arrays, same idea as using awk.
Code:
#!/bin/ksh
#script
typeset -A values
oifs="$IFS"
while read line
do
    IFS="|"
    set -A flds $line
    IFS="$oifs"
    values["${flds[1]}"]=1
done
# 
echo "${!values[@]}"  
# or
a="${!values[@]}"  
# comma separated
echo "${a// /,}"

./script < infile
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Need distinct values from command in a script

Hello, I am using below command srvctl config service -d cmdbut cmdbut_01 (P):/devoragridcn_01/app/oracle> srvctl config service -d cmdbut Service name: boms10.world Service is enabled Server pool: cmdbut_boms10.world Cardinality: 1 Disconnect: false Service role: PRIMARY Management... (7 Replies)
Discussion started by: Vishal_dba
7 Replies

2. Shell Programming and Scripting

Command to identify distinct field value

Could anyone help me on a command to identify distinct values from a particular column ?my input file has header. So i need a command in which we pass Column1 as parameter. For eg my input is like Column1 Column2 Column3 ... (4 Replies)
Discussion started by: krish123
4 Replies

3. Shell Programming and Scripting

Find distinct values

Hi, I have two files of the following format file1 chr1:345-456 chr2:123-456 chr2:455-678 chr3:456-789 chr3:444-555 file2 chr1:345-456 chr2:123-456 chr3:456-789 output (2 Replies)
Discussion started by: jacobs.smith
2 Replies

4. Shell Programming and Scripting

Get the distinct of a particular field

From the below ps output , I want the distinct values of the third field (ie. I need the distinct PPIDs) $ ps -ef UID PID PPID C STIME TTY TIME CMD root 1 0 0 Nov 10 - 48:49 /etc/init root 1769576 1 0 Nov 10 - 0:07... (1 Reply)
Discussion started by: polavan
1 Replies

5. Shell Programming and Scripting

average of distinct values with awk

Hi guys, I am not an expert in shell and I need help with awk command. I have a file with values like 200 1 1 200 7 2 200 6 3 200 5 4 300 3 1 300 7 2 300 6 3 300 4 4 I need resulting file with averages of... (3 Replies)
Discussion started by: saif
3 Replies

6. Shell Programming and Scripting

distinct values of all the fields

I am a beginner to scripting, please help me in this regard. How do I create a script that provides a count of distinct values of all the fields in the pipe delimited file ? I have 20 different files with multiple columns in each file. I needed to write a generic script where I give the number... (2 Replies)
Discussion started by: vukkusila
2 Replies

7. UNIX for Dummies Questions & Answers

distinct values of all the fields

I am a beginner to scripting, please help me in this regard. How do I create a script that provides a count of distinct values of all the fields in the pipe delimited file ? I have 20 different files with multiple columns in each file. I needed to write a generic script where I give the number... (1 Reply)
Discussion started by: vukkusila
1 Replies

8. Shell Programming and Scripting

Select distinct values from a flat file

Hi , I have a similar problem. Please can anyone help me with a shell script or a perl. I have a flat file like this fruit country apple germany apple india banana pakistan banana saudi mango india I want to get a output like fruit country apple ... (7 Replies)
Discussion started by: smalya
7 Replies

9. Shell Programming and Scripting

grep distinct values

this is a little more complex than that. I have a text file and I need to find all the distinct words that appear in a line after the word TABLESPACE when I grep for just the word tablespace, I get: how do i parse this a little better so i have a smaller file to read? This is just an... (4 Replies)
Discussion started by: guessingo
4 Replies

10. Shell Programming and Scripting

Loop through only the distinct values in a file

Datafile has the following data seperated by : FIELD1:FIELD2:FIELD3 D1:/opt/9.1.9:Y D2:/opt/10.1.10:Y D3:/opt/9.1.9:Y D4:/opt/8.1.8:Y D5:/opt/8.1.8:Y D6:/opt/9.1.9:Y D7:/opt/9.1.9:Y D8:/opt/10.1.10:Y D9:/opt/9.1.9:Y D10:/opt/10.1.10:Y I want to do some operations only on the distinct... (2 Replies)
Discussion started by: pbekal
2 Replies
Login or Register to Ask a Question