Command to identify distinct field value


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Command to identify distinct field value
# 1  
Old 04-09-2013
Command to identify distinct field value

Could anyone help me on a command to identify distinct values from a particular column ?my input file has header. So i need a command in which we pass Column1 as parameter.
For eg my input is like
Column1 Column2 Column3
a 11 abc
a 22 abc
b 33 edf
c 44 ghi

I require a output like

Column1

a
b
c

Last edited by krish123; 04-09-2013 at 01:29 AM..
# 2  
Old 04-09-2013
Here is an awk program:
Code:
printf "Enter column name: "
read col

awk -v C="$col" '
        NR == 1 {
                        for ( i = 1; i <= NF; i++ )
                        {
                                if ( $i == C )
                                        break;
                        }
        }
        NR > 1 {
                        A[$i]
        }
        END {
                for ( k in A )
                        print k
} ' file

Output:
Code:
$ ./krish
Enter column name: Column1
a
b
c

# 3  
Old 04-09-2013
This version does not eliminate duplicates:
Code:
awk 'NR==1 {for(i=1; $i!=C; i++) if(i>NF) exit; next} {print $i}' C=Column3 file

Without the next it will also print the header.
# 4  
Old 04-09-2013
Two comments on Yoda's proposal:
No header printed.
The order of lines is not predictable when using for ( k in A), which may not show up in small files.
Try this, based on Yoda's proposal:
Code:
awk     'NR == 1        {for (i = 1; i <= NF; i++ ) if ( $i == C ) break }
         LL[$i]         {next}
                        {print $i; LL[$i]++}
        ' C="$col" file
Column1
a
b
c

# 5  
Old 04-09-2013
OR..

Code:
awk -v COL="$var" 'NR==1 {for(i=1;i<=NF;i++){if($i==COL){break;}}}
		NR>1 && !A[$i]++ {print $i}' file

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Assistance with my Find command to identify last part of a file name and report the name found

Hello Forum, We have two bootstraps of Chef in our environment which are identified by colour: /var/chef/cache/cookbooks/bootstrap_cookbooks_version_green and /var/chef/cache/cookbooks/bootstrap_cookbooks_version_red I'm attempting to identify which version is installed based on the name... (11 Replies)
Discussion started by: greavette
11 Replies

2. Shell Programming and Scripting

Help with Getting distinct record count from a .dat file using UNIX command

Hi, I have a .dat file with contents like the below: Input file ============SEQ NO-1: COLUMN1========== 9835619 7152815 ============SEQ NO-2: COLUMN2 ========== 7615348 7015548 9373086 ============SEQ NO-3: COLUMN3=========== 9373086 Expected Output: (I just... (1 Reply)
Discussion started by: MS06
1 Replies

3. Shell Programming and Scripting

Command/script to match a field and print the next field of each line in a file.

Hello, I have a text file in the below format: Source Destination State Lag Status CQA02W2K12pl:D:\CAQA ... (10 Replies)
Discussion started by: pocodot
10 Replies

4. Shell Programming and Scripting

Trying to find the distinct lines using uniq command

Platform :Oracle Linux 6.4 Shell : bash The below file has 7 lines , some of them are duplicates. There are only 3 distinct lines. But why is the uniq command still showing 7 ? I just want the distinct lines to be returned. $ cat test.txt SELECT FC.COORD_SET_ID FROM OM_ORDER_FLOW F, -... (2 Replies)
Discussion started by: kraljic
2 Replies

5. Shell Programming and Scripting

Need distinct values from command in a script

Hello, I am using below command srvctl config service -d cmdbut cmdbut_01 (P):/devoragridcn_01/app/oracle> srvctl config service -d cmdbut Service name: boms10.world Service is enabled Server pool: cmdbut_boms10.world Cardinality: 1 Disconnect: false Service role: PRIMARY Management... (7 Replies)
Discussion started by: Vishal_dba
7 Replies

6. Shell Programming and Scripting

Command to identify mail attachment

Hi, I need to send a email with attachment from windows system to a unix system. when ever a new email arrived the unix system has to check and extract or download the attachment then move to seprate folder. For identifying the attachment and downloading any command available. Thanks. (1 Reply)
Discussion started by: nag_sathi
1 Replies

7. Shell Programming and Scripting

Get the distinct of a particular field

From the below ps output , I want the distinct values of the third field (ie. I need the distinct PPIDs) $ ps -ef UID PID PPID C STIME TTY TIME CMD root 1 0 0 Nov 10 - 48:49 /etc/init root 1769576 1 0 Nov 10 - 0:07... (1 Reply)
Discussion started by: polavan
1 Replies

8. Shell Programming and Scripting

Identify high values "ÿ" in a text file using Unix command

I have high values (such as ÿÿÿÿ) in a text file contained in an Unix AIX server. I need to identify all the records which are having these high values and also get the position/column number in the record structure if possible. Is there any Unix command by which this can be done to : 1.... (5 Replies)
Discussion started by: devina
5 Replies

9. Shell Programming and Scripting

Getting Distinct values from second field in a file....

Hi I have a pipe delimited file. I am trying to grab the DISTINCT value from the second field. The file is something like: 1233|apple|ron 1234|apple|elephant 1235|egg|man the output I am trying to get from second field is apple,egg (apple coming only once) Thanks simi (4 Replies)
Discussion started by: simi28
4 Replies

10. UNIX for Dummies Questions & Answers

Identify duplicate words in a line using command

Hi, Let me explain the problem clearly: Let the entries in my file be: lion,tiger,bear apple,mango,orange,apple,grape unix,windows,solaris,windows,linux red,blue,green,yellow orange,maroon,pink,violet,orange,pink Can we detect the lines in which one of the words(separated by field... (8 Replies)
Discussion started by: srinivasan_85
8 Replies
Login or Register to Ask a Question