Sponsored Content
Top Forums Shell Programming and Scripting Find duplicates in 2 & 3rd column and their ID Post 303000187 by busyboy on Thursday 6th of July 2017 04:52:51 AM
Old 07-06-2017
Find duplicates in 2 & 3rd column and their ID

with below given format,

I have been trying to find out all IDs for those entries with duplicate names in 2nd and 3rd columns and their count like how many time duplication happened for any name if any,


Code:
0.237788 Aaban Aahva
0.291066 Aabheer Aahlaad
0.845814 Aabid Aahan
0.152208 Aadam Aagneya
0.585537 Aadan Aagney
0.193475 Aadarsh Aagam
0.810623 Aadavan Aadvik
0.173531 Aadesh Aadrian
0.484983 Aadhan Aaditya
0.151863 Aadhira Aaditeya
0.366957 Aadi Aadit
0.491736 Aadidev Aadir
0.910094 Aadil Aadinath
0.265257 Aadinath Aadil
0.893188 Aadir Aadidev
0.220351 Aadit Aadi
0.631798 Aaditeya Aadhira
0.571077 Aaditya Aadhan
0.332158 Aadrian Aadesh
0.104455 Aadvik Aadavan
0.502931 Aagam Aadarsh
0.567394 Aagney Aadan
0.854165 Aagneya Aadam
0.0401409 Aahan Aabid
0.108022 Aahlaad Aabheer
0.639396 Aahva Aaban
0.291066  Aadil Aadinath
0.845814  Aadinath Aadil
0.152208  Aadir Aadidev
0.585537  Aadit Aadi
0.193475  Aaditeya Aadhira
0.810623  Aaditya Aadhan
0.173531  Aadrian Aadesh
0.484983  Aadvik Aadavan
0.151863  Aagam Aadarsh
0.366957  Aagney Aadan
0.491736  Aagneya Aadam
0.910094   Aahan Aabid
0.265257  Aahlaad Aabheer
0.893155  Aahva Aaban
0.193443  Aaditeya Aadhira
0.810667  Aaditya Aadhan
0.173545  Aadrian Aadesh
0.484934  Aadvik Aadavan
0.151862  Aagam Aadarsh
0.366954  Aagney Aadan
0.491736  Aagneya Aadam
0.910094   Aahan Aabid
0.265232  Aahlaad Aabheer
0.893134  Aahva Aaban

REgards,
Nasir

Moderator's Comments:
Mod Comment Use code tags please, thanks.

Last edited by zaxxon; 07-06-2017 at 06:05 AM..
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Find duplicates in the first column of text file

Hello, My text file has input of the form abc dft45.xml ert rt653.xml abc ert57.xml I need to write a perl script/shell script to find duplicates in the first column and write it into a text file of the form... abc dft45.xml abc ert57.xml Can some one help me plz? (5 Replies)
Discussion started by: gameboy87
5 Replies

2. Shell Programming and Scripting

need to remove duplicates based on key in first column and pattern in last column

Given a file such as this I need to remove the duplicates. 00060011 PAUL BOWSTEIN ad_waq3_921_20100826_010517.txt 00060011 PAUL BOWSTEIN ad_waq3_921_20100827_010528.txt 0624-01 RUT CORPORATION ad_sade3_10_20100827_010528.txt 0624-01 RUT CORPORATION ... (13 Replies)
Discussion started by: script_op2a
13 Replies

3. Shell Programming and Scripting

AWK script to create max value of 3rd column, grouping by first column

Hi, I need an awk script (or whatever shell-construct) that would take data like below and get the max value of 3 column, when grouping by the 1st column. clientname,day-of-month,max-users ----------------------------------- client1,20120610,5 client2,20120610,2 client3,20120610,7... (3 Replies)
Discussion started by: ckmehta
3 Replies

4. Shell Programming and Scripting

Find duplicates in column 1 and merge their lines (awk?)

Hi, I have a file (sorted by sort) with 8 tab delimited columns. The first column contains duplicated fields and I need to merge all these identical lines. My input file: comp100002 aaa bbb ccc ddd eee fff ggg comp100003 aba aba aba aba aba aba aba comp100003 fff fff fff fff fff fff fff... (5 Replies)
Discussion started by: falcox
5 Replies

5. UNIX for Dummies Questions & Answers

Search word in 3rd column and move it to next column (4th)

Hi, I have a file with +/- 13000 lines and 4 column. I need to search the 3rd column for a word that begins with "SAP-" and move/skip it to the next column (4th). Because the 3rd column need to stay empty. Thanks in advance.:) 89653 36891 OTR-60 SAP-2 89653 36892 OTR-10 SAP-2... (2 Replies)
Discussion started by: AK47
2 Replies

6. Shell Programming and Scripting

Find smallest & largest in every column

Dear All, I have input like this, J_15TEST_ASH05_33A22.13885.txt: $$ 1 MAKE SP1501 1 1 4 6101 7392 2 2442 2685 18 3201 4008 20 120 4158 J_15TEST_ASH05_33A22.13885.txt: $$ 1 MAKE SP1502 1 1 4 5125 6416 2 ... (4 Replies)
Discussion started by: attila
4 Replies

7. Shell Programming and Scripting

Changing values only in 3rd column and 4th column

#cat file testing test! nipw asdkjasjdk ok! what !ok host server1 check_ssh_disk!102.56.1.101!30!50!/ other host server 2 des check_ssh_disk!192.6.1.10!40!30!/ #grep check file| awk -F! '{print $3,$4}'|awk '{gsub($1,"",$1)}1' 50 30 # Output: (6 Replies)
Discussion started by: kenshinhimura
6 Replies

8. Shell Programming and Scripting

Solution for replacement of 4th column with 3rd column in a file using awk/sed preserving delimters

input "A","B","C,D","E","F" "S","T","U,V","W","X" "AA","BB","CC,DD","EEEE","FFF" required output: "A","B","C,D","C,D","F" "S", T","U,V","U,V","X" "AA","BB","CC,DD","CC,DD","FFF" tried using awk but double quotes not preserving for every field. any help to solve this is much... (5 Replies)
Discussion started by: khblts
5 Replies

9. Shell Programming and Scripting

awk to Sum columns when other column has duplicates and append one column value to another with Care

Hi Experts, Please bear with me, i need help I am learning AWk and stuck up in one issue. First point : I want to sum up column value for column 7, 9, 11,13 and column15 if rows in column 5 are duplicates.No action to be taken for rows where value in column 5 is unique. Second point : For... (1 Reply)
Discussion started by: as7951
1 Replies

10. UNIX for Beginners Questions & Answers

UNIX script to compare 3rd column value with first column and display

Hello Team, My source data (INput) is like below EPIC1 router EPIC2 Targetdefinition Exp1 Expres rtr1 Router SQL SrcQual Exp1 Expres rtr1 Router EPIC1 Targetdefinition My output like SQL SrcQual Exp1 Expres Exp1 Expres rtr1 Router rtr1 Router EPIC1 Targetdefinition... (5 Replies)
Discussion started by: sekhar.lsb
5 Replies
LS(1)							      General Commands Manual							     LS(1)

NAME
ls - list contents of directory SYNOPSIS
ls [ -acdfgiloqrstu1ACLFR ] name ... DESCRIPTION
For each directory argument, ls lists the contents of the directory; for each file argument, ls repeats its name and any other information requested. By default, the output is sorted alphabetically. When no argument is given, the current directory is listed. When several arguments are given, the arguments are first sorted appropriately, but file arguments are processed before directories and their contents. There are a large number of options: -l List in long format, giving mode, number of links, owner, size in bytes, and time of last modification for each file. (See below.) If the file is a special file the size field will instead contain the major and minor device numbers. If the file is a symbolic link the pathname of the linked-to file is printed preceded by ``->''. -o Include the file flags in a long (-l) output. -g Include the group ownership of the file in a long output. -t Sort by time modified (latest first) instead of by name. -a List all entries; in the absence of this option, entries whose names begin with a period (.) are not listed. -s Give size in kilobytes of each file. -d If argument is a directory, list only its name; often used with -l to get the status of a directory. -L If argument is a symbolic link, list the file or directory the link references rather than the link itself. -r Reverse the order of sort to get reverse alphabetic or oldest first as appropriate. -u Use time of last access instead of last modification for sorting (with the -t option) and/or printing (with the -l option). -c Use time of file creation for sorting or printing. -i For each file, print the i-number in the first column of the report. -f Output is not sorted. -F cause directories to be marked with a trailing `/', sockets with a trailing `=', symbolic links with a trailing `@', and executable files with a trailing `*'. -R recursively list subdirectories encountered. -1 force one entry per line output format; this is the default when output is not to a terminal. -C force multi-column output; this is the default when output is to a terminal. -q force printing of non-graphic characters in file names as the character `?'; this is the default when output is to a terminal. The mode printed under the -l option contains 11 characters which are interpreted as follows: the first character is d if the entry is a directory; b if the entry is a block-type special file; c if the entry is a character-type special file; l if the entry is a symbolic link; s if the entry is a socket, or - if the entry is a plain file. The next 9 characters are interpreted as three sets of three bits each. The first set refers to owner permissions; the next refers to per- missions to others in the same user-group; and the last to all others. Within each set the three characters indicate permission respec- tively to read, to write, or to execute the file as a program. For a directory, `execute' permission is interpreted to mean permission to search the directory. The permissions are indicated as follows: r if the file is readable; w if the file is writable; x if the file is executable; - if the indicated permission is not granted. The group-execute permission character is given as s if the file has the set-group-id bit set; likewise the user-execute permission charac- ter is given as s if the file has the set-user-id bit set. The last character of the mode (normally `x' or `-') is t if the 1000 bit of the mode is on. See chmod(1) for the meaning of this mode. When the sizes of the files in a directory are listed, a total count of blocks, including indirect blocks is printed. FILES
/etc/passwd to get user id's for `ls -l'. /etc/group to get group id's for `ls -g'. BUGS
Newline and tab are considered printing characters in file names. The output device is assumed to be 80 columns wide. The option setting based on whether the output is a teletype is undesirable as ``ls -s'' is much different than ``ls -s | lpr''. On the other hand, not doing this setting would make old shell scripts which used ls almost certain losers. 3rd Berkeley Distribution December 20, 1994 LS(1)
All times are GMT -4. The time now is 09:06 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy