Sponsored Content
Top Forums Shell Programming and Scripting Print Only second Duplicate entry in the file Post 302254282 by ravi_rn on Tuesday 4th of November 2008 05:06:19 AM
Old 11-04-2008
Print Only second Duplicate entry in the file

I have file where it contains 2 columns. In two columns the first column is repeated more than once. I wanted to take the unique record in first column
and the corresponding second column value .

The below is the example of the file:

8244100320012955|000b063471a4
8244100320012955|00080e5a6624
8244100330007664|0004bde2ac84
8244100330007664|00080e429228
8244100330011369|0004bd0ec236
8244100330011369|0012c9a1c44a
8244100330016046|00080e0b46c4
8244100330016046|000b06cb18fe

In the above example i need the output as
8244100320012955|00080e5a6624
8244100330007664|00080e429228
8244100330011369|0012c9a1c44a
8244100330016046|000b06cb18fe

I am using the following command
cat f1.txt | cut -d"|" -f 1,2 |sort |uniq -d|wc -l

but the above command is failed

Please help me.

Thank you very much in advance.
 

10 More Discussions You Might Find Interesting

1. HP-UX

Hazardous Duplicate Cron Entry?

Hi All, How to prevent starting of processes that have duplicate entries in cron file, i have written a shell script to validate with "ps |grep" command before starting the process, but still when same process started at same time, it may not be able to detect the existing process. Sample... (3 Replies)
Discussion started by: nag_sundaram
3 Replies

2. UNIX for Dummies Questions & Answers

Remove duplicate entry in one line

Can anyone help me how can i print only the unique entry in a line? MI_AP MI_AP MI_CM MI_MF RC_NAP MBS_AP SF_RAN MBS_AP NT_CAR so that it will on output the one unique entry per line. MI_AP MI_CM MI_MF RC_NAP MBS_AP SF_RAN NT_CAR I can't find the same situation on the knowledge... (5 Replies)
Discussion started by: kharen11
5 Replies

3. Shell Programming and Scripting

Not able to print variable value in crontab entry

Hi, I just stuckup to echo a variable that is assigned in .profile file from a crontab entry. I have a entry in .profile as, CONF_FILE_PATH=/home/test/conf export CONF_FILE_PATH I am able to echo it in command line but I am not able to print $CONF_FILE_PATH when i scheduled it in... (4 Replies)
Discussion started by: naren_0101bits
4 Replies

4. UNIX for Dummies Questions & Answers

Delete duplicate lines and print to file

OK, I have read several things on how to do this, but can't make it work. I am writing this to a vi file then calling it as an awk script. So I need to search a file for duplicate lines, delete duplicate lines, then write the result to another file, say /home/accountant/files/docs/nodup ... (2 Replies)
Discussion started by: bfurlong
2 Replies

5. Shell Programming and Scripting

Need to delete duplicate lease entry

Hi *, I need to delete duplicate lease entries in file according to MAC/IP. I'm having tempfile which contains many lease info and need to have one entry for each IP(not more than that), if it contains more than one entry for same set, need to be deleted that entry... EX: lease... (4 Replies)
Discussion started by: SMNK
4 Replies

6. UNIX for Dummies Questions & Answers

print duplicate number

input chr1 10 100 chr1 10 100 chr1 20 200 output chr1 10 100 2 chr1 20 200 1 (1 Reply)
Discussion started by: repinementer
1 Replies

7. Shell Programming and Scripting

Print duplicate lines

I have a file where some of the lines are duplicates. How do I use bash to print all the lines that have duplicates? (2 Replies)
Discussion started by: locoroco
2 Replies

8. Shell Programming and Scripting

checking duplicate entry in file

Hi i have a file like 110.10 120.10 -1120 110.10 and the lines are having more than 10k. do we have anycommand to check the duplicate entries in the file. I applied the while loop by greping each line with whole file, but it is taking huge amount of time as the file size is large. ... (5 Replies)
Discussion started by: saluja.deepak
5 Replies

9. Shell Programming and Scripting

awk Help: Filter Multiple Entry & print in one line.

AWK Gurus, data: srvhcm01 AZSCI srvhcm01 AZSDB srvhcm01 BZSDB srvhcm01 E2QDI31 srvhcm01 YPDCI srvhcm01 YPDDB srvhcm01 UV2FSCR srvhcm01 UV2FSBI srvhcm01 UV2FSXI srvhcm01 UV2FSUC srvhcm01 UV2FSEP srvhcm01 UV2FSRE srvhcm01 NASCI srvhcm01 NASDB srvhcm01 UV2FSSL srvhcm01 UV2FSDI (7 Replies)
Discussion started by: rveri
7 Replies

10. Shell Programming and Scripting

Deleting duplicate glosses in a dictionary entry

I am working on an Urdu to Hindi dictionary and I have created the following file structure: Headword=Gloss1,Gloss2,Gloss3 i.e. glosses delimited by a comma. It so happens that in some cases (around 6000+ in a file of over 200,000+ the glosses are duplicated. Since this may be a... (3 Replies)
Discussion started by: gimley
3 Replies
COLUMN(1)							   User Commands							 COLUMN(1)

NAME
column - columnate lists SYNOPSIS
column [options] file... DESCRIPTION
The column utility formats its input into multiple columns. Rows are filled before columns. Input is taken from file or, by default, from standard input. Empty lines are ignored. OPTIONS
-c, --columns width Output is formatted to a width specified as number of characters. -t, --table Determine the number of columns the input contains and create a table. Columns are delimited with whitespace, by default, or with the characters supplied using the separator. Table output is useful for pretty-printing. -s, --separator separators Specify possible table delimiters (default is whitespace). -o, --output-separator separators Specify table output delimiter (default is two whitespaces). -x, --fillrows Fill columns before filling rows. -h, --help Print help and exit. ENVIRONMENT
The environment variable COLUMNS is used to determine the size of the screen if no other information is available. EXAMPLES
sed 's/#.*//' /etc/fstab | column -t BUGS
The util-linux version 2.23 changed -s option to be non-greedy, for example: $ printf "a:b:c 1::3 " | column -t -s ':' old output: a b c 1 3 new output (since util-linux 2.23) a b c 1 3 SEE ALSO
colrm(1), ls(1), paste(1), sort(1) HISTORY
The column command appeared in 4.3BSD-Reno. AVAILABILITY
The column command is part of the util-linux package and is available from ftp://ftp.kernel.org/pub/linux/utils/util-linux/. util-linux October 2010 COLUMN(1)
All times are GMT -4. The time now is 06:57 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy