Sponsored Content
Top Forums Shell Programming and Scripting How to extract duplicate rows Post 302159367 by bobbygsk on Thursday 17th of January 2008 03:20:00 PM
Old 01-17-2008
How to extract duplicate rows

I have searched the internet for duplicate row extracting.
All I have seen is extracting good rows or eliminating duplicate rows.

How do I extract duplicate rows from a flat file in unix.
I'm using Korn shell on HP Unix.

For.eg.
FlatFile.txt
========
123:456:678
123:456:678

123:456:876
345:457:987
345:457:987

345:123:745

The output should be
OutPutFile.txt
============
123:456:678
345:457:987

I appreciate your help in advance. Thanks
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Extract duplicate fields in rows

I have a input file with formating: 6000000901 ;36200103 ;h3a01f496 ; 2000123605 ;36218982 ;heefa1328 ; 2000273132 ;36246985 ;h08c5cb71 ; 2000041207 ;36246985 ;heef75497 ; Each fields is seperated by semi-comma. Sometime, the second files is... (6 Replies)
Discussion started by: anhtt
6 Replies

2. Shell Programming and Scripting

duplicate rows in a file

hi all can anyone please let me know if there is a way to find out duplicate rows in a file. i have a file that has hundreds of numbers(all in next row). i want to find out the numbers that are repeted in the file. eg. 123434 534 5575 4746767 347624 5575 i want 5575 please help (3 Replies)
Discussion started by: infyanurag
3 Replies

3. HP-UX

How to get Duplicate rows in a file

Hi all, I have written one shell script. The output file of this script is having sql output. In that file, I want to extract the rows which are having multiple entries(duplicate rows). For example, the output file will be like the following way. ... (7 Replies)
Discussion started by: raghu.iv85
7 Replies

4. Shell Programming and Scripting

How to extract duplicate rows

Hi! I have a file as below: line1 line2 line2 line3 line3 line3 line4 line4 line4 line4 I would like to extract duplicate lines (not unique, triplicate or quadruplicate lines). Output will be as below: line2 line2 I would appreciate if anyone can help. Thanks. (4 Replies)
Discussion started by: chromatin
4 Replies

5. Shell Programming and Scripting

Duplicate rows in a text file

notes: i am using cygwin and notepad++ only for checking this and my OS is XP. #!/bin/bash typeset -i totalvalue=(wc -w /cygdrive/c/cygwinfiles/database.txt) typeset -i totallines=(wc -l /cygdrive/c/cygwinfiles/database.txt) typeset -i columnlines=`expr $totalvalue / $totallines` awk -F' ' -v... (5 Replies)
Discussion started by: whitecross
5 Replies

6. Shell Programming and Scripting

Delete duplicate rows

Hi, This is a followup to my earlier post him mno klm 20 76 . + . klm_mango unix_00000001; alp fdc klm 123 456 . + . klm_mango unix_0000103; her tkr klm 415 439 . + . klm_mango unix_00001043; abc tvr klm 20 76 . + . klm_mango unix_00000001; abc def klm 83 84 . + . klm_mango... (5 Replies)
Discussion started by: jacobs.smith
5 Replies

7. Shell Programming and Scripting

Extract and count number of Duplicate rows

Hi All, I need to extract duplicate rows from a file and write these bad records into another file. And need to have a count of these bad records. i have a command awk ' {s++} END { for(i in s) { if(s>1) { print i } } }' ${TMP_DUPE_RECS}>>${TMP_BAD_DATA_DUPE_RECS}... (5 Replies)
Discussion started by: Arun Mishra
5 Replies

8. Shell Programming and Scripting

How to duplicate rows using awk or any other method?

I want to duplicate each row in my file Egfile.txt Name State Age Jack NJ 34 John MA 23 Jessica FL 45 I want the code to produce this output Name State Age Jack NJ 34 Jack NJ 34 John MA 23 John MA 23 Jessica FL 45 Jessica FL 45 (6 Replies)
Discussion started by: sidnow
6 Replies

9. Shell Programming and Scripting

Extract duplicate rows with conditions

Gents Can you help please. Input file 5490921425 1 7 1310342 54909214251 5490921425 2 1 1 54909214252 5491120937 1 1 3 54911209371 5491120937 3 1 1 54911209373 5491320785 1 ... (4 Replies)
Discussion started by: jiam912
4 Replies

10. Shell Programming and Scripting

Extract and exclude rows based on duplicate values

Hello I have a file like this: > cat examplefile ghi|NN603762|eee mno|NN607265|ttt pqr|NN613879|yyy stu|NN615002|uuu jkl|NN607265|rrr vwx|NN615002|iii yzA|NN618555|ooo def|NN190486|www BCD|NN628717|ppp abc|NN190486|qqq EFG|NN628717|aaa HIJ|NN628717|sss > I can sort the file by... (5 Replies)
Discussion started by: CHoggarth
5 Replies
strfmon(3C)						   Standard C Library Functions 					       strfmon(3C)

NAME
strfmon - convert monetary value to string SYNOPSIS
#include <monetary.h> ssize_t strfmon(char *restrict s, size_t maxsize, const char *restrict format, ...); DESCRIPTION
The strfmon() function places characters into the array pointed to by s as controlled by the string pointed to by format. No more than max- size bytes are placed into the array. The format is a character string that contains two types of objects: plain characters, which are simply copied to the output stream, and conversion specifications, each of which results in the fetching of zero or more arguments which are converted and formatted. The results are undefined if there are insufficient arguments for the format. If the format is exhausted while arguments remain, the excess arguments are simply ignored. A conversion specification consists of the following sequence: o a % character o optional flags o optional field width o optional left precision o optional right precision o a required conversion character that determines the conversion to be performed. Flags One or more of the following optional flags can be specified to control the conversion: =f An = followed by a single character f which is used as the numeric fill character. The fill character must be representable in a single byte in order to work with precision and width counts. The default numeric fill character is the space character. This flag does not affect field width filling which always uses the space character. This flag is ignored unless a left precision (see below) is specified. ^ Do not format the currency amount with grouping characters. The default is to insert the grouping characters if defined for the current locale. + or ( Specify the style of representing positive and negative currency amounts. Only one of `+' or `(' may be specified. If `+' is spec- ified, the locale's equivalent of + and `-' are used (for example, in the U.S.A.: the empty string if positive and `-' if nega- tive). If `(' is specified, negative amounts are enclosed within parentheses. If neither flag is specified, the `+' style is used. ! Suppress the currency symbol from the output conversion. - Specify the alignment. If this flag is present all fields are left-justified (padded to the right) rather than right-justified. Field Width w A decimal digit string w specifying a minimum field width in bytes in which the result of the conversion is right-justified (or left-justified if the flag `-' is specified). The default is zero. Left Precision #n A `#' followed by a decimal digit string n specifying a maximum number of digits expected to be formatted to the left of the radix character. This option can be used to keep the formatted output from multiple calls to the strfmon() aligned in the same columns. It can also be used to fill unused positions with a special character as in $***123.45. This option causes an amount to be format- ted as if it has the number of digits specified by n. If more than n digit positions are required, this conversion specification is ignored. Digit positions in excess of those actually required are filled with the numeric fill character (see the =f flag above). If grouping has not been suppressed with the `^' flag, and it is defined for the current locale, grouping separators are inserted before the fill characters (if any) are added. Grouping separators are not applied to fill characters even if the fill character is a digit. To ensure alignment, any characters appearing before or after the number in the formatted output such as currency or sign symbols are padded as necessary with space characters to make their positive and negative formats an equal length. Right Precision .p A period followed by a decimal digit string p specifying the number of digits after the radix character. If the value of the right precision p is zero, no radix character appears. If a right precision is not included, a default specified by the current locale is used. The amount being formatted is rounded to the specified number of digits prior to formatting. Conversion Characters The conversion characters and their meanings are: i The double argument is formatted according to the locale's international currency format (for example, in the U.S.A.: USD 1,234.56). n The double argument is formatted according to the locale's national currency format (for example, in the U.S.A.: $1,234.56). % Convert to a %; no argument is converted. The entire conversion specification must be %%. Locale Information The LC_MONETARY category of the program's locale affects the behavior of this function including the monetary radix character (which may be different from the numeric radix character affected by the LC_NUMERIC category), the grouping separator, the currency symbols and formats. The international currency symbol should be in conformance with the ISO 4217: 1987 standard. RETURN VALUES
If the total number of resulting bytes (including the terminating null byte) is not more than maxsize, strfmon() returns the number of bytes placed into the array pointed to by s, not including the terminating null byte. Otherwise, -1 is returned, the contents of the array are indeterminate, and errno is set to indicate the error. ERRORS
The strfmon() function will fail if: ENOSYS The function is not supported. E2BIG Conversion stopped due to lack of space in the buffer. USAGE
The behavior of strfmon() in an SUSv3-conforming application differs from its behavior in a non-conforming application as follows: o With the conversion 'i', strfmon() uses information set to int_p_cs_precedes, int_n_cs_precedes, int_p_sep_by_space, int_n_sep_by_space, int_p_sign_posn, and int_n_sign_posn of the current locale instead of p_cs_precedes, n_cs_precedes, p_sep_by_space, n_sep_by_space, p_sign_posn, and n_sign_posn, respectively. o With the conversion 'i', strfmon() uses the fourth character of the string set to int_curr_symbol of the current locale instead of a space for int_p_sep_by_space and int_n_sep_by_space. o When the value of p_sep_by_space, n_sep_by_space, int_p_sep_by_space, or int_n_sep_by_space is set to 2 in the current locale, strf- mon() separates the currency symbol from the sign string by a space, if adjacent; otherwise, strfmon() separates the sign string from the value by a space. EXAMPLES
Example 1: A sample output of strfmon(). Given a locale for the U.S.A. and the values 123.45, -123.45, and 3456.781: +------------------------------------------------------------------------+ | Conversion Output Comments | |Specification | |%n $123.45 default formatting | | -$123.45 | | $3,456.78 | |%11n $123.45 right align within an 11 | | -$123.45 character field | | $3,456.78 | |%#5n $123.45 aligned columns for values | | -$123.45 up to 99,999 | | $3,456.78 | |%=*#5n $***123.45 specify a fill character | | -$***123.45 | | $*3,456.78 | |%=0#5n $000123.45 fill characters do not use | | -$000123.45 grouping even if the fill | | $03,456.78 character is a digit | |%^#5n $123.45 disable the grouping | | -$123.45 separator | | $3456.78 | |%^#5.0n $123 round off to whole units | | -$123 | | $3457 | |%^#5.4n $123.4500 increase the precision | | -$123.4500 | | $3456.7810 | |%(#5n 123.45 use an alternative | | ($123.45) pos/neg style | | $3,456.78 | |%!(#5n 123.45 disable the currency | | (123.45) symbol | | 3,456.78 | +------------------------------------------------------------------------+ ATTRIBUTES
See attributes(5) for descriptions of the following attributes: +-----------------------------+-----------------------------+ | ATTRIBUTE TYPE | ATTRIBUTE VALUE | +-----------------------------+-----------------------------+ |CSI |Enabled | +-----------------------------+-----------------------------+ |Interface Stability |Standard | +-----------------------------+-----------------------------+ |MT-Level |MT-Safe with exceptions | +-----------------------------+-----------------------------+ Th strfmon() function can be used safely in multithreaded applications, as long as setlocale(3C) is not called to change the locale. SEE ALSO
localeconv(3C), setlocale(3C), attributes(5), standards(5) SunOS 5.10 1 Nov 2003 strfmon(3C)
All times are GMT -4. The time now is 06:23 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy