Sponsored Content
Operating Systems Solaris How to remove duplicate records with out sort Post 302171291 by svenkatareddy on Thursday 28th of February 2008 05:09:19 AM
Old 02-28-2008
How to remove duplicate records with out sort

Can any one give me command How to delete duplicate records with out sort.

Suppose if the records like below:

345,bcd,789
123,abc,456
234,abc,456
712,bcd,789

out tput should be
345,bcd,789
123,abc,456

Key for the records is 2nd and 3rd fields.fields are seperated by colon(,).
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Remove all instances of duplicate records from the file

Hi experts, I am new to scripting. I have a requirement as below. File1: A|123|NAME1 A|123|NAME2 B|123|NAME3 File2: C|123|NAME4 C|123|NAME5 D|123|NAME6 1) I have 2 merge both the files. 2) need to do a sort ( key fields are first and second field) 3) remove all the instances... (3 Replies)
Discussion started by: vukkusila
3 Replies

2. Shell Programming and Scripting

How to remove duplicate records with out sort

Can any one give me command How to delete duplicate records with out sort. Suppose if the records like below: 345,bcd,789 123,abc,456 234,abc,456 712,bcd,789 out tput should be 345,bcd,789 123,abc,456 Key for the records is 2nd and 3rd fields.fields are seperated by colon(,). (19 Replies)
Discussion started by: svenkatareddy
19 Replies

3. Shell Programming and Scripting

Remove duplicate records

I want to remove the records based on duplicate. I want to remove if two or more records exists with combination fields. Those records should not come once also file abc.txt ABC;123;XYB;HELLO; ABC;123;HKL;HELLO; CDE;123;LLKJ;HELLO; ABC;123;LSDK;HELLO; CDF;344;SLK;TEST key fields are... (7 Replies)
Discussion started by: svenkatareddy
7 Replies

4. Shell Programming and Scripting

Remove Duplicate Records

Hi frinds, Need your help. item , color ,desc ==== ======= ==== 1,red ,abc 1,red , a b c 2,blue,x 3,black,y 4,brown,xv 4,brown,x v 4,brown, x v I have to elemnet the duplicate rows on the basis of item. the final out put will be 1,red ,abc (6 Replies)
Discussion started by: imipsita.rath
6 Replies

5. Shell Programming and Scripting

Sort and Remove Duplicate on file

How do we sort and remove duplicate on column 1,2 retaining the record with maximum date (in feild 3) for the file with following format. aaa|1234|2010-12-31 aaa|1234|2010-11-10 bbb|345|2011-01-01 ccc|346|2011-02-01 bbb|345|2011-03-10 aaa|1234|2010-01-01 Required Output ... (5 Replies)
Discussion started by: mabarif16
5 Replies

6. Shell Programming and Scripting

Remove somewhat Duplicate records from a flat file

I have a flat file that contains records similar to the following two lines; 1984/11/08 7 700000 123456789 2 1984/11/08 1941/05/19 7 700000 123456789 2 The 123456789 2 represents an account number, this is how I identify the duplicate record. The ### signs represent... (4 Replies)
Discussion started by: jolney
4 Replies

7. Shell Programming and Scripting

Remove duplicate lines based on field and sort

I have a csv file that I would like to remove duplicate lines based on field 1 and sort. I don't care about any of the other fields but I still wanna keep there data intact. I was thinking I could do something like this but I have no idea how to print the full line with this. Please show any method... (8 Replies)
Discussion started by: cokedude
8 Replies

8. Shell Programming and Scripting

Remove duplicate chars and sort string [SED]

Hi, INPUT: DCBADD OUTPUT: ABCD The SED script should alphabetically sort the chars in the string and remove the duplicate chars. (5 Replies)
Discussion started by: jds93
5 Replies

9. Shell Programming and Scripting

Remove duplicate records

Hi, i am working on a script that would remove records or lines in a flat file. The only difference in the file is the "NOT NULL" word. Please see below example of the input file. INPUT FILE:> CREATE a ( TRIAL_CLIENT NOT NULL VARCHAR2(60), TRIAL_FUND NOT NULL... (3 Replies)
Discussion started by: reignangel2003
3 Replies

10. Shell Programming and Scripting

Remove duplicate lines, sort it and save it as file itself

Hi, all I have a csv file that I would like to remove duplicate lines based on 1st field and sort them by the 1st field. If there are more than 1 line which is same on the 1st field, I want to keep the first line of them and remove the rest. I think I have to use uniq or something, but I still... (8 Replies)
Discussion started by: refrain
8 Replies
mdbGeneral(5)							 The m17n Library						     mdbGeneral(5)

NAME
mdbGeneral - General Format DESCRIPTION
The mdatabase_load() function returns the data specified by tags in the form of plist if the first tag is not Mchartable nor Mcharset. The keys of the returned plist are limited to Minteger, Msymbol, Mtext, and Mplist. The type of the value is unambiguously determined by the corresponding key. If the key is Minteger, the value is an integer. If the key is Msymbol, the value is a symbol. And so on. A number of expressions are possible to represent a plist. For instance, we can use the form (K1:V1, K2:V2, ..., Kn:Vn) to represent a plist whose first property key and value are K1 and V1, second key and value are K2 and V2, and so on. However, we can use a simpler expression here because the types of plists used in the m17n database are fairly restricted. Hereafter, we use an expression, which is similar to S-expression, to represent a plist. (Actually, the default database loader of the m17n library is designed to read data files written in this expression.) The expression consists of one or more elements. Each element represents a property, i.e. a single element of a plist. Elements are separated by one or more whitespaces, i.e. a space (code 32), a tab (code 9), or a newline (code 10). Comments begin with a semicolon (;) and extend to the end of the line. The key and the value of each property are determined based on the type of the element as explained below. o INTEGER An element that matches the regular expression -?[0-9]+ or 0[xX][0-9A-Fa-f]+ represents a property whose key is Minteger. An element matching the former expression is interpreted as an integer in decimal notation, and one matching the latter is interpreted as an integer in hexadecimal notation. The value of the property is the result of interpretation. For instance, the element 0xA0 represents a property whose value is 160 in decimal. o SYMBOL An element that matches the regular expression [^-0-9(]([^()]|.)+ represents a property whose key is Msymbol. In the element, , , , and e are replaced with tab (code 9), newline (code 10), carriage return (code 13), and escape (code 27) respectively. Other characters following a backslash is interpreted as it is. The value of the property is the symbol having the resulting string as its name. For instance, the element abc def represents a property whose value is the symbol having the name 'abc def'. o MTEXT An element that matches the regular expression '([^']|')*' represents a property whose key is Mtext. The backslash escape explained above also applies here. r, each part in the element matching the regular expression [xX][0-9A-Fa-f][0-9A-Fa-f] is replaced with its hexadecimal interpretation. After having resolved the backslash escapes, the byte sequence between the double quotes is interpreted as a UTF-8 sequence and decoded into an M-text. This M-text is the value of the property. o PLIST Zero or more elements surrounded by a pair of parentheses represent a property whose key is Mplist. Whitespaces before and after a parenthesis can be omitted. The value of the property is a plist, which is the result of recursive interpretation of the elements between the parentheses. SYNTAX NOTATION
In an explanation of a plist format of data, a BNF-like notation is used. In the notation, non-terminals are represented by a string of uppercase letters (including '-' in the middle), terminals are represented by a string surrounded by '"'. Special non-terminals INTEGER, SYMBOL, MTEXT and PLIST represents property integer, symbol, M-text, or plist respectively. EXAMPLE
Here is an example of database data that is read into a plist of this simple format: DATA-FORMAT ::= [ INTEGER | SYMBOL | MTEXT | FUNC ] * FUNC ::= '(' FUNC-NAME FUNC-ARG * ')' FUNC-NAME ::= SYMBOL FUNC-ARG ::= INTEGER | SYMBOL | MTEXT | '(' FUNC-ARG ')' For instance, a data file that contains this text matches the above syntax: abc 123 (pqr 0xff) "m and is read into this plist: 1st element: key: Msymbol, value: abc 2nd element: key: Minteger, value: 123 3rd element: key: Mplist, value: a plist of these elements: 1st element: key Msymbol, value: pgr 2nd element: key Minteger, value: 255 4th element: key: Mtext, value: m"text 5th element: key: Mplist, value: a plist of these elements: 1st element: key: Msymbol, value: __ 2nd element: key: Mplist, value: a plist of these elements: 1st element: key: Mtext, value: string 2nd element: key: Msymbol, value: xyz 3rd element: key: Minteger, value: -456 COPYRIGHT
Copyright (C) 2001 Information-technology Promotion Agency (IPA) Copyright (C) 2001-2011 National Institute of Advanced Industrial Science and Technology (AIST) Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License <http://www.gnu.org/licenses/fdl.html>. Version 1.6.2 12 Jan 2011 mdbGeneral(5)
All times are GMT -4. The time now is 01:48 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy