Removing duplicates in fixed width file which has multiple key columns Post: 302745161

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Combining Two fixed width columns to a variable length file

Hi, I have two files. File1: File1 contains two fixed width columns ID of 15 characters length and Name is of 100 characters length. ID Name 1-43<<11 spaces>>Swapna<<94 spaces>> 1-234<<10 spaces>>Mani<<96 spaces>> 1-3456<<9 spaces>>Kapil<<95 spaces>> File2: ...

2. Shell Programming and Scripting

Removing \n within a fixed width record

I am trying to remove a line feed (\n) within a fixed width record. I tried the tr -d �\n' command, but it also removes the record delimiter. Is there a way to remove the line feed without removing the record delimiter?

3. Shell Programming and Scripting

Removing inserted newlines from a fileld of fixed width file.

Hi champs! I have a fixed width file in which the records appear like this 11111 <fixed spaces such as 6> description for 11111 <fixed spaces such as 6> some more field to the record of 11111 22222 <fixed spaces such as 6> description for 22222 <fixed spaces such as 6> some more field to the...

4. Shell Programming and Scripting

Printing Fixed Width Columns

Hi everyone, I have been working on a pretty laborious shellscript (with bash) the last couple weeks that parses my firewall policies (from a Juniper) for me and creates a nifty little columned output. It does so using awk on a line by line basis to pull out the appropriate pieces of each...

5. UNIX for Dummies Questions & Answers

Remove duplicates based on a column in fixed width file

Hi, How to output the duplicate record to another file. We say the record is duplicate based on a column whose position is from 2 and its length is 11 characters. The file is a fixed width file. ex of Record: DTYU12333567opert tjhi kkklTRG9012 The data in bold is the key on which...

6. UNIX for Dummies Questions & Answers

Removing duplicates based on key

Hi, I have the input file with the below data: 12345|12|34 12345|13|23 3456|12|90 15670|12|13 12345|10|14 3456|12|13 I need to remove the duplicates based on the first field only. I need the output like: 12345|12|34 3456|12|90 15670|12|13 The first field needs to be unique .

7. Shell Programming and Scripting

How to parse fixed-width columns which may include empty fields?

I am trying to selectively display several columns from a db2 query, which gives me a fixed-width output (partial output listed here): --------- -------------------------- ------------ ------ 000 0000000000198012 702 29 000 0000000000198013 ...

8. Shell Programming and Scripting

Remove Duplicates on multiple Key Columns and get the Latest Record from Date/Time Column

Hi Experts , we have a CDC file where we need to get the latest record of the Key columns Key Columns will be CDC_FLAG and SRC_PMTN_I and fetch the latest record from the CDC_PRCS_TS Can we do it with a single awk command. Please help....

9. Shell Programming and Scripting

Removing duplicates from delimited file based on 2 columns

Hi guys,Got a bit of a bind I'm in. I'm looking to remove duplicates from a pipe delimited file, but do so based on 2 columns. Sounds easy enough, but here's the kicker... Column #1 is a simple ID, which is used to identify the duplicate. Once dups are identified, I need to only keep the one...

LEARN ABOUT DEBIAN

tcfmgr

TCFMGR(1)							   Tokyo Cabinet							 TCFMGR(1)

NAME

       tcfmgr - the command line utility of the fixed-length database API

DESCRIPTION

       The  command `tcfmgr' is a utility for test and debugging of the fixed-length database API and its applications.  `path' specifies the path
       of a database file.  `width' specifies the width of the value of each record.  `limsiz' specifies the limit  size  of  the  database  file.
       `key' specifies the key of a record.  `value' specifies the value of a record.  `file' specifies the input file.

	      tcfmgr create path [width [limsiz]]
		     Create a database file.
	      tcfmgr inform [-nl|-nb] path
		     Print miscellaneous information to the standard output.
	      tcfmgr put [-nl|-nb] [-sx] [-dk|-dc|-dai|-dad] path key value
		     Store a record.
	      tcfmgr out [-nl|-nb] [-sx] path key
		     Remove a record.
	      tcfmgr get [-nl|-nb] [-sx] [-px] [-pz] path key
		     Print the value of a record.
	      tcfmgr list [-nl|-nb] [-m num] [-pv] [-px] [-rb lkey ukey] [-ri str] path
		     Print keys of all records, separated by line feeds.
	      tcfmgr optimize [-nl|-nb] path [width [limsiz]]
		     Optimize a database file.
	      tcfmgr importtsv [-nl|-nb] [-sc] path [file]
		     Store records of TSV in each line of a file.
	      tcfmgr version
		     Print the version information of Tokyo Cabinet.

       Options feature the following.

	      -nl : enable the option `FDBNOLCK'.
	      -nb : enable the option `FDBLCKNB'.
	      -sx : the input data is evaluated as a hexadecimal data string.
	      -dk : use the function `tcfdbputkeep' instead of `tcfdbput'.
	      -dc : use the function `tcfdbputcat' instead of `tcfdbput'.
	      -dai : use the function `tcfdbaddint' instead of `tcfdbput'.
	      -dad : use the function `tcfdbadddouble' instead of `tcfdbput'.
	      -px : the output data is converted into a hexadecimal data string.
	      -pz : do not append line feed at the end of the output.
	      -m num : specify the maximum number of the output.
	      -pv : print values of records also.
	      -rb lkey ukey : specify the range of keys.
	      -ri str : specify the interval notation of keys.
	      -sc : normalize keys as lower cases.

       This command returns 0 on success, another on failure.

SEE ALSO

       tcftest(1), tcfmttest(1), tcfdb(3), tokyocabinet(3)

Man Page							    2011-02-12								 TCFMGR(1)