Sponsored Content
Top Forums Shell Programming and Scripting Removing duplicates in fixed width file which has multiple key columns Post 302745043 by saj on Sunday 16th of December 2012 11:47:12 AM
Old 12-16-2012
Removing duplicates in fixed width file which has multiple key columns

Hi All ,

I have a requirement where I need to remove duplicates from a fixed width file which has multiple key columns .Also , need to capture the duplicate records into another file .

File has 8 columns.
Key columns are col1 and col2.
Col1 has the length of 8 col 2 has the length of 3.

Please help...
 

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Combining Two fixed width columns to a variable length file

Hi, I have two files. File1: File1 contains two fixed width columns ID of 15 characters length and Name is of 100 characters length. ID Name 1-43<<11 spaces>>Swapna<<94 spaces>> 1-234<<10 spaces>>Mani<<96 spaces>> 1-3456<<9 spaces>>Kapil<<95 spaces>> File2: ... (4 Replies)
Discussion started by: manneni prakash
4 Replies

2. Shell Programming and Scripting

Removing \n within a fixed width record

I am trying to remove a line feed (\n) within a fixed width record. I tried the tr -d ‘\n' command, but it also removes the record delimiter. Is there a way to remove the line feed without removing the record delimiter? (10 Replies)
Discussion started by: CKT_newbie88
10 Replies

3. Shell Programming and Scripting

Removing inserted newlines from a fileld of fixed width file.

Hi champs! I have a fixed width file in which the records appear like this 11111 <fixed spaces such as 6> description for 11111 <fixed spaces such as 6> some more field to the record of 11111 22222 <fixed spaces such as 6> description for 22222 <fixed spaces such as 6> some more field to the... (8 Replies)
Discussion started by: enigma_1
8 Replies

4. Shell Programming and Scripting

Printing Fixed Width Columns

Hi everyone, I have been working on a pretty laborious shellscript (with bash) the last couple weeks that parses my firewall policies (from a Juniper) for me and creates a nifty little columned output. It does so using awk on a line by line basis to pull out the appropriate pieces of each... (4 Replies)
Discussion started by: cixelsyd
4 Replies

5. UNIX for Dummies Questions & Answers

Remove duplicates based on a column in fixed width file

Hi, How to output the duplicate record to another file. We say the record is duplicate based on a column whose position is from 2 and its length is 11 characters. The file is a fixed width file. ex of Record: DTYU12333567opert tjhi kkklTRG9012 The data in bold is the key on which... (1 Reply)
Discussion started by: Qwerty123
1 Replies

6. UNIX for Dummies Questions & Answers

Removing duplicates based on key

Hi, I have the input file with the below data: 12345|12|34 12345|13|23 3456|12|90 15670|12|13 12345|10|14 3456|12|13 I need to remove the duplicates based on the first field only. I need the output like: 12345|12|34 3456|12|90 15670|12|13 The first field needs to be unique . (4 Replies)
Discussion started by: pandeesh
4 Replies

7. Shell Programming and Scripting

How to parse fixed-width columns which may include empty fields?

I am trying to selectively display several columns from a db2 query, which gives me a fixed-width output (partial output listed here): --------- -------------------------- ------------ ------ 000 0000000000198012 702 29 000 0000000000198013 ... (9 Replies)
Discussion started by: ahsh79
9 Replies

8. Shell Programming and Scripting

Remove Duplicates on multiple Key Columns and get the Latest Record from Date/Time Column

Hi Experts , we have a CDC file where we need to get the latest record of the Key columns Key Columns will be CDC_FLAG and SRC_PMTN_I and fetch the latest record from the CDC_PRCS_TS Can we do it with a single awk command. Please help.... (3 Replies)
Discussion started by: vijaykodukula
3 Replies

9. Shell Programming and Scripting

Removing duplicates from delimited file based on 2 columns

Hi guys,Got a bit of a bind I'm in. I'm looking to remove duplicates from a pipe delimited file, but do so based on 2 columns. Sounds easy enough, but here's the kicker... Column #1 is a simple ID, which is used to identify the duplicate. Once dups are identified, I need to only keep the one... (2 Replies)
Discussion started by: kevinprood
2 Replies
DBIx::Class::Schema::Loader::RelBuilder(3pm)		User Contributed Perl Documentation	      DBIx::Class::Schema::Loader::RelBuilder(3pm)

NAME
DBIx::Class::Schema::Loader::RelBuilder - Builds relationships for DBIx::Class::Schema::Loader SYNOPSIS
See DBIx::Class::Schema::Loader and DBIx::Class::Schema::Loader::Base. DESCRIPTION
This class builds relationships for DBIx::Class::Schema::Loader. This is module is not (yet) for external use. METHODS
new Arguments: $loader object generate_code Arguments: [ [ local_moniker1 (scalar), fk_info1 (arrayref), uniq_info1 (arrayref) ] [ local_moniker2 (scalar), fk_info2 (arrayref), uniq_info2 (arrayref) ] ... ] This generates the code for the relationships of each table. "local_moniker" is the moniker name of the table which had the REFERENCES statements. The fk_info arrayref's contents should take the form: [ { local_table => 'some_table', local_moniker => 'SomeTable', local_columns => [ 'col2', 'col3' ], remote_table => 'another_table_moniker', remote_moniker => 'AnotherTableMoniker', remote_columns => [ 'col5', 'col7' ], }, { local_table => 'some_other_table', local_moniker => 'SomeOtherTable', local_columns => [ 'col1', 'col4' ], remote_table => 'yet_another_table_moniker', remote_moniker => 'YetAnotherTableMoniker', remote_columns => [ 'col1', 'col2' ], }, # ... ], The uniq_info arrayref's contents should take the form: [ [ uniq_constraint_name => [ 'col1', 'col2' ], ], [ another_uniq_constraint_name => [ 'col1', 'col2' ], ], ], This method will return the generated relationships as a hashref keyed on the class names. The values are arrayrefs of hashes containing method name and arguments, like so: { 'Some::Source::Class' => [ { method => 'belongs_to', arguments => [ 'col1', 'Another::Source::Class' ], { method => 'has_many', arguments => [ 'anothers', 'Yet::Another::Source::Class', 'col15' ], ], 'Another::Source::Class' => [ # ... ], # ... } AUTHOR
See "AUTHOR" in DBIx::Class::Schema::Loader and "CONTRIBUTORS" in DBIx::Class::Schema::Loader. LICENSE
This library is free software; you can redistribute it and/or modify it under the same terms as Perl itself. perl v5.14.2 2012-06-08 DBIx::Class::Schema::Loader::RelBuilder(3pm)
All times are GMT -4. The time now is 01:07 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy