Removing duplicates in fixed width file which has multiple key columns Post: 302745047

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Combining Two fixed width columns to a variable length file

Hi, I have two files. File1: File1 contains two fixed width columns ID of 15 characters length and Name is of 100 characters length. ID Name 1-43<<11 spaces>>Swapna<<94 spaces>> 1-234<<10 spaces>>Mani<<96 spaces>> 1-3456<<9 spaces>>Kapil<<95 spaces>> File2: ...

2. Shell Programming and Scripting

Removing \n within a fixed width record

I am trying to remove a line feed (\n) within a fixed width record. I tried the tr -d �\n' command, but it also removes the record delimiter. Is there a way to remove the line feed without removing the record delimiter?

3. Shell Programming and Scripting

Removing inserted newlines from a fileld of fixed width file.

Hi champs! I have a fixed width file in which the records appear like this 11111 <fixed spaces such as 6> description for 11111 <fixed spaces such as 6> some more field to the record of 11111 22222 <fixed spaces such as 6> description for 22222 <fixed spaces such as 6> some more field to the...

4. Shell Programming and Scripting

Printing Fixed Width Columns

Hi everyone, I have been working on a pretty laborious shellscript (with bash) the last couple weeks that parses my firewall policies (from a Juniper) for me and creates a nifty little columned output. It does so using awk on a line by line basis to pull out the appropriate pieces of each...

5. UNIX for Dummies Questions & Answers

Remove duplicates based on a column in fixed width file

Hi, How to output the duplicate record to another file. We say the record is duplicate based on a column whose position is from 2 and its length is 11 characters. The file is a fixed width file. ex of Record: DTYU12333567opert tjhi kkklTRG9012 The data in bold is the key on which...

6. UNIX for Dummies Questions & Answers

Removing duplicates based on key

Hi, I have the input file with the below data: 12345|12|34 12345|13|23 3456|12|90 15670|12|13 12345|10|14 3456|12|13 I need to remove the duplicates based on the first field only. I need the output like: 12345|12|34 3456|12|90 15670|12|13 The first field needs to be unique .

7. Shell Programming and Scripting

How to parse fixed-width columns which may include empty fields?

I am trying to selectively display several columns from a db2 query, which gives me a fixed-width output (partial output listed here): --------- -------------------------- ------------ ------ 000 0000000000198012 702 29 000 0000000000198013 ...

8. Shell Programming and Scripting

Remove Duplicates on multiple Key Columns and get the Latest Record from Date/Time Column

Hi Experts , we have a CDC file where we need to get the latest record of the Key columns Key Columns will be CDC_FLAG and SRC_PMTN_I and fetch the latest record from the CDC_PRCS_TS Can we do it with a single awk command. Please help....

9. Shell Programming and Scripting

Removing duplicates from delimited file based on 2 columns

Hi guys,Got a bit of a bind I'm in. I'm looking to remove duplicates from a pipe delimited file, but do so based on 2 columns. Sounds easy enough, but here's the kicker... Column #1 is a simple ID, which is used to identify the duplicate. Once dups are identified, I need to only keep the one...

LEARN ABOUT LINUX

shtool-table

SHTOOL-TABLE.TMP(1)					      GNU Portable Shell Tool					       SHTOOL-TABLE.TMP(1)

NAME

       shtool-table - GNU shtool pretty-print a field-separated list

SYNOPSIS

       shtool table [-F|--field-sep sep] [-w|--width width] [-c|--columns cols] [-s|--strip strip] strsepstr...

DESCRIPTION

       This pretty-prints a list of strings as a table.

OPTIONS

       The following command line options are available.

       -F, --field-sep sep
	   Separate columns using sep. Default is ":".

       -w, --width width
	   Width of each column. Default is 15 characters.

       -c, --columns cols
	   Number of columns. Default is 3.

       -s, --strip strip
	   Strip off any characters past strip. Default is 79.

EXAMPLE

	#   shell script
	shtool table -F , -w 5 -c 4 "1,2,3,4,5,6,7,8,9,10,11,12"

HISTORY

       The GNU shtool table command was originally written by Ralf S.  Engelschall <rse@engelschall.com> in 1999 for GNU shtool.

SEE ALSO

       shtool(1), tr(1), fmt(1), sh(1), awk(1), sed(1).

18-Jul-2008							   shtool 2.0.8 					       SHTOOL-TABLE.TMP(1)