Sponsored Content
Top Forums Shell Programming and Scripting Removing duplicates in fixed width file which has multiple key columns Post 302745045 by Don Cragun on Sunday 16th of December 2012 11:56:31 AM
Old 12-16-2012
Please give a sample input file (showing field contents and separators), and provide the outputs that you expect to get from that input. Please use code tags when you post the input and output files.
 

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Combining Two fixed width columns to a variable length file

Hi, I have two files. File1: File1 contains two fixed width columns ID of 15 characters length and Name is of 100 characters length. ID Name 1-43<<11 spaces>>Swapna<<94 spaces>> 1-234<<10 spaces>>Mani<<96 spaces>> 1-3456<<9 spaces>>Kapil<<95 spaces>> File2: ... (4 Replies)
Discussion started by: manneni prakash
4 Replies

2. Shell Programming and Scripting

Removing \n within a fixed width record

I am trying to remove a line feed (\n) within a fixed width record. I tried the tr -d ‘\n' command, but it also removes the record delimiter. Is there a way to remove the line feed without removing the record delimiter? (10 Replies)
Discussion started by: CKT_newbie88
10 Replies

3. Shell Programming and Scripting

Removing inserted newlines from a fileld of fixed width file.

Hi champs! I have a fixed width file in which the records appear like this 11111 <fixed spaces such as 6> description for 11111 <fixed spaces such as 6> some more field to the record of 11111 22222 <fixed spaces such as 6> description for 22222 <fixed spaces such as 6> some more field to the... (8 Replies)
Discussion started by: enigma_1
8 Replies

4. Shell Programming and Scripting

Printing Fixed Width Columns

Hi everyone, I have been working on a pretty laborious shellscript (with bash) the last couple weeks that parses my firewall policies (from a Juniper) for me and creates a nifty little columned output. It does so using awk on a line by line basis to pull out the appropriate pieces of each... (4 Replies)
Discussion started by: cixelsyd
4 Replies

5. UNIX for Dummies Questions & Answers

Remove duplicates based on a column in fixed width file

Hi, How to output the duplicate record to another file. We say the record is duplicate based on a column whose position is from 2 and its length is 11 characters. The file is a fixed width file. ex of Record: DTYU12333567opert tjhi kkklTRG9012 The data in bold is the key on which... (1 Reply)
Discussion started by: Qwerty123
1 Replies

6. UNIX for Dummies Questions & Answers

Removing duplicates based on key

Hi, I have the input file with the below data: 12345|12|34 12345|13|23 3456|12|90 15670|12|13 12345|10|14 3456|12|13 I need to remove the duplicates based on the first field only. I need the output like: 12345|12|34 3456|12|90 15670|12|13 The first field needs to be unique . (4 Replies)
Discussion started by: pandeesh
4 Replies

7. Shell Programming and Scripting

How to parse fixed-width columns which may include empty fields?

I am trying to selectively display several columns from a db2 query, which gives me a fixed-width output (partial output listed here): --------- -------------------------- ------------ ------ 000 0000000000198012 702 29 000 0000000000198013 ... (9 Replies)
Discussion started by: ahsh79
9 Replies

8. Shell Programming and Scripting

Remove Duplicates on multiple Key Columns and get the Latest Record from Date/Time Column

Hi Experts , we have a CDC file where we need to get the latest record of the Key columns Key Columns will be CDC_FLAG and SRC_PMTN_I and fetch the latest record from the CDC_PRCS_TS Can we do it with a single awk command. Please help.... (3 Replies)
Discussion started by: vijaykodukula
3 Replies

9. Shell Programming and Scripting

Removing duplicates from delimited file based on 2 columns

Hi guys,Got a bit of a bind I'm in. I'm looking to remove duplicates from a pipe delimited file, but do so based on 2 columns. Sounds easy enough, but here's the kicker... Column #1 is a simple ID, which is used to identify the duplicate. Once dups are identified, I need to only keep the one... (2 Replies)
Discussion started by: kevinprood
2 Replies
LINTIAN-INFO(1) 					User Contributed Perl Documentation					   LINTIAN-INFO(1)

NAME
lintian-info - give detailed information about Lintian's error tags SYNOPSIS
lintian-info [log-file...] lintian-info --tags tag ... DESCRIPTION
The lintian-info command parses the output of the lintian command and gives verbose information about the listed Lintian error tags, parses a Lintian override file and gives verbose information about the tags included, or (if given the -t or --tags option) explains a given tag or tags. If no log-file is specified on the command line, this command expects its input on stdin. Thus, the output of lintian can either be piped through lintian-info or a log file produced by lintian can be processed with this command. (Note, though, that the lintian command has a command line option -i to display the same results as lintian-info, so you will not normally need to pipe the output of lintian into this command.) OPTIONS
-a, --annotate Read from standard input or any files specified on the command line and search the input for lines formatted like Lintian override entries. For each one that was found, display verbose information about that tag. -h, --help Display usage information and exit. --profile prof Use the severities from the vendor profile prof when displaying tags. If the profile name does not contain a slash, the default profile for than vendor is chosen. If not specified, lintian-info loads the best profile for the current vendor. Please Refer to the Lintian User Manual for the full documentation of profiles. -t, --tags Rather than treating them as log file names, treat any command-line options as tag names and display the descriptions of each tag. EXIT STATUS
If -t or --tags was given and one or more of the tags specified were unknown, this command returns the exit code 1. Otherwise, it always returns with exit code 0. SEE ALSO
lintian(1) AUTHORS
Niels Thykier <niels@thykier.net> Richard Braakman <dark@xs4all.nl> Christian Schwarz <schwarz@monet.m.isar.de> perl v5.14.2 2013-02-16 LINTIAN-INFO(1)
All times are GMT -4. The time now is 12:35 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy