12-16-2012
Removing duplicates in fixed width file which has multiple key columns
Hi All ,
I have a requirement where I need to remove duplicates from a fixed width file which has multiple key columns .Also , need to capture the duplicate records into another file .
File has 8 columns.
Key columns are col1 and col2.
Col1 has the length of 8 col 2 has the length of 3.
Please help...
9 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
Hi, I have two files.
File1:
File1 contains two fixed width columns ID of 15 characters length and Name is of 100 characters length.
ID Name
1-43<<11 spaces>>Swapna<<94 spaces>>
1-234<<10 spaces>>Mani<<96 spaces>>
1-3456<<9 spaces>>Kapil<<95 spaces>>
File2:
... (4 Replies)
Discussion started by: manneni prakash
4 Replies
2. Shell Programming and Scripting
I am trying to remove a line feed (\n) within a fixed width record. I tried the tr -d ‘\n' command, but it also removes the record delimiter. Is there a way to remove the line feed without removing the record delimiter? (10 Replies)
Discussion started by: CKT_newbie88
10 Replies
3. Shell Programming and Scripting
Hi champs!
I have a fixed width file in which the records appear like this
11111 <fixed spaces such as 6> description for 11111 <fixed spaces such as 6> some more field to the record of 11111
22222 <fixed spaces such as 6> description for 22222 <fixed spaces such as 6> some more field to the... (8 Replies)
Discussion started by: enigma_1
8 Replies
4. Shell Programming and Scripting
Hi everyone,
I have been working on a pretty laborious shellscript (with bash) the last couple weeks that parses my firewall policies (from a Juniper) for me and creates a nifty little columned output. It does so using awk on a line by line basis to pull out the appropriate pieces of each... (4 Replies)
Discussion started by: cixelsyd
4 Replies
5. UNIX for Dummies Questions & Answers
Hi,
How to output the duplicate record to another file. We say the record is duplicate based on a column whose position is from 2 and its length is 11 characters.
The file is a fixed width file.
ex of Record:
DTYU12333567opert tjhi kkklTRG9012
The data in bold is the key on which... (1 Reply)
Discussion started by: Qwerty123
1 Replies
6. UNIX for Dummies Questions & Answers
Hi,
I have the input file with the below data:
12345|12|34
12345|13|23
3456|12|90
15670|12|13
12345|10|14
3456|12|13
I need to remove the duplicates based on the first field only.
I need the output like:
12345|12|34
3456|12|90
15670|12|13
The first field needs to be unique . (4 Replies)
Discussion started by: pandeesh
4 Replies
7. Shell Programming and Scripting
I am trying to selectively display several columns from a db2 query, which gives me a fixed-width output (partial output listed here):
--------- -------------------------- ------------ ------
000 0000000000198012 702 29
000 0000000000198013 ... (9 Replies)
Discussion started by: ahsh79
9 Replies
8. Shell Programming and Scripting
Hi Experts ,
we have a CDC file where we need to get the latest record of the Key columns
Key Columns will be CDC_FLAG and SRC_PMTN_I
and fetch the latest record from the CDC_PRCS_TS
Can we do it with a single awk command.
Please help.... (3 Replies)
Discussion started by: vijaykodukula
3 Replies
9. Shell Programming and Scripting
Hi guys,Got a bit of a bind I'm in. I'm looking to remove duplicates from a pipe delimited file, but do so based on 2 columns. Sounds easy enough, but here's the kicker...
Column #1 is a simple ID, which is used to identify the duplicate.
Once dups are identified, I need to only keep the one... (2 Replies)
Discussion started by: kevinprood
2 Replies
LEARN ABOUT X11R4
msguniq
MSGUNIQ(1) GNU MSGUNIQ(1)
NAME
msguniq - unify duplicate translations in message catalog
SYNOPSIS
msguniq [OPTION] [INPUTFILE]
DESCRIPTION
Unifies duplicate translations in a translation catalog. Finds duplicate translations of the same message ID. Such duplicates are invalid
input for other programs like msgfmt, msgmerge or msgcat. By default, duplicates are merged together. When using the --repeated option,
only duplicates are output, and all other messages are discarded. Comments and extracted comments will be cumulated, except that if
--use-first is specified, they will be taken from the first translation. File positions will be cumulated. When using the --unique
option, duplicates are discarded.
Mandatory arguments to long options are mandatory for short options too.
Input file location:
INPUTFILE
input PO file
-D, --directory=DIRECTORY
add DIRECTORY to list for input files search
If no input file is given or if it is -, standard input is read.
Output file location:
-o, --output-file=FILE
write output to specified file
The results are written to standard output if no output file is specified or if it is -.
Message selection:
-d, --repeated
print only duplicates
-u, --unique
print only unique messages, discard duplicates
Input file syntax:
-P, --properties-input
input file is in Java .properties syntax
--stringtable-input
input file is in NeXTstep/GNUstep .strings syntax
Output details:
-t, --to-code=NAME
encoding for output
--use-first
use first available translation for each message, don't merge several translations
-e, --no-escape
do not use C escapes in output (default)
-E, --escape
use C escapes in output, no extended chars
--force-po
write PO file even if empty
-i, --indent
write the .po file using indented style
--no-location
do not write '#: filename:line' lines
-n, --add-location
generate '#: filename:line' lines (default)
--strict
write out strict Uniforum conforming .po file
-p, --properties-output
write out a Java .properties file
--stringtable-output
write out a NeXTstep/GNUstep .strings file
-w, --width=NUMBER
set output page width
--no-wrap
do not break long message lines, longer than the output page width, into several lines
-s, --sort-output
generate sorted output
-F, --sort-by-file
sort output by file location
Informative output:
-h, --help
display this help and exit
-V, --version
output version information and exit
AUTHOR
Written by Bruno Haible.
REPORTING BUGS
Report bugs to <bug-gnu-gettext@gnu.org>.
COPYRIGHT
Copyright (C) 2001-2007 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law.
SEE ALSO
The full documentation for msguniq is maintained as a Texinfo manual. If the info and msguniq programs are properly installed at your
site, the command
info msguniq
should give you access to the complete manual.
GNU gettext-tools 0.17 November 2007 MSGUNIQ(1)