I am doing KSH script to remove duplicate lines in a file. Let say the file has format below.
FileA
1253-6856
3101-4011
1827-1356
1822-1157
1822-1157
1000-1410
1000-1410
1822-1231
1822-1231
3101-4011
1822-1157
1822-1231
and I want to simply it with no duplicate line as file... (5 Replies)
I have a log file "logreport" that contains several lines as seen below:
04:20:00 /usr/lib/snmp/snmpdx: Agent snmpd appeared dead but responded to ping
06:38:08 /usr/lib/snmp/snmpdx: Agent snmpd appeared dead but responded to ping
07:11:05 /usr/lib/snmp/snmpdx: Agent snmpd appeared dead but... (18 Replies)
Hi please help me how to remove duplicate lines in any file.
I have a file having huge number of lines.
i want to remove selected lines in it.
And also if there exists duplicate lines, I want to delete the rest & just keep one of them.
Please help me with any unix commands or even fortran... (7 Replies)
greetings,
i'm hoping there is a way to cat a file, remove duplicate lines and send that output to a new file. the file will always vary but be something similar to this:
please keep in mind that the above could be eight occurrences of each hostname or it might simply have another four of an... (2 Replies)
Hi,
I have two files with below data::
file1:-
123|aaa|ppp
445|fff|yyy
999|ttt|jjj
555|hhh|hhh
file2:-
445|fff|yyy
555|hhh|hhh
The records present in file1, not present in file 2 should be writtent to the out put file.
output:-
123|aaa|ppp
999|ttt|jjj
Is there any one line... (3 Replies)
Hey guys, need some help to fix this script. I am trying to remove all the duplicate lines in this file.
I wrote the following script, but does not work. What is the problem?
The output file should only contain five lines:
Later! (5 Replies)
Hi,
I have a csv file which contains some millions of lines in it.
The first line(Header) repeats at every 50000th line. I want to remove all the duplicate headers from the second occurance(should not remove the first line).
I don't want to use any pattern from the Header as I have some... (7 Replies)
Dear community,
I have to remove duplicate lines from a file contains a very big ammount of rows (milions?) based on 1st and 3rd columns
The data are like this:
Region 23/11/2014 09:11:36 41752
Medio 23/11/2014 03:11:38 4132
Info 23/11/2014 05:11:09 4323... (2 Replies)
Hi, all
I have a csv file that I would like to remove duplicate lines based on 1st field and sort them by the 1st field. If there are more than 1 line which is same on the 1st field, I want to keep the first line of them and remove the rest. I think I have to use uniq or something, but I still... (8 Replies)
Discussion started by: refrain
8 Replies
LEARN ABOUT NETBSD
msguniq
MSGUNIQ(1) GNU MSGUNIQ(1)NAME
msguniq - unify duplicate translations in message catalog
SYNOPSIS
msguniq [OPTION] [INPUTFILE]
DESCRIPTION
Unifies duplicate translations in a translation catalog. Finds duplicate translations of the same message ID. Such duplicates are invalid
input for other programs like msgfmt, msgmerge or msgcat. By default, duplicates are merged together. When using the --repeated option,
only duplicates are output, and all other messages are discarded. Comments and extracted comments will be cumulated, except that if
--use-first is specified, they will be taken from the first translation. File positions will be cumulated. When using the --unique
option, duplicates are discarded.
Mandatory arguments to long options are mandatory for short options too.
Input file location:
INPUTFILE
input PO file
-D, --directory=DIRECTORY
add DIRECTORY to list for input files search
If no input file is given or if it is -, standard input is read.
Output file location:
-o, --output-file=FILE
write output to specified file
The results are written to standard output if no output file is specified or if it is -.
Message selection:
-d, --repeated
print only duplicates
-u, --unique
print only unique messages, discard duplicates
Input file syntax:
-P, --properties-input
input file is in Java .properties syntax
--stringtable-input
input file is in NeXTstep/GNUstep .strings syntax
Output details:
-t, --to-code=NAME
encoding for output
--use-first
use first available translation for each message, don't merge several translations
-e, --no-escape
do not use C escapes in output (default)
-E, --escape
use C escapes in output, no extended chars
--force-po
write PO file even if empty
-i, --indent
write the .po file using indented style
--no-location
do not write '#: filename:line' lines
-n, --add-location
generate '#: filename:line' lines (default)
--strict
write out strict Uniforum conforming .po file
-p, --properties-output
write out a Java .properties file
--stringtable-output
write out a NeXTstep/GNUstep .strings file
-w, --width=NUMBER
set output page width
--no-wrap
do not break long message lines, longer than the output page width, into several lines
-s, --sort-output
generate sorted output
-F, --sort-by-file
sort output by file location
Informative output:
-h, --help
display this help and exit
-V, --version
output version information and exit
AUTHOR
Written by Bruno Haible.
REPORTING BUGS
Report bugs to <bug-gnu-gettext@gnu.org>.
COPYRIGHT
Copyright (C) 2001-2005 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICU-
LAR PURPOSE.
SEE ALSO
The full documentation for msguniq is maintained as a Texinfo manual. If the info and msguniq programs are properly installed at your
site, the command
info msguniq
should give you access to the complete manual.
GNU gettext-tools 0.14.4 April 2005 MSGUNIQ(1)