need Shell script for Sort BASED ON FIRST FIELD and PRINT THE WHOLE FILE WITHOUT DUPLICATES Post: 302409024

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Shell script to remove duplicates lines in a file

Hi, I am writing a shell script that needs to remove duplicate lines within a file by category. example: section a a c b a section b a b a c I need to remove the duplicates within th category with out removing the duplicates from the 2 different sections (one of the a's in section...

2. Shell Programming and Scripting

Remove duplicate lines based on field and sort

I have a csv file that I would like to remove duplicate lines based on field 1 and sort. I don't care about any of the other fields but I still wanna keep there data intact. I was thinking I could do something like this but I have no idea how to print the full line with this. Please show any method...

3. UNIX for Dummies Questions & Answers

remove duplicates based on a field and criteria

Hi, I have a file with fields like below: A;XYZ;102345;222 B;XYZ;123243;333 C;ABC;234234;444 D;MNO;103345;222 E;DEF;124243;333 desired output: C;ABC;234234;444 D;MNO;103345;222 E;DEF;124243;333 ie, if the 4rth field is a duplicate.. i need only those records where...

4. Shell Programming and Scripting

Remove duplicates based on a field's value

Hi All, I have a text file with three columns. I would like a simple script that removes lines in which column 1 has duplicate entries, but use the largest value in column 3 to decide which one to keep. For example: Input file: 12345a rerere.rerere len=23 11111c fsdfdf.dfsdfdsf len=33 ...

5. Shell Programming and Scripting

Trying to remove duplicates based on field and row

I am trying to see if I can use awk to remove duplicates from a file. This is the file: -==> Listvol <== deleting /vol/eng_rmd_0941 deleting /vol/eng_rmd_0943 deleting /vol/eng_rmd_0943 deleting /vol/eng_rmd_1006 deleting /vol/eng_rmd_1012 rearrange /vol/eng_rmd_0943 ...

6. Shell Programming and Scripting

Help in modifying a PERL script to sort Singletons and Duplicates

I have a large database which has the following structure a=b where a is one language and b is the other and = is the delimiter Since the data treats of language, homographs occur i.e. the same word on the left hand side can map in two different entries to two different glosses on the right...

7. Shell Programming and Scripting

Command/script to match a field and print the next field of each line in a file.

Hello, I have a text file in the below format: Source Destination State Lag Status CQA02W2K12pl:D:\CAQA ...

8. UNIX for Beginners Questions & Answers

Sort and remove duplicates in directory based on first 5 columns:

I have /tmp dir with filename as: 010020001_S-FOR-Sort-SYEXC_20160229_2212101.marker 010020001_S-FOR-Sort-SYEXC_20160229_2212102.marker 010020001-S-XOR-Sort-SYEXC_20160229_2212104.marker 010020001-S-XOR-Sort-SYEXC_20160229_2212105.marker 010020001_S-ZOR-Sort-SYEXC_20160229_2212106.marker...

9. Shell Programming and Scripting

Using awk to print output based on first field.

Hi Folks, I have one requirement, There is one file, which contains two fields. Based on first field, I need to print an output. Example will be more suitable. Input file like this. abc 5 abc 10 xyz 6 xyz 9 xyz 10 mnp 10 mnp 12 mnp 6

LEARN ABOUT REDHAT

msguniq

MSGUNIQ(1)								GNU								MSGUNIQ(1)

NAME

       msguniq - unify duplicate translations in message catalog

SYNOPSIS

       msguniq [OPTION] [INPUTFILE]

DESCRIPTION

       Unifies duplicate translations in a translation catalog.  Finds duplicate translations of the same message ID.  Such duplicates are invalid
       input for other programs like msgfmt, msgmerge or msgcat.  By default, duplicates are merged together.  When using the  --repeated  option,
       only  duplicates  are  output,  and  all  other	messages are discarded.  Comments and extracted comments will be cumulated, except that if
       --use-first is specified, they will be taken from the first translation.  File positions  will  be  cumulated.	When  using  the  --unique
       option, duplicates are discarded.

       Mandatory arguments to long options are mandatory for short options too.

   Input file location:
       INPUTFILE
	      input PO file

       -D, --directory=DIRECTORY
	      add DIRECTORY to list for input files search

       If no input file is given or if it is -, standard input is read.

   Output file location:
       -o, --output-file=FILE
	      write output to specified file

       The results are written to standard output if no output file is specified or if it is -.

   Message selection:
       -d, --repeated
	      print only duplicates

       -u, --unique
	      print only unique messages, discard duplicates

   Output details:
       -t, --to-code=NAME
	      encoding for output

       --use-first
	      use first available translation for each message, don't merge several translations

       -e, --no-escape
	      do not use C escapes in output (default)

       -E, --escape
	      use C escapes in output, no extended chars

       --force-po
	      write PO file even if empty

       -i, --indent
	      write the .po file using indented style

       --no-location
	      do not write '#: filename:line' lines

       -n, --add-location
	      generate '#: filename:line' lines (default)

       --strict
	      write out strict Uniforum conforming .po file

       -w, --width=NUMBER
	      set output page width

       --no-wrap
	      do not break long message lines, longer than the output page width, into several lines

       -s, --sort-output
	      generate sorted output

       -F, --sort-by-file
	      sort output by file location

   Informative output:
       -h, --help
	      display this help and exit

       -V, --version
	      output version information and exit

AUTHOR

       Written by Bruno Haible.

REPORTING BUGS

       Report bugs to <bug-gnu-gettext@gnu.org>.

COPYRIGHT

       Copyright (C) 2001-2002 Free Software Foundation, Inc.
       This is free software; see the source for copying conditions.  There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICU-
       LAR PURPOSE.

SEE ALSO

       The full documentation for msguniq is maintained as a Texinfo manual.  If the info and msguniq programs	are  properly  installed  at  your
       site, the command

	      info msguniq

       should give you access to the complete manual.

GNU gettext 0.11.4						     July 2002								MSGUNIQ(1)