Removing dupicate lines in the file ..(they are not continuous)


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Removing dupicate lines in the file ..(they are not continuous)
# 8  
Old 09-29-2005
Quote:
Originally Posted by Srini75
hi vgersh99,


Thanks for your help,

Can you please explan me what actually is happening when we execute the above command ??


i.e nawk '!a[$NF]++' file.txt

Thanks
Srini
  1. What's unique for all your records/lines?
  2. Where does this unique field occur in the record/line?
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Removing lines from a file

I have a file `/tmp/wrk` containing filenames with paths. I want to remove filenames from this file, for example remove all filenames containing alja cagr cavt clta cmdo or corl remove all filenames containing data for days in region `d.2016.001` to `d.2016.207` remove all filenames... (10 Replies)
Discussion started by: kristinu
10 Replies

2. Shell Programming and Scripting

Removing lines from a file

Hi, I have a linux server that was hacked and I have a bunch of files that sporadically contain the following lines through out the file: <?php eval(base64_decode("Xxxxxxxxxxxxxx/xxxxxxxx")); I did't put the exact lines of the file in this post. The "Xxxx" are random letters/numbers.... (8 Replies)
Discussion started by: nck
8 Replies

3. UNIX for Dummies Questions & Answers

removing several lines from a file

Hi folks, I have a long string of DNA sequences, and I need to remove several lines, as well as the line directly following them. For example, here is a sample of my starting material: >548::GY31UMJ02DLYEH rank=0007170 x=1363.5 y=471.0 length=478... (1 Reply)
Discussion started by: kkohl78
1 Replies

4. Shell Programming and Scripting

removing lines from file

Hi I have many files all with 1 field per line as in 12345 abcde john.paul.net 6789101 how do I remove ceratin lines from these files. Have tried sed but sed wrecks my head! Many thanks in advance for any help (9 Replies)
Discussion started by: rob171171
9 Replies

5. Shell Programming and Scripting

Removing Lines From a File

Hi Does anybody know of a command that will enable me to remove all entries in a file that have the format (name & time) more testfile anthony 2003 anthonyr 2008 amorel 15:00 anthonyp 14:35 anthonyp 14:35 anthonyr 2008 ardean 13:28 arlene 2003 arlenem 08:15 arlenem 08:15... (5 Replies)
Discussion started by: jamba1
5 Replies

6. Shell Programming and Scripting

Removing the first and last lines in a file

Hi Gurus, I'm a little new to UNIX. How can I do remove the first and last line in a file? Say, supppose I have a file as below: Code: 1DMA 400002BARRIE 401002CALGARY/LETHBRI 402002CARLETON 500001PORTLAND-AUBRN 501001NEW YORK, NY 502001BINGHAMTON, NY ... (2 Replies)
Discussion started by: naveendronavall
2 Replies

7. AIX

Removing the first and last lines in a file

Hi Gurus, I'm a little new to UNIX. How can I do remove the first and last line in a file? Say, supppose I have a file as below: 1DMA 400002BARRIE 401002CALGARY/LETHBRI 402002CARLETON 500001PORTLAND-AUBRN 501001NEW YORK, NY ... (1 Reply)
Discussion started by: naveendronavall
1 Replies

8. UNIX for Dummies Questions & Answers

Removing lines from a file

I'm trying to find a command which will allow me to remove a range of lines (2-4) from a .dat file from the command line without opening the file. Someone mentioned using the ex command? Does anyone have any ideas? thanks (6 Replies)
Discussion started by: computersaysno
6 Replies

9. Shell Programming and Scripting

Removing lines within a file

Hi There, I've written a script that processes a data file on our system. Basically the script reads a post code from a list file, looks in the data file for the first occurrence (using grep) and reads the line number. It then tails the data file, with the line number just read, and outputs to a... (3 Replies)
Discussion started by: tookers
3 Replies

10. Shell Programming and Scripting

Removing lines from a file

Hello i have 2 files file1 and file2 as shown below file1 110010000000206|567810008161509 110010000000207|567810072227627 110010000000208|567811368851555 110010000000209|567811422513652 110010000000210|567812130217683 110010000000211|567813220211182 110010000000212|567813449322589... (4 Replies)
Discussion started by: PradeepRed
4 Replies
Login or Register to Ask a Question
SGML2TXT(1)						      General Commands Manual						       SGML2TXT(1)

NAME
sgml2txt - create plain text output from a LinuxDoc DTD SGML source file SYNOPSIS
sgml2txt [generic-option...] [--manpage] [--filter] [--blanks=n] file[.sgml] DESCRIPTION
sgml2txt is an old and obsoleted form of the text converter command of LinuxDoc-Tools. It is recommended to switch the new form linuxdoc -B text now. It converts a LinuxDoc DTD SGML source file to ASCII, ISO-8859-1, or EUC-JP output. Output will appear in file.txt where file is the name of the SGML source file. The attribute/value pair "output=txt" is set for conditionals. OPTIONS
sgml2txt accepts all the generic options described in linuxdoc(1), and the following specific options: --manpage, -m Outputs a groff source file, suitable for formatting with groff -man for man pages --filter, -f Remove backspace-overstrikes from the intermediate form generated by groff(1). --pass, -P The argument of the pass option is added to the command-line options handed to groff(1). --blanks=n, -b Set the limit of continuous blank lines for generating the output document. The default limit is 3. if 0 (zero) is specified, the result have many continuous blank lines. file The SGML source file, named either file or file.sgml FILES
Many files and executables in /usr/share/linuxdoc-tools are used. BUGS
None known. AUTHOR
Originally written by Greg Hankins <greg.hankins@cc.gatech.edu>, based on scripts by Tom Gordon and Alexander Horz, and later rewritten by Cees de Groot <cg@cdegroot.com> for SGML-Tools (v1). Currently maintained by Taketoshi Sano <sano@debian.org> for Linuxdoc-Tools. SEE ALSO
linuxdoc(1), sgml2html(1), sgml2info(1), sgml2latex(1), sgml2lyx(1), sgml2rtf(1), sgmlcheck(1). 16 May 2000 SGML2TXT(1)