Sponsored Content
Top Forums Shell Programming and Scripting Shell script to remove duplicates lines in a file Post 302291500 by cfajohnson on Wednesday 25th of February 2009 10:05:14 PM
Old 02-25-2009
Code:
awk '/section/ { section = $2 } !x[section,$0]++' "$FILE"

 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Shell script: Remove blank lines

Hi gurus, I have this file with blank lines in it. How do i remove them in shell? I tried these commands but not working: sed '/^ *$/d' or sed '/^$/d' Anybody has a better idea pls? Also there are lines which starts with a single space, how do we remove the space in those lines?... (3 Replies)
Discussion started by: gholdbhurg
3 Replies

2. Shell Programming and Scripting

shell script to remove all lines from a file before a line starting with pattern

hi,, i hav a file with many lines.i need to remove all lines before a line begginning with a specific pattern from the file because these lines are not required. Can u help me out with either a perl script or shell script example:- if file initially contains lines: a b c d .1.2 d e f... (2 Replies)
Discussion started by: raksha.s
2 Replies

3. Shell Programming and Scripting

How to delete lines in a file that have duplicates or derive the lines that aper once

Input: a b b c d d I need: a c I know how to get this (the lines that have duplicates) : b d sort file | uniq -d But i need opossite of this. I have searched the forum and other places as well, but have found solution for everything except this variant of the problem. (3 Replies)
Discussion started by: necroman08
3 Replies

4. Shell Programming and Scripting

need Shell script for Sort BASED ON FIRST FIELD and PRINT THE WHOLE FILE WITHOUT DUPLICATES

Can some one provide me a shell script. I have file with many columns and many rows. need to sort the first column and then remove the duplicates records if exists.. finally print the full data with first coulm as unique. Sort BASED ON FIRST FIELD and remove the duplicates if exists... (2 Replies)
Discussion started by: tuffEnuff
2 Replies

5. Shell Programming and Scripting

Script to remove duplicates

Hi I need a script that removes the duplicate records and write it to a new file for example I have a file named test.txt and it looks like abcd.23 abcd.24 abcd.25 qwer.25 qwer.26 qwer.98 I want to pick only $1 and compare with the next record and the output should be abcd.23... (6 Replies)
Discussion started by: antointoronto
6 Replies

6. Shell Programming and Scripting

How to remove some xml tag lines using shell script

I have existing XML file as below, now based on input string in shell script on workordercode i need to create a seprate xml file for e.g if we pass the input string as 184851 then it find the tag data from <workOrder>..</workOrder> and write to a new file and similarly next time if i pass the... (3 Replies)
Discussion started by: balrajg
3 Replies

7. UNIX for Dummies Questions & Answers

script to remove duplicates per line

Hello experts! I'd like a way to remove duplicates per line. Strings are enclosed in brackets, and I would prefer to maintain the order of the file: example input (56)(63) (56)(70)(56)(70)(24) (25)(78) (12)(33)(12) (10) (10) desired output (56)(63) (56)(70)(24) (25)(78)... (5 Replies)
Discussion started by: torchij
5 Replies

8. UNIX for Dummies Questions & Answers

Remove duplicates lines in a files

I have a file called FILE cat FILE 11/11/2012 11/11/2012 12/11/2012 15/11/2012 need to remove the duplicates dates ( ie 11/11/2012 is present two times i need remove one duplicates date ) Need outputs like this 11/11/2012 12/11/2012 15/11/2012 I have tried using awk... (8 Replies)
Discussion started by: Venkatesh1
8 Replies

9. Shell Programming and Scripting

How can I remove some xml tag lines using shell script?

Hi All, My name is Prathyu and I am working as a ETL develper. I have one requirement to create a XML file based on the provided XSD file. As per the Datastage standards Key(repeatable) field does not contain any Null values so I am inserting some dummy tag line to that XML file. ... (14 Replies)
Discussion started by: Prathyu
14 Replies

10. Shell Programming and Scripting

How to remove duplicates in C shell Array?

Please help me on this My script name is uniqueArray.csh #!/bin/csh set ARRAY = ( one teo three one three ) set ARRAY = ( $ARRAY one five three five ) How to remove the duplicates in this array ,sort and save those in the same variable or different variable. Thanks in the advance ... (3 Replies)
Discussion started by: SA_Palani
3 Replies
MAN(7)							 Miscellaneous Information Manual						    MAN(7)

NAME
man - macros to typeset manual SYNOPSIS
nroff -man file ... troff -man file ... DESCRIPTION
These macros are used to lay out pages of this manual. A skeleton page may be found in the file /usr/man/man.template. Any text argument t may be zero to six words. Quotes may be used to include blanks in a `word'. If text is empty, special treatment is applied to the next input line with text to be printed. In this way .I may be used to italicize a whole line, or .SM may be followed by .B to make small bold letters. A prevailing indent distance is remembered between successive indented paragraphs, and is reset to default value upon reaching a non- indented paragraph. Default units for indents i are ens. Type font and size are reset to default values before each paragraph, and after processing font and size setting macros. These strings are predefined by -man: *R `(Reg)', trademark symbol in troff. *S Change to default type size. FILES
/usr/share/tmac/tmac.an /usr/man/man.template SEE ALSO
troff(1), man(1) BUGS
Relative indents don't nest. REQUESTS
Request Cause If no Explanation Break Argument .B t no t=n.t.l.*Text t is bold. .BI t no t=n.t.l. Join words of t alternating bold and italic. .BR t no t=n.t.l. Join words of t alternating bold and Roman. .DT no .5i 1i...Restore default tabs. .HP i yes i=p.i.* Set pre- vailing indent to i. Begin paragraph with hanging indent. .I t no t=n.t.l. Text t is italic. .IB t no t=n.t.l. Join words of t alternating italic and bold. .IP x i yes x="" Same as .TP with tag x. .IR t no t=n.t.l. Join words of t alternating italic and Roman. .LP yes - Same as .PP. .PD d no d=.4v Interparagraph distance is d. .PP yes - Begin paragraph. Set prevailing indent to .5i. .RE yes - End of relative indent. Set prevailing indent to amount of starting .RS. .RB t no t=n.t.l. Join words of t alternating Roman and bold. .RI t no t=n.t.l. Join words of t alternating Roman and italic. .RS i yes i=p.i. Start relative indent, move left margin in distance i. Set prevailing indent to .5i for nested indents. .SH t yes t=n.t.l. Subhead. .SM t no t=n.t.l. Text t is small. .TH n c x v m yes -Begin page named n of chapter c; x is extra commentary, e.g. `local', for page foot center; v alters page foot left, e.g. `4th Berkeley Distribution'; m alters page head center, e.g. `Brand X Programmer's Manual'. Set prevailing indent and tabs to .5i. .TP i yes i=p.i. Set prevailing indent to i. Begin indented paragraph with hanging tag given by next text line. If tag doesn't fit, place it on separate line. * n.t.l. = next text line; p.i. = prevailing indent 7th Edition October 22, 1996 MAN(7)
All times are GMT -4. The time now is 12:45 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy