How to delete lines in a file that have duplicates or derive the lines that aper once Post: 302335014

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

delete blank lines or lines with spaces only

hi there i'm trying to delete blank lines and or lines with spaces only from a series of files in an directory. to do so, i'm using this: for files in `ls /users/myname/pesop* 2>/dev/null` do grep -v ^$ $files > newfile mv newfile $files done now, this works great for blank lines but...

2. Shell Programming and Scripting

Shell script to remove duplicates lines in a file

Hi, I am writing a shell script that needs to remove duplicate lines within a file by category. example: section a a c b a section b a b a c I need to remove the duplicates within th category with out removing the duplicates from the 2 different sections (one of the a's in section...

3. Shell Programming and Scripting

sed problem - delete all lines until a match on 2 lines

First of all, I know this can be more eassily done with perl or other scripting languages but, that's not the issue. I need this in sed. (or wander if it's possible ) I got a file (trace file to recreate the control file from oracle for the dba boys) which contains some lines another line...

4. UNIX for Dummies Questions & Answers

How get only required lines & delete the rest of the lines in file

Hiiii I have a file which contains huge data as a.dat: PDE 1990 1 9 18 51 28.90 24.7500 95.2800 118.0 6.1 0.0 BURMA event name: 010990D time shift: 7.3000 half duration: 5.0000 latitude: 24.4200 longitude: 94.9500 depth: 129.6000 Mrr: ...

5. Shell Programming and Scripting

need to delete all lines from a group of files except the 1st 2 lines

Hello, I have a group of text files with many lines in each file. I need to delete all the lines in each and only leave 2 lines in each file.

6. UNIX for Advanced & Expert Users

In a huge file, Delete duplicate lines leaving unique lines

Hi All, I have a very huge file (4GB) which has duplicate lines. I want to delete duplicate lines leaving unique lines. Sort, uniq, awk '!x++' are not working as its running out of buffer space. I dont know if this works : I want to read each line of the File in a For Loop, and want to...

7. Shell Programming and Scripting

search and replace, when found, delete multiple lines, add new set of lines?

hey guys, I tried searching but most 'search and replace' questions are related to one liners. Say I have a file to be replaced that has the following: $ cat testing.txt TESTING AAA BBB CCC DDD EEE FFF GGG HHH ENDTESTING This is the input file:

8. Shell Programming and Scripting

Sed/awk to delete single lines that aren't touching other lines

Hello, I'm trying to figure out how to use sed or awk to delete single lines in a file. By single, I mean lines that are not touching any other lines (just one line with white space above and below). Example: one two three four five six seven eight I want it to look like:

9. Shell Programming and Scripting

Delete 40 lines after every 24 lines from a file

Hello, I have file of more than 10000 lines. I want to delete 40 lines after every 20 lines. e.g from a huge file, i want to delete line no from 34 - 74, then 94 - 134 etc and so on. Please let me know how i can do it. Best regards,

10. UNIX for Beginners Questions & Answers

Delete multiple lines between blank lines containing two patterns

Hi all, I'm looking for a way (sed or awk) to delete multiple lines between blank lines containing two patterns ex: user: alpha parameter_1 = 15 parameter_2 = 1 parameter_3 = 0 user: alpha parameter_1 = 15 parameter_2 = 1 parameter_3 = 0 user: alpha parameter_1 = 16...

LEARN ABOUT OPENSOLARIS

fmt

fmt(1)								   User Commands							    fmt(1)

NAME

       fmt - simple text formatters

SYNOPSIS

       fmt [-cs] [-w width | -width] [inputfile]...

DESCRIPTION

       fmt  is a simple text formatter that fills and joins lines  to produce output lines of (up to) the number of characters specified in the -w
       width option. The default  width is 72. fmt concatenates the inputfiles listed as arguments. If none are given, fmt formats text  from  the
       standard input.

       Blank lines are preserved in the output, as is the spacing between words. fmt does not fill nor split lines beginning with a `.' (dot), for
       compatibility with nroff(1). Nor does it fill or split a set of contiguous non-blank lines which is determined to be  a	mail  header,  the
       first line of which must begin with "From".

       Indentation is preserved in the output, and input lines with differing indentation are not joined (unless -c is used).

       fmt can also be used as an in-line text filter for vi(1). The  vi command:

       !}fmt

       reformats the text between the cursor location and the end of the paragraph.

OPTIONS

       -c		    Crown  margin  mode.  Preserve the indentation of the first two lines within a paragraph, and align the left margin of
			    each subsequent line with that of the second line. This is useful for tagged paragraphs.

       -s		    Split lines only. Do not join short lines to form longer ones. This prevents sample lines of code, and other such for-
			    matted text, from being unduly combined.

       -w width | -width    Fill output lines to up to	width columns.

OPERANDS

       inputfile    Input file.

ENVIRONMENT VARIABLES

       See environ(5) for a description of the LC_CTYPE environment variable that affects the execution of fmt.

ATTRIBUTES

       See attributes(5) for descriptions of the following attributes:

       +-----------------------------+-----------------------------+
       |      ATTRIBUTE TYPE	     |	    ATTRIBUTE VALUE	   |
       +-----------------------------+-----------------------------+
       |Availability		     |SUNWcsu			   |
       +-----------------------------+-----------------------------+

SEE ALSO

       nroff(1), vi(1), attributes(5), environ(5)

NOTES

       The -width option is acceptable for BSD compatibility, but it may go away in future releases.

SunOS 5.11							    9 May 1997								    fmt(1)

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

delete blank lines or lines with spaces only

Discussion started by: vascobrito

2. Shell Programming and Scripting

Shell script to remove duplicates lines in a file

Discussion started by: RichElks

3. Shell Programming and Scripting

sed problem - delete all lines until a match on 2 lines

Discussion started by: plelie2

4. UNIX for Dummies Questions & Answers

How get only required lines & delete the rest of the lines in file

Discussion started by: reva

5. Shell Programming and Scripting

need to delete all lines from a group of files except the 1st 2 lines

Discussion started by: script_op2a

6. UNIX for Advanced & Expert Users

In a huge file, Delete duplicate lines leaving unique lines

Discussion started by: krishnix

7. Shell Programming and Scripting

search and replace, when found, delete multiple lines, add new set of lines?

Discussion started by: DeuceLee