Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

removing duplicate blank lines

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting removing duplicate blank lines
# 1  
Old 07-31-2008
removing duplicate blank lines
Hi,

how to remove the blank lines from the file only If we have more than one blank line.

thanks
rameez
# 2  
Old 07-31-2008
Code:
awk '/^$/ { if (blank++) next; print }1' file

# 3  
Old 07-31-2008
It woks fine.

Can you please explain the following part in the command
'{ if (blank++) next; print }1'
# 4  
Old 07-31-2008
The variable blank will be zero initially, but non-zero (i.e. true) on subsequent runs.

The 1 is an awk idiom which matches and prints lines not handled by previous conditions.

Actually you should change it to '{ blank=0; print }' in order for the script to work correctly on files where there are multiple sequences of adjacent empty lines. Sorry for missing that.
# 5  
Old 07-31-2008
Thanks...thats great!
# 6  
Old 07-31-2008
Another way with awk yet :
Code:
awk 'NR==1 || NF || prvNF {print} {prvNF=NF}'

Jean-Pierre.
# 7  
Old 07-31-2008
Sorry, it's still not right. Should test before I post. Apologies.

Code:
awk '/^$/{ if (! blank++) print; next } { blank=0; print }'file

(What a grand way to increase my post count ...)
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Removing blank lines

Hi, my input file is like this I want to remove the blank line. "/home/rit/sandbox/garuda/data/test/woman/T_RITK_F0008_ML_100080039.lab" r a N e l a k sh a m . "/home/rit/sandbox/garuda/data/test/woman/T_RITK_F0008_ML_100070453.lab" a v a s (4 Replies)
Discussion started by: sreejithalokkan
4 Replies

2. Shell Programming and Scripting

Removing blank lines from a file

Hi All, How do i remove continuos blank lines from a file. I have a file with data: abc; def; ghi; jkl; mno; pqr; In the above file, there are two blank lines. I want to remove, one out of them. My output should look like: (2 Replies)
Discussion started by: raosr020
2 Replies

3. UNIX for Dummies Questions & Answers

Removing blank lines not working

In my bash script I want to echo lines in a file and ensure no blank lines are echoed:for i in $(cat txt) do echo $i | sed 's/|/ /g;s/ SEARCHTERM$//g;s/ /\r\n/g;s/^$/d' done Keep in mind this is a fragment so ignore the fact that the for loop is not closed. When I add the "s/^$/d' per... (12 Replies)
Discussion started by: MaindotC
12 Replies

4. UNIX for Dummies Questions & Answers

Removing blank lines in a file

Hi I have a text file that has blank lines at different places. How to remove all the blank lines in a file? Thanks Ashok (3 Replies)
Discussion started by: ashok.k
3 Replies

5. Shell Programming and Scripting

Removing Duplicate Lines per Section

Hello, I am in need of removing duplicate lines from within a file per section. File: ABC1 012345 header ABC2 7890-000 ABC3 012345 Header Table ABC4 ABC5 593.0000 587.4800 ABC5 593.5000 587.6580 <= dup need to remove ABC5 593.5000 ... (5 Replies)
Discussion started by: petersf
5 Replies

6. Shell Programming and Scripting

PERL: removing blank lines from multiple files

Hi Guru's , I have a whole bunch of files in /var/tmp that i need to strip any blank lines from, so ive written the following script to identify the lines (which works perfectly).. but i wanted to know, how can I actually strip the identified lines from the actual source files ?? my... (11 Replies)
Discussion started by: hcclnoodles
11 Replies

7. Shell Programming and Scripting

Removing duplicates from string (not duplicate lines)

please help me in getting following: Input Desired output x="foo" foo x="foo foo" foo x="foo foo" foo x="foo abc foo" foo abc x="foo foo1 foo2" foo foo1 foo2 I need to remove duplicated from string.. (8 Replies)
Discussion started by: vickylife
8 Replies

8. Shell Programming and Scripting

removing the duplicate lines in a file

Hi, I need to concatenate three files in to one destination file.In this if some duplicate data occurs it should be deleted. eg: file1: ----- data1 value1 data2 value2 data3 value3 file2: ----- data1 value1 data4 value4 data5 value5 file3: ----- data1 value1 data4 value4 (3 Replies)
Discussion started by: Sharmila_P
3 Replies

9. Shell Programming and Scripting

Removing Blank Lines

Hi i have the below lines from a file 7538 PRGRP450800PERSONAL SOAP AND BATH ADDITIV 7036 PRGRP450800PERSONAL SOAP AND BATH ADDITIV 7036 PRGRP450800PERSONAL SOAP AND BATH ADDITIV 7036... (3 Replies)
Discussion started by: dhanamurthy
3 Replies

10. UNIX for Dummies Questions & Answers

removing duplicate lines from a file

Hi, I am trying to remove duplicate lines from a file. For example the contents of example.txt is: this is a test 2342 this is a test 34343 this is a test 43434 and i want to remove the "this is a test" lines only and end up with the numbers in the file, that is, end up with: 2342... (4 Replies)
Discussion started by: ocelot
4 Replies
Login or Register to Ask a Question