unable to filter out blank lines ^$ matches nothing


 
Thread Tools Search this Thread
Top Forums UNIX for Advanced & Expert Users unable to filter out blank lines ^$ matches nothing
# 1  
Old 01-30-2008
Question unable to filter out blank lines ^$ matches nothing

Hi
I've got a file with a lot of blank lines in it or are they?...I tried to remove them by using

grep -v ^$ <filename>

This did not remove anything....I then tried the reverse to see if ^$ indeed matches anything...so I tried

$ grep ^$ grepres.txt
$ grep ^\s*$ grepres.txt

This matched nothing...


..I am a bit of a noob at this so if anyone out there can figure out how to do this using regular expressions...I would really appreciate it...

I've attached the file here

Thanks in advance...
# 2  
Old 01-30-2008
To remove blank lines using sed:

Code:
sed '/^$/d' infile > outfile

# 3  
Old 01-30-2008
If u want to use grep then...

grep . grepres.txt
# 4  
Old 01-30-2008
Hi.

My experience has been that I have often received files that contain spaces and TABS in otherwise empty lines. I call these visually empty lines. Usually we want to get rid of these as well as lines which have no characters at all.

Here is what I usually use:
Code:
#!/usr/bin/env sh

# @(#) dbl      Delete blank ( visually empty ) lines.
# If POSIX character class does not work, then use
# [ SPACE TAB ].

grep -v '^[[:space:]]*$' $*
# grep -v '^[   ]*$' $*

exit $?

Here is a demonstration script:
Code:
#!/bin/bash -

# @(#) s1       Demonstrate visually empty line processing.

echo "(Versions displayed with local utility \"version\")"
version >/dev/null 2>&1 && version =o $(_eat $0 $1) cat grep

FILE=${1-data1}

echo
echo " Input file $FILE:"
cat -A $FILE

echo
echo " Processed with grep '.':"
grep . $FILE

echo
echo " Processing with dbl:"
./dbl $FILE

exit 0

Which produces:
Code:
% ./s1
(Versions displayed with local utility "version")
Linux 2.6.11-x1
GNU bash 2.05b.0
cat (coreutils) 5.2.1
grep (GNU grep) 2.5.1

 Input file data1:
First line.$
Next line has no characters.$
$
Next line has 1 space.$
 $
Next line has 1 TAB.$
^I$
Penultimate line.$
Last line.$

 Processed with grep '.':
First line.
Next line has no characters.
Next line has 1 space.

Next line has 1 TAB.

Penultimate line.
Last line.

 Processing with dbl:
First line.
Next line has no characters.
Next line has 1 space.
Next line has 1 TAB.
Penultimate line.
Last line.

Best wishes ... cheers, drl
# 5  
Old 01-31-2008
MySQL

Hi

..sorry didnt mention in my first post that sed '/^$/d' didnt work...

...grep . filename selects everything dosent it..?

....grep -v '^[[:space:]]*$' worked perfectly..

thing is there was a space in the line...dunno why \s*$ didnt work....

thanks a lot drl...
# 6  
Old 02-08-2008
This file contain no empty lines. You got some whitespaces in there.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Delete multiple lines between blank lines containing two patterns

Hi all, I'm looking for a way (sed or awk) to delete multiple lines between blank lines containing two patterns ex: user: alpha parameter_1 = 15 parameter_2 = 1 parameter_3 = 0 user: alpha parameter_1 = 15 parameter_2 = 1 parameter_3 = 0 user: alpha parameter_1 = 16... (3 Replies)
Discussion started by: ce9888
3 Replies

2. Shell Programming and Scripting

Tried many options but unable to delete blank lines from text file

Hi, I tried the following options but was unable to delete blank lines from file Input file = temp.hash.txt temp.hash.txt content 90 0 89.56 0 0 57575.4544 56.89 (9 Replies)
Discussion started by: uuuunnnn
9 Replies

3. Shell Programming and Scripting

To check Blank Lines, Blank Records and Junk Characters in a File

Hi All Need Help I have a file with the below format (ABC.TXT) : ®¿¿ABCDHEJJSJJ|XCBJSKK01|M|7348974982790 HDFLJDKJSKJ|KJALKSD02|M|7378439274898 KJHSAJKHHJJ|LJDSAJKK03|F|9898982039999 (cont......) I need to write a script where it will check for : blank lines (between rows,before... (6 Replies)
Discussion started by: chatwithsaurav
6 Replies

4. UNIX for Advanced & Expert Users

Delete blank spaces and blank lines in a file

Hi Gurus, Somebody can say me how to delete blank spaces and blank lines in a file unix, please. Thank you for advanced. (10 Replies)
Discussion started by: systemoper
10 Replies

5. UNIX for Dummies Questions & Answers

Finding lines with a regular expression, replacing them with blank lines

So the tag for this forum says all newbies welcome... All I want to do is go through my file and find lines which contain a given string of characters then replace these with a blank line. I really tried to find a simple command to do this but failed. Here's what I did come up with though: ... (2 Replies)
Discussion started by: Golpette
2 Replies

6. Shell Programming and Scripting

sed show lines text between 2 blank lines

I have a file like blah blah blah blah this is the text I need, which might be between 1-4 lines, but always has a blank line above and below it, and is at the end of the text file the code tags don't show the trailing blank line. I started by deleting the last blank line with: ... (2 Replies)
Discussion started by: unclecameron
2 Replies

7. Shell Programming and Scripting

Help in replacing two blank lines with two lines of diff data

Hi.. I'm facing a trouble in replacing two blank lines in a file using shell script... I used sed to search a line and insert two blank lines after the searchd line using the following sed command. sed "/data/{G;G;}/" filename . In the file, after data tag, two lines got inserted blank lines..... (4 Replies)
Discussion started by: arjun_arippa
4 Replies

8. Shell Programming and Scripting

Delete blank lines, if blank lines are more than one using shell

Hi, Consider a file named "testfile" The contents of file are as below first line added for test second line added for test third line added for test fourth line added for test fifth line added for test (5 Replies)
Discussion started by: anil8103
5 Replies

9. Shell Programming and Scripting

remove blank lines and merge lines in shell

Hi, I'm not a expert in shell programming, so i've come here to take help from u gurus. I'm trying to tailor a csv file that i got to make it work for the LOAD FROM command. I've a datatable csv of the below format - --in file format xx,xx,xx ,xx , , , , ,,xx, xxxx,, ,, xxx,... (11 Replies)
Discussion started by: dvah
11 Replies

10. UNIX for Dummies Questions & Answers

How to count lines - ignoring blank lines and commented lines

What is the command to count lines in a files, but ignore blank lines and commented lines? I have a file with 4 sections in it, and I want each section to be counted, not including the blank lines and comments... and then totalled at the end. Here is an example of what I would like my... (6 Replies)
Discussion started by: kthatch
6 Replies
Login or Register to Ask a Question