Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: unable to filter out blank lines ^$ matches nothing
Top Forums UNIX for Advanced & Expert Users unable to filter out blank lines ^$ matches nothing Post 302162866 by drl on Wednesday 30th of January 2008 11:35:46 AM
Old 01-30-2008
Hi.

My experience has been that I have often received files that contain spaces and TABS in otherwise empty lines. I call these visually empty lines. Usually we want to get rid of these as well as lines which have no characters at all.

Here is what I usually use:
Code:
#!/usr/bin/env sh

# @(#) dbl      Delete blank ( visually empty ) lines.
# If POSIX character class does not work, then use
# [ SPACE TAB ].

grep -v '^[[:space:]]*$' $*
# grep -v '^[   ]*$' $*

exit $?

Here is a demonstration script:
Code:
#!/bin/bash -

# @(#) s1       Demonstrate visually empty line processing.

echo "(Versions displayed with local utility \"version\")"
version >/dev/null 2>&1 && version =o $(_eat $0 $1) cat grep

FILE=${1-data1}

echo
echo " Input file $FILE:"
cat -A $FILE

echo
echo " Processed with grep '.':"
grep . $FILE

echo
echo " Processing with dbl:"
./dbl $FILE

exit 0

Which produces:
Code:
% ./s1
(Versions displayed with local utility "version")
Linux 2.6.11-x1
GNU bash 2.05b.0
cat (coreutils) 5.2.1
grep (GNU grep) 2.5.1

 Input file data1:
First line.$
Next line has no characters.$
$
Next line has 1 space.$
 $
Next line has 1 TAB.$
^I$
Penultimate line.$
Last line.$

 Processed with grep '.':
First line.
Next line has no characters.
Next line has 1 space.

Next line has 1 TAB.

Penultimate line.
Last line.

 Processing with dbl:
First line.
Next line has no characters.
Next line has 1 space.
Next line has 1 TAB.
Penultimate line.
Last line.

Best wishes ... cheers, drl
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

How to count lines - ignoring blank lines and commented lines

What is the command to count lines in a files, but ignore blank lines and commented lines? I have a file with 4 sections in it, and I want each section to be counted, not including the blank lines and comments... and then totalled at the end. Here is an example of what I would like my... (6 Replies)
Discussion started by: kthatch
6 Replies

2. Shell Programming and Scripting

remove blank lines and merge lines in shell

Hi, I'm not a expert in shell programming, so i've come here to take help from u gurus. I'm trying to tailor a csv file that i got to make it work for the LOAD FROM command. I've a datatable csv of the below format - --in file format xx,xx,xx ,xx , , , , ,,xx, xxxx,, ,, xxx,... (11 Replies)
Discussion started by: dvah
11 Replies

3. Shell Programming and Scripting

Delete blank lines, if blank lines are more than one using shell

Hi, Consider a file named "testfile" The contents of file are as below first line added for test second line added for test third line added for test fourth line added for test fifth line added for test (5 Replies)
Discussion started by: anil8103
5 Replies

4. Shell Programming and Scripting

Help in replacing two blank lines with two lines of diff data

Hi.. I'm facing a trouble in replacing two blank lines in a file using shell script... I used sed to search a line and insert two blank lines after the searchd line using the following sed command. sed "/data/{G;G;}/" filename . In the file, after data tag, two lines got inserted blank lines..... (4 Replies)
Discussion started by: arjun_arippa
4 Replies

5. Shell Programming and Scripting

sed show lines text between 2 blank lines

I have a file like blah blah blah blah this is the text I need, which might be between 1-4 lines, but always has a blank line above and below it, and is at the end of the text file the code tags don't show the trailing blank line. I started by deleting the last blank line with: ... (2 Replies)
Discussion started by: unclecameron
2 Replies

6. UNIX for Dummies Questions & Answers

Finding lines with a regular expression, replacing them with blank lines

So the tag for this forum says all newbies welcome... All I want to do is go through my file and find lines which contain a given string of characters then replace these with a blank line. I really tried to find a simple command to do this but failed. Here's what I did come up with though: ... (2 Replies)
Discussion started by: Golpette
2 Replies

7. UNIX for Advanced & Expert Users

Delete blank spaces and blank lines in a file

Hi Gurus, Somebody can say me how to delete blank spaces and blank lines in a file unix, please. Thank you for advanced. (10 Replies)
Discussion started by: systemoper
10 Replies

8. Shell Programming and Scripting

To check Blank Lines, Blank Records and Junk Characters in a File

Hi All Need Help I have a file with the below format (ABC.TXT) : ®¿¿ABCDHEJJSJJ|XCBJSKK01|M|7348974982790 HDFLJDKJSKJ|KJALKSD02|M|7378439274898 KJHSAJKHHJJ|LJDSAJKK03|F|9898982039999 (cont......) I need to write a script where it will check for : blank lines (between rows,before... (6 Replies)
Discussion started by: chatwithsaurav
6 Replies

9. Shell Programming and Scripting

Tried many options but unable to delete blank lines from text file

Hi, I tried the following options but was unable to delete blank lines from file Input file = temp.hash.txt temp.hash.txt content 90 0 89.56 0 0 57575.4544 56.89 (9 Replies)
Discussion started by: uuuunnnn
9 Replies

10. UNIX for Beginners Questions & Answers

Delete multiple lines between blank lines containing two patterns

Hi all, I'm looking for a way (sed or awk) to delete multiple lines between blank lines containing two patterns ex: user: alpha parameter_1 = 15 parameter_2 = 1 parameter_3 = 0 user: alpha parameter_1 = 15 parameter_2 = 1 parameter_3 = 0 user: alpha parameter_1 = 16... (3 Replies)
Discussion started by: ce9888
3 Replies

LEARN ABOUT CENTOS

look

LOOK(1) 							   User Commands							   LOOK(1)

NAME

       look - display lines beginning with a given string

SYNOPSIS

       look [options] string [file]

DESCRIPTION

       The  look  utility  displays  any  lines  in file which contain string.	As look performs a binary search, the lines in file must be sorted
       (where sort(1) got the same options -d and/or -f that look is invoked with).

       If file is not specified, the file /usr/share/dict/words is used, only alphanumeric characters are compared  and  the  case  of	alphabetic
       characters is ignored.

OPTIONS

       -a, --alternative
	      Use the alternative dictionary file.

       -d, --alphanum
	      Use normal dictionary character set and order, i.e. only alphanumeric characters are compared.  (This is on by default if no file is
	      specified.)

       -f, --ignore-case
	      Ignore the case of alphabetic characters.  (This is on by default if no file is specified.)

       -t, --terminate character
	      Specify a string termination character, i.e. only the characters in string up to and including the first occurrence of character are
	      compared.

       -h, --help
	      Display help text and exit.

       -V, --version
	      Output version information and exit.

       The look utility exits 0 if one or more lines were found and displayed, 1 if no lines were found, and >1 if an error occurred.

EXAMPLE

	      sort -d /etc/passwd -o /tmp/look.dict
	      look -t: root:foobar /tmp/look.dict

FILES

       /usr/share/dict/words
	   the dictionary

       /usr/share/dict/web2
	   the alternative dictionary

SEE ALSO

       grep(1), sort(1)

COMPATIBILITY

       The  original  manual  page stated that tabs and blank characters participated in comparisons when the alphanum option was specified.  This
       was incorrect, and the current man page matches the historic implementation.

HISTORY

       The look utility appeared in Version 7 AT&T Unix.

AVAILABILITY

       The look command is part of the util-linux package and is available from ftp://ftp.kernel.org/pub/linux/utils/util-linux/.

util-linux							     June 2011								   LOOK(1)
All times are GMT -4. The time now is 10:18 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy