PERL: removing blank lines from multiple files


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting PERL: removing blank lines from multiple files
# 8  
Old 09-19-2009
Quote:
Originally Posted by ilikecows
Look at my above post. It does just that.
Thanks, but i really cant believe a perl one liner exists to do this, but the only way to do this omside an actual perl script is to run system() to run sed, then run system() again to run cp to copy the file back over


surely ?
# 9  
Old 09-19-2009
You can write something like this (untested).
Backup your files first!


Code:
#! /usr/bin/env perl

use warnings;
use strict;

my @files   = glob '/tmp/*'; # adjust the glob!
my $logfile = $0 . '.log';

open LH, '>', $logfile or die "open $logfile: $!\n";

for (@files) {
    {
        local ( $^I, @ARGV ) = ( '', $_ );
        while (<>) {
            print LH $ARGV, ": record ", $., "\n" and next if /^\s*$/;
            print;
        }
    }
}

close LH or warn "close $logfile: $!\n";


Last edited by radoulov; 09-19-2009 at 04:18 PM..
# 10  
Old 09-19-2009
Quote:
Originally Posted by hcclnoodles
...
..the reason i have to go through each file line by line is because i need to identify exactly which line number in which file is returning blank ...
...
That's not a good enough reason.
You do not have to
(1) open
(2) read and
(3) close a file

just to find out which line number is blank. Sure, you can, but you don't have to.
If you do, then it's a convoluted way of doing things. The sed program didn't open, read, and close the file (at least explicitly).

Quote:
...
to identify exactly which line number in which file is returning blank and send it to a log file.
...
Send the blank line to a log file ? This wasn't the requirement in your original post.

Quote:
...
there is no need for me to actually fix these issues within this actual script but i thought it make sense to do so if possible
...
Not sure I understand this. If there is no need to fix it, then why do you even want to do it ?
As for the possibility, you opened the file for reading (the default). And you wanted to write to a file. They are not the same thing.

Quote:
...
I read on another site that I can set my file location within my script to say $file and then run this
Code:
my $file = "/var/tmp/0629AN1208.net";
$file =~ s/^\n//g;

but that doesnt seem to work,
...
Of course it doesn't, if you understand what you are trying to do. "$file" is just a variable, not an actual file. And the "s///" operator simply works on the value of that variable. Your variable did not have a newline character at the beginning, so it wasn't removed. If it had, then it would've been removed -

Code:
$ 
$ perl -le '$file = "
/var/tmp/0629AN1208.net"; print "Before: $file"; $file =~ s/^\n//g; print "After : $file"'
Before: 
/var/tmp/0629AN1208.net
After : /var/tmp/0629AN1208.net
$ 
$

Perl is simply doing what it was told to do.
And if you are thinking that the "s///" operator will actually open a file and remove newlines, just because the variable was called "$file", then you need to brush up on the basics first.

Quote:
...
whereas if i run the equivalent one liner from the command line (see below) it works perfectly .

Code:
perl -pi -e "s/^\n//" /var/tmp/0629AN1208.net

...
That one-liner writes to the file because of that qualifier "-i". Check the perl documentation. You could make it work on a hierarchy of files, but the solution would be more verbose than that sed solution.

tyler_durden
# 11  
Old 09-20-2009
Here is what you can do:

1. A seperate script using sed to remove the blank lines
2. Call sed from within your perl script with system
3. The perl -i route in the above post
4. Dump each line of the file to an array except blank lines, open the files again for writing, and clobber them with the contents of the array
5. Create a new file, open it for writing, add an else to the if ( /^$/ ) that counts blank lines, and print the non blanks to the new file. Then overwrite the original with the new one.

1,2, and 3 are much easier than 4 or 5.
# 12  
Old 09-20-2009
thank you for your responses, I have been learning perl for the grand total of 5 weeks now...so go easy on me Smilie

i will look at using the sed solution instead

thanks again
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Removing carriage returns from multiple lines in multiple files of different number of columns

Hello Gurus, I have a multiple pipe separated files which have records going over multiple Lines. End of line separator is \n and records going over multiple lines have <CR> as separator. below is example from one file. 1|ABC DEF|100|10 2|PQ RS T|200|20 3| UVWXYZ|300|30 4| GHIJKL|400|40... (7 Replies)
Discussion started by: dJHa
7 Replies

2. Shell Programming and Scripting

Removing multiple lines from input file, if multiple lines match a pattern.

GM, I have an issue at work, which requires a simple solution. But, after multiple attempts, I have not been able to hit on the code needed. I am assuming that sed, awk or even perl could do what I need. I have an application that adds extra blank page feeds, for multiple reports, when... (7 Replies)
Discussion started by: jxfish2
7 Replies

3. Shell Programming and Scripting

Removing blank lines

Hi, my input file is like this I want to remove the blank line. "/home/rit/sandbox/garuda/data/test/woman/T_RITK_F0008_ML_100080039.lab" r a N e l a k sh a m . "/home/rit/sandbox/garuda/data/test/woman/T_RITK_F0008_ML_100070453.lab" a v a s (4 Replies)
Discussion started by: sreejithalokkan
4 Replies

4. Shell Programming and Scripting

Removing blank lines from a file

Hi All, How do i remove continuos blank lines from a file. I have a file with data: abc; def; ghi; jkl; mno; pqr; In the above file, there are two blank lines. I want to remove, one out of them. My output should look like: (2 Replies)
Discussion started by: raosr020
2 Replies

5. UNIX for Dummies Questions & Answers

Removing blank lines not working

In my bash script I want to echo lines in a file and ensure no blank lines are echoed:for i in $(cat txt) do echo $i | sed 's/|/ /g;s/ SEARCHTERM$//g;s/ /\r\n/g;s/^$/d' done Keep in mind this is a fragment so ignore the fact that the for loop is not closed. When I add the "s/^$/d' per... (12 Replies)
Discussion started by: MaindotC
12 Replies

6. UNIX for Dummies Questions & Answers

Removing Lines Shared by Multiple Files

Hey everyone, I have a question about comparing two files. I have two lists of files. The first list, todo.csv, lists a series of compounds my supervisor wants me to perform calculations on. The second list, done.csv, lists a series of compounds that I have already performed calculations on.... (2 Replies)
Discussion started by: Stuart Ness
2 Replies

7. UNIX for Dummies Questions & Answers

Removing blank lines in a file

Hi I have a text file that has blank lines at different places. How to remove all the blank lines in a file? Thanks Ashok (3 Replies)
Discussion started by: ashok.k
3 Replies

8. Shell Programming and Scripting

removing duplicate blank lines

Hi, how to remove the blank lines from the file only If we have more than one blank line. thanks rameez (8 Replies)
Discussion started by: rameezrajas
8 Replies

9. Shell Programming and Scripting

Removing Blank Lines

Hi i have the below lines from a file 7538 PRGRP450800PERSONAL SOAP AND BATH ADDITIV 7036 PRGRP450800PERSONAL SOAP AND BATH ADDITIV 7036 PRGRP450800PERSONAL SOAP AND BATH ADDITIV 7036... (3 Replies)
Discussion started by: dhanamurthy
3 Replies

10. Shell Programming and Scripting

Adding 3 Lines to Multiple c and h files with perl

Hello, i need some help with a perl script. i need to add the lines: #ifdef LOGALLOC #include "logalloc.h" #endif // LOGALLOC To all c and h files in a project with subdirectories. Logalloc is a tool to log all *alloc and free's in a text file, it defines the *alloc funtions new.... (2 Replies)
Discussion started by: Lazzar
2 Replies
Login or Register to Ask a Question