Sponsored Content
Top Forums Shell Programming and Scripting Reading the file line by line in Perl Post 302589055 by birei on Tuesday 10th of January 2012 03:59:08 PM
Old 01-10-2012
Not sure about what are header and footer but I hope you can avoid the 'out of memory' message:
Code:
$ cat script.pl
use warnings;
use strict;

die qq[Usage: perl $0 <input-file> <output-good-file> <output-bad-file>\n] unless @ARGV == 3;

open my $bad_fh, ">", pop @ARGV or die qq[ERROR: $!\n];
open my $good_fh, ">", pop @ARGV or die qq[ERROR: $!\n];
open my $input_fh, "<", pop @ARGV or die qq[ERROR: $!\n];

my ($fields_processed, $flipflop);

while ( my $line = <$input_fh> ) {
        chomp $line;

        ## Header.
        if ( $flipflop = ( $line =~ m/\A(?i)start-of-file/ .. $line =~ m/\A(?i)start-of-fields/ ) ) {
                next if $flipflop == 1 || $flipflop =~ /E0\Z/;
                printf $good_fh qq[%s\n], $line;
                next;
        }

        ## Footer.
        if ( $fields_processed ) {
                if ( $flipflop = ( $line =~ m/\A(?i)end-of-fields/ .. eof ) ) {
                        next if $flipflop == 1;
                        printf $good_fh qq[%s\n], $line;
                }
        }

        my @f = split /\|/, $line, 25;

        if ( @f < 25 ) {
                next;
        }
        else {
                $fields_processed = 1;
        }

        if ( ( $f[21] eq "N.A.")  && ( $f[22] == 0 || $f[22] eq " ") && ( $f[23] == 0  ||  $f[23] eq " ") ) {
                printf $bad_fh qq[%s\n], $line;
        } 
        else {
                printf $good_fh qq[%s\n], $line;
        }
}

Regards,
Birei
This User Gave Thanks to birei For This Post:
 

9 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

perl - file reading - last line not displayed

Hi, Here is something that am trying with perl #! /opt/third-party/bin/perl open(fh, "s") || die "unable to open the file <small>"; @ch = (); $i = 0; while( $content = <fh> ) { if( $i <= 5 ) { push(@ch, $content); $i++; } else { $i = 1; foreach(@ch) { (8 Replies)
Discussion started by: matrixmadhan
8 Replies

2. Shell Programming and Scripting

Reading each line of a file in perl script

HI I need to read each line (test.txt) and store it in a array (@test) How to do it in perl. Suppose i have a file test.txt. I have to read each line of the test.txt file and store it in a array @test. How to do it in perl. Regards Harikrishna (3 Replies)
Discussion started by: Harikrishna
3 Replies

3. Shell Programming and Scripting

Reading a file line by line and processing for each line

Hi, I am a beginner in shell scripting. I have written the following script, which is supposed to process the while loop for each line in the sid_home.txt file. But I'm getting the 'end of file' unexpected for the last line. The file sid_home.txt gets generated as expected, but the script... (6 Replies)
Discussion started by: sagarparadkar
6 Replies

4. Shell Programming and Scripting

[Solved] Problem in reading a file line by line till it reaches a white line

So, I want to read line-by-line a text file with unknown number of files.... So: a=1 b=1 while ; do b=`sed -n '$ap' test` a=`expr $a + 1` $here do something with b etc done the problem is that sed does not seem to recognise the $a, even when trying sed -n ' $a p' So, I cannot read... (3 Replies)
Discussion started by: hakermania
3 Replies

5. UNIX for Dummies Questions & Answers

Parsing file, reading each line to variable, evaluating date/time stamp of each line

So, the beginning of my script will cat & grep a file with the output directed to a new file. The data I have in this file needs to be parsed, read and evaluated. Basically, I need to identify the latest date/time stamp and then calculate whether or not it is within 15 minutes of the current... (1 Reply)
Discussion started by: hynesward
1 Replies

6. Shell Programming and Scripting

Reading text file, comparing a value in a line, and placing only part of the line in a variable?

I need some help. I would like to read in a text file. Take a variable such as ROW-D-01, compare it to what's in one line in the text file such as PROD/VM/ROW-D-01 and only input PROD/VM into a variable without the /ROW-D-01. Is this possible? any help is appreciated. (2 Replies)
Discussion started by: xChristopher
2 Replies

7. Shell Programming and Scripting

Comparison of fields then increment a counter reading line by line in a file

Hi, i have a scenario were i should compare a few fields from each line then increment a variable based on that. Example file 989878|8999|Y|0|Y|N|V 989878|8999|Y|0|N|N|V 989878|8999|Y|2344|Y|N|V i have 3 conditions to check and increment a variable on every line condition 1 if ( $3... (4 Replies)
Discussion started by: selvankj
4 Replies

8. Shell Programming and Scripting

Reading line by line from live log file using while loop and considering only those lines start from

Hi, I want to read a live log file line by line and considering those line which start from time stamp; Below code I am using, which read line but throws an exception when comparing line that does not contain error code tail -F /logs/COMMON-ERROR.log | while read myline; do... (2 Replies)
Discussion started by: ketanraut
2 Replies

9. UNIX for Beginners Questions & Answers

Reading a file line by line and print required lines based on pattern

Hi All, i want to write a shell script read below file line by line and want to exclude the lines which contains empty value for MOUNTPOINT field. i am using centos 7 Operating system. want to read below file. # cat /tmp/d5 NAME="/dev/sda" TYPE="disk" SIZE="60G" OWNER="root"... (4 Replies)
Discussion started by: balu1234
4 Replies
install::TempContent::Objects::mod_perl-2.0.9::docs::apiUseraContrinstall::TempContent::Objects::mod_perl-2.0.9::docs::api::Apache2::SubProcess(3)

NAME
Apache2::SubProcess -- Executing SubProcesses under mod_perl Synopsis use Apache2::SubProcess (); use Config; use constant PERLIO_IS_ENABLED => $Config{useperlio}; # pass @ARGV / read from the process $command = "/tmp/argv.pl"; @argv = qw(foo bar); $out_fh = $r->spawn_proc_prog($command, @argv); $output = read_data($out_fh); # pass environment / read from the process $command = "/tmp/env.pl"; $r->subprocess_env->set(foo => "bar"); $out_fh = $r->spawn_proc_prog($command); $output = read_data($out_fh); # write to/read from the process $command = "/tmp/in_out_err.pl"; ($in_fh, $out_fh, $err_fh) = $r->spawn_proc_prog($command); print $in_fh "hello "; $output = read_data($out_fh); $error = read_data($err_fh); # helper function to work w/ and w/o perlio-enabled Perl sub read_data { my ($fh) = @_; my $data; if (PERLIO_IS_ENABLED || IO::Select->new($fh)->can_read(10)) { $data = <$fh>; } return defined $data ? $data : ''; } # pass @ARGV but don't ask for any communication channels $command = "/tmp/argv.pl"; @argv = qw(foo bar); $r->spawn_proc_prog($command, @argv); Description "Apache2::SubProcess" provides the Perl API for running and communicating with processes spawned from mod_perl handlers. At the moment it's possible to spawn only external program in a new process. It's possible to provide other interfaces, e.g. executing a sub-routine reference (via "B::Deparse") and may be spawn a new program in a thread (since the APR api includes API for spawning threads, e.g. that's how it's running mod_cgi on win32). API
"spawn_proc_prog" Spawn a sub-process and return STD communication pipes: $r->spawn_proc_prog($command); $r->spawn_proc_prog($command, @argv); $out_fh = $r->spawn_proc_prog($command); $out_fh = $r->spawn_proc_prog($command, @argv); ($in_fh, $out_fh, $err_fh) = $r->spawn_proc_prog($command); ($in_fh, $out_fh, $err_fh) = $r->spawn_proc_prog($command, @argv); obj: $r ( "Apache2::RequestRec object" ) arg1: $command ( string ) The command to be "$exec()"'ed. opt arg2: "@argv" ( ARRAY ref ) A reference to an array of arguments to be passed to the process as the process' "ARGV". ret: ... In VOID context returns no filehandles (all std streams to the spawned process are closed). In SCALAR context returns the output filehandle of the spawned process (the in and err std streams to the spawned process are closed). In LIST context returns the input, outpur and error filehandles of the spawned process. since: 2.0.00 It's possible to pass environment variables as well, by calling: $r->subprocess_env->set($key => $value); before spawning the subprocess. There is an issue with reading from the read filehandle ($in_fh)): A pipe filehandle returned under perlio-disabled Perl needs to call select() if the other end is not fast enough to send the data, since the read is non-blocking. A pipe filehandle returned under perlio-enabled Perl on the other hand does the select() internally, because it's really a filehandle opened via ":APR" layer, which internally uses APR to communicate with the pipe. The way APR is implemented Perl's select() cannot be used with it (mainly because select() wants fileno() and APR is a crossplatform implementation which hides the internal datastructure). Therefore to write a portable code, you want to use select for perlio-disabled Perl and do nothing for perlio-enabled Perl, hence you can use something similar to the "read_data()" wrapper shown in the Synopsis section. Several examples appear in the Synopsis section. "spawn_proc_prog()" is similar to "fork()", but provides you a better framework to communicate with that process and handles the cleanups for you. But that means that just like "fork()" it gives you a different process, so you don't use the current Perl interpreter in that new process. If you try to use that method or fork to run a high-performance parallel processing you should look elsewhere. You could try Perl threads, but they are very expensive to start if you have a lot of things loaded into memory (since "perl_clone()" dups almost everything in the perl land, but the opcode tree). In the mod_perl "paradigm" this is much more expensive than fork, since normally most of the time we have lots of perl things loaded into memory. Most likely the best solution here is to offload the job to PPerl or some other daemon, with the only added complexity of communication. To spawn a completely independent process, which will be able to run after Apache has been shutdown and which won't prevent Apache from restarting (releasing the ports Apache is listening to) call spawn_proc_prog() in a void context and make the script detach and close/reopen its communication streams. For example, spawn a process as: use Apache2::SubProcess (); $r->spawn_proc_prog ('/path/to/detach_script.pl', $args); and the /path/to/detach_script.pl contents are: # file:detach_script.pl #!/usr/bin/perl -w use strict; use warnings; use POSIX 'setsid'; chdir '/' or die "Can't chdir to /: $!"; open STDIN, '/dev/null' or die "Can't read /dev/null: $!"; open STDOUT, '+>>', '/path/to/apache/error_log' or die "Can't write to /dev/null: $!"; open STDERR, '>&STDOUT' or die "Can't dup stdout: $!"; setsid or die "Can't start a new session: $!"; # run your code here or call exec to another program reopening (or closing) the STD streams and called "setsid()" makes sure that the process is now fully detached from Apache and has a life of its own. "chdir()" ensures that no partition is tied, in case you need to remount it. See Also mod_perl 2.0 documentation. Copyright mod_perl 2.0 and its core modules are copyrighted under The Apache Software License, Version 2.0. Authors The mod_perl development team and numerous contributors. perl v5.18.2 install::TempContent::Objects::mod_perl-2.0.9::docs::api::Apache2::SubProcess(3)
All times are GMT -4. The time now is 05:05 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy