Merge broken lines


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Merge broken lines
# 15  
Old 04-16-2012
Thanks Smilie birei is there anyway where we can run this script only if the file size of any particular file in grep directory is zero and if it is zero only then create a backup file and then move the edited new file to another directory and backed up files to some other dir??
# 16  
Old 04-16-2012
I didn't fully understand you.

Avoid execution of the script if exists any file without content, and then, what?
# 17  
Old 04-16-2012
What i need is... If in the grep folder there is a file of size zero which i see when i do ls. Only then this perl script should run. When there is a file of size 0, it should execute the script and then move the modified original file to another directory and the back up file to some other directory.
# 18  
Old 04-16-2012
Try next one with your own parameters to the script (it only accepts a suffix as back-up extension):
Code:
$ cat script2.pl 
use warnings;
use strict;
use Getopt::Long;
use File::Copy qw( move );
use File::Spec;

my ($destdir, $bakdir);

GetOptions(
        q[destdir=s] => \$destdir,
        q[bakdir=s] => \$bakdir,
) or die qq[ERROR reading arguments\n];

die qq[Incorrect arguments\n] 
        unless defined $destdir &&
                   defined $bakdir &&
                   -d $destdir &&
                   -d $bakdir;

unless ( grep { -z } @ARGV ) {
        die qq[There is no file of zero size\n];
}

@ARGV = grep { ! -z } @ARGV;

my @prev_line;

while ( <> ) {
        chomp;

        if ( /\A\*\*/ .. eof ) {
                next if m/\A\s*\Z/;
                if ( m/¦/ ) {
                        if ( @prev_line ) {
                                printf qq[%s\n], pop @prev_line;
                        }
                        push @prev_line, $_;
                        next;
                }

                printf qq[%s\n], (pop @prev_line || qq[]) . $_;
                next;
        }

        printf qq[%s\n], $_;
}
continue {
        if ( eof ) {
                move( 
                        $ARGV . $^I,
                        File::Spec->catfile( $bakdir, $ARGV . $^I ) 
                ) || warn qq[Couldn't move file $ARGV . $^I\n];

                move(
                        $ARGV,
                        File::Spec->catfile( $destdir, $ARGV )
                ) || warn qq[Couldn't move file $ARGV\n];

        }
}
$ perl -i.bak script2.pl --destdir=./dest --bakdir=./bak /tmp/grep/*

# 19  
Old 04-17-2012
birei will this avoid taking back up of files which are proper? i dont need back up of files which are proper....
# 20  
Old 04-17-2012
Which files are proper? I don't know your tree structure of files.
# 21  
Old 04-17-2012
ok will tell u the exact story :

The files will appear in a folder called "processing" in a partular directory. The script should target this folder . I was just considering grep folder as an example. What happens is files appear in this processing folder every minute or so. they get processed and move to a folder called sent. Now if there is an error as u know ( blank line or name appearing on next line ) in any of the files they get stuck in processing folder and do not get to sent folder. The files are something lik this after the processing :

Code:
GURD0906
GURD0906trim$$
GURD0906Service

Now the file with size 0 will be GURD0906trim$$ ... but we cant edit this file as this will not contain anything. We need to edit GURD0906 file which has the error. So when the script runs it should find the file with size 0... consider the first 8 characters of that file and edit it to remove the error. This file should be moved back to the a folder called "IN" which is in the same place as the processing directory. all the other files like

Code:
GURD0906trim$$
GURD0906Service

and the the backed up original file GURD0906.bak

should be moved to a folder called "failed reports" which is also next to processing folder.

And the most important part is if there is no file with size 0. then nothing should be done and backup file should not be created. otherwise we will have disk space issue.

This is the whole thing.

Last edited by Scrutinizer; 04-17-2012 at 12:14 PM.. Reason: code tags
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Joining broken lines and removing empty lines

Hi - I have req to join broken lines and remove empty lines but should NOT be in one line. It has to be as is line by line. The challenge here is there is no end of line/start of line char. thanks in advance Source:- 2003-04-34024|04-10-2003|Claims|Claim|01-13-2003|Air Bag:Driver;... (7 Replies)
Discussion started by: Jackceasar123
7 Replies

2. Shell Programming and Scripting

Merge Lines

Hello I have an input as below this is test we are( ) one end of description I am looking for output this is test we are () one end of description (2 Replies)
Discussion started by: Tomlight
2 Replies

3. Shell Programming and Scripting

Joining broken lines with awk or perl

Hi, I have a huge file with sql broken statements like: PP3697HB @@@@0 <<<<<<Record has been deleted as per PP3697HB>>>>>> FROM sys.xtab_ref rc,sys.xtab_sys f,sys.domp ur WHE RE rc.milf = ur.milf AND rc.molf = f.molf AND ur.dept = 'SWIT'AND ur .department = 'IND' AND share = '2' AND... (4 Replies)
Discussion started by: som.nitk
4 Replies

4. Shell Programming and Scripting

Merge lines

Hello I have a file with CAR 23 COLOR 12 CAR 44 COLOR 12 CAR 55 COLOR 20 SIZE BIG CAR 56 CAR 57 COLOR 11 How can merge the CAR and the COLOR + SIZE (if there are COLOR,SIZE) CAR 23 COLOR 12 CAR 44 COLOR 12 CAR 55 COLOR 20 SIZE BIG CAR 56 CAR 57 COLOR 11 Every line begin in... (4 Replies)
Discussion started by: sharong
4 Replies

5. UNIX for Dummies Questions & Answers

How to identify broken lines in a file?

Hi, I have a 100 byte length fixed width file . In that three rows are broken and went off to next line. How can I identify the broken lines? E.g. ABCD1234MNRD4321 abcd1234mnrd 4321 As you can see in my example my second row with small case alphabets is broken... (5 Replies)
Discussion started by: okkadu
5 Replies

6. Shell Programming and Scripting

Joining broken lines

I have a plain test file with a delimeter ''. In this file some lines are broken into two. The first part of these broken line will have 6 columns and the second part will have 4. These broken lines will be consicutive. I want to join the two consicutive lines which are having 6 fields and 4... (8 Replies)
Discussion started by: ratheeshjulk
8 Replies

7. Shell Programming and Scripting

remove blank lines and merge lines in shell

Hi, I'm not a expert in shell programming, so i've come here to take help from u gurus. I'm trying to tailor a csv file that i got to make it work for the LOAD FROM command. I've a datatable csv of the below format - --in file format xx,xx,xx ,xx , , , , ,,xx, xxxx,, ,, xxx,... (11 Replies)
Discussion started by: dvah
11 Replies

8. Shell Programming and Scripting

Merge two lines

Hi I have two lines of data formatted as displayed below shop, price, remark, date a,#N/A,order to -fd, 20091011 and would like it to be shop:a price:#N/A remark:order to -fd date:20091011 How can I do it? Many thanks (2 Replies)
Discussion started by: lalelle
2 Replies

9. Shell Programming and Scripting

awk / shell - Fix broken lines and data

Gurus, I am struggling with a issue and thought I could use some of your expertise. Need Help with this I have a flat file that has millions of records 24|john|account ~ info |56| 25|kuo|account ~ journal |58| 27|kim|account ~ journal |59| 28|San|account ~ journal |60|... (3 Replies)
Discussion started by: rimss
3 Replies
Login or Register to Ask a Question