Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

In PErl script: need to read the data one file and generate multiple files based on the data

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting In PErl script: need to read the data one file and generate multiple files based on the data
# 1  
Old 03-08-2018
In PErl script: need to read the data one file and generate multiple files based on the data
We have the data looks like below in a log file.
I want to generat files based on the string between two hash(#) symbol like below

Source:

Code:
#ext1#test1.tale2 drop
#ext1#test11.tale21 drop
#ext1#test123.tale21 drop
#ext2#test1.tale21 drop
#ext2#test12.tale21 drop
#ext3#test11.tale21 drop
#ext3#test123.tale21 drop
#ext4#test1.tale21 drop
#ext4#test124.tale21 drop

Desired output :

ext1.log
Code:
test1.tale2 drop
test11.tale21 drop
test123.tale21 drop

ext2.log
Code:
test1.tale21 drop
test12.tale21 drop

ext3.log
Code:
test11.tale21 drop
test123.tale21 drop

ext4.log
Code:
test1.tale21 drop
test124.tale21 drop

Moderator's Comments:
Mod Comment Please use CODE tags as required by forum rules!
Last edited by RudiC; 03-08-2018 at 05:56 AM.. Reason: Added CODE tags.
# 2  
Old 03-08-2018
If not necessarily perl:
Code:
while IFS="#" read _ file data
do
   echo $data >> "$file".log
done < Source

# 3  
Old 03-08-2018
Code:
# use_sanjeev_g.file.pl
use strict;
use warnings;

my $last_name = '';
my $writeout;
while(<>) {
    if (/^#(\w+)#(.+)$/) {
        if ($last_name ne $1) {
            close $writeout if $writeout;
            open($writeout, '>', "$1.log");
        }
        $last_name = $1;
        print $writeout "$2\n" if $writeout;
    }
}
close $writeout or die;

Code:
perl use_sanjeev_g.file.pl sanjeev_g.file

Code:
$ ls ext[0-9]*.log | while read f; do printf "file: $f\n--------------\n";cat $f; echo; done

Output:
Code:
file: ext1.log
--------------
test1.tale2 drop
test11.tale21 drop
test123.tale21 drop

file: ext2.log
--------------
test1.tale21 drop
test12.tale21 drop

file: ext3.log
--------------
test11.tale21 drop
test123.tale21 drop

file: ext4.log
--------------
test1.tale21 drop
test124.tale21 drop

This User Gave Thanks to Aia For This Post:
Sanjeev G
# 4  
Old 03-09-2018
Thanks Aia, It working fine for below condition.
If we want to add/append few lines in all the the file which was generated,Could you please help with the logic.

want to add below data in all the output file

Example :


Code:
EXTRACT e1_bill1
SETENV (ORACLE_SID = "TV19DBM")
SETENV (ORACLE_HOME = /opt/oracle/product/11.2.0.4/db_1)
SETENV (NLS_LANG = "AMERICAN_AMERICA.US7ASCII")
 
--USERID GGSCHEMA, PASSWORD , &
USERID 4444444A@xxxxxx, PASSWORD "xxxxxx"
 
TRANLOGOPTIONS DBLOGREADER
TRANLOGOPTIONS excludeuser GGSCHEMA
TRANLOGOPTIONS excludeuser esg_dba
nocompressupdates -- Include the full record
nocompressdeletes -- Include the full record
-- To avoid need to delete log manually[but may need to enable it in Dataguard Environemnt]
--TRANLOGOPTIONS LOGRETENTION DISABLED
-- Cache Manger parameter to limit memory utilization by extract
CACHEMGR CACHESIZE 1GB
-- Must be used together and directs Extract to fetch data from the source table if it cannot fetch
FETCHOPTIONS USELATESTVERSION, NOUSESNAPSHOT
--Warns for a long running transaction
WARNLONGTRANS 1h, CHECKINTERVAL 10m


Moderator's Comments:
Mod Comment Please use CODE tags as required by forum rules!
Last edited by RudiC; 03-09-2018 at 09:01 AM.. Reason: Added CODE tags.
# 5  
Old 03-09-2018
It is best to create a different thread if the request/inquire is different than the original post.
Code:
while(<>) {
    print $_;
}

while (<DATA>) {
    print $_;
}

__DATA__
EXTRACT e1_bill1
SETENV (ORACLE_SID = "TV19DBM")
SETENV (ORACLE_HOME = /opt/oracle/product/11.2.0.4/db_1)
SETENV (NLS_LANG = "AMERICAN_AMERICA.US7ASCII")

--USERID GGSCHEMA, PASSWORD , &
USERID 4444444A@xxxxxx, PASSWORD "xxxxxx"

TRANLOGOPTIONS DBLOGREADER
TRANLOGOPTIONS excludeuser GGSCHEMA
TRANLOGOPTIONS excludeuser esg_dba
nocompressupdates -- Include the full record
nocompressdeletes -- Include the full record
-- To avoid need to delete log manually[but may need to enable it in Dataguard Environemnt]
--TRANLOGOPTIONS LOGRETENTION DISABLED
-- Cache Manger parameter to limit memory utilization by extract
CACHEMGR CACHESIZE 1GB
-- Must be used together and directs Extract to fetch data from the source table if it cannot fetch
FETCHOPTIONS USELATESTVERSION, NOUSESNAPSHOT
--Warns for a long running transaction
WARNLONGTRANS 1h, CHECKINTERVAL 10m

# 6  
Old 03-13-2018
take backup of old logs and generate new logs
Hi Aia,

Thanks for the reply.....
Appreciate your help for the below issue.
Im using below code.....I dont want to attach the logs when I ran the perl twice...I just want to take backup with today date and generate new logs...What I need to do for the below scirpt..............

1)if logs exist it should move the logs with extention of today date nad need to generate new one
2) only for first log file coming perfectly and for remaining logfiles and input.txt not adding in the outputfile.

Script :

Code:
my $last_name = '';
my $writeout;
 
while(<>) {
    if (/^#(\w+)#(.+)$/) {
        if ($last_name ne $1) {
            close $writeout if $writeout;
            open($writeout, '>>', "$1.log");
        }
        $last_name = $1;
if ($. == 1) {
print $writeout "Extract $1 \n";  
my $filename="input.txt";
open (my $ip, "<" , $filename) || die ("Can't open file input.txt");
while ( <$ip> ) {
    next if  (/^$/);
    print $writeout  "$_";
}
close $ip;
}
        print $writeout "$2\n" if $writeout;
    }
}
close $writeout or die;

input files :
Code:
more sanj.txt

#ext1#test1.tale2 drop
#ext1#test11.tale21 drop
#ext1#test123.tale21 drop
#ext2#test1.tale21 drop
#ext2#test12.tale21 drop
#ext3#test11.tale21 drop
#ext3#test123.tale21 drop
#ext4#test1.tale21 drop
#ext4#test124.tale21 drop
#ext1#test1.tale2 drop




Code:
more input.txt

1.1.1.1
2.2.2.2


Code:
ls ext[0-9]*.log | while read f; do printf "file: $f\n--------------\n";cat $f; echo; done


file: ext1.log

  Extract ext1 
1.1.1.1
2.2.2.2
test1.tale2 drop
test11.tale21 drop
test123.tale21 drop
test1.tale2 drop

file: ext2.log
--------------
  test1.tale21 drop
test12.tale21 drop

file: ext3.log
--------------
 test11.tale21 drop
test123.tale21 drop

file: ext4.log
--------------
  test1.tale21 drop
test124.tale21 drop

Thanks,
G sanjeev Kumar
Last edited by Sanjeev G; 03-14-2018 at 02:24 AM.. Reason: adding in corrected format.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Generate files and use csv data to replace multiple variables in a template

I have a source csv file consists of first field as variable name, and the rest are site-specific information (converted from excel file, where site -specific values in columns). I am trying to create a file for every site using a template and replace the multiple variables with values from the... (3 Replies)
Discussion started by: apalex
3 Replies

2. Shell Programming and Scripting

Read multiple text files and copy data to csv

hi i need to extract lines from multiple files to a csv file. for example, i have these 3 files file1.txt date:29dec1980 caller:91245824255 called:8127766 file2.txt date:11apr2014 caller:9155584558 called:8115478 file3.txt date:25jun2015 caller:445225552 called:8117485 (30 Replies)
Discussion started by: lp.descamps
30 Replies

3. Shell Programming and Scripting

Generate Join clause based on key data

Hi, I have a file pk.txt which has pk data in following format TableName | PK Employee | id Contact|name,country My Output should be Employee | t1.id=s.id Contact| t1.name=s.name AND t1.country=s.country I started of like this: for LIST in `cat pk.txt` do... (5 Replies)
Discussion started by: wahi80
5 Replies

4. Shell Programming and Scripting

Need a UNIX/perl script to read and write the data

Hi, I have on Designdocument in that information is stored with in tabular format.I need Perl/unix script to read and write the data using perl script? Regards, Ravi (4 Replies)
Discussion started by: toravi.pentaho
4 Replies

5. Shell Programming and Scripting

Need a perl script to read and write the data

Hi, I have on Designdocument in that information is stored with in tabular format.I need Perlscript to read and write the datausing perl script? Regards, Ravi (0 Replies)
Discussion started by: toravi.pentaho
0 Replies

6. Shell Programming and Scripting

Generate tabular data based on a column value from an existing data file

Hi, I have a data file with : 01/28/2012,1,1,98995 01/28/2012,1,2,7195 01/29/2012,1,1,98995 01/29/2012,1,2,7195 01/30/2012,1,1,98896 01/30/2012,1,2,7083 01/31/2012,1,1,98896 01/31/2012,1,2,7083 02/01/2012,1,1,98896 02/01/2012,1,2,7083 02/02/2012,1,1,98899 02/02/2012,1,2,7083 I... (1 Reply)
Discussion started by: himanish
1 Replies

7. Shell Programming and Scripting

Read multiple files, parse data and append to a file

Hi..Can anyone suggest a simple way of achieving this. I have several files which ends with extension .vcf . I will give example with two files In the below files, we are interested in File 1: 38 107 C 3 T 6 C/T 38 241 C 4 T 5 C/T 38 247 T 4 C 5 T/C 38 259 T 3 C 6 T/C... (8 Replies)
Discussion started by: empyrean
8 Replies

8. Shell Programming and Scripting

Read Data from Config file using Perl

Hi All, Can anyone please explain me how to read data from config file in Perl. Suppose i have a config file named cfile. The data in config file is name=parth lname=mittal user=2007 hostname=fluoride username=parthmittal password=XXXXXX account=unix url=www.unix.com ... (2 Replies)
Discussion started by: parthmittal2007
2 Replies

9. Shell Programming and Scripting

generate report based on data in files.

Hi All, I need to develop a shell script which does sanity check of a data file, as below. 1. For DATE columns, it should check if date is given in proper format or not? For example, if format of date is expected as DD-MON-YYYY HH24:MI:SS and we received the date in formation like DDMMYYYY HH24,... (1 Reply)
Discussion started by: ace_friends22
1 Replies

10. Shell Programming and Scripting

Read the data from multiple files and sum the value

Hi all, I have a requirement where i have to read multiple files using Shell Script in Korn Shell. each file will have the 3rd line as the amount field, i have to read this amount field and sum it for all the files. any idea on how to achieve this?? (i think i can achieve it using a loop,... (9 Replies)
Discussion started by: nvuradi
9 Replies
Login or Register to Ask a Question