Help fix my garbage - File Split Program in Perl


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Help fix my garbage - File Split Program in Perl
# 1  
Old 06-26-2009
Help fix my garbage - File Split Program in Perl

Hi,

I have the following, it doesn't work and I know it's crap code.

The objective is to split a file with a givin number of codes such as:

01,02,03,...,99

Then return all records with each seperate identifier in a new file.

The files being split have lrecl=500, recfm=F, and I am using two fields in these files, one at byte 377 with length of 2 (this is the segment identifier) and a second at byte 393 with length of 25 (this is the keycode for the segment) which is in general shorted then 25 charaters and needs to being included in the output filename.

Here is my garbage:
Code:
 
#!/usr/bin/perl
 
($ProgName = $0) =~ s%.*/%%;
print STDOUT "Enter the number of segments to split\n";
$splitnum = <STDIN>;
chomp($splitnum);
if ($#ARGV>0) { print STDERR " *** One file at a time, please!\n", exit 1; }
if ($#ARGV<0) { print STDERR " *** Sorry, need a filename!\n", exit 1; }
print STDOUT "\n$ProgName: Beginning $splitnum way split on $ARGV[0]\n\n";
open INFILE, "<$ARGV[0]" or die "\n$ProgName: *** Can't open data file '$ARGV[0]': $!\n\n";
while (<INFILE>)
 {
  $counter++;
  if ($counter%100000 == 0) { print "$counter\n"; }
  my $seg = substr($_,376,2);
  my $key = substr($_,392,25);
  chop($key);
  $a = 1;
  while($i<=$splitnum)
   {
    $oflename = "OFLE$key";
    open $oflename, ">$ARGV[0].$key.dat" or die "\n$ProgName: *** Can't open data file '$ARGV[0].$key.dat'";
    if ($a <~ m/10/i) { $b="0$a"; if ($seg =~ m/$b/i) { print $oflename; } }
    if ($a >=~ m/10/i) { if ($seg =~ m/$a/i) { print $oflename; } }
    $a++
   }
 }

I appreciate the help and am also open to any other suggestions to use other shell scripts, I was thinking awk could do this easily, but don't know that well either.

Thanks!
# 2  
Old 06-26-2009
This is not a perl answer, but you do know that csplit will do this for you.

Try man csplit.

If this is not a practical app you are developing, but programming learning, go for it.
# 3  
Old 06-26-2009
Use the Perl debugger

By far the easiest way to find out why Perl code isn't working the way you think it should is to use the Perl debugger.
# 4  
Old 06-26-2009
Thanks for your input, I'm not familiar with csplit and after reading the manual I'm still not certain how to utilize it to do what I need without passing the command multiple times which I would rather not do especially when some file can have up to 99 segments and over 100million lines. I was looking at awk and I figured out something but it is also not working 100% how I'd like

Code:
awk '!/^$/{
key=substr($0,393,25)
print $0 > key".dat"
}' testmail.dat

Here I'm trying to do the same as mentioned above and the splitting works, but I can't get the output nameing convention the way I want. I need the output filename formatted as 'input filename'.$key.'dat' and I need the $key variable trimmed on extranious white spaces after the actual value ends as it is not always 25 characters long. Also how can I turn this into a self contained script so I can use it as a single command such as $ awksplit testmail.dat

Thanks again!!!
# 5  
Old 06-27-2009
Hi.

Welcome to the forum.

Using phrases like it doesn't work gives us very little information. If it worked, you would probably not be here, so to say it is not working just is not helpful.

However, I (and I suspect others) would not be likely to wade through your code. Asking specific questions is is the way to get the best help here and in most other forums.

If I needed to work with fixed-length data, I would use perl's read function instead of the <> syntax:
Code:
       read FILEHANDLE,SCALAR,LENGTH,OFFSET
       read FILEHANDLE,SCALAR,LENGTH
               Attempts to read LENGTH characters of data into variable SCALAR
               from the specified FILEHANDLE.  Returns the number of
               characters actually read, 0 at end of file, or undef if there
               was an error (in the latter case $! is also set). 

-- excerpt from perldoc -f read

On the other hand, if awk works for you, great.

When I am tackling a problem, I usually create a small data set for testing. I would also make sure it was small enough to post in a forum, because posting sample data and expected output is is a good way to attract answers. In your case, I would not post the long records, but I would make up a dataset with lrecl=50 or so, with perhaps 10 records or whatever is a representative sample.

Best wishes ... cheers, drl
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

PERL: split 2 part from singel file.

Hi, I would like to split single fine into two array .. Example: file.txt --------------Installation -------------------- #GXTOOL=GxTools-20130501.tar.gz GCSS=GExpLinux-BE-3700.0.12.37.tar.gz TOP=TOPLinux-BE-3700.0.6.21.tar.gz GHDER=GHDERLinux-BE-3700.0.6.20.tar.gz... (2 Replies)
Discussion started by: Mani_apr08
2 Replies

2. Shell Programming and Scripting

Perl : to split the tags from xml file

I do have an xml sheet as below where I need the perl script to filter only the hyperlink tags. <cols><col min="1" max="1" width="30.5703125" customWidth="1"/><col min="2" max="2" width="7.140625" bestFit="1" customWidth="1"/> <col min="3" max="3" width="32.28515625" bestFit="1"... (3 Replies)
Discussion started by: scriptscript
3 Replies

3. UNIX for Dummies Questions & Answers

[Solved] Perl Question - split function with csv file

Hi all, I have a csv file that appears as follows: ,2013/03/26,2012/12/26,4,1,"2017/09/26,5.75%","2017/09/26,1,2018/09/26,1,2019/09/26,1,2020/09/26,1,2021/09/26,1",,,2012/12/26,now when i use the split function like this: my @f = split/,/; the split function will split the data that is... (2 Replies)
Discussion started by: WongSifu
2 Replies

4. Shell Programming and Scripting

perl script to split the text file after every 4th field

I had a text file(comma seperated values) which contains as below 196237,ram,25-May-06,ram.kiran@xyz.com,204183,Pavan,4-Jun-07,Pavan.Desai@xyz.com,237107,ram Chandra,15-Mar-10,ram.krishna@xyz.com ... (3 Replies)
Discussion started by: giridhar276
3 Replies

5. Shell Programming and Scripting

How do I split file into pieces with PERL?

How do I split file into pieces with PERL? IE file.txt head 1 2 3 4 end head 5 6 7 8 9 end n so on (7 Replies)
Discussion started by: 3junior
7 Replies

6. Shell Programming and Scripting

Perl Split for text in file

Hi all I have written Perl script to swap the strings in the second a third column from a text file. My input file format is : the|empty|the|det lake|empty|lake|conj_and was|empty|was|auxpass drained|empty|drained|conj_and birds|empty|bird|s|nn The expected output file format is... (11 Replies)
Discussion started by: my_Perl
11 Replies

7. Shell Programming and Scripting

how to get split output of a file, using perl script

Hi, I have file: data.log.1 ### s1 main.build.3495 main.build.199 main.build.3408 ###s2 main.build.3495 main.build.3408 main.build.199 I want to read this file and store in two arrays in Perl. I have following command, which is working fine on command prompt. perl -n -e... (1 Reply)
Discussion started by: ashvini
1 Replies

8. Shell Programming and Scripting

perl help to split big verilog file into smaller ones for each module

Hi I have a big verilog file with multiple modules. Each module begin with the code word 'module <module-name>(ports,...)' and end with the 'endmodule' keyword. Could you please suggest the best way to split each of these modules into multiple files? Thank you for the help. Example of... (7 Replies)
Discussion started by: return_user
7 Replies

9. Solaris

Help me fix sdtfontadm program

Hi all. I am trying to add font to use however sdtfontadm fail so many times regardless of using GUI or CDE. It look likes a very regular error. It release "Segamentation Fault (core dump) ./sdtfontadm " The serious of this problem is its hard to find a way without reinstalling the... (3 Replies)
Discussion started by: tien86
3 Replies

10. Shell Programming and Scripting

How to split a text after fix character count

HI, I want to split a text after certain fix character count in text. For eg: My file is containing text like: AURBJR,AURCID,AURVID,CHANDV,DAMNEW,DHMMAN,GANGAN,GARKHE,GOREGA,JEJKHA,JEJSHI,JINTUR,JMKKUS,JUNAWA,KALKAL,KHOJEW,KUNJIR,MAGARP,MAHAD, in this i want to print text after each... (5 Replies)
Discussion started by: vikash.rastogi
5 Replies
Login or Register to Ask a Question