Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Splitting a file into several smaller files using perl
Top Forums Shell Programming and Scripting Splitting a file into several smaller files using perl Post 302615521 by ramky79 on Thursday 29th of March 2012 03:06:28 PM
Old 03-29-2012
Splitting a file into several smaller files using perl
Hi,
I'm trying to split a large file into several smaller files
the script will have two input arguments argument1=filename and argument2=no of files to be split.

In my large input file I have a header followed by 100009 records
The first line is a header; I want this header in all my splitted files

Here is what I have done so far
Code:
#!/usr/bin/perl
use File::Basename;

$inputfile=@ARGV[0];
$nof=@ARGV[1];                              # nof - no of files to split
($filename,$dir,$ext) = fileparse($inputfile,'\..*');
$header=`cat $inputfile | head -1`;
$NOLIF=`cat $inputfile | wc -l`;         # NOLIF - no of lines in file
$NOARIF=$NOLIF-1;                         # NOARIF - no of actual records in file
$NORPF=$NOARIF/$not;                    # NORPF - no of records per file
$NNORPF=`printf "%1.f\n" $NORPF`;   # NNORPF - new no of records per file

$count=0;
$filenum=0;

while (<$inputfile>) {
if ( $count == 0 ) {
     $nfilename = $filename._.$filenum.$ext;
     open( FILE, ">> $nfilename" );
     print( FILE "$header\n" );
     print( FILE "$_" );
     $count++;
  #} elsif ( $count == $NUM_LINES ) {
  } elsif ( $count == $NNORPF ) {
     close( FILE );
     $count = 0;
     $file_num++;
  } else {
     # just write the line!
     print( FILE "$_" );
     $count++;
  }
}

Here is my challenge:
Say I'm splitting my large input file into 10 files
so the first 9 files should have 10001 records and last should have 10000 records.

how do i get this working.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Splitting a Larger File Into Mutiple Smaller ones.

Hello.. Iam in need to urgent help with the below. Have data-file with 40,567 and need to split them into multiple files with smaller line-count. Iam aware of "split" command with -l option which allows you to specify the no of lines in smaller files ,with the target file-name pattern... (1 Reply)
Discussion started by: madhubt_1982
1 Replies

2. UNIX for Dummies Questions & Answers

splitting the large file into smaller files

hi all im new to this forum..excuse me if anythng wrong. I have a file containing 600 MB data in that. when i do parse the data in perl program im getting out of memory error. so iam planning to split the file into smaller files and process one by one. can any one tell me what is the code... (1 Reply)
Discussion started by: vsnreddy
1 Replies

3. Shell Programming and Scripting

splitting text file into smaller ones

Hello We have a text file with 400,000 lines and need to split into multiple files each with 5000 lines ( will result in 80 files) Got an idea of using head and tail commands to do that with a loop but looked not efficient. Please advise the simple and yet effective way to do it. TIA... (3 Replies)
Discussion started by: prvnrk
3 Replies

4. Shell Programming and Scripting

perl help to split big verilog file into smaller ones for each module

Hi I have a big verilog file with multiple modules. Each module begin with the code word 'module <module-name>(ports,...)' and end with the 'endmodule' keyword. Could you please suggest the best way to split each of these modules into multiple files? Thank you for the help. Example of... (7 Replies)
Discussion started by: return_user
7 Replies

5. Shell Programming and Scripting

Help with splitting a large text file into smaller ones

Hi Everyone, I am using a centos 5.2 server as an sflow log collector on my network. Currently I am using inmons free sflowtool to collect the packets sent by my switches. I have a bash script running on an infinate loop to stop and start the log collection at set intervals - currently one... (2 Replies)
Discussion started by: lord_butler
2 Replies

6. Shell Programming and Scripting

How to split a file into smaller files

Hi, I have a big text file with m columns and n rows. The format is like: STF123450001000200030004STF123450005000600070008STF123450009001000110012 STF234560345002208330154STF234590705620600070080STF234567804094562357688 STF356780001000200030004STF356780005000600070080STF356780800094562657687... (2 Replies)
Discussion started by: wintersnow2011
2 Replies

7. Shell Programming and Scripting

Sed: Splitting A large File into smaller files based on recursive Regular Expression match

I will simplify the explaination a bit, I need to parse through a 87m file - I have a single text file in the form of : <NAME>house........ SOMETEXT SOMETEXT SOMETEXT . . . . </script> MORETEXT MORETEXT . . . (6 Replies)
Discussion started by: sumguy
6 Replies

8. Shell Programming and Scripting

Splitting a file and creating new files using Perl script

Hi All, I am new to Scripting language. I want to split a file and create several subfiles using Perl script. Example : File format : Sourcename ID Date Nbr SU IMYFDJ 9/17/2012 5552159976555 SU BWZMIG 9/14/2012 1952257857887 AR PEHQDF 11/26/2012 ... (13 Replies)
Discussion started by: Deepak9870
13 Replies

9. Shell Programming and Scripting

Splitting xml file into several xml files using perl

Hi Everyone, I'm new here and I was checking this old post: /shell-programming-and-scripting/180669-splitting-file-into-several-smaller-files-using-perl.html (cannot paste link because of lack of points) I need to do something like this but understand very little of perl. I also check... (4 Replies)
Discussion started by: mcosta
4 Replies

10. Shell Programming and Scripting

Splitting a text file into smaller files with awk, how to create a different name for each new file

Hello, I have some large text files that look like, putrescine Mrv1583 01041713302D 6 5 0 0 0 0 999 V2000 2.0928 -0.2063 0.0000 N 0 0 0 0 0 0 0 0 0 0 0 0 5.6650 0.2063 0.0000 N 0 0 0 0 0 0 0 0 0 0 0 0 3.5217 ... (3 Replies)
Discussion started by: LMHmedchem
3 Replies

LEARN ABOUT SUNOS

auditstat

auditstat(1M)						  System Administration Commands					     auditstat(1M)

NAME

       auditstat - display kernel audit statistics

SYNOPSIS

       auditstat [-c count] [-h numlines] [-i interval] [-n] [-v]

DESCRIPTION

       auditstat displays kernel audit statistics. The fields displayed are as follows:

       aud	The total number of audit records processed by the audit(2) system call.

       ctl	This field is obsolete.

       drop	The  total  number  of	audit  records that have been dropped. Records are dropped according to the kernel audit policy. See audi-
		ton(2), AUDIT_CNT policy for details.

       enq	The total number of audit records put on the kernel audit queue.

       gen	The total number of audit records that have been constructed (not the number written).

       kern	The total number of audit records produced by user processes (as a result of system calls).

       mem	The total number of Kbytes of memory currently in use by the kernel audit module.

       nona	The total number of non-attributable audit records that have been constructed. These are audit records that are  not  attributable
		to any particular user.

       rblk	The total number of times that auditsvc(2) has blocked waiting to process audit data.

       tot	The total number of Kbytes of audit data written to the audit trail.

       wblk	The total number of times that user processes blocked on the audit queue at the high water mark.

       wrtn	The  total  number of audit records written. The difference between enq and wrtn is the number of outstanding audit records on the
		audit queue that have not been written.

OPTIONS

       -c count        Display the statistics a total of count times. If count is equal to zero, statistics are displayed  indefinitely.   A  time
		       interval must be specified.

       -h numlines     Display	a  header  for every  numlines of statistics printed. The default is to display the header every 20 lines. If num-
		       lines is equal to zero, the header is never displayed.

       -i interval     Display the statistics every interval where interval is the number of seconds to sleep between each collection.

       -n	       Display the number of kernel audit events currently configured.

       -v	       Display the version number of the kernel audit module software.

EXIT STATUS

       auditstat returns 0 upon success and 1 upon failure.

ATTRIBUTES

       See attributes(5) for descriptions of the following attributes:

       +-----------------------------+-----------------------------+
       |      ATTRIBUTE TYPE	     |	    ATTRIBUTE VALUE	   |
       +-----------------------------+-----------------------------+
       |Availability		     |SUNWcsu			   |
       +-----------------------------+-----------------------------+

SEE ALSO

       auditconfig(1M), praudit(1M), bsmconv(1M), audit(2), auditon(2), auditsvc(2), attributes(5)

NOTES

       The functionality described in this man page is available only if the Basic Security Module (BSM) has been enabled.  See   bsmconv(1M)  for
       more information.

SunOS 5.10							    6 May 1993							     auditstat(1M)
All times are GMT -4. The time now is 12:24 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy