Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Splitting a file and creating new files using Perl script

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Splitting a file and creating new files using Perl script
# 1  
Old 06-19-2013
Splitting a file and creating new files using Perl script
Hi All,

I am new to Scripting language.
I want to split a file and create several subfiles using Perl script.

Example :
File format :
Code:
Sourcename		ID			Date	               Nbr	
SU				IMYFDJ		9/17/2012	   5552159976555	
SU				BWZMIG		9/14/2012	   1952257857887	
AR				PEHQDF		11/26/2012	   0442045903874	
AR				ELIOAA		12/31/2012	   0442121341024

I want to split this files by Sourcename.
As this has two source name, so it will be splitting into two.

File 1 (after splitting) :

Code:
Sourcename		ID				Date	       Nbr	
SU				IMYFDJ			9/17/2012	5552159976555	
SU				BWZMIG			9/14/2012	1952257857887

File 2 (after splitting) :

Code:
Sourcename		ID				Date	       Nbr	
AR				PEHQDF			11/26/2012	0442045903874	
AR				ELIOAA			12/31/2012	0442121341024

Thanks a lot in advance,
Deepak
Last edited by Franklin52; 06-19-2013 at 07:06 AM.. Reason: Please use code tags
# 2  
Old 06-19-2013
Hi Deepak,

Code:
 
perl -F"\s+" -anle 'system("echo \"$_\" >> $F[0]'.txt'");' filename

# 3  
Old 06-19-2013
Hi Pravin,

I ran that code given by you, it provided me four splitting files. Two files without below headings.

Code:
Sourcename ID Date Nbr

Another file with only this heading and one blank text file.

But I need this heading in the two splitted files.

Can you help me on this ?

-Deepak
# 4  
Old 06-19-2013
A Perl program doing the job
file025 (tabs have been replaces by spaces):
Code:
Sourcename ID Date Nbr
SU IMYFDJ 9/17/2012  5552159976555
SU BWZMIG 9/14/2012   1952257857887
AR PEHQDF 11/26/2012   0442045903874
AR ELIOAA 12/31/2012   0442121341024

Program :
Code:
#!/usr/bin/perl -w
use strict;

my $cur_dir = $ENV{PWD};
my $filename = "$cur_dir/$ARGV[0]";
my ($record,$header,$i,@fields,%files);

open(FILEIN,"<$filename") or die"open: $!";
while( defined( $record = <FILEIN> ) ) {
  chomp $record;
  $header=$record if (!defined $header);

  $i++;
  if($i > 1) {
    @fields=split(/ /,$record);
    # if file do not exits, create it and write header
    if(! exists( $files{$fields[0]}) ) {
      $files{$fields[0]} = "$fields[0].file";
      open (FILEOUT, "> /tmp/$files{$fields[0]}") ||
        die "FATAL: cannot open \"$files{$fields[0]}\" for writing: $!\n";
      print FILEOUT "$header\n";
      close(FILEOUT);
    }
    # append record to file
    open (FILEOUT, ">> /tmp/$files{$fields[0]}") ||
      die "FATAL: cannot open \"$files{$fields[0]}\" for writing: $!\n";
    print FILEOUT "$record\n";
    close(FILEOUT);
  }
}
close(FILEIN);

Outputs :
Code:
/tmp %cat SU.file
Sourcename ID Date Nbr
SU IMYFDJ 9/17/2012  5552159976555
SU BWZMIG 9/14/2012   1952257857887
/tmp %cat AR.file
Sourcename ID Date Nbr
AR PEHQDF 11/26/2012   0442045903874
AR ELIOAA 12/31/2012   0442121341024

# 5  
Old 06-20-2013
Hi Fundix,

Thanks a lot for your help..its working fine Smilie.

In the file if I am using delimiter '|' instead of space,
It is giving output file name as single letter.

Example :

Code:
     $i++;
  if($i > 1) {
    @fields=split(/|/,$record);
    # if file do not exits, create it and write header
    if(! exists( $files{$fields[0]}) ) {
      $files{$fields[0]} = "$fields[0].file";
      open (FILEOUT, "> /tmp/$files{$fields[0]}") ||
        die "FATAL: cannot open \"$files{$fields[0]}\" for writing: $!\n";
      print FILEOUT "$header\n";
      close(FILEOUT);

Result :

Filename :
Code:
S.file

Sometimes I have Sourcename more than 2 letters also.

-Deepak
# 6  
Old 06-20-2013
Hi,

guessing your file looks like :
Code:
Sourcename ID Date Nbr
SU|IMYFDJ|9/17/2012|5552159976555
SU|BWZMIG|9/14/2012|1952257857887
AR|PEHQDF|11/26/2012|0442045903874
AR|ELIOAA|12/31/2012|0442121341024

you must escape the pipe "|" in the splitting line that way :
Code:
@fields=split(/\|/,$record);

# 7  
Old 06-20-2013
Hi Fundix,

Thanks a lot.. u are awesome Smilie.. Its working fine.

I have another requirement that is to zip the splitted files after creating each of them.

Can that be done in perl script too ?

-Deepak
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Creating script to read many files and load into database via single control file

Hi, I have many files but with only 2 names , I want to load the data of that file into database through sqlldr with single control file. how can i do that ????? Example: switch_file switch_file billing_file billing_file now these files should be loaded into same database but different... (1 Reply)
Discussion started by: niti_sharma
1 Replies

2. Shell Programming and Scripting

Script for splitting file of records into multiple files

Hello I have a file of following format HDR 1234 abc qwerty abc def ghi jkl HDR 4567 xyz qwerty abc def ghi jkl HDR 890 mno qwerty abc def ghi jkl HDR 1234 abc qwerty abc def ghi jkl HDR 1234 abc qwerty abc def ghi jkl -Need to split this into multiple files based on tag... (8 Replies)
Discussion started by: wincrazy
8 Replies

3. Shell Programming and Scripting

Splitting xml file into several xml files using perl

Hi Everyone, I'm new here and I was checking this old post: /shell-programming-and-scripting/180669-splitting-file-into-several-smaller-files-using-perl.html (cannot paste link because of lack of points) I need to do something like this but understand very little of perl. I also check... (4 Replies)
Discussion started by: mcosta
4 Replies

4. Shell Programming and Scripting

Splitting a file into several smaller files using perl

Hi, I'm trying to split a large file into several smaller files the script will have two input arguments argument1=filename and argument2=no of files to be split. In my large input file I have a header followed by 100009 records The first line is a header; I want this header in all my... (9 Replies)
Discussion started by: ramky79
9 Replies

5. UNIX for Advanced & Expert Users

Splitting a file into small files

Hi Folks, Please help me in solving the problem. I want to write script in order to split a file into small pieces and send it automatically through mail. Ex. The file name is CALM*.txt . It is around 50 MB. I want to split the file into 20 MB 2-3 smaller files and send (like uuencode) it... (6 Replies)
Discussion started by: piyushbhashkar
6 Replies

6. Shell Programming and Scripting

Data Splitting into two files from one file

I have a file as: I/P File: Ground Car 2009 Lib 2008 Lib 2003 Ground Car 2009 Ground Car 2003 Car 2005 Car 2003 Car 2005 Sita 2900 2006 Car 2007 I have to split the file into two: - one for names and second for years. O/p1 (Names): Ground Car (3 Replies)
Discussion started by: karumudi7
3 Replies

7. Shell Programming and Scripting

Splitting file into 2 files ?

Hi extending to one of my previous posted query .... I am using nawk -v invar1="$aa" '{print > ("ABS\_"((/\|/)?"A\_":"B\_")invar1"\_NETWORKID.txt")}' spfile.txt to get 2 different files based on split condition i.e. "|" Similar to invar1 variable in nawk I also need one more variable... (18 Replies)
Discussion started by: shekharjchandra
18 Replies

8. Shell Programming and Scripting

Splitting the files via shell script

Hi all, We have 102 flat files created by Informatica from 102 tables. These 102 files contain pharmcy details. There are a total of 450 pharmcyids.The naming convention for the flat file is ODS_<TABLE NAME>_yyyymmdd_timestamp.dat. Each flat file may contain data for 450 pharmacies which is... (2 Replies)
Discussion started by: Maya_Pillai
2 Replies

9. Shell Programming and Scripting

Splitting files from one file

Hi, I have an input file like: 111 abcdefgh asdfghjk dfghjkl 222 aaaaaaa bbbbbb 333 djfhfgjktitjhgfkg 444 djdhfjkhfjkghjkfg hsbfjksdbhjkgherjklg fjkhfjklsahjgh fkrjkgnj I want to read this input file and make separate output files with the header as numric value like "111"... (9 Replies)
Discussion started by: saltysumi
9 Replies

10. Shell Programming and Scripting

Creating loop for a script -Perl

Hi Guyz I designed a script that can compare 2 columns(values) of single file and gives the closest numbers to the first column by comparing the numbers in first column with second and it works in a single file. Now I'm trying to design a new script with 2 objectives for 2 files (not a single... (4 Replies)
Discussion started by: repinementer
4 Replies
Login or Register to Ask a Question