Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Splitting data file
Top Forums Shell Programming and Scripting Splitting data file Post 59985 by zazzybob on Thursday 6th of January 2005 10:51:38 AM
Old 01-06-2005
There's probably an easier way to do it, but I'd do something like this with Perl...
Code:
#!/usr/bin/perl

# Just modify the following two variables
$NUM_LINES=10;
$HEADER_STRING="This is my header";

$count=0;
$file_num=0;

while (<>) {
  if ( $count == 0 ) {
     $filename = join("","file",$file_num);
     open( FILE, ">> $filename" );
     print( FILE "$HEADER_STRING\n" );
     print( FILE "$_" );
     $count++;
  } elsif ( $count == $NUM_LINES ) {
     close( FILE );
     $count = 0;
     $file_num++;
  } else {
     # just write the line!
     print( FILE "$_" );
     $count++;
  }
}

Then make it executable and call with
$ my_perl_script.pl my_large_file

This will split your large file into as many 10 data line files (i.e. 11 lines including the header) as required.

Just modify the $NUM_LINES and $HEADER_STRING variables to suit your needs.

Cheers
ZB
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Splitting the data and storing it into 2 variables

Hi, I have a file in the given format: |aaa |hhh |fat hat chat |make sure I need to get store in a variable,say 'error' the value of the string 'fat hat chat' In short, echo $error should give the result fat hat chat Is this possible using awk? Also, can this be split.For... (10 Replies)
Discussion started by: jisha
10 Replies

2. Shell Programming and Scripting

Data Splitting into two files from one file

I have a file as: I/P File: Ground Car 2009 Lib 2008 Lib 2003 Ground Car 2009 Ground Car 2003 Car 2005 Car 2003 Car 2005 Sita 2900 2006 Car 2007 I have to split the file into two: - one for names and second for years. O/p1 (Names): Ground Car (3 Replies)
Discussion started by: karumudi7
3 Replies

3. UNIX for Dummies Questions & Answers

Splitting Data in File

I have a file with the below Data 1,nj@ny@pa@caa 2,ct 3,ca@vaa@txI want the output to be 1,nj 1,ny 1,pa 1,caa 2,ct 3,ca 3,vaa 3,tx I need to split the second column based on @ as delimiter The number of delimiters is unknown (4 Replies)
Discussion started by: traininfa
4 Replies

4. Shell Programming and Scripting

Splitting the data in a column into several columns

Hi, I have the following input file 32895901-d17f-414c-ac93-3e7e0f5ec240 AND @GDF_INPUT 73b129e1-1fa9-4c0d-b95b-4682e5389612 AUS @GDF_INPUT 40f82e88-d1ff-4ce2-9b8e-d827ddb39447 BEL @GDF_INPUT 36e9c3f1-042a-43a4-a80e-4a3bc2513d01 BGR @GDF_INPUT I want to split column 3 into two columns:... (1 Reply)
Discussion started by: ramky79
1 Replies

5. Shell Programming and Scripting

perl : splitting the data into 2 different variables

I have a perl variable which contains the below value. $var1 = "2% / 51%" Now I would like to split the data into 2 different variables. For example $part1 = 2 $part2 = 51 Could anyone please help me in this regard ? Regards, GS (4 Replies)
Discussion started by: giridhar276
4 Replies

6. Shell Programming and Scripting

Help me pls : splitting single file in unix into different files based on data

I have a file in unix with sample data as follows : -------------------------------------------------------------- -------------------------------------------------------------- {30001002|XXparameter|Layout|$ I want this file to be splitted into different files and corresponding to the sample... (54 Replies)
Discussion started by: Ravindra Swan
54 Replies

7. UNIX for Dummies Questions & Answers

Splitting data into new records

Hi, My file is seperated with ";" delimiter, after 13 delimiter i want to put the data in new line... eg: My current file:- a;b;c;d;e;f;g;h;e;f;h;s;t;a;i;o;q;t;q;r;yu;f;sz;f;t;r........... i want o/p as:- a;b;c;d;e;f;g;h;e;f;h;s;t a;i;o;q;t;q;r;yu;f;sz;f;t;r How to achieve ths,... (2 Replies)
Discussion started by: gnnsprapa
2 Replies

8. UNIX for Dummies Questions & Answers

Extracting data from one file, based on another file (splitting)

Dear All, I have two files but want to extract data from one based on another... can you please help me file 1 David Tom Ellen and file 2 David|0010|testnamez|resultsz David|0004|testnamex|resultsx Tom|0010|testnamez|resultsz Tom|0004|testnamex|resultsx Ellen|0010|testnamez|resultsz... (12 Replies)
Discussion started by: A-V
12 Replies

9. Shell Programming and Scripting

Splitting the Data using awk

Hello All, I have a comma delimiter file with 10 columns. I took the desired data but from $4 I need to split into two columns as 3+7 bytes. awk -F"," -v OFS=',' '{print $2,$3,$4}' foo.txt 42366,11/10/2014,5012418769 42366,11/10/2014,2046955672 42366,11/10/2014,2076802951 ... (3 Replies)
Discussion started by: karumudi7
3 Replies

10. Shell Programming and Scripting

awk issue splitting a fixed-width file containing line feed in data

Hi Forum. I have the following script that splits a large fixed-width file into smaller multiple fixed-width files based on input segment type. The main command in the script is: awk -v search_col_pos=$search_col_pos -v search_str_len=$search_str_len -v segment_type="$segment_type"... (8 Replies)
Discussion started by: pchang
8 Replies

LEARN ABOUT HPUX

split

split(1)						      General Commands Manual							  split(1)

NAME

       split - split a file into pieces

SYNOPSIS

       line_count] suffix_length] [file [name]]

       nsuffix_length] [file [name]]

   Obsolescent
       n] [file [name]]

DESCRIPTION

       reads  file  and  writes  it  in  pieces  (default  1000 lines) onto a set of output files.  The name of the first output file is name with
       appended, and so on lexicographically, up to (only ASCII letters are used, a maximum of 676 files).  If no output name  is  given,  is  the
       default.

       If no input file is given, or if is given instead, the standard input file is used.

   Options
       recognizes the following command-line options and arguments:

	      The input file is split into pieces
			     line_count lines in size.

	      suffix_length  letters  are  used  to  form the suffix of the output filenames.  This option allows creation of more than 676 output
			     files.  The output file names created cannot exceed the maximum file name length allowed in the directory	containing
			     the files.

	      The input file is split into pieces
			     n bytes in size.

	      The input file is split into pieces
			     n x 1024 bytes in size.  No space separates the n from the

	      The input file is split into pieces
			     n x 1048576 bytes in size.  No space separates the n from the

	      The input file is split into pieces
			     n lines in size.  This option is obsolescent and is equivalent to using the option.

EXTERNAL INFLUENCES

   Environment Variables
       determines the locale for the interpretation of text as single- and/or multi-byte characters.

       determines the language in which messages are displayed.

       If  or  is  not specified in the environment or is set to the empty string, the value of is used as a default for each unspecified or empty
       variable.  If is not specified or is set to the empty string, a default of "C" (see lang(5)) is used instead of

       If any internationalization variable contains an invalid setting, behaves as if all internationalization variables are  set  to	"C".   See
       environ(5).

   International Code Set Support
       Single- and multi-byte character code sets are supported.

SEE ALSO

       csplit(1).

STANDARDS CONFORMANCE

																	  split(1)
All times are GMT -4. The time now is 02:17 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy