Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Modification of perl script to split a large file into chunks of 5000 chracters
Top Forums Shell Programming and Scripting Modification of perl script to split a large file into chunks of 5000 chracters Post 303017073 by gimley on Tuesday 8th of May 2018 10:34:08 PM
Old 05-08-2018
Modification of perl script to split a large file into chunks of 5000 chracters
I have a perl script which splits a large file into chunks.The script is given below
Code:
use strict;
use warnings;
open (FH, "<monolingual.txt") or die "Could not open source file. $!";
my $i = 0;
while (1) {
    my $chunk;
	print "process part $i\n";
	open(OUT, ">part$i.log") or die "Could not open destination file";
	$i ++;
	if (!eof(FH)) {
		read(FH, $chunk, 5000);
		print OUT $chunk;
	} 
	if (!eof(FH)) {
		$chunk = <FH>;
		print OUT $chunk;
	}
	close(OUT);
	last if eof(FH);
}

I want the script to create chunks of 5000 characters or a bit less but not more than that.
How do I modify the chunk size to ensure that each chunk is of 5000 characters. When I run it some chunks are more than 5000 characters.
Many thanks for your kind help
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Split A Large File

Hi, I have a large file(csv format) that I need to split into 2 files. The file looks something like Original_file.txt first name, family name, address a, b, c, d, e, f, and so on for over 100,00 lines I need to create two files from this one file. The condition is i need to ensure... (4 Replies)
Discussion started by: nbvcxzdz
4 Replies

2. HP-UX

Need to split a large data file using a Unix script

Greetings all: I am still new to Unix environment and I need help with the following requirement. I have a large sequential file sorted on a field (say store#) that is being split into several smaller files, one for each store. That means if there are 500 stores, there will be 500 files. This... (1 Reply)
Discussion started by: SAIK
1 Replies

3. Shell Programming and Scripting

Split Large File

HI, i've to split a large file which inputs seems like : Input file name_file.txt 00001|AAAA|MAIL|DATEOFBIRTHT|....... 00001|AAAA|MAIL|DATEOFBIRTHT|....... 00002|BBBB|MAIL|DATEOFBIRTHT|....... 00002|BBBB|MAIL|DATEOFBIRTHT|....... 00003|CCCC|MAIL|DATEOFBIRTHT|.......... (1 Reply)
Discussion started by: AMARA
1 Replies

4. Shell Programming and Scripting

how to get split output of a file, using perl script

Hi, I have file: data.log.1 ### s1 main.build.3495 main.build.199 main.build.3408 ###s2 main.build.3495 main.build.3408 main.build.199 I want to read this file and store in two arrays in Perl. I have following command, which is working fine on command prompt. perl -n -e... (1 Reply)
Discussion started by: ashvini
1 Replies

5. Shell Programming and Scripting

Split file into chunks of low & high byte

Hi guys, i have a question about spliting a binary file into 2 chunks. First chunk with all high bytes and the second one with all low bytes. What unix tools can i use? And how can this be performed? I looked in manpages of split and dd but this does not help. Thanks (2 Replies)
Discussion started by: basta
2 Replies

6. Shell Programming and Scripting

Split a large file

I have a 3 GB text file that I would like to split. How can I do this? It's a giant comma-separated list of numbers. I would like to make it into about 20 files of ~100 MB each, with a custom header and footer. The file can only be split on commas, but they're plentiful. Something like... (3 Replies)
Discussion started by: CRGreathouse
3 Replies

7. Shell Programming and Scripting

perl script to split the text file after every 4th field

I had a text file(comma seperated values) which contains as below 196237,ram,25-May-06,ram.kiran@xyz.com,204183,Pavan,4-Jun-07,Pavan.Desai@xyz.com,237107,ram Chandra,15-Mar-10,ram.krishna@xyz.com ... (3 Replies)
Discussion started by: giridhar276
3 Replies

8. Shell Programming and Scripting

Split a large array into small chunks

Hi, I need to split a large array "@sharedArray" into 10 small arrays. The arrays should be like @sharedArray1,@sharedArray2,@sharedArray3...so on.. Can anyone help me with the logic to do so :(:confused: (6 Replies)
Discussion started by: rkrish
6 Replies

9. UNIX for Beginners Questions & Answers

Split large file into smaller files without disturbing the entry chunks

Dears, Need you help with the below file manipulation. I want to split the file into 8 smaller files but without cutting/disturbing the entries (meaning every small file should start with a entry and end with an empty line). It will be helpful if you can provide a one liner command for this... (12 Replies)
Discussion started by: Kamesh G
12 Replies

10. UNIX for Beginners Questions & Answers

Trying To Split a Large File

Trying to split a 35gb file into 1000mb parts. My research shows I should you this. split -b 1000m file.txt and my return is "split: cannot open 'crunch1.txt' for reading: No such file or directory" so I tried split -b 1000m Documents/Wordlists/file.txt and I get nothing other than the curser just... (3 Replies)
Discussion started by: sub terra
3 Replies

LEARN ABOUT CENTOS

sieveshell

SIEVESHELL(1)						User Contributed Perl Documentation					     SIEVESHELL(1)

NAME

       sieveshell - remotely manipulate sieve scripts

SYNOPSIS

       sieveshell [--user=user] [--authname=authname] [--realm=realm] [--exec=script] server[:port]

       sieveshell --help

DESCRIPTION

       sieveshell allows users to manipulate their scripts on a remote server.	It works via MANAGESIEVE, a work in progress.

       The following commands are recognized:

	   list list scripts on server.

	   put <filename> upload script to server.

	   get <name> [<filename>] get script. if no filename display to stdout

	   delete <name> delete script.

	   activate <name> activate script.

	   deactivate deactivate all scripts.

OPTIONS

       -u user, --user=user
	   The authorization name to request; by default, derived from the authentication credentials.

       -a authname, --authname=authname
	   The user to use for authentication (defaults to current user).

       -r realm, --realm=realm
	   The realm to attempt authentication in.

       -e script, --exec=script
	   Instead of working interactively, run commands from script, and exit when done.

REFERENCES

       [MANAGESIEVE] Martin, T.; "A Protocol for Remotely Managing Sieve Scripts", draft-ietf-managesieve-03.txt, Mirapoint, Inc.; May 2001, work
       in progress.

AUTHOR

       Tim Martin <tmartin@mirapoint.com>, and the rest of the Cyrus team <cyrus-bugs@andrew.cmu.edu>.

perl v5.16.3							    2014-06-10							     SIEVESHELL(1)
All times are GMT -4. The time now is 06:38 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy