script for splitting file


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting script for splitting file
# 1  
Old 04-11-2011
script for splitting file

Can anyone help me in giving a script for the below scenario

I have file from the source increamenting in size...

I require to write a script witch will move the data to the new file once the file reaches 50MB of size .
This needs If the first file reaches 50MB then my script has to generate a new file which again store the data upto to 50MB size and so on..... untill the
end of the file.


1st file -- 50MB
2nd file -- 50MB
3nd file -- 50MB
------
------------
Thanks
# 2  
Old 04-11-2011
Take a look at the 'split' command, such as,
Code:
split -b 50MB orig_file prefix

NOTE: not tested!
# 3  
Old 04-11-2011
Man Page for split (OpenSolaris Section 1) - The UNIX and Linux Forums

Usually, the trick is disconnecting the file from the application that is writing it. Sometimes, make the file a named pipe and control copying from that pipe as you please. Named pipes are a little tricky, with different behaviors for open for read and write in different orders. Play with one in the shell to find which goes first -- I forget.
# 4  
Old 04-11-2011
The way I understood is that you want to be in a loop where there is a main file that keeps having records inserted into it and
when it reaches 50Mb, you want to copy into another file, then empty the main file.

You decide:
Code:
#!/bin/ksh
typeset -i mLastLine mSize
mSize=$(wc -c < xxx)
if [[ ${mSize} -ge 50000000 ]]; then
  cp xxx xxx_50k
  mLastLine=$(wc -l < xxx_50k)+1
  tail +${mLastLine} xxx > xxx
fi

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Splitting a text file into smaller files with awk, how to create a different name for each new file

Hello, I have some large text files that look like, putrescine Mrv1583 01041713302D 6 5 0 0 0 0 999 V2000 2.0928 -0.2063 0.0000 N 0 0 0 0 0 0 0 0 0 0 0 0 5.6650 0.2063 0.0000 N 0 0 0 0 0 0 0 0 0 0 0 0 3.5217 ... (3 Replies)
Discussion started by: LMHmedchem
3 Replies

2. Shell Programming and Scripting

Script for splitting file of records into multiple files

Hello I have a file of following format HDR 1234 abc qwerty abc def ghi jkl HDR 4567 xyz qwerty abc def ghi jkl HDR 890 mno qwerty abc def ghi jkl HDR 1234 abc qwerty abc def ghi jkl HDR 1234 abc qwerty abc def ghi jkl -Need to split this into multiple files based on tag... (8 Replies)
Discussion started by: wincrazy
8 Replies

3. UNIX for Dummies Questions & Answers

File splitting script help

Hi All, I have file in my system with below data PP1234512345671234567CABC PP1234512345671234567BABC PP1234512345671234567BABC PP1234512345671234567CABC PP1234512345671234567DABC PP1234512345671234567EABC PP1234512345671234567DABC PP1234512345671234567EABC... (10 Replies)
Discussion started by: ibrar Ahmad
10 Replies

4. Shell Programming and Scripting

Execution of loop :Splitting a single file into multiple .dat file

hdr=$(cut -c1 $path$file|head -1)#extract header”H” trl=$(cut -c|path$file|tail -1)#extract trailer “T” SplitFile=$(cut -c 50-250 $path 1$newfile |sed'$/ *$//' head -1')# to trim white space and extract table name If; then # start loop if it is a header While read I #read file Do... (4 Replies)
Discussion started by: SwagatikaP1
4 Replies

5. Shell Programming and Scripting

Splitting XML file on basis of line number into multiple file

Hi All, I have more than half million lines of XML file , wanted to split in four files in a such a way that top 7 lines should be present in each file on top and bottom line of should be present in each file at bottom. from the 8th line actual record starts and each record contains 15 lines... (14 Replies)
Discussion started by: ajju
14 Replies

6. Shell Programming and Scripting

Splitting a file and creating new files using Perl script

Hi All, I am new to Scripting language. I want to split a file and create several subfiles using Perl script. Example : File format : Sourcename ID Date Nbr SU IMYFDJ 9/17/2012 5552159976555 SU BWZMIG 9/14/2012 1952257857887 AR PEHQDF 11/26/2012 ... (13 Replies)
Discussion started by: Deepak9870
13 Replies

7. Shell Programming and Scripting

Splitting a file in to multiple files and passing each individual file to a command

I have an input file with contents like: MainFile.dat: 12247689|7896|77698080 16768900|hh78|78959390 12247689|7896|77698080 16768900|hh78|78959390 12247689|7896|77698080 16768900|hh78|78959390 12247689|7896|77698080 16768900|hh78|78959390 12247689|7896|77698080 16768900|hh78|78959390 ... (4 Replies)
Discussion started by: rkrish
4 Replies

8. UNIX for Dummies Questions & Answers

Is there any way of splitting the script (Noob Here).

I m writing a script to check Server Hardening. The problem is whenever i add new point it grows and it become very tedious to edit the script file. Is there any way of making them separate and call them from one base script? Is it possible to define global variable that can be accessed via... (5 Replies)
Discussion started by: pinga123
5 Replies

9. Shell Programming and Scripting

File splitting and grouping using unix script

Hello All, I have a small problem with file group/splitting and I am trying to get the best way to perform this in unix. I am trying with awk but need some suggestion what would be the best and fastest way to-do it. Here is the problem. I have a fixed length file with filled with product... (4 Replies)
Discussion started by: nandhan11891
4 Replies

10. Shell Programming and Scripting

Help with shell script - splitting

Hi, I need to split the file lines in below format as. Input file : Sample.txt <Rule expression="DeliverToCompID IS NULL" invert="true"> <Rule field="PossDupFlag" value="Y" > <Rule expression="OrdStatus = '2' AND OrigClOrdID IS NULL"> Output... (5 Replies)
Discussion started by: manosubsulo
5 Replies
Login or Register to Ask a Question