Splitting a file into unequal parts


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Splitting a file into unequal parts
# 1  
Old 09-08-2009
Splitting a file into unequal parts

How do I split a file into many parts but with different amounts of lines per part? I looked at the split command but that only splits evenly.

I'd like a range specified to determine how many lines each output file should have.

For example, if the input file has 1000 lines and the range is 200 - 300, one possible output would be that the files have 247, 212, 275, and 266 lines each.

It doesn't have to come out exactly like the above example. A "leftover" file is alright. If the following were chosen for each amount then there will be leftover lines:
295 (file1) + 297 (file2) + 300 (file3) + 108 leftover (file4)

Last edited by revax; 09-08-2009 at 03:22 PM..
# 2  
Old 09-08-2009
with some mathematics, you can do something like:

Code:
echo "Enter 3 random numbers in which you want to split the file."
read num1 num2 num3

filename="file.main"

total=$(wc -l $filename | awk '{print $1}')

start_1=1
end_1=$num1

start_2=$(echo "${end_1}+1" | bc)
end_2=$(echo "${start_2}+$num2" | bc)

start_3=$(echo "${end_2}+1" | bc)
end_3=$(echo "${start_3}+${num3}" | bc)

start_4=$(echo "${end_3}+1" | bc)
end_4=$total


sed -n "${start_1},${end_1}p" $filename > file1
sed -n "${start_2},${end_2}p" $filename > file2
sed -n "${start_3},${end_3}p" $filename > file3
sed -n "${start_4},${end_4}p" $filename > file4

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How many studies have unequal values for each pair?

I have several Studies (s) which has points (p) having Values (v). My goal is to determine for each pair of points, how many studies have different values ( if available ). Study Point Value 1 p1 value1 1 p2 value2 1 p3 value1 1 p4 value3 1 p5 value3 2 p2 value1 2 p4 value1 3 p1 value1... (5 Replies)
Discussion started by: senhia83
5 Replies

2. Shell Programming and Scripting

Incrementing parts of ten digits number by parts

I have number in file which contains date and serial number: 2013101000. The last two digits are serial number (00). So maximum of serial number is 100. After reaching 100 it becomes 00 with incrementing 10 which is day with max 31. after reaching 31 it becomes 00 and increments 10... (31 Replies)
Discussion started by: Natalie
31 Replies

3. Shell Programming and Scripting

Combine two parts of a file

Hello All, I have a file like this APPLY ( 'INSERT INTO brdcst_media_cntnt ( cntnt_id ,brdcst_media_cntnt_cd ,cntnt_prvdr_cd ,data_src_type_cd ,cntnt_titl_nm ,cntnt_desc ,batch_dt ,batch_id ) VALUES ( :cntnt_id (3 Replies)
Discussion started by: nnani
3 Replies

4. Shell Programming and Scripting

Newline between unequal record fields

Assume the following 5 records (field separator is a space): 0903 0903 0910 0910 0910 0910 0910 0910 0917 0917 0917 0917 0924 1001 1001 1001 1001 1008 1008 1008 1008 1015 1015 1015 1015 1022 1029 1029 1029 1029 1105 1105 1105 1105 1112 1112 1112 1112 1119 1126 1126 1126 1126 1203 1203 1203 1203... (8 Replies)
Discussion started by: tree
8 Replies

5. Shell Programming and Scripting

AWK splitting a string of equal parts of 500 chars

Hi , can someone help me how to make an AWK code for splitting a string of equal parts of 500 chars in while loop? Thank you! (4 Replies)
Discussion started by: sanantonio7777
4 Replies

6. Shell Programming and Scripting

perl, splitting out specific parts of the string

Hi there, I have an output from a command like this # ypcat -k netgroup.byuser| grep steven steven.* users_main,users_sysadmin,users_global,users_backup_team and wanted to pull the 'users' netgroups returned into a perl array, that will look like this users_main... (2 Replies)
Discussion started by: rethink
2 Replies

7. Shell Programming and Scripting

Extracting parts of a file.

Hello, I have a XML file as below and i would like to extract all the lines between <JOB & </JOB> for every such occurance. The number of lines between them is not fixed. Anyways to do this awk? ============ <JOB APR="1" AUG="1" DEC="1" FEB="1" JAN="1" JUL="1" JUN="1" MAR="1" MAY="1"... (3 Replies)
Discussion started by: srivat79
3 Replies

8. Shell Programming and Scripting

Splitting huge XML Files into fixsized wellformed parts

Hi, I need to split xml-files with sizes greater than 2 gb into smaler chunks. As I dont want to end up with billions of files, I want those splitted files to have configurable sizes like 250 MB. Each file should be well formed having an exact copy of the header (and footer as the closing of the... (0 Replies)
Discussion started by: Malapha
0 Replies

9. Programming

Splitting the console into 2 "parts"

Good morning, I would like to make a little interface for an application which will split the console in 2 "parts": 1 top area to print messages & 1 command prompt. The printed messages are asynchronous to the command prompt, what I mean is that while while the user types a command a... (2 Replies)
Discussion started by: jonas.gabriel
2 Replies

10. Shell Programming and Scripting

getting parts of a file

Hello, I'm trying to retreive certain bits of info from a file. the file contains a list like this info1:info2:info3:info4 info1:info2:info3:info4 info1:info2:info3:info4 info1:info2:info3:info4 how do i pick out only info2 or only info3 without the others? Thanks (11 Replies)
Discussion started by: bebop1111116
11 Replies
Login or Register to Ask a Question