Split single file into n number of files


 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers Split single file into n number of files
# 1  
Old 09-06-2011
Split single file into n number of files

Hi,

I am new to unix. we have a requirement here to split a single file into multiples files based on the number of people available for processing. So i tried my hand at writing some code as below.
Code:
#!/bin/bash
var1=`wc -l $filename`
var2=$var1/$splitno
split -l $var2 $1

Please help me to get the code right manner...

Thanks

Last edited by Franklin52; 09-07-2011 at 03:38 AM.. Reason: Please use code tags for code and data samples, thank you
# 2  
Old 09-06-2011
Change the value of "5" as per ur requirement
Code:
$ split -5 singlefile

This User Gave Thanks to jayan_jay For This Post:
# 3  
Old 09-06-2011
Jay, Thanks for replying but what I am trying to achieve is split based on the number of people available..suppose if there are 3 people then I need the file to be split into 3 files, if there are 5 people then there needs to be 5 output files...
# 4  
Old 09-06-2011
Here "n" - number of users. In case the line count of infile is not exactly divisible by the number of users, then it ll create a new file with the remainder lines.
Try this ..
Code:
$ wc -l infile | nawk -v n=3 '{print "expr "$1" / "n"|bc" }' | sh | sed 's,^,split -,g;s,$, infile,g' | sh

This User Gave Thanks to jayan_jay For This Post:
# 5  
Old 09-07-2011
@Jay ,

I am still not able to get the above code to run... I am on Windows 7 with Cygwin installed and when I tried to execute the above code in a script.. I am recieving the following error...Smilie

"./split.sh: line 2: $'sh\r' : command not found
sh: line 1: bc command not found "

My split.sh contains
Code:
#!/bin/bash
wc -l 1.txt | awk -v n=3 '{print "expr "$1" / "n"|bc" }' | sh | sed 's,^,split -,g;s,$, 1.txt,g' | sh

Please help me out...

Last edited by Franklin52; 09-07-2011 at 03:38 AM.. Reason: Please use code tags for code and data samples, thank you
# 6  
Old 09-07-2011
Try...
Code:
$ seq 13 > infile

$ awk '{a[NR]=$0}END{for(i=1;i<=NR;i++)print a[i] > "outfile" 1+int(n*((i-1)/NR))}' n=3 infile

$ head outfile*
==> outfile1 <==
1
2
3
4
5

==> outfile2 <==
6
7
8
9

==> outfile3 <==
10
11
12
13

$

This User Gave Thanks to Ygor For This Post:
# 7  
Old 09-07-2011
@YGOR

Thanks for the help ...it worked for me....
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Split a single file into multiple files based on a value.

Hi All, I have the sales_data.csv file in the directory as below. SDDCCR; SOM ; MD6546474777 ;05-JAN-16 ABC ; KIRAN ; CB789 ;04-JAN-16 ABC ; RAMANA; KS566767477747 ;06-JAN-16 ABC ; KAMESH; A33535335 ;04-JAN-16 SDDCCR; DINESH; GD6674474747 ;08-JAN-16... (4 Replies)
Discussion started by: ROCK_PLSQL
4 Replies

2. UNIX for Dummies Questions & Answers

Split files into smaller ones with 1000 hierarchies in a single file.

input file: AD,00,--,---,---,---,---,---,---,--,--,--- AM,000,---,---,---,---,---,--- AR, ,---,--,---,--- AA,---,---,---,--- AT,--- AU,---,---,--- AS,---,--- AP,---,---,--- AI,--- AD,00,---,---,---, ,---,---,---,---,---,--- AM,000,---,---,--- AR,... (6 Replies)
Discussion started by: kcdg859
6 Replies

3. Shell Programming and Scripting

Split single file into multiple files using pattern matching

I have one single shown below and I need to break each ST|850 & SE to separate file using unix script. Below example should create 3 files. We can use ST & SE to filter as these field names will remain same. Please advice with the unix code. ST|850 BEG|PO|1234 LIN|1|23 SE|4 ST|850... (3 Replies)
Discussion started by: prasadm
3 Replies

4. UNIX for Dummies Questions & Answers

How to separate a single column file into files of the same size (i.e. number of rows)?

I have a text file with 1,000,000 rows (It is a single column text file of numbers). I would like to separate the text file into 100 files of equal size (i.e. number of rows). The first file will contain the first 10,000 rows, the second row will contain the second 10,000 rows (rows 10,001-20,000)... (2 Replies)
Discussion started by: evelibertine
2 Replies

5. Shell Programming and Scripting

split a csv file into specified number of files (not lines)

hi, i really need it ...it's not simple to explain but as it's part of a crontab i can't split the file manually...and the file can change every day so the lines are not a good base. example: how to split 1 csv file in 15 files? thank you very much regards :b: (4 Replies)
Discussion started by: 7stars
4 Replies

6. Shell Programming and Scripting

Split the single file lines into multiple files

Let's assume that I have a file name called ‘A' and it has 100 lines in it and would like to split these 100 lines into 4 files as specified bellow. INPUT: Input file name A 1 2 3 4 5 6 7 8 9 ........100 Output: 4 output files (x,y,z,w) File x should contains (Skip 4 lines)... (15 Replies)
Discussion started by: subbarao25
15 Replies

7. Shell Programming and Scripting

Split single file into multiple files based on the number in the column

Dear All, I would like to split a file of the following format into multiple files based on the number in the 6th column (numbers 1, 2, 3...): ATOM 1 N GLY A 1 -3.198 27.537 -5.958 1.00 0.00 N ATOM 2 CA GLY A 1 -2.199 28.399 -6.617 1.00 0.00 ... (3 Replies)
Discussion started by: tomasl
3 Replies

8. UNIX for Dummies Questions & Answers

split a file into a specified number of files

I have been googling on the 'split' unix command to see if it can split a large file into 'n' number of files. Can anyone spare an example or a code snippet? Thanks, - CB (2 Replies)
Discussion started by: ChicagoBlues
2 Replies

9. UNIX for Dummies Questions & Answers

split a single sql file into multiple files

Hi,I have a single sql file containing many create table ddl's.Example: CREATE TABLE sec_afs ( rpt_per_typ_c char(1) NOT NULL, rpt_per_typ_t varchar(20) NULL, LOCK ALLPAGES go EXEC sp_primarykey 'sec_afs', rpt_per_typ_c go GRANT SELECT ON sec_afs TO developer_read_only... (5 Replies)
Discussion started by: smarter_aries
5 Replies

10. Shell Programming and Scripting

split single file into many

Hello, We have a text file with more than 1500 paragraphs. There is a blank line to separate paragraphs. We need to create one text file (with any name but unique) per paragraph. In other words, how can i extract each paragraph and create a separate file with those contents (so 1500 text... (3 Replies)
Discussion started by: prvnrk
3 Replies
Login or Register to Ask a Question