Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Split a file into multiple files

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Split a file into multiple files
# 22  
Old 12-30-2009
No you are wrong ..I alraedy exute the code and get 3 file only. with correct o/p

see below

Code:
cat filetoBeSplit.dat
|1|2|3|4|5|
|1|2|8|4|6|
|Trailer1|||||
|1|2|3|
|Trailer2|||
|3|4|5|6|
|3|4|5|7|
|3|4|5|8|
|Trailer2|||



Code:
cat filesplit0.dat
|1|2|3|4|5|
|1|2|8|4|6|
|Trailer1|||||

Code:
cat filesplit1.dat
|1|2|3|
|Trailer2|||

Code:
cat filesplit2.dat
|3|4|5|6|
|3|4|5|7|
|3|4|5|8|
|Trailer2|||


why it is wrong when you excute..did you copy/paste the code as I write it.

V.important note:- you need to delete the old files before executing the code again.

if you want to add new reqexp add it in the box in below
Code:
/usr/xpg4/bin/awk  -F"|" -v n=0 '
($2 ~/^[TFZD]/){print > > "filesplit"n".dat" ; close("filesplit"n".dat");n++;next}{print > "filesplit"n".dat"}' filetoBeSplit.dat


SmilieSmilie
Last edited by ahmad.diab; 12-30-2009 at 09:00 AM..
# 23  
Old 12-30-2009
hi scritunizer,

it wrks fine, but it generates file like 1.out, 2.out, and so on,
actually i ve to generate files /test1/filename1.out, /test2/filename2.out (there is no such specific format) n they can be created at different location after split...how can i give file names in awk?

---------- Post updated at 07:09 AM ---------- Previous update was at 07:04 AM ----------

this is my filetobesplit.dat:

Code:
|1|2|3|4|5|
|1|2|3|4|4|
|1|2|3|4|3|
|T|one||||
|1|2|3|4|5|6|7|8|9|
|2|3|4|5|6|7|8|9|1|
|D|three|||||
|4|
|5|
|6|
|Z|four||||

after split following files r getting generated with wrong data.

Code:
-rw-r--r--  1 r245347 fwsAPP      36 2009-12-30 07:07 filesplit0.dat
-rw-r--r--  1 r245347 fwsAPP      40 2009-12-30 07:07 filesplit1.dat
-rw-r--r--  1 r245347 fwsAPP      12 2009-12-30 07:07 filesplit2.dat
-rw-r--r--  1 r245347 fwsAPP      12 2009-12-30 07:07 filesplit2.dat0
-rw-r--r--  1 r245347 fwsAPP      14 2009-12-30 07:07 filesplit1.dat0
-rw-r--r--  1 r245347 fwsAPP      11 2009-12-30 07:07 filesplit0.dat0

============

Code:
/testDir> cat filesplit0.dat
|1|2|3|4|5|
|1|2|3|4|4|
|1|2|3|4|3|
/testDir> cat filesplit1.dat
|1|2|3|4|5|6|7|8|9|
|2|3|4|5|6|7|8|9|1|
/testDir> cat filesplit2.dat
|4|
|5|
|6|
/testDir> cat filesplit2.dat0
|Z|four||||
/testDir> cat filesplit1.dat0
|D|three|||||
/testDir> cat filesplit0.dat0
|T|one||||



---------- Post updated at 07:58 AM ---------- Previous update was at 07:09 AM ----------

hi, ahmed,

its nt wrking still at my end ...

can u try the following data file:

Code:
|1|2|3|4|5|
|1|2|3|4|4|
|1|2|3|4|3|
|T|one||||
|1|2|3|4|5|6|7|8|9|
|2|3|4|5|6|7|8|9|1|
|D|three|||||
|4|
|5|
|6|
|Z|four||||

Last edited by Franklin52; 12-30-2009 at 09:20 AM.. Reason: Please use code tags!!
# 24  
Old 12-30-2009
Hi, pparthji

As per my understanding the script is reading the input file only once,
i.e. 1 line at a time so there shud not be any performance issue with this.
Further the grep command is run only against a line of a file, so i dont think it will always match.

Quote:
Originally Posted by pparthji
Hi xoops,

in your script, script is reading the file again and again which hampers the performance. and besides that, grep command returns the all the matched patterns, for eg,


Code:
|1|2|3|
|T||||
|1|2|
|T||||
|1||2|3|4|5|
|T1||||

In this case. grep will always start from first.
Code:
 
#!/bin/bash
i=1
IFS=$'\n'
for line in `cat $1`
do
  echo $line >> filesplit${i}.dat
  $(echo ${line} | egrep -q "T|D|Z|F")
  if [ $? -eq 0 ] ; then i=$(($i+1)) ;fi
done

Code:
 
 
 
>> cat filetoBeSplit
|1|2|3|4|5|
|1|2|3|4|4|
|1|2|3|4|3|
|T|one||||
|1|2|3|4|5|6|7|8|9|
|2|3|4|5|6|7|8|9|1|
|D|three|||||
|4|
|5|
|6|
|Z|four||||
 
> sh script.sh filetoBeSplit
 
> more *.dat
::::::::::::::
filesplit1.dat
::::::::::::::
|1|2|3|4|5|
|1|2|3|4|4|
|1|2|3|4|3|
|T|one||||
::::::::::::::
filesplit2.dat
::::::::::::::
|1|2|3|4|5|6|7|8|9|
|2|3|4|5|6|7|8|9|1|
|D|three|||||
::::::::::::::
filesplit3.dat
::::::::::::::
|4|
|5|
|6|
|Z|four||||

# 25  
Old 12-30-2009
Quote:
Originally Posted by pparthji
hi scritunizer,

it wrks fine, but it generates file like 1.out, 2.out, and so on,
actually i ve to generate files /test1/filename1.out, /test2/filename2.out (there is no such specific format) n they can be created at different location after split...how can i give file names in awk?

---------- Post updated at 07:09 AM ---------- Previous update was at 07:04 AM ----------

this is my filetobesplit.dat:

Code:
|1|2|3|4|5|
|1|2|3|4|4|
|1|2|3|4|3|
|T|one||||
|1|2|3|4|5|6|7|8|9|
|2|3|4|5|6|7|8|9|1|
|D|three|||||
|4|
|5|
|6|
|Z|four||||

after split following files r getting generated with wrong data.

Code:
-rw-r--r--  1 r245347 fwsAPP      36 2009-12-30 07:07 filesplit0.dat
-rw-r--r--  1 r245347 fwsAPP      40 2009-12-30 07:07 filesplit1.dat
-rw-r--r--  1 r245347 fwsAPP      12 2009-12-30 07:07 filesplit2.dat
-rw-r--r--  1 r245347 fwsAPP      12 2009-12-30 07:07 filesplit2.dat0
-rw-r--r--  1 r245347 fwsAPP      14 2009-12-30 07:07 filesplit1.dat0
-rw-r--r--  1 r245347 fwsAPP      11 2009-12-30 07:07 filesplit0.dat0

============

Code:
/testDir> cat filesplit0.dat
|1|2|3|4|5|
|1|2|3|4|4|
|1|2|3|4|3|
/testDir> cat filesplit1.dat
|1|2|3|4|5|6|7|8|9|
|2|3|4|5|6|7|8|9|1|
/testDir> cat filesplit2.dat
|4|
|5|
|6|
/testDir> cat filesplit2.dat0
|Z|four||||
/testDir> cat filesplit1.dat0
|D|three|||||
/testDir> cat filesplit0.dat0
|T|one||||



---------- Post updated at 07:58 AM ---------- Previous update was at 07:09 AM ----------

hi, ahmed,

its nt wrking still at my end ...

can u try the following data file:

Code:
|1|2|3|4|5|
|1|2|3|4|4|
|1|2|3|4|3|
|T|one||||
|1|2|3|4|5|6|7|8|9|
|2|3|4|5|6|7|8|9|1|
|D|three|||||
|4|
|5|
|6|
|Z|four||||

With the data you provide my code still working find...

code:-

Code:
bash-3.00$ rm filesplit*.dat

bash-3.00$ /usr/xpg4/bin/awk  -F"|" -v n=0 '($2 ~/^[TFZD]/){print > > "filesplit"n".dat";close("filesplit"n".dat");n++;next}{print > "filesplit"n".dat"}' filetoBeSplit.dat

just copy/paste the commands
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Split file into multiple files using awk

I have following file: FHEAD0000000001RTLG20161205110959201612055019 THEAD...... TCUST..... TITEM.... TTEND... TTAIL... THEAD...... TCUST..... TITEM.... TITEM..... TTEND... TTAIL... FTAIL<number of lines in file- 10 digits;prefix 0><number of lines in file-2 - 10 digits- perfix 0>... (6 Replies)
Discussion started by: amitdaf
6 Replies

2. Shell Programming and Scripting

Split a .csv File into Multiple Files

Hi guys, I have a requirement where i need to split a .csv file into multiple files. Say for example i have data.csv file and i have splitted that into multiple files based on some conditions i.e first file should have 100, last file 50 and other files 1000 each. Am passing the values in... (2 Replies)
Discussion started by: azherkn3
2 Replies

3. Shell Programming and Scripting

Split file into multiple files using delimiter

Hi, I have a file which has many URLs delimited by space. Now i want them to move to separate files each one holding 10 URLs per file. http://3276.e-printphoto.co.uk/guardian http://abdera.apache.org/ http://abdera.apache.org/docs/api/index.html I have used the below code to arrange... (6 Replies)
Discussion started by: vel4ever
6 Replies

4. Shell Programming and Scripting

Split a file into multiple files with an extension

Hi I have a file with 100 million rows. I want to split them into 1000 subfiles and name them from 1.xls to 1000.xls.. Can I do it in awk? Thanks, (8 Replies)
Discussion started by: Diya123
8 Replies

5. Shell Programming and Scripting

Split file in unix into multiple files

Hi Gurus I have to split the incoming source file into multiple file. File contains some unwanted XML tags also . Files looks like some XML tags FILEHEADERABC 12 -- --- ---- EOF some xml tags xxxFILEHEADERABC 13 -- --- ---- EOF I have to ignore XML tags and only split file... (6 Replies)
Discussion started by: manish2608
6 Replies

6. Shell Programming and Scripting

split file into multiple files

Hi, I have a file of the following syntax that has around 120K records that are tab separated. input.txt abc def klm 20 76 . + . klm_mango unix_00000001; abc def klm 83 84 . + . klm_mango unix_0000103; abc def klm 415 439 . + . klm_mango unix_00001043; I am looking for an awk oneliner... (2 Replies)
Discussion started by: jacobs.smith
2 Replies

7. Shell Programming and Scripting

Split file into multiple files

Hi I have a file that has multiple sequences; the sequence name is the line starting with '>'. It looks like below: infile.txt: >HE_ER tttggtgccttgactcggattgggggacctcccttgggagatcaatcccctgtcctcctgctctttgctc cgtgaaaaggatccacctatgacctctagtcctcagacccaccagcccaaggaacatctcaccaatttca >M7B_Ho_sap... (2 Replies)
Discussion started by: jdhahbi
2 Replies

8. UNIX for Dummies Questions & Answers

How to split multiple records file in n files

Hello, Each record has a lenght of 7 characters I have 2 types of records 010 and 011 There is no character of end of line. For example my file is like that : 010hello 010bonjour011both 011sisters I would like to have 2 files 010.txt (2 records) hello bonjour and ... (1 Reply)
Discussion started by: jeuffeu
1 Replies

9. UNIX for Dummies Questions & Answers

split a file into multiple files

Hi All, I have a file ABC.txt and I need to split this file on every 250 rows. And the file name should be ABC1.txt , ABC2.txt and so on. I tried with split command split -l 250 <filename> '<filename>' but the file name returned was ABC.txtaa ABC.txtab. Please... (8 Replies)
Discussion started by: kumar66
8 Replies

10. Shell Programming and Scripting

Split a file into multiple files

I have a file ehich has multiple create statements as create abc 123 one two create xyz 456 four five create nnn 666 six four I want to separte each create statement in seperate files (3 Replies)
Discussion started by: glamo_2312
3 Replies
Login or Register to Ask a Question