Split a big file into multiple files using awk


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Split a big file into multiple files using awk
# 1  
Old 09-17-2014
Split a big file into multiple files using awk

this thread is a continuation from previous thread
Split a big file into multiple files based on first four characters

..I am using awk to split file and I have a syntax error while executing the below code

Code:
[awk '!/^$/{ a=substr($0,1,4) print $0 > "SBSCR." a ".txt"}' harsha1.txt
 syntax error The source line is 1.
 The error context is
                !/^$/{ a=substr($0,1,4) >>>  print <<<  $0 > "SBSCR." a ".txt"}
 awk: The statement cannot be correctly parsed.
 The source line is 1.
]

I am using AIX 7.2 and this code worked fine previously.I am an awk dummy..please help what the synax error..please help me debug the code.

below is how my data text file looks like

Image - TinyPic - Free Image Hosting, Photo Sharing & Video Hosting

Last edited by Don Cragun; 09-17-2014 at 07:00 AM.. Reason: Add closing CODE tag, change IMG tags to URL tags.
# 2  
Old 09-17-2014
Try:
Code:
awk '!/^$/{ a=substr($0,1,4) print $0 > ("SBSCR." a ".txt")}' harsha1.txt

Note, however, that awk has a limited number of file descriptors available for use in creating output files. You may need to add logic to your code to close the previous output file when you switch to a new output file.
# 3  
Old 09-17-2014
I have actually tried that but this time..It does not throw any error but takes ever to run and eventually I have to cancel the execution

---------- Post updated at 02:34 PM ---------- Previous update was at 08:36 AM ----------

I tried this and it worked
Code:
[$ awk '{print > substr ($0,1,4)}' file]..

Can someone help me in formatting this awk statement so that my output file name looks like ABCD.substr ($0,1,4).txt

Last edited by rbatte1; 09-18-2014 at 08:16 AM.. Reason: Added [/CODE] tag
# 4  
Old 09-17-2014
I apologize; I must have been asleep when I looked at your code yesterday. Change:
Code:
awk '!/^$/{ a=substr($0,1,4) print $0 > "SBSCR." a ".txt"}' harsha1.txt

to:
Code:
awk '!/^$/{ a=substr($0,1,4); print $0 > ("SBSCR." a ".txt")}' harsha1.txt

You need both the semicolon and the parentheses to turn this into a working and portable awk command.
These 2 Users Gave Thanks to Don Cragun For This Post:
# 5  
Old 09-18-2014
that worked great.thanks a lot
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Solaris

Split a big file system to several files

Gents Actually I have question and i need your support. I have this NAS file system mounted as /coresys has size of 7 TB I need to Split this file system into several file systems as mount points I mean how to can I Split it professionally to different NAS mount points how to can I decide... (2 Replies)
Discussion started by: AbuAliiiiiiiiii
2 Replies

2. Shell Programming and Scripting

Split file into multiple files using awk

I have following file: FHEAD0000000001RTLG20161205110959201612055019 THEAD...... TCUST..... TITEM.... TTEND... TTAIL... THEAD...... TCUST..... TITEM.... TITEM..... TTEND... TTAIL... FTAIL<number of lines in file- 10 digits;prefix 0><number of lines in file-2 - 10 digits- perfix 0>... (6 Replies)
Discussion started by: amitdaf
6 Replies

3. Shell Programming and Scripting

Split a big file into multiple files based on first four characters

I have a requirement to split a huge file to smaller text files based on first four characters which look like ABCD 1234 DFGH RREX : : : : : 0000 Each of these records are OF EQUAL bytes with a different internal layout based on the above first digit identifier.. Any help to start... (5 Replies)
Discussion started by: etldev
5 Replies

4. Shell Programming and Scripting

awk script to split file into multiple files based on many columns

So I have a space delimited file that I'd like to split into multiple files based on multiple column values. This is what my data looks like 1bc9A02 1 10 1000 FTDLNLVQALRQFLWSFRLPGEAQKIDRMMEAFAQRYCQCNNGVFQSTDTCYVLSFAIIMLNTSLHNPNVKDKPTVERFIAMNRGINDGGDLPEELLRNLYESIKNEPFKIPELEHHHHHH 1ku1A02 1 10... (9 Replies)
Discussion started by: viored
9 Replies

5. Shell Programming and Scripting

How to split file into multiple files using awk based on 1 field in the file?

Good day all I need some helps, say that I have data like below, each field separated by a tab DATE NAME ADDRESS 15/7/2012 LX a.b.c 15/7/2012 LX1 a.b.c 16/7/2012 AB a.b.c 16/7/2012 AB2 a.b.c 15/7/2012 LX2 a.b.c... (2 Replies)
Discussion started by: alexyyw
2 Replies

6. Emergency UNIX and Linux Support

Combine multiple Files into one big file

Hi Ppl, I have a requirement like i will be getting files of huge size daily and if the file size is so huge ,the files will be split into many parts and sent.The first file will have the header details followed by detail records and the consecutive files will have detail records and the last... (11 Replies)
Discussion started by: ganesh_248
11 Replies

7. Shell Programming and Scripting

Need to Split Big XML into multiple xmls

Hi friends.. We have urgent requirement.We need to split the big xml having multiple orders into multiple xmls having each order in each xml. For Example In input XMl will be in following format with multiple line orders.. <OrderDetail BillToKey="20100805337" Createuserid="CreateGuestOrder"... (8 Replies)
Discussion started by: dprakash
8 Replies

8. UNIX for Advanced & Expert Users

Split a big file into two others files

Hello, i have a very big file that has more then 80 MBytes (100MBytes). So with my CVS Application I cannot commit this file (too Big) because it must have < 80 MBytes. How can I split this file into two others files, i think the AIX Unix command : split -b can do that, buit how is the right... (2 Replies)
Discussion started by: steiner
2 Replies

9. Shell Programming and Scripting

Split line to multiple files Awk/Sed/Shell Script help

Hi, I need help to split lines from a file into multiple files. my input look like this: 13 23 45 45 6 7 33 44 55 66 7 13 34 5 6 7 87 45 7 8 8 9 13 44 55 66 77 8 44 66 88 99 6 I want to split every 3 lines from this file to be written to individual files. (3 Replies)
Discussion started by: saint2006
3 Replies

10. Shell Programming and Scripting

Help Needed : Split one big file to multiple files

Hi friends, I have data in flat file as following, first filed is the customer number. We have almost 50-100 customers in the system 100 ABC A123 100 BVC D234 100 BNC N324 200 CBC A122 200 AVC D294 200 HNC N324 300 GBC A173 300 FVC D234 300 DNC N344 I want to split the file and... (5 Replies)
Discussion started by: monicasgupta
5 Replies
Login or Register to Ask a Question