Sponsored Content
Top Forums Shell Programming and Scripting Help needed - Split large file into smaller files based on pattern match Post 302758061 by frustrated1 on Friday 18th of January 2013 11:40:00 AM
Old 01-18-2013
Quote:
Originally Posted by vgersh99
Code:
awk '/START/{close(x);x="F"++i;}{print > x;}' inputfile


Code:
awk '/START/{close(x);x="F"++i;}{print > x;}' inputfile 
awk: too many output files 10
 record number 610

Hi - thanks for responding quickly

Tried that but same error. It creates output files from 0 to 9 with some output but fails with error above.

Any ideas?

---------- Post updated at 04:40 PM ---------- Previous update was at 04:30 PM ----------

Hi - Found some error - as I was using solaris version of awk - so now using the POSIX awk


This works
Code:
$ /usr/xpg4/bin/awk '/CP START/{close(x);x="F"++i;}{print > x;}' inputfile

however this fails:
Code:
$ /usr/xpg4/bin/awk '/CP START ACCOUNT/{close(x);x="F"++i;}{print > x;}' inputfile 
/usr/xpg4/bin/awk: line 0 (NR=1): output file "": No such file or directory


How can I get the above to succeed
Also - can I get the split to only split the file after say 20 matches of "CP START ACCOUNT" ?

Last edited by vgersh99; 01-18-2013 at 12:38 PM.. Reason: once again - please start using code tags!
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

splitting the large file into smaller files

hi all im new to this forum..excuse me if anythng wrong. I have a file containing 600 MB data in that. when i do parse the data in perl program im getting out of memory error. so iam planning to split the file into smaller files and process one by one. can any one tell me what is the code... (1 Reply)
Discussion started by: vsnreddy
1 Replies

2. Shell Programming and Scripting

Split a file into multiple files based on the input pattern

I have a file with lines something like. ...... 123_start ...... ....... 123_end .... ..... 456_start ...... ..... 456_end .... ..... 789_start .... .... 789_end (6 Replies)
Discussion started by: abinash
6 Replies

3. Shell Programming and Scripting

Splitting large file into multiple files in unix based on pattern

I need to write a shell script for below scenario My input file has data in format: qwerty0101TWE 12345 01022005 01022005 datainala alanfernanded 26 qwerty0101mXZ 12349 01022005 06022008 datainalb johngalilo 28 qwerty0101TWE 12342 01022005 07022009 datainalc hitalbert 43 qwerty0101CFG 12345... (19 Replies)
Discussion started by: jimmy12
19 Replies

4. Shell Programming and Scripting

Split large file into smaller file

hi Guys i need some help here.. i have a file which has > 800,000 lines in it. I need to split this file into smaller files with 25000 lines each. please help thanks (1 Reply)
Discussion started by: sitaldip
1 Replies

5. Shell Programming and Scripting

split XML file into multiple files based on pattern

Hello, I am using awk to split a file into multiple files using command: nawk '{ if ( $1 == "<process" ) { n=split($2, arr, "\""); file=arr } print > file }' processes.xml <process name="Process1.process"> ... (3 Replies)
Discussion started by: chiru_h
3 Replies

6. Shell Programming and Scripting

Sed: Splitting A large File into smaller files based on recursive Regular Expression match

I will simplify the explaination a bit, I need to parse through a 87m file - I have a single text file in the form of : <NAME>house........ SOMETEXT SOMETEXT SOMETEXT . . . . </script> MORETEXT MORETEXT . . . (6 Replies)
Discussion started by: sumguy
6 Replies

7. UNIX for Dummies Questions & Answers

Split a huge 7 GB File Based on Pattern into 4 files

Hi, I have a Huge 7 GB file which has around 1 million records, i want to split this file into 4 files to contain around 250k messages each. Please help me as Split command cannot work here as it might miss tags.. Format of the file is as below <!--###### ###### START-->... (6 Replies)
Discussion started by: KishM
6 Replies

8. Shell Programming and Scripting

Split Large Files Based On Row Pattern..

Hi all. I've tried searching the web but could not find similar problem to mine. I have one large file to be splitted into several files based on the matching pattern found in each row. For example, let's say the file content: ... (13 Replies)
Discussion started by: aimy
13 Replies

9. UNIX for Dummies Questions & Answers

Split large file to smaller fastly

hi , I have a requirement input file: 1 1111111111111 108 1 1111111111111 109 1 1111111111111 109 1 1111111111111 110 1 1111111111111 111 1 1111111111111 111 1 1111111111111 111 1 1111111111111 112 1 1111111111111 112 1 1111111111111 112 The output should be, (19 Replies)
Discussion started by: mechvijays
19 Replies

10. UNIX for Beginners Questions & Answers

Split large file into smaller files without disturbing the entry chunks

Dears, Need you help with the below file manipulation. I want to split the file into 8 smaller files but without cutting/disturbing the entries (meaning every small file should start with a entry and end with an empty line). It will be helpful if you can provide a one liner command for this... (12 Replies)
Discussion started by: Kamesh G
12 Replies
APSFILTER-BUG(1)					    BSD General Commands Manual 					  APSFILTER-BUG(1)

NAME
apsfilter-bug -- create a half-automatic bug report for apsfilter SYNOPSIS
apsfilter-bug OPTIONS
none DESCRIPTION
With the apsfilter-bug script you can automatically create a bug report for apsfilter(1). A template report file is created which you can edit with your favourite editor ( emacs(1) by default, or whatever is entered in the environment variable EDITOR ). You will see a couple of lines starting with '#'; these lines will be removed before the report is sent. So if you want to use the '#' char- acter at the beginning of a line, just indent it with a space. A very important piece of the bug report is the debugging log created by aps2file(1), probably with a command like aps2file -D -o /dev/null [-P...] [-Z...] [input] 2> log.txt Unless you don't even have a chance to create a debugging log, it is crucial for serious bug hunting. After you have filled in as much (useful) information as possible, you must save the file under its original name, then exit your editor. If you invoked apsfilter-bug by accident, or if you need some additional system information to include into the bug report, just exit your editor without saving the file. NOTES
In addition to the stuff you've edited, some more information is appended to the bug report automatically: apsfilter version 7.2.6-stable ghostscript version (from 'gs --version') system id (from 'uname -a') configure options --prefix=/usr --sysconfdir=/etc --mandir=/usr/share/man --with-awk=/usr/bin/awk --with-sendmail=/usr/sbin/sendmail shell executable /bin/bash awk executable /usr/bin/awk sendmail executable /usr/sbin/sendmail FILES
/usr/bin/aps2file script to create debugging output SEE ALSO
apsfilter(1), aps2file(1) BUGS
See apsfilter software center - http://www.apsfilter.org/ - for new versions, bugfixes and known bugs. Please use the new tool apsfilter-bug(1) to release bug- or problem reports. It automatically presents you a form in an editor window which asks you some standard questions. If you save and quit the editor session, then this report is sent automatically via e-mail to the proper apsfilter mailinglist. If apsfilter fails to print something or prints it in a way you wouldn't expect and you want to report an apsfilter error then please save the debugging output of one print session using the new aps2file(1) utility by typing aps2file -D -Z options file > /dev/null 2> file.debug and including the debugging output in the file file.debug into the edit session of the apsfilter-bug utility, so that it is included into the mail to the apsfilter mailinglist. Please note that you need to run /bin/sh (Bourne Shell), bash or a compatible shell, so that the above mentioned output redirection works. Under C-shell (/bin/csh) or tcsh it would't work. If you don't know, then simply make sure you use the Bournce shell by typing /bin/sh or bash, then you should have no problems with redirection of stdout and stderr (> /dev/null 2> file.debug). DOCUMENTATION
See official apsfilter homepage http://www.apsfilter.org/handbook.html Apsfilter Handbook including the Frequently Asked Questions (FAQ) USER FORUM
Please send questions to the official apsfilter help channel apsfilter-help@apsfilter.org. The above section BUGS and the file HOWTO-BUGREPORTS tells you how to report bugs. If you want to know how to troubleshoot your apsfilter installation, please read the manpage aps2file(1) and apsfilter-bug(1) as well as the Apsfilter Handbook carefully. HISTORY
The apsfilter-bug manpage has been written by Michael LoBin <phallobst@web.de> and first appeared in apsfilter V 7.1.0. BSD
Dec 26, 2001 BSD
All times are GMT -4. The time now is 07:21 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy