|
|||||||
| Forums | Search Forums | Register | Forum Rules | Man Pages | Albums | FAQ | Members | Calendar | Search | Today's Posts | Mark Forums Read |
| Shell Programming and Scripting Post questions about KSH, CSH, SH, BASH, PERL, PHP, SED, AWK and OTHER shell scripts and shell scripting languages here. |
|
|
|
Thread Tools | Search this Thread | Display Modes |
|
#1
01-18-2013
|
||||
|
||||
|
Help needed - Split large file into smaller files based on pattern match
Help needed urgently please. I have a large file - a few hundred thousand lines. Sample Code:
CP START ACCOUNT 1234556 name 1 CP END ACCOUNT CP START ACCOUNT 2224444 name 1 CP END ACCOUNT CP START ACCOUNT 333344444 name 1 CP END ACCOUNT I need to split this file each time "CP START ACCOUNT" is matched. Preferably I would split it every 20 times this is matched and output to smaller files. I was trying something like the below but could do with help obviously Code:
awk '/START/{x="F"++i;}{print > x;}' inputfile
awk: too many output files 10
record number 610Also need to replace START above with CP START ACCOUNT Can anyone help urgently?
Last edited by vgersh99; 01-18-2013 at 11:25 AM.. Reason: code tags, please! |
| Sponsored Links | ||
|
|
#2
01-18-2013
|
||||
|
||||
|
Code:
awk '/START/{close(x);x="F"++i;}{print > x;}' inputfile |
| Sponsored Links | ||
|
|
#3
01-18-2013
|
|||
|
|||
|
Quote:
Code:
awk '/START/{close(x);x="F"++i;}{print > x;}' inputfile
awk: too many output files 10
record number 610Hi - thanks for responding quickly Tried that but same error. It creates output files from 0 to 9 with some output but fails with error above. Any ideas? ---------- Post updated at 04:40 PM ---------- Previous update was at 04:30 PM ---------- Hi - Found some error - as I was using solaris version of awk - so now using the POSIX awk This works Code:
$ /usr/xpg4/bin/awk '/CP START/{close(x);x="F"++i;}{print > x;}' inputfilehowever this fails: Code:
$ /usr/xpg4/bin/awk '/CP START ACCOUNT/{close(x);x="F"++i;}{print > x;}' inputfile
/usr/xpg4/bin/awk: line 0 (NR=1): output file "": No such file or directoryHow can I get the above to succeed Also - can I get the split to only split the file after say 20 matches of "CP START ACCOUNT" ? Last edited by vgersh99; 01-18-2013 at 11:38 AM.. Reason: once again - please start using code tags! |
|
#4
01-18-2013
|
||||
|
||||
|
Code:
awk '/START/{close(x);x=("F" ++i)}{print > x;}' inputfileif on Solaris, using nawk instead of awk |
| Sponsored Links | |
|
|
#5
01-18-2013
|
|||
|
|||
|
Thanks, can you help on the previous comment as well ? This works Code:
$ /usr/xpg4/bin/awk '/CP START/{close(x);x="F"++i;}{print > x;}' inputfilehowever this fails: Code:
$ /usr/xpg4/bin/awk '/CP START ACCOUNT/{close(x);x="F"++i;}{print > x;}' inputfile
/usr/xpg4/bin/awk: line 0 (NR=1): output file "": No such file or directoryHow can I get the above to succeed Also - can I get the split to only split the file after say 20 matches of "CP START ACCOUNT" ? Last edited by vgersh99; 01-18-2013 at 12:00 PM.. Reason: third warning - start using code tags! |
| Sponsored Links | |
|
|
#6
01-18-2013
|
||||
|
||||
|
Another approach is using a BASH script, but this is gonna run slower than awk: Code:
#!/bin/bash
c=0
while read line
do
if [[ "$line" =~ "^CP START ACCOUNT" ]]
then
c=$(( c + 1 ))
echo "$line" >> F${c}.txt
else
echo "$line" >> F${c}.txt
fi
done < inputfile |
| Sponsored Links | |
|
|
#7
01-18-2013
|
|||
|
|||
|
Give this a try, extending vgersh99's proposal Code:
awk '/^CP START ACCOUNT/ {if (!(n%20)) {close (fn); fn=("F" ++i)}; n++}
{print > fn;}
' fileLast edited by vgersh99; 01-18-2013 at 05:02 PM.. |
| Sponsored Links | ||
|
|
| Thread Tools | Search this Thread |
| Display Modes | |
More UNIX and Linux Forum Topics You Might Find Helpful
|
||||
| Thread | Thread Starter | Forum | Replies | Last Post |
| split XML file into multiple files based on pattern | chiru_h | Shell Programming and Scripting | 3 | 01-10-2012 04:17 PM |
| Split large file into smaller file | sitaldip | Shell Programming and Scripting | 1 | 08-05-2011 04:59 AM |
| Splitting large file into multiple files in unix based on pattern | jimmy12 | Shell Programming and Scripting | 19 | 07-06-2011 03:14 AM |
| Split a file into multiple files based on the input pattern | abinash | Shell Programming and Scripting | 6 | 01-16-2011 02:45 PM |
| splitting the large file into smaller files | vsnreddy | UNIX for Dummies Questions & Answers | 1 | 11-16-2008 08:09 PM |
|
|
Linear Mode
