Sponsored Content
Top Forums Shell Programming and Scripting Help needed - Split large file into smaller files based on pattern match Post 302758049 by frustrated1 on Friday 18th of January 2013 11:20:02 AM
Old 01-18-2013
Help needed - Split large file into smaller files based on pattern match

Help needed urgently please.

I have a large file - a few hundred thousand lines.

Sample
Code:
CP START ACCOUNT
1234556
name 1
CP END ACCOUNT
CP START ACCOUNT
2224444
name 1
CP END ACCOUNT
CP START ACCOUNT
333344444
name 1
CP END ACCOUNT


I need to split this file each time "CP START ACCOUNT" is matched.

Preferably I would split it every 20 times this is matched and output to smaller files.


I was trying something like the below but could do with help obviously


Code:
awk '/START/{x="F"++i;}{print > x;}' inputfile
awk: too many output files 10
 record number 610


Also need to replace START above with CP START ACCOUNT


Can anyone help urgently?
Moderator's Comments:
Mod Comment
Please use code tags when posting data and code samples!

Last edited by vgersh99; 01-18-2013 at 12:25 PM.. Reason: code tags, please!
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

splitting the large file into smaller files

hi all im new to this forum..excuse me if anythng wrong. I have a file containing 600 MB data in that. when i do parse the data in perl program im getting out of memory error. so iam planning to split the file into smaller files and process one by one. can any one tell me what is the code... (1 Reply)
Discussion started by: vsnreddy
1 Replies

2. Shell Programming and Scripting

Split a file into multiple files based on the input pattern

I have a file with lines something like. ...... 123_start ...... ....... 123_end .... ..... 456_start ...... ..... 456_end .... ..... 789_start .... .... 789_end (6 Replies)
Discussion started by: abinash
6 Replies

3. Shell Programming and Scripting

Splitting large file into multiple files in unix based on pattern

I need to write a shell script for below scenario My input file has data in format: qwerty0101TWE 12345 01022005 01022005 datainala alanfernanded 26 qwerty0101mXZ 12349 01022005 06022008 datainalb johngalilo 28 qwerty0101TWE 12342 01022005 07022009 datainalc hitalbert 43 qwerty0101CFG 12345... (19 Replies)
Discussion started by: jimmy12
19 Replies

4. Shell Programming and Scripting

Split large file into smaller file

hi Guys i need some help here.. i have a file which has > 800,000 lines in it. I need to split this file into smaller files with 25000 lines each. please help thanks (1 Reply)
Discussion started by: sitaldip
1 Replies

5. Shell Programming and Scripting

split XML file into multiple files based on pattern

Hello, I am using awk to split a file into multiple files using command: nawk '{ if ( $1 == "<process" ) { n=split($2, arr, "\""); file=arr } print > file }' processes.xml <process name="Process1.process"> ... (3 Replies)
Discussion started by: chiru_h
3 Replies

6. Shell Programming and Scripting

Sed: Splitting A large File into smaller files based on recursive Regular Expression match

I will simplify the explaination a bit, I need to parse through a 87m file - I have a single text file in the form of : <NAME>house........ SOMETEXT SOMETEXT SOMETEXT . . . . </script> MORETEXT MORETEXT . . . (6 Replies)
Discussion started by: sumguy
6 Replies

7. UNIX for Dummies Questions & Answers

Split a huge 7 GB File Based on Pattern into 4 files

Hi, I have a Huge 7 GB file which has around 1 million records, i want to split this file into 4 files to contain around 250k messages each. Please help me as Split command cannot work here as it might miss tags.. Format of the file is as below <!--###### ###### START-->... (6 Replies)
Discussion started by: KishM
6 Replies

8. Shell Programming and Scripting

Split Large Files Based On Row Pattern..

Hi all. I've tried searching the web but could not find similar problem to mine. I have one large file to be splitted into several files based on the matching pattern found in each row. For example, let's say the file content: ... (13 Replies)
Discussion started by: aimy
13 Replies

9. UNIX for Dummies Questions & Answers

Split large file to smaller fastly

hi , I have a requirement input file: 1 1111111111111 108 1 1111111111111 109 1 1111111111111 109 1 1111111111111 110 1 1111111111111 111 1 1111111111111 111 1 1111111111111 111 1 1111111111111 112 1 1111111111111 112 1 1111111111111 112 The output should be, (19 Replies)
Discussion started by: mechvijays
19 Replies

10. UNIX for Beginners Questions & Answers

Split large file into smaller files without disturbing the entry chunks

Dears, Need you help with the below file manipulation. I want to split the file into 8 smaller files but without cutting/disturbing the entries (meaning every small file should start with a entry and end with an empty line). It will be helpful if you can provide a one liner command for this... (12 Replies)
Discussion started by: Kamesh G
12 Replies
MC-TOOL(1)							     Utilities								MC-TOOL(1)

NAME
mc-tool - manipulate Mission Control accounts from the command line SYNOPSIS
mc-tool list mc-tool add PROFILE DISPLAY-NAME [PARAMETER-SETTINGS...] mc-tool add MANAGER/PROTOCOL [PARAMETER-SETTINGS...] mc-tool update ACCOUNT [PARAMETER-SETTINGS...] mc-tool display ACCOUNT DISPLAY-NAME mc-tool nick ACCOUNT NICKNAME mc-tool service ACCOUNT SERVICE-NAME mc-tool icon ACCOUNT ICON mc-tool show ACCOUNT mc-tool get ACCOUNT PARAMETER [PARAMETER...] mc-tool auto-presence ACCOUNT STATUS [MESSAGE] mc-tool request ACCOUNT STATUS [MESSAGE] mc-tool auto-connect ACCOUNT on|off mc-tool remove ACCOUNT DESCRIPTION
mc-tool's usage depends on its first argument (the "command") SPECIFYING ACCOUNTS Where an account name is needed, it may be given as a full object path (e.g. /org/freedesktop/Telepathy/Account/gabble/jabber/example), or as the unique account-specific suffix (e.g. gabble/jabber/example). SPECIFYING PARAMETER SETTINGS Many commands take arguments that represent Telepathy connection manager parameters (see the Telepathy Specification for details of these). Each argument has the following form: TYPE:NAME=VALUE where TYPE is either int, uint, bool, or string (other parameter types exist, but are currently unsupported by mc-tool), NAME is the name of the connection manager parameter (e.g. account or password), and VALUE is the parameter's value (with integers in decimal, and booleans as 0 or 1). The update command also accepts arguments of the form clear:NAME which delete the named parameter from the account configuration. LIST mc-tool list lists the available accounts. ADD mc-tool add adds an account. The connection manager and protocol can either be given as the name of a Mission Control profile, (e.g. gtalk), or as a slash-separated connection manager name and protocol (e.g. gabble/jabber). DISPLAY mc-tool display ACCOUNT DISPLAY-NAME sets the DisplayName property of ACCOUNT to DISPLAY-NAME. NICK mc-tool nick ACCOUNT NICKNAME sets the Nickname property of ACCOUNT to NICKNAME. SERVICE mc-tool service ACCOUNT SERVICE-NAME sets the Service property of ACCOUNT to SERVICE-NAME. ICON mc-tool icon ACCOUNT ICON sets the Icon property of ACCOUNT to ICON. SHOW mc-tool show ACCOUNT shows information about ACCOUNT. GET mc-tool get ACCOUNT PARAMETER [PARAMETER...] shows the values of the given parameters of ACCOUNT. ENABLE mc-tool enable ACCOUNT sets the Enabled property of ACCOUNT to True. DISABLE mc-tool disable ACCOUNT sets the Enabled property of ACCOUNT to False. AUTO-PRESENCE mc-tool auto-presence ACCOUNT STATUS [MESSAGE] sets the AutomaticPresence property of ACCOUNT to STATUS with optional message MESSAGE. REQUEST mc-tool request ACCOUNT STATUS [MESSAGE] sets the RequestedPresence property of ACCOUNT to STATUS with optional message MESSAGE; this is how you tell an account to connect or go offline, for example. AUTO-CONNECT mc-tool auto-connect ACCOUNT on sets the ConnectAutomatically property of ACCOUNT to True. mc-tool auto-connect ACCOUNT off sets it to False. Telepathy July 2009 MC-TOOL(1)
All times are GMT -4. The time now is 07:27 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy