Sponsored Content
Top Forums Shell Programming and Scripting Help needed - Split large file into smaller files based on pattern match Post 302758079 by Yoda on Friday 18th of January 2013 11:44:56 AM
Old 01-18-2013
Another approach is using a BASH script, but this is gonna run slower than awk:
Code:
#!/bin/bash

c=0
while read line
do
        if [[ "$line" =~ "^CP START ACCOUNT" ]]
        then
                c=$(( c + 1 ))
                echo "$line" >> F${c}.txt
        else
                echo "$line" >> F${c}.txt
        fi
done < inputfile

 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

splitting the large file into smaller files

hi all im new to this forum..excuse me if anythng wrong. I have a file containing 600 MB data in that. when i do parse the data in perl program im getting out of memory error. so iam planning to split the file into smaller files and process one by one. can any one tell me what is the code... (1 Reply)
Discussion started by: vsnreddy
1 Replies

2. Shell Programming and Scripting

Split a file into multiple files based on the input pattern

I have a file with lines something like. ...... 123_start ...... ....... 123_end .... ..... 456_start ...... ..... 456_end .... ..... 789_start .... .... 789_end (6 Replies)
Discussion started by: abinash
6 Replies

3. Shell Programming and Scripting

Splitting large file into multiple files in unix based on pattern

I need to write a shell script for below scenario My input file has data in format: qwerty0101TWE 12345 01022005 01022005 datainala alanfernanded 26 qwerty0101mXZ 12349 01022005 06022008 datainalb johngalilo 28 qwerty0101TWE 12342 01022005 07022009 datainalc hitalbert 43 qwerty0101CFG 12345... (19 Replies)
Discussion started by: jimmy12
19 Replies

4. Shell Programming and Scripting

Split large file into smaller file

hi Guys i need some help here.. i have a file which has > 800,000 lines in it. I need to split this file into smaller files with 25000 lines each. please help thanks (1 Reply)
Discussion started by: sitaldip
1 Replies

5. Shell Programming and Scripting

split XML file into multiple files based on pattern

Hello, I am using awk to split a file into multiple files using command: nawk '{ if ( $1 == "<process" ) { n=split($2, arr, "\""); file=arr } print > file }' processes.xml <process name="Process1.process"> ... (3 Replies)
Discussion started by: chiru_h
3 Replies

6. Shell Programming and Scripting

Sed: Splitting A large File into smaller files based on recursive Regular Expression match

I will simplify the explaination a bit, I need to parse through a 87m file - I have a single text file in the form of : <NAME>house........ SOMETEXT SOMETEXT SOMETEXT . . . . </script> MORETEXT MORETEXT . . . (6 Replies)
Discussion started by: sumguy
6 Replies

7. UNIX for Dummies Questions & Answers

Split a huge 7 GB File Based on Pattern into 4 files

Hi, I have a Huge 7 GB file which has around 1 million records, i want to split this file into 4 files to contain around 250k messages each. Please help me as Split command cannot work here as it might miss tags.. Format of the file is as below <!--###### ###### START-->... (6 Replies)
Discussion started by: KishM
6 Replies

8. Shell Programming and Scripting

Split Large Files Based On Row Pattern..

Hi all. I've tried searching the web but could not find similar problem to mine. I have one large file to be splitted into several files based on the matching pattern found in each row. For example, let's say the file content: ... (13 Replies)
Discussion started by: aimy
13 Replies

9. UNIX for Dummies Questions & Answers

Split large file to smaller fastly

hi , I have a requirement input file: 1 1111111111111 108 1 1111111111111 109 1 1111111111111 109 1 1111111111111 110 1 1111111111111 111 1 1111111111111 111 1 1111111111111 111 1 1111111111111 112 1 1111111111111 112 1 1111111111111 112 The output should be, (19 Replies)
Discussion started by: mechvijays
19 Replies

10. UNIX for Beginners Questions & Answers

Split large file into smaller files without disturbing the entry chunks

Dears, Need you help with the below file manipulation. I want to split the file into 8 smaller files but without cutting/disturbing the entries (meaning every small file should start with a entry and end with an empty line). It will be helpful if you can provide a one liner command for this... (12 Replies)
Discussion started by: Kamesh G
12 Replies
RDUP-UP(1)							       rdup								RDUP-UP(1)

NAME
rdup-up - update a directory tree with a rdup archive SYNOPSIS
rdup-up [OPTION]... DIRECTORY DESCRIPTION
With rdup-up you can update an (possibly) existing directory structure with a rdup archive. The rdup archive has to be given to rdup-up's standard input. Username and uids rdup outputs both the username and uid, the receiving system (which may be a totally different system) checks if the username and uid match. If the username and uid don't match the (numeric) uid is used on the file. The same holds true for the groupname and gid. OPTIONS
-n Do a dry-run and do not create anything on disk. -t Create DIRECTORY (ala mkdir -p) if it does not exist. -s N Strip N path components from a pathname. If the resulting pathname is empty after this operation it is skipped. Be careful however with the following structure: /foo /foo/bar /foo/bar/bla.txt /foo/blork/bla.txt With rdup-up -s2 this will leave: <empty> <empty> /bla.txt /bla.txt And the last 'bla.txt' will overwrite the previous one, this will happen without warnings. -r PATH This option is related to the -s option, but works different. The string PATH is removed from (the beginning of) each pathname. With -r /home/backup the pathname /home/backup/bin/mycmd becomes /bin/mycmd. The same could be done with -s 2, but then you need to count the slashes. Note -s is always performed before -r. -v Be more verbose and echo the processed files to standard output. -vv Be even more verbose and echo processed file and the uid and gid information to standard output. -T Show a table of contents of the rdup stream received (ala tar -tf -). With -T the directory argument is optional. -T unsets any verbose (-v) options. -h A short help message. -V Show the version. EXIT CODE
rdup-up return a zero exit code on success, otherwise 1 is returned. AUTHOR
Written by Miek Gieben. REPORTING BUGS
Report bugs to <miek@miek.nl>. SEE ALSO
http:/www.miek.nl/projects/rdup/ is the main site of rdup. Also see rdup(1), rdup-tr(1) and rdup-backups(7). COPYRIGHT
Copyright (C) 2005-2010 Miek Gieben. This is free software. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. Licensed under the GPL version 3. See the file LICENSE in the source distribution of rdup. 1.1.11 13 Dec 2008 RDUP-UP(1)
All times are GMT -4. The time now is 08:03 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy