The UNIX and Linux Forums  

Go Back   The UNIX and Linux Forums > Top Forums > Shell Programming and Scripting
Google UNIX.COM


Shell Programming and Scripting Post questions about KSH, CSH, SH, BASH, PERL, PHP, SED, AWK and OTHER shell scripts here.

More UNIX and Linux Forum Topics You Might Find Helpful
Thread Thread Starter Forum Replies Last Post
Split large file and add header and footer to each small files ashish4422 Shell Programming and Scripting 7 07-07-2008 11:13 AM
Splitting text file to several other files using sed. JeffV Shell Programming and Scripting 3 03-14-2008 12:34 PM
splitting files based on text in the file matrix1067 Shell Programming and Scripting 1 01-30-2006 05:45 PM
Splitting large files Rvbs Shell Programming and Scripting 2 12-07-2005 05:29 AM
Splitting a large log file simmonet UNIX for Dummies Questions & Answers 8 09-19-2001 01:14 AM

Reply
 
Submit Tools LinkBack Thread Tools Search this Thread Display Modes
  #1  
Old 06-07-2005
Registered User
 

Join Date: Jun 2005
Posts: 10
Splitting large file into small files

Hi,

I need to split a large file into small files based on a string.

At different palces in the large I have the string ^Job.
I need to split the file into different files starting from ^Job to the last character before the next ^Job.
Also all the small files should be automatically named.

Please suggest.

Thanks,
Chandra
Reply With Quote
Forum Sponsor
  #2  
Old 06-07-2005
vgersh99's Avatar
Moderator
 

Join Date: Feb 2005
Location: Boston, MA
Posts: 3,029
Quote:
Originally Posted by dncs
Hi,

I need to split a large file into small files based on a string.

At different palces in the large I have the string ^Job.
I need to split the file into different files starting from ^Job to the last character before the next ^Job.
Also all the small files should be automatically named.

Please suggest.

Thanks,
Chandra
man csplit
Reply With Quote
  #3  
Old 06-07-2005
Registered User
 

Join Date: Jun 2005
Posts: 10
I have gone thru the Man csplit help but not sure how to use it for my requirement.
Reply With Quote
  #4  
Old 06-08-2005
Bughunter Extraordinaire
 

Join Date: May 2005
Location: In the leftmost byte of /dev/kmem
Posts: 1,262
You have omitted some necessary information:

1. can the string appear only on the begin or in the middle of lines too?

2. Is the split-string to appear in the output too or is it to be stripped?

3. is the line containing the string considered to go to the new part of the file or is the file to be split exactly at the search string? That is, in the following example:

Code:
This is the first part
still first part ^Job This is the new part
is "still first part" considered to go to first or second part? Is it even possible, according to question 1?

Assuming the split-string can appear anywhere in the file and lines are to be split exactly where the split-string occurs the solution is:

Code:
#!/bin/ksh

typeset srcfile="file"
typeset -i cnt=1
typeset line=""

exec 3>${srcfile}.part${cnt}                # define our output file
cat $srcfile | while read line ; do
                                            # we have a line with a splitter
     if [ $(print - "$line" | grep -c "\^Job") -gt 0 ] ; then
          print -u3 "${line%%^Job*}"        # put the part of the line before
                                            # the splitter to the old output
          exec 3>&-                         # close output
          (( cnt += 1 ))
          exec 3>${srcfile}.part${cnt}      # open the next part
          print -u3 "${line##*^Job}"        # output part of line after the
                                            # occurence of the splitter
     else                                   # this is a regular line, just print
          print -u3 "$line"
     fi
done
exec 3>&-                                   # close last output file

exit 0
The reason for the "exec"s is them making output to various files easier than the hassle with redirections IMHO.

To make the script more general I'd prefer putting the split-string into a variable and feed that by a commandline option. This is left as an exercise to the reader.

bakunin
Reply With Quote
  #5  
Old 06-08-2005
Registered User
 

Join Date: Jun 2005
Posts: 10
Hi bakunin,
Thanks for your reply.
I will try the solution provided by you.
To provide more details , the file I am going to split will be having multiple Purchase Orders and after executing the script I should have 'n' number of files one for each PO.
It looks like below
^Job
PO No1:
<Po Details>
^Job
PO No1:
<Po Details>
......
I think the logic provided by you should fit for my requirement.
Thanks for your help.
Chandra
Reply With Quote
Google The UNIX and Linux Forums
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes




All times are GMT -7. The time now is 11:54 PM.


Powered by: vBulletin, Copyright ©2000 - 2006, Jelsoft Enterprises Limited.
The UNIX and Linux Forums Content Copyright ©1993-2008. All Rights Reserved.Ad Management by RedTyger Visit The Complex Event Processing Blog

Content Relevant URLs by vBSEO 3.2.0