Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Split one file to Multiple file with report basis in unix

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Split one file to Multiple file with report basis in unix
# 1  
Old 02-03-2011
MySQL Split one file to Multiple file with report basis in unix
Hi, Please help on this. i want split the below file(11020111.CLT) to more files with some condition. Smilie
1) %s stating of the report
2) %e ending of the report

example starting of the report:
Code:
%sAEGONCA| |MUMBAI | :EXPC|N|D

ending of the report
Code:
%eAEGONCA| |MUMBAI | :EXPC

3)so the between and start&end lines going to one file,then EACH REPORT name is city name BOMBA0001.txt and next REPORT name BOMBA0002.txt..etc .
4) then compare the report header in the .CLT file
like:
Code:
%sAEGONCA| |MUMBAI | :EXPC|N|D

eample:
Code:
%sAEGONCA| |MUMBAI | :EXPC|N|D
%sAGENTCA| |MUMBAI | :EXPC|Y|D

once above is equal...this two report going to generate one report file.

File: 11020111.CLT
Code:
DRAWEE LOC.NAME|DEPOSIT DATE|ACTIVATION DATE|DAY|AMOUNT|CLIENT CODE|PRODUCT|CBP.NO|EXP.CLEARING DATE
MUMBAI|29/01/2011|29/01/2011|Tue| 21643.00|AEGONCA|aaaa|110290361013 |01/02/2011
MUMBAI|29/01/2011|29/01/2011|Tue| 142.00|AEGONCA|aaaa|110290361014 |01/02/2011
BHUBANESHWAR (ORI.)|31/01/2011|31/01/2011|Tue| 70751.00|AEGONCA|CCLR|110310758005 |01/02/2011
ERNAKULAM (KERALA)|31/01/2011|31/01/2011|Tue| 68980.00|AEGONCA|CCLR|110310905015 |01/02/2011
AHMEDABAD|31/01/2011|31/01/2011|Tue| 108000.00|AEGONCA|aaaa|110310924024 |01/02/2011
BHOPAL (M.P.)|31/01/2011|31/01/2011|Tue| 97832.00|AEGONCA|aaaa|110310959001 |01/02/2011
%eAEGONCA| |MUMBAI | :EXPC
 
%sAGENTCA| |MUMBAI | :EXPC|N|D
DRAWEE LOC.NAME|DEPOSIT DATE|ACTIVATION DATE|DAY|AMOUNT|CLIENT CODE|PRODUCT|CBP.NO|EXP.CLEARING DATE
UDIPI(KAR) CASH|31/01/2011|31/01/2011|Tue| 250.00|AGENTCA|aaaa|110310667003 |01/02/2011
MANGALORE(KAR) CASH|31/01/2011|31/01/2011|Tue| 350.00|AGENTCA|aaaa|110310839008 |01/02/2011
ERNAKULAM (KERALA)|31/01/2011|31/01/2011|Tue| 250.00|AGENTCA|aaaa|110310905004 |01/02/2011
ERNAKULAM (KERALA)|31/01/2011|31/01/2011|Tue| 600.00|AGENTCA|aaaa|110310905005 |01/02/2011
ERNAKULAM (KERALA)|31/01/2011|31/01/2011|Tue| 1500.00|AGENTCA|aaaa|110310905006 |01/02/2011
%eAGENTCA| |MUMBAI | :EXPC
 
%sPFRDAXX| |MUMBAI | :EXPC|N|D

DRAWEE LOC.NAME|DEPOSIT DATE|ACTIVATION DATE|DAY|AMOUNT|CLIENT CODE|PRODUCT|CBP.NO|EXP.CLEARING DATE
CALCUTTA|31/01/2011|31/01/2011|Tue| 15000.00|PFRDAXX|aaaa|110311499006 |01/02/2011
%ePFRDAXX| |MUMBAI | :EXPC
 
%sAGENTCA| |MUMBAI | :EXPC|Y|D
DRAWEE LOC.NAME|DEPOSIT DATE|ACTIVATION DATE|DAY|AMOUNT|CLIENT CODE|PRODUCT|CBP.NO|EXP.CLEARING DATE
UDIPI(KAR) CASH|31/01/2011|31/01/2011|Tue| 2502.00|AGENTCA|aaaa|110310667003 |01/02/2011
MANGALORE(KAR) CASH|31/01/2011|31/01/2011|Tue| 3050.00|AGENTCA|aaaa|110310839008 |01/02/2011
ERNAKULAM (KERALA)|31/01/2011|31/01/2011|Tue| 2250.00|AGENTCA|aaaa|110310905004 |01/02/2011
ERNAKULAM (KERALA)|31/01/2011|31/01/2011|Tue| 600.00|AGENTCA|aaaa|110310905005 |01/02/2011
ERNAKULAM (KERALA)|31/01/2011|31/01/2011|Tue| 1520.00|AGENTCA|aaaa|110310905006 |01/02/2011
%eAGENTCA| |MUMBAI | :EXPC

11020111.TXT (2.5 KB) 
Last edited by Scott; 02-06-2011 at 10:52 AM.. Reason: Code tags
# 2  
Old 02-03-2011
The inner sed write a sed script for the outer sed, which in a second pass puts your subsections into like named files:
Code:
sed $(
 sed -n '
   s/^\%s\([^|]*| |\([^ |]*\) *| :EXPC$/\/^&$\/,\/&|.*\/w "\1_\2.txt"/
  ' $file_in
 ) $file_in

# 3  
Old 02-03-2011
Hi ,Thanks for writing me...

i want the report file like below from 11020111.CLT.

MUMBAI0001.txt
Code:
%sAEGONCA| |MUMBAI | :EXPC|N|D
 
DRAWEE LOC.NAME|DEPOSIT DATE|ACTIVATION DATE|DAY|AMOUNT|CLIENT CODE|PRODUCT|CBP.NO|EXP.CLEARING DATE
MUMBAI|29/01/2011|29/01/2011|Tue| 21643.00|AEGONCA|aaaa|110290361013 |01/02/2011
MUMBAI|29/01/2011|29/01/2011|Tue| 142.00|AEGONCA|aaaa|110290361014 |01/02/2011
BHUBANESHWAR (ORI.)|31/01/2011|31/01/2011|Tue| 70751.00|AEGONCA|CCLR|110310758005 |01/02/2011
ERNAKULAM (KERALA)|31/01/2011|31/01/2011|Tue| 68980.00|AEGONCA|CCLR|110310905015 |01/02/2011
AHMEDABAD|31/01/2011|31/01/2011|Tue| 108000.00|AEGONCA|aaaa|110310924024 |01/02/2011
BHOPAL (M.P.)|31/01/2011|31/01/2011|Tue| 97832.00|AEGONCA|aaaa|110310959001 |01/02/2011
%eAEGONCA| |MUMBAI | :EXPC

MUMBAI002.TXT
Code:
%sAGENTCA| |MUMBAI | :EXPC|N|D
DRAWEE LOC.NAME|DEPOSIT DATE|ACTIVATION DATE|DAY|AMOUNT|CLIENT CODE|PRODUCT|CBP.NO|EXP.CLEARING DATE
UDIPI(KAR) CASH|31/01/2011|31/01/2011|Tue| 250.00|AGENTCA|aaaa|110310667003 |01/02/2011
MANGALORE(KAR) CASH|31/01/2011|31/01/2011|Tue| 350.00|AGENTCA|aaaa|110310839008 |01/02/2011
ERNAKULAM (KERALA)|31/01/2011|31/01/2011|Tue| 250.00|AGENTCA|aaaa|110310905004 |01/02/2011
ERNAKULAM (KERALA)|31/01/2011|31/01/2011|Tue| 600.00|AGENTCA|aaaa|110310905005 |01/02/2011
ERNAKULAM (KERALA)|31/01/2011|31/01/2011|Tue| 1500.00|AGENTCA|aaaa|110310905006 |01/02/2011
%eAGENTCA| |MUMBAI | :EXPC
 
%sAGENTCA| |MUMBAI | :EXPC|Y|D
DRAWEE LOC.NAME|DEPOSIT DATE|ACTIVATION DATE|DAY|AMOUNT|CLIENT CODE|PRODUCT|CBP.NO|EXP.CLEARING DATE
UDIPI(KAR) CASH|31/01/2011|31/01/2011|Tue| 2502.00|AGENTCA|aaaa|110310667003 |01/02/2011
MANGALORE(KAR) CASH|31/01/2011|31/01/2011|Tue| 3050.00|AGENTCA|aaaa|110310839008 |01/02/2011
ERNAKULAM (KERALA)|31/01/2011|31/01/2011|Tue| 2250.00|AGENTCA|aaaa|110310905004 |01/02/2011
ERNAKULAM (KERALA)|31/01/2011|31/01/2011|Tue| 600.00|AGENTCA|aaaa|110310905005 |01/02/2011
ERNAKULAM (KERALA)|31/01/2011|31/01/2011|Tue| 1520.00|AGENTCA|aaaa|110310905006 |01/02/2011
%eAGENTCA| |MUMBAI | :EXPC

MUMBAI003.TXT
Code:
%sPFRDAXX| |MUMBAI | :EXPC|N|D
 
DRAWEE LOC.NAME|DEPOSIT DATE|ACTIVATION DATE|DAY|AMOUNT|CLIENT CODE|PRODUCT|CBP.NO|EXP.CLEARING DATE
CALCUTTA|31/01/2011|31/01/2011|Tue| 15000.00|PFRDAXX|aaaa|110311499006 |01/02/2011
%ePFRDAXX| |MUMBAI | :EXPC

PLEASE HELP ON THIS. thanks in advance.Smilie

---------- Post updated at 03:16 PM ---------- Previous update was at 03:12 PM ----------

This script is not working...pls help
Last edited by Scott; 02-06-2011 at 10:53 AM.. Reason: Code tags
# 4  
Old 02-03-2011
Try...
Code:
awk -F '|' '/^%s/{t=$3;gsub(/ /,"",t);c=++a[t];f=sprintf("%05s%04d.txt",t,c)}{print $0>f}' 11020111.TXT

# 5  
Old 02-05-2011
Thanks.. this script is not working....

---------- Post updated at 01:39 PM ---------- Previous update was at 01:30 PM ----------

Pls help on this...Pls
# 6  
Old 02-05-2011
Bumping up posts or double posting is not permitted in these forums.

Please read the rules, which you agreed to when you registered, if you have not already done so.

You may receive an infraction for this. If so, don't worry, just try to follow the rules more carefully. The infraction will expire in the near future

Thank You.

The UNIX and Linux Forums.
# 7  
Old 02-05-2011
Lets do this in stages. If you grep -n, you can get the line numbers for each start and end, and from them, you can sed -n '123,456p' file to extract the data between those line numbers inclusive. If you need to adjust the line numbers up or down, you can "ln=$(( $ln + 1 ))". Start with:
Code:
rct=0
grep -n '| :EXPC|' $file | while read ln
do
 sed '
   1,'#\"$ln"'d
   /| :EXPC$/,$d
  ' $file >report_$(( $rct += 1 ))
done

Of course, if you want to read every data line into a shell or PERL script, you can do it in one pass. Unless you have a lot of data, or a demanding boss or professor, it is not worth the effort.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Split into multiple files by using Unique columns in a UNIX file

I have requirement to split below file (sample.csv) into multiple files by using the unique columns (first 3 are unique columns) sample.csv 123|22|56789|ABCDEF|12AB34|2019-07-10|2019-07-10|443.3400|1|1 123|12|5679|BCDEFG|34CD56|2019-07-10|2019-07-10|896.7200|1|2... (3 Replies)
Discussion started by: RVSP
3 Replies

2. UNIX for Beginners Questions & Answers

Split large file into 24 small files on one hour basis

I Have a large file with 24hrs log in the below format.i need to split the large file in to 24 small files on one hour based.i.e ex:from 09:55 to 10:55,10:55-11:55 can any one help me on this.! ... (20 Replies)
Discussion started by: Raghuram717
20 Replies

3. UNIX for Beginners Questions & Answers

Split a txt file on the basis of line number

I have to split a file containing 100 lines to 5 files say from lines ,1-20 ,21-30 ,31-40 ,51-60 ,61-100 Here is i can do it for 2 file but how to handle it for more than 2 files awk 'NR < 21{ print >> "a"; next } {print >> "b" }' $input_file Please advidse. Thanks (4 Replies)
Discussion started by: abhaydas
4 Replies

4. Linux

Split a large textfile (one file) into multiple file to base on ^L

Hi, Anyone can help, I have a large textfile (one file), and I need to split into multiple file to break each file into ^L. My textfile ========== abc company abc address abc contact ^L my company my address my contact my skills ^L your company your address ========== (3 Replies)
Discussion started by: fspalero
3 Replies

5. Shell Programming and Scripting

Splitting XML file on basis of line number into multiple file

Hi All, I have more than half million lines of XML file , wanted to split in four files in a such a way that top 7 lines should be present in each file on top and bottom line of should be present in each file at bottom. from the 8th line actual record starts and each record contains 15 lines... (14 Replies)
Discussion started by: ajju
14 Replies

6. Shell Programming and Scripting

How to split file into multiple files using awk based on 1 field in the file?

Good day all I need some helps, say that I have data like below, each field separated by a tab DATE NAME ADDRESS 15/7/2012 LX a.b.c 15/7/2012 LX1 a.b.c 16/7/2012 AB a.b.c 16/7/2012 AB2 a.b.c 15/7/2012 LX2 a.b.c... (2 Replies)
Discussion started by: alexyyw
2 Replies

7. Shell Programming and Scripting

Split a file into multiple files based on first two digits of file.

Hi , I do have a fixedwidth flatfile that has data for 10 different datasets each identified by the first two digits in the flatfile. 01 in the first two digit position refers to Set A 02 in the first two digit position refers to Set B and so on I want to genrate 10 different files from my... (6 Replies)
Discussion started by: okkadu
6 Replies

8. Shell Programming and Scripting

Split file in unix into multiple files

Hi Gurus I have to split the incoming source file into multiple file. File contains some unwanted XML tags also . Files looks like some XML tags FILEHEADERABC 12 -- --- ---- EOF some xml tags xxxFILEHEADERABC 13 -- --- ---- EOF I have to ignore XML tags and only split file... (6 Replies)
Discussion started by: manish2608
6 Replies

9. Shell Programming and Scripting

Help with split one file content into multiple different file

Input file: cat input_file.txt data_1 2342 data_3 242 data_1 3546 data_5 458 data_10 342 data_30 42 data_10 346 content_50 48 content_1 2343 Desired output: cat output_file1.txt data_1 2342 data_3 242 data_1 3546 data_5 458 (2 Replies)
Discussion started by: perl_beginner
2 Replies

10. Shell Programming and Scripting

Help with split the file content into multiple different file

Input file content: NA_10001 XA_10081 NG_10015 AC_1321.1 . . Desired output file: ls *.txt NA_10001.txt XA_10081.txt NG_10015.txt AC_1321.1.txt cat NA_10001.txt NA_10001 cat XA_10081.txt XA_10081 (1 Reply)
Discussion started by: perl_beginner
1 Replies
Login or Register to Ask a Question