The UNIX and Linux Forums  
Hello and Welcome from United States to the UNIX and Linux Forums! Thank You for Visiting and Joining Our Global Community.

Go Back   The UNIX and Linux Forums > Top Forums > Shell Programming and Scripting
divide a single file with different Weboffercodes into different files with each of o divide a single file with different Weboffercodes into different files with each of o
.
google unix.com

Forums Register Forum RulesLinksAlbums FAQ Members List Calendar Search Today's Posts Mark Forums Read


Shell Programming and Scripting Post questions about KSH, CSH, SH, BASH, PERL, PHP, SED, AWK and OTHER shell scripts and shell scripting languages here.

More UNIX and Linux Forum Topics You Might Find Helpful
Thread Thread Starter Forum Replies Last Post
Compare 2 files for a single column and output differences samit_9999 UNIX for Dummies Questions & Answers 1 04-23-2008 12:02 PM
Executing Multiple .SQL Files from Single Shell Script file anushilrai Shell Programming and Scripting 3 04-07-2008 10:09 AM
how to rename multiple files with a single command tayyabq8 Shell Programming and Scripting 5 03-18-2008 04:04 PM
how to divide single large log file into multiple files. kamleshm Shell Programming and Scripting 1 01-15-2008 07:33 PM
divide with awk tontal Shell Programming and Scripting 1 10-09-2006 07:23 AM

Closed Thread
English Japanese Spanish French German Portuguese Italian Dutch Swedish Russian Norwegian Hungarian Hebrew Danish Powered by Powered by Google
 
LinkBack Thread Tools Search this Thread Rate Thread Display Modes
  #1 (permalink)  
Old 03-31-2008
enigma_83 enigma_83 is offline
Registered User
  
 

Join Date: Mar 2008
Posts: 7
Question divide a single file with different Weboffercodes into different files with each of o
Here is the format of my file; I do not have the delimiter in the file for the data to be separated.

Each line in the file is in the following format. File contains the data of different WebOfferCodes

Item Code | Account Number | Card Number | Source code | WebOfferCode

12digits | 10 Digits | 15 digits | 10 letters | 20 letters
Format of file:-

3412222240171M41006422373273564021006SDFGHJRSGJBBBBBBBBBBBBBBBBBBBB
0009546640171M41006400373273564531345TTTTPAPAPBBBBBBBBBBBBBBBBBBBB
0009546640171M41006433373273564021006YYYYPAPAPHHHHHHHHHHHHHHHHHH
3412222240171M41006455373273564021116PAPAPAPAPBBBBBBBBBBBBBBBBBBBB
0009546640171M41006400373273564531345UUUUPAPAPHHHHHHHHHHHHHHHHHH
0009546640171M41006477373273564021006SDFGHJRSGJBBBBBBBBBBBBBBBBBBBBB

Item Code: 12digits

Account Number: 10 Digits

Card Number: 15 digits

Source_code:10 letters

WebOfferCode: 20 letters

I need the optimal ways to sort this file of million records.So i want to divide the single file with multiple WebOffercode into distinct file containing data of 1 Weboffercode each

Validation to be made:-

Card Member fields should not be null
Web Offer code fields should not be null
  #2 (permalink)  
Old 03-31-2008
aigles's Avatar
aigles aigles is online now Forum Advisor  
Registered User
  
 

Join Date: Apr 2004
Location: Bordeaux, France
Posts: 1,416
A possible solution :
Code:
$ cat web.awk
#!/usr/bin/awk -f

BEGIN {
   ErrorCode = "invalid";
}

{
   #Item Code | Account Number | Card Number | Source code | WebOfferCode
   #12digits | 10 Digits | 15 digits | 10 letters | 20 letters

   ItemCode      = substr($0,  1, 12);
   AccountNumber = substr($0, 13, 10);
   CardNumber    = substr($0, 23, 15);
   SourceCode    = substr($0, 38, 10);
   WebOfferCode  = substr($0, 48, 20);


   cn  = CardNumber  ; gsub(/[0 ]/, "", cn  );
   woc = WebOfferCode; gsub(/[0 ]/, "", woc );
   if (cn == "" || woc == "")
      OutputFile = "woc_" ErrorCode ".dat"
   else
      OutputFile = "woc_" WebOfferCode ".dat"
   print $0 >> OutputFile;

   printf("%-28s : %s|%s|%s|%s|%s\n", OutputFile, ItemCode, AccountNumber, CardNumber, SourceCode, WebOfferCode);
}

$ cat web.dat
3412222240171M41006422373273564021006SDFGHJRSGJBBBBBBBBBBBBBBBBBBBB
0009546640171M41006400373273564531345TTTTPAPAP
0009546640171M41006400373273564531345TTTTPAPAP00000000000000000000
0009546640171M41006400               TTTTPAPAPBBBBBBBBBBBBBBBBBBBB
0009546640171M41006400000000000000000TTTTPAPAPBBBBBBBBBBBBBBBBBBBB
0009546640171M41006400373273564531345TTTTPAPAPBBBBBBBBBBBBBBBBBBBB
0009546640171M41006433373273564021006YYYYPAPAPHHHHHHHHHHHHHHHHHH
3412222240171M41006455373273564021116PAPAPAPAPBBBBBBBBBBBBBBBBBBBB
0009546640171M41006400373273564531345UUUUPAPAPHHHHHHHHHHHHHHHHHH
0009546640171M41006477373273564021006SDFGHJRSGJBBBBBBBBBBBBBBBBBBBBB
$ web.awk web.dat
woc_BBBBBBBBBBBBBBBBBBBB.dat : 341222224017|1M41006422|373273564021006|SDFGHJRSGJ|BBBBBBBBBBBBBBBBBBBB
woc_invalid.dat              : 000954664017|1M41006400|373273564531345|TTTTPAPAP |
woc_invalid.dat              : 000954664017|1M41006400|373273564531345|TTTTPAPAP0|0000000000000000000
woc_invalid.dat              : 000954664017|1M41006400|               |TTTTPAPAPB|BBBBBBBBBBBBBBBBBBB
woc_invalid.dat              : 000954664017|1M41006400|000000000000000|TTTTPAPAPB|BBBBBBBBBBBBBBBBBBB
woc_BBBBBBBBBBBBBBBBBBB.dat  : 000954664017|1M41006400|373273564531345|TTTTPAPAPB|BBBBBBBBBBBBBBBBBBB
woc_HHHHHHHHHHHHHHHHH.dat    : 000954664017|1M41006433|373273564021006|YYYYPAPAPH|HHHHHHHHHHHHHHHHH
woc_BBBBBBBBBBBBBBBBBBB.dat  : 341222224017|1M41006455|373273564021116|PAPAPAPAPB|BBBBBBBBBBBBBBBBBBB
woc_HHHHHHHHHHHHHHHHH.dat    : 000954664017|1M41006400|373273564531345|UUUUPAPAPH|HHHHHHHHHHHHHHHHH
woc_BBBBBBBBBBBBBBBBBBBB.dat : 000954664017|1M41006477|373273564021006|SDFGHJRSGJ|BBBBBBBBBBBBBBBBBBBB
$ ls -1 woc_*.dat
woc_BBBBBBBBBBBBBBBBBBB.dat
woc_BBBBBBBBBBBBBBBBBBBB.dat
woc_HHHHHHHHHHHHHHHHH.dat
woc_invalid.dat
$
With gawk, you can also do :
Code:
$ cat web2.awk
#!/usr/bin/gawk -f

BEGIN {
   FIELDWIDTHS = "12 10 15 10 20";
   ErrorCode = "invalid";
}

{
   #Item Code | Account Number | Card Number | Source code | WebOfferCode
   #12digits | 10 Digits | 15 digits | 10 letters | 20 letters

   ItemCode      = $1;
   AccountNumber = $2;
   CardNumber    = $3;
   SourceCode    = $4;
   WebOfferCode  = $5;


   cn  = CardNumber  ; gsub(/[0 ]/, "", cn  );
   woc = WebOfferCode; gsub(/[0 ]/, "", woc );
   if (cn == "" || woc == "")
      OutputFile = "woc_" ErrorCode ".dat"
   else
      OutputFile = "woc_" WebOfferCode ".dat"
   print $0 >> OutputFile;

   # DebuG
   printf("%-28s : %s|%s|%s|%s|%s\n", OutputFile, ItemCode, AccountNumber, CardNumber, SourceCode, WebOfferCode);
}
$
Jean-Pierre.
  #3 (permalink)  
Old 04-01-2008
enigma_83 enigma_83 is offline
Registered User
  
 

Join Date: Mar 2008
Posts: 7
Hey!!!
Thanks a lot your reply is of great help to me...
  #4 (permalink)  
Old 04-02-2008
summer_cherry summer_cherry is offline Forum Advisor  
Registered User
  
 

Join Date: Jun 2007
Location: Beijing China
Posts: 1,078
Code:
awk '{
t=substr($0,48)
if(file[t]=="")
        file[t]=$0
else
        file[t]=sprintf("%s\n%s",file[t],$0)
}
END{
for (i in file)
{
        print file[i] > i
        close(i)
}
}' filename
  #5 (permalink)  
Old 04-02-2008
enigma_83 enigma_83 is offline
Registered User
  
 

Join Date: Mar 2008
Posts: 7
Join the data in two files
hello friendz...

I have multiple around 6 files in the folder I have to load the each file to the different tables in DB2 database.
List of tables is present in a file called table_list.txt file
MR_Detail_1

MR_Detail_4

MR_Detail_5

Note:-

LET.MRTEST.BBBB.txt present in xyz folder should be loaded into MR_Detail_1 table

LET.MRTEST.CCCC.txt present in xyz folder should be loaded into MR_detail_4 table

LET.MRTEST.DDDD.txt present in xyz folder should be loaded into MR_detail_5 table


There are around 6 columns in the database and the data comes from the file.

Format for the file

3412222240171M41006422373273564021006SDFGHJRSGJBBBBBBBBBBBBBBBBBBBB

0009546640171M41006400373273564531345TTTTPAPAPABBBBBBBBBBBBBBBBBBBB

Each file contains this type of data .How will I write my shell script so that my code picks the file automatically from xyz folder and keep loading on the Table_list.txt.There should me mapping between the Files and tables

What do you suggest shall I keep this mapping between file name and table in 1 file and read this file..how can do this

like creating this format file

LET.MRTEST.BBBB.txt MR_DETAIL_1
LET.MRTEST.BBBB.txt MR_DETAIL_4
LET.MRTEST.BBBB.txt MR_DETAIL_5

AND READING these elements and substituting in my loadutility command how can I do this...please help me
Closed Thread

Bookmarks

« sendmail return code | Date Function »
Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes Rate This Thread
Hybrid Mode Hybrid Mode
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts
BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On



All times are GMT -4. The time now is 12:49 PM.

The UNIX and Linux Forums - Learn UNIX and UNIX Commands RSS Feeds - Contact Us - The UNIX and Linux Forums - Learn UNIX and UNIX Commands - Archive - Top

Powered by: vBulletin, Copyright ©2000 - 2006, Jelsoft Enterprises Limited. Language Translations Powered by .
vBCredits v1.4 Copyright ©2007 - 2008, PixelFX Studios
The UNIX and Linux Forums Content Copyright ©1993-2009. All Rights Reserved.Ad Management by RedTyger

Content Relevant URLs by vBSEO 3.2.0