Sponsored Content
Top Forums Shell Programming and Scripting Split Large Files Based On Row Pattern.. Post 302876645 by aimy on Tuesday 26th of November 2013 12:44:25 AM
Old 11-26-2013
Thanks a lot Don. The filename now worked perfectly.

Regarding the header produced as txt file. Actually that was based on the real file which actually got extra info on top:
Code:
HD|20131126_104934|1
MONTH,ACCOUNT_NO,CUSTOMER_NAME,SEGMENT_CODE,COST_CENTER,BILLED_DATE,BILL_FREQ,BILL_START_DATE,BILL_END_DATE,BILLED_RENTAL,BILLED_EARNED,PREVIOUS_BILLED_EARNED,REMAINING_UNEARNED_BALANCE,TOTAL_REVENUE_EARNED
08,D919518500104,HENG POH MING,R20,YRAC33,BP 04,Monthly,04/08/13,03/09/13,25,22.58,0,2.42,22.58
08,A100027860305,PANG KAM SENG,S10,YNDSAA,BP 13,Monthly,13/07/13,12/08/13,25,15.32,9.68,0,25
08,A100026920403,LIM YOKE TIN,S10,YNDSAA,BP 13,Monthly,13/07/13,12/08/13,111,68.03,42.97,0,111
08,D925038340109,SITI SHARAH BINTI OTHMAN,R20,YRAC33,BP 04,Monthly,04/07/13,03/08/13,25,22.58,2.42,0,25
08,D217414580206,ABDOL ADI BIN BABA,R20,YRACMM,BP 19,Monthly,19/08/13,18/09/13,85,35.64,0,49.36,35.64
08,A100015330204,CHUA THIAN SONG,S10,YNDSAA,BP 13,Monthly,13/08/13,12/09/13,45,27.58,0,17.42,27.58
08,A100011760104,TOYO ENTERPRISE,S10,YNDSAA,BP 13,Monthly,13/08/13,12/09/13,45,27.58,0,17.42,27.58
08,A100036551202,LEE HUA FONG,S10,YNDSAA,BP 13,Monthly,13/07/13,12/08/13,113,69.26,43.74,0,113
08,A600022365604,LIM HOOI LENG,R30,YRACPP,BP 19,Monthly,19/07/13,18/08/13,135,56.61,78.39,0,135
08,A100036551202,LEE HUA FONG,S10,YNDSAA,BP 13,Monthly,13/08/13,12/09/13,113,69.26,0,43.74,69.26
08,N351670650204,TING SIEW KIONG,R30,YRAUSI,BP 04,Monthly,04/07/13,03/08/13,115,103.87,11.13,0,115
08,N351670650204,TING SIEW KIONG,R30,YRAUSI,BP 04,Monthly,04/08/13,03/09/13,115,103.87,0,11.13,103.87
08,D919518500104,HENG POH MING,R20,YRAC33,BP 04,Monthly,04/07/13,03/08/13,25,22.58,2.42,0,25
08,Y102551020201,LIM LIAN HOCK,S10,YNDSJJ,BP 04,Monthly,04/08/13,03/09/13,20,18.06,0,1.94,18.06
08,D925038340109,SITI SHARAH BINTI OTHMAN,R20,YRAC33,BP 04,Monthly,04/08/13,03/09/13,25,22.58,0,2.42,22.58
08,D214091570207,CHUAH ANG TUAN,R60,YRACAA,BP 19,Monthly,19/08/13,18/09/13,113,47.38,0,65.62,47.38
08,Y502630980403,BAN SENG CHAN SDN BHD,S10,YNDSPP,BP 04,Monthly,04/07/13,03/08/13,45,40.65,4.35,0,45

And thus, it will create BILLED_DATE.txt as well.

So, in other words the actual header is actually belongs to the 2nd line, not the 1st.

Ermm... Smilie
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

split large file based on field criteria

I have a file containing date/time sorted data of the form ... 2009/06/10,20:59:59.950,XAG/USD,Q,1,1115, 14.3025,100,1,1 2009/06/10,20:59:59.950,XAG/USD,Q,1,1116, 14.3026,125,1,1 2009/06/10,20:59:59.950,XAG/USD,R,0,0, , 0,0,0 2009/06/10,20:59:59.950,XAG/USD,R,1,0, 14.1910,100,1,1... (6 Replies)
Discussion started by: asriva
6 Replies

2. Shell Programming and Scripting

Split large file based on last digit from a column

Hello, What's the best way to split a large into multiple files based on the last digit in the first column. input file: f 2738483300000x0y03772748378831x1y13478378358383x2y23743878383802x3y33787828282820x4y43748838383881x5y5 Desired Output: f0 3738483300000x0y03787828282820x4y4 f1... (9 Replies)
Discussion started by: alain.kazan
9 Replies

3. Shell Programming and Scripting

Split a file into multiple files based on the input pattern

I have a file with lines something like. ...... 123_start ...... ....... 123_end .... ..... 456_start ...... ..... 456_end .... ..... 789_start .... .... 789_end (6 Replies)
Discussion started by: abinash
6 Replies

4. Shell Programming and Scripting

Splitting large file into multiple files in unix based on pattern

I need to write a shell script for below scenario My input file has data in format: qwerty0101TWE 12345 01022005 01022005 datainala alanfernanded 26 qwerty0101mXZ 12349 01022005 06022008 datainalb johngalilo 28 qwerty0101TWE 12342 01022005 07022009 datainalc hitalbert 43 qwerty0101CFG 12345... (19 Replies)
Discussion started by: jimmy12
19 Replies

5. Shell Programming and Scripting

split XML file into multiple files based on pattern

Hello, I am using awk to split a file into multiple files using command: nawk '{ if ( $1 == "<process" ) { n=split($2, arr, "\""); file=arr } print > file }' processes.xml <process name="Process1.process"> ... (3 Replies)
Discussion started by: chiru_h
3 Replies

6. Shell Programming and Scripting

Problem with splitting large file based on pattern

Hi Experts, I have to split huge file based on the pattern to create smaller files. The pattern which is expected in the file is: Master..... First... second.... second... third.. third... Master... First.. second... third... Master... First... second.. second.. second..... (2 Replies)
Discussion started by: saisanthi
2 Replies

7. Shell Programming and Scripting

Help needed - Split large file into smaller files based on pattern match

Help needed urgently please. I have a large file - a few hundred thousand lines. Sample CP START ACCOUNT 1234556 name 1 CP END ACCOUNT CP START ACCOUNT 2224444 name 1 CP END ACCOUNT CP START ACCOUNT 333344444 name 1 CP END ACCOUNT I need to split this file each time "CP START... (7 Replies)
Discussion started by: frustrated1
7 Replies

8. UNIX for Dummies Questions & Answers

Split a huge 7 GB File Based on Pattern into 4 files

Hi, I have a Huge 7 GB file which has around 1 million records, i want to split this file into 4 files to contain around 250k messages each. Please help me as Split command cannot work here as it might miss tags.. Format of the file is as below <!--###### ###### START-->... (6 Replies)
Discussion started by: KishM
6 Replies

9. UNIX for Advanced & Expert Users

Split one file to many based on pattern

Hello All, I have records in a file in a pattern A,B,B,B,B,K,A,B,B,K Is there any command or simple logic I can pull out records into multiple files based on A record? I want output as File1: A,B,B,B,B,K File2: A,B,B,K (9 Replies)
Discussion started by: deal1dealer
9 Replies

10. Shell Programming and Scripting

Split files based on row delimiter count

I have a huge file (around 4-5 GB containing 20 million rows) which has text like: <EOFD>11<EOFD>22<EORD>2<EOFD>2222<EOFD>3333<EORD>3<EOFD>44<EOFD>55<EORD>66<EOFD>888<EOFD>9999<EORD> Actually above is an extracted file from a Sql Server with each field delimited by <EOFD> and each row ends... (8 Replies)
Discussion started by: amvip
8 Replies
BACKUP_ADDDUMP(8)					       AFS Command Reference						 BACKUP_ADDDUMP(8)

NAME
backup_adddump - Defines a dump level in the dump hierarchy SYNOPSIS
backup adddump -dump <dump level name>+ [-expires <expiration date>+] [-localauth] [-cell <cell name>] [-help] backup addd -d <dump level name>+ [-e <expiration date>+] [-l] [-c <cell name>] [-h] DESCRIPTION
The backup adddump command creates one or more dump levels in the dump hierarchy stored in the Backup Database, and optionally assigns an expiration date to each one. All of the dump levels in the Backup Database collectively constitute the dump hierarchy. Use the -expires argument to associate an expiration date with each dump level. When the Backup System subsequently creates a dump at the dump level, it uses the specified value to derive the dump's expiration date, which it records on the label of the tape (or backup data file). The Backup System refuses to overwrite a tape until after the latest expiration date of any dump that the tape contains, unless the backup labeltape command is used to relabel the tape. If a dump level does not have an expiration date, the Backup System treats dumps created at the level as expired as soon as it creates them. (Note that the Backup System does not automatically remove a dump's record from the Backup Database when the dump reaches its expiration date, but only if the tape that contains the dump is recycled or relabeled. To remove expired and other obsolete dump records, use the backup deletedump command.) Define either an absolute or relative expiration date: o An absolute expiration date defines the month/day/year (and, optionally, hour and minutes) at which a dump expires. If the expiration date predates the dump creation time, the Backup System immediately treats the dump as expired. o A relative date defines the number of years, months, or days (or a combination of the three) after the dump's creation that it expires. When the Backup System creates a dump at the dump level, it calculates an actual expiration date by adding the relative date to the start time of the dump operation. OPTIONS
-dump <dump level name>+ Names each dump level to add to the dump hierarchy. Precede full dump level names with a slash (for example, "/full"). Indicate an incremental dump level by preceding it with an ordered list of the dump levels directly above it in the hierarchy (its parent dump levels); use the slash as a separator. The parent dump levels must already exist. For example, the dump levels "/full" and "/full/incremental1" must exist when the incremental dump level "/full/incremental1/incremental2" is created. Dump level names can have any number of levels, but cannot exceed 256 characters in length, including the slashes. The maximum length for any single level (the text between slashes) is 28 characters, not including the preceding slash. All alphanumeric characters are allowed in dump level names. Do not use the period ("."), however, because it is the separator between the volume set name and dump level name in the dump name assigned automatically by the backup dump command. It is best not to include other metacharacters either; if using them, enclose them in double quotes (" ") when issuing the backup adddump command outside interactive mode. -expires <expiration date>+ Defines the absolute or relative expiration date to associate with each dump level named by the -dump argument. Absolute expiration dates have the following format: [at] {NEVER | <mm>/<dd>/<yyyy> [<hh>:<MM>] } where the optional word at is followed either by the string "NEVER", which indicates that dumps created at the dump level never expire, or by a date value with a required portion (<mm> for month, <dd> for day, and <yyyy> for year) and an optional portion (<hh> for hours and <MM> for minutes). Omit the hh:MM portion to use the default of midnight (00:00 hours), or provide a value in 24-hour format (for example, "20:30" is 8:30 p.m.). Valid values for the year range from 1970 to 2037; higher values are not valid because the latest possible date in the standard UNIX representation is in February 2038. The command interpreter automatically reduces later dates to the maximum value. Relative expiration dates have the following format: [in] [<years>y] [<months>m] [<days>d] where the optional word in is followed by at least one of a number of years (maximum 9999) followed by the letter "y", a number of months (maximum 12) followed by the letter "m", or a number of days (maximum 31) followed by the letter "d". If providing more than one of the three, list them in the indicated order. If the date that results from adding the relative expiration value to a dump's creation time is later than the latest possible date in the UNIX time representation, the Backup System automatically reduces it to that date. -localauth Constructs a server ticket using a key from the local /etc/openafs/server/KeyFile file. The backup command interpreter presents it to the Backup Server, Volume Server and VL Server during mutual authentication. Do not combine this flag with the -cell argument. For more details, see backup(8). -cell <cell name> Names the cell in which to run the command. Do not combine this argument with the -localauth flag. For more details, see backup(8). -help Prints the online help for this command. All other valid options are ignored. EXAMPLES
The following command defines a full dump called "/1999" with a relative expiration date of one year: % backup adddump -dump /1999 -expires in 1y The following command defines an incremental dump called "/sunday1/monday"1 with a relative expiration date of 13 days: % backup adddump -dump /sunday1/monday1 -expires in 13d The following command defines two dump incremental dump levels, "/Monthly/Week1" and "/Monthly/Week2". Their parent, the full dump level "/Monthly", must already exist. The expiration date for both levels is 12:00 a.m. on 1 January 2000. % backup adddump -dump /Monthly/Week1 /Monthly/Week2 -expires at 01/01/2000 PRIVILEGE REQUIRED
The issuer must be listed in the /etc/openafs/server/UserList file on every machine where the Backup Server is running, or must be logged onto a server machine as the local superuser "root" if the -localauth flag is included. SEE ALSO
backup(8), backup_deldump(8), backup_deletedump(8), backup_listdumps(8), backup_setexp(8) COPYRIGHT
IBM Corporation 2000. <http://www.ibm.com/> All Rights Reserved. This documentation is covered by the IBM Public License Version 1.0. It was converted from HTML to POD by software written by Chas Williams and Russ Allbery, based on work by Alf Wachsmann and Elizabeth Cassell. OpenAFS 2012-03-26 BACKUP_ADDDUMP(8)
All times are GMT -4. The time now is 06:01 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy