Sponsored Content
Top Forums Shell Programming and Scripting Split Large Files Based On Row Pattern.. Post 302876645 by aimy on Tuesday 26th of November 2013 12:44:25 AM
Old 11-26-2013
Thanks a lot Don. The filename now worked perfectly.

Regarding the header produced as txt file. Actually that was based on the real file which actually got extra info on top:
Code:
HD|20131126_104934|1
MONTH,ACCOUNT_NO,CUSTOMER_NAME,SEGMENT_CODE,COST_CENTER,BILLED_DATE,BILL_FREQ,BILL_START_DATE,BILL_END_DATE,BILLED_RENTAL,BILLED_EARNED,PREVIOUS_BILLED_EARNED,REMAINING_UNEARNED_BALANCE,TOTAL_REVENUE_EARNED
08,D919518500104,HENG POH MING,R20,YRAC33,BP 04,Monthly,04/08/13,03/09/13,25,22.58,0,2.42,22.58
08,A100027860305,PANG KAM SENG,S10,YNDSAA,BP 13,Monthly,13/07/13,12/08/13,25,15.32,9.68,0,25
08,A100026920403,LIM YOKE TIN,S10,YNDSAA,BP 13,Monthly,13/07/13,12/08/13,111,68.03,42.97,0,111
08,D925038340109,SITI SHARAH BINTI OTHMAN,R20,YRAC33,BP 04,Monthly,04/07/13,03/08/13,25,22.58,2.42,0,25
08,D217414580206,ABDOL ADI BIN BABA,R20,YRACMM,BP 19,Monthly,19/08/13,18/09/13,85,35.64,0,49.36,35.64
08,A100015330204,CHUA THIAN SONG,S10,YNDSAA,BP 13,Monthly,13/08/13,12/09/13,45,27.58,0,17.42,27.58
08,A100011760104,TOYO ENTERPRISE,S10,YNDSAA,BP 13,Monthly,13/08/13,12/09/13,45,27.58,0,17.42,27.58
08,A100036551202,LEE HUA FONG,S10,YNDSAA,BP 13,Monthly,13/07/13,12/08/13,113,69.26,43.74,0,113
08,A600022365604,LIM HOOI LENG,R30,YRACPP,BP 19,Monthly,19/07/13,18/08/13,135,56.61,78.39,0,135
08,A100036551202,LEE HUA FONG,S10,YNDSAA,BP 13,Monthly,13/08/13,12/09/13,113,69.26,0,43.74,69.26
08,N351670650204,TING SIEW KIONG,R30,YRAUSI,BP 04,Monthly,04/07/13,03/08/13,115,103.87,11.13,0,115
08,N351670650204,TING SIEW KIONG,R30,YRAUSI,BP 04,Monthly,04/08/13,03/09/13,115,103.87,0,11.13,103.87
08,D919518500104,HENG POH MING,R20,YRAC33,BP 04,Monthly,04/07/13,03/08/13,25,22.58,2.42,0,25
08,Y102551020201,LIM LIAN HOCK,S10,YNDSJJ,BP 04,Monthly,04/08/13,03/09/13,20,18.06,0,1.94,18.06
08,D925038340109,SITI SHARAH BINTI OTHMAN,R20,YRAC33,BP 04,Monthly,04/08/13,03/09/13,25,22.58,0,2.42,22.58
08,D214091570207,CHUAH ANG TUAN,R60,YRACAA,BP 19,Monthly,19/08/13,18/09/13,113,47.38,0,65.62,47.38
08,Y502630980403,BAN SENG CHAN SDN BHD,S10,YNDSPP,BP 04,Monthly,04/07/13,03/08/13,45,40.65,4.35,0,45

And thus, it will create BILLED_DATE.txt as well.

So, in other words the actual header is actually belongs to the 2nd line, not the 1st.

Ermm... Smilie
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

split large file based on field criteria

I have a file containing date/time sorted data of the form ... 2009/06/10,20:59:59.950,XAG/USD,Q,1,1115, 14.3025,100,1,1 2009/06/10,20:59:59.950,XAG/USD,Q,1,1116, 14.3026,125,1,1 2009/06/10,20:59:59.950,XAG/USD,R,0,0, , 0,0,0 2009/06/10,20:59:59.950,XAG/USD,R,1,0, 14.1910,100,1,1... (6 Replies)
Discussion started by: asriva
6 Replies

2. Shell Programming and Scripting

Split large file based on last digit from a column

Hello, What's the best way to split a large into multiple files based on the last digit in the first column. input file: f 2738483300000x0y03772748378831x1y13478378358383x2y23743878383802x3y33787828282820x4y43748838383881x5y5 Desired Output: f0 3738483300000x0y03787828282820x4y4 f1... (9 Replies)
Discussion started by: alain.kazan
9 Replies

3. Shell Programming and Scripting

Split a file into multiple files based on the input pattern

I have a file with lines something like. ...... 123_start ...... ....... 123_end .... ..... 456_start ...... ..... 456_end .... ..... 789_start .... .... 789_end (6 Replies)
Discussion started by: abinash
6 Replies

4. Shell Programming and Scripting

Splitting large file into multiple files in unix based on pattern

I need to write a shell script for below scenario My input file has data in format: qwerty0101TWE 12345 01022005 01022005 datainala alanfernanded 26 qwerty0101mXZ 12349 01022005 06022008 datainalb johngalilo 28 qwerty0101TWE 12342 01022005 07022009 datainalc hitalbert 43 qwerty0101CFG 12345... (19 Replies)
Discussion started by: jimmy12
19 Replies

5. Shell Programming and Scripting

split XML file into multiple files based on pattern

Hello, I am using awk to split a file into multiple files using command: nawk '{ if ( $1 == "<process" ) { n=split($2, arr, "\""); file=arr } print > file }' processes.xml <process name="Process1.process"> ... (3 Replies)
Discussion started by: chiru_h
3 Replies

6. Shell Programming and Scripting

Problem with splitting large file based on pattern

Hi Experts, I have to split huge file based on the pattern to create smaller files. The pattern which is expected in the file is: Master..... First... second.... second... third.. third... Master... First.. second... third... Master... First... second.. second.. second..... (2 Replies)
Discussion started by: saisanthi
2 Replies

7. Shell Programming and Scripting

Help needed - Split large file into smaller files based on pattern match

Help needed urgently please. I have a large file - a few hundred thousand lines. Sample CP START ACCOUNT 1234556 name 1 CP END ACCOUNT CP START ACCOUNT 2224444 name 1 CP END ACCOUNT CP START ACCOUNT 333344444 name 1 CP END ACCOUNT I need to split this file each time "CP START... (7 Replies)
Discussion started by: frustrated1
7 Replies

8. UNIX for Dummies Questions & Answers

Split a huge 7 GB File Based on Pattern into 4 files

Hi, I have a Huge 7 GB file which has around 1 million records, i want to split this file into 4 files to contain around 250k messages each. Please help me as Split command cannot work here as it might miss tags.. Format of the file is as below <!--###### ###### START-->... (6 Replies)
Discussion started by: KishM
6 Replies

9. UNIX for Advanced & Expert Users

Split one file to many based on pattern

Hello All, I have records in a file in a pattern A,B,B,B,B,K,A,B,B,K Is there any command or simple logic I can pull out records into multiple files based on A record? I want output as File1: A,B,B,B,B,K File2: A,B,B,K (9 Replies)
Discussion started by: deal1dealer
9 Replies

10. Shell Programming and Scripting

Split files based on row delimiter count

I have a huge file (around 4-5 GB containing 20 million rows) which has text like: <EOFD>11<EOFD>22<EORD>2<EOFD>2222<EOFD>3333<EORD>3<EOFD>44<EOFD>55<EORD>66<EOFD>888<EOFD>9999<EORD> Actually above is an extracted file from a Sql Server with each field delimited by <EOFD> and each row ends... (8 Replies)
Discussion started by: amvip
8 Replies
pal(1)							      General Commands Manual							    pal(1)

NAME
pal - calendar with events SYNOPSIS
pal [options] DESCRIPTION
pal is a command-line calendar utility. It displays a cal(1)-like calendar and events specified in text files. OPTIONS
The following options are provided by pal: -d date Show events on the given date. Valid formats for date include: dd, mmdd, yyyymmdd, 'yesterday', 'today', 'tomorrow', 'n days away', 'n days ago', first two letters of weekday, 'next ' followed by first two letters of weekday, 'last ' followed by first two letters of weekday, '1 Jan 2000', 'Jan 1 2000'. -r n Display events occurring in the next n days (counting today). By default, n is 0 and no events are displayed. For example, using -r 1 makes pal display events occurring today. If -d is used too, the range is relative to date instead of the current date. -r p-n Display a list of events occurring in the past p days (not counting today) and the next n days (counting today). For example -r 1-1 will show yesterday's and today's events. If -d is used too, the range is relative to date instead of the current date. -s regex Search for any occurrences of an event matching the regular expression (regex) occurring in the range of dates specified with -r. This command searches both the event description and the type of event (specified at the top of a calendar file). This search is case insensitive. -x n Expunge events that are n or more days old if they do not occur again in the future. pal will not expunge events from the calendars loaded from /usr/share/pal; even if you are root and you have added events to the calendars that are not recurring. When -x is used with -v, the events that are expunged will be displayed. -c n Display a calendar with n lines (default: 5). -f file Load file instead of ~/.pal/pal.conf. -u username Load /home/username/.pal/pal.conf instead of ~/.pal/pal.conf. -p palfile Override the .pal files loaded from pal.conf. This will only load palfile. For convenience, if palfile is a relative path, pal looks for the file relative from ~/.pal/, if not found, it tries relative to /usr/share/pal/, if not found it tries relative to your current directory. (This behavior might change in the future.) Using an absolute path will work as you expect it to. -m Manage events interactively. Events can be added, modified and deleted with this interface. --color Force use of colors, regardless of terminal type. --nocolor Do not use colors, regardless of terminal type. --mail Generates output readable by sendmail by adding "From:" and "Subject:" fields and forcing --nocolor. For example, you could mail yourself a reminder of the upcoming events in the next week with pal --mail -r 7 | sendmail username. Note: For the calendar to appear correctly, make sure your email client is using a fixed width font. --html Generates a HTML calendar suitable for display on a web page. It does not generate a complete HTML document so that you can add your website's header and footer around the calendar. The number of months shown on the calendar can be adjusted with -c. You will need to use Cascading Style Sheets (CSS) to change how the calendar appears; if you do not use a style sheet, the calendar will not have any borders. See /usr/share/doc/pal/examples/example.css for an example style. SECURITY NOTE: If you set up pal so it is being executed server-side, it is recommended that you do not allow web page visitors to directly change the parameters sent to pal. Allowing users to pass strange parameters (such as extremely long ones) can be a security risk. --latex Generates a LaTeX source for a calendar that can be used to generate a printer-friendly DVI (run "pal --latex > file.tex; latex file.tex"), PostScript or PDF (run "pal --latex > file.tex; pdflatex file.tex"). The number of months shown on the calendar can be adjusted with -c. -v Verbose output. --version Display version information. -h, --help Display a help message. EVENT DESCRIPTIONS
Years since year YYYY pal will replace !YYYY! (where YYYY is a year) with the current year minus YYYY. This feature is particularly useful for birthdays. For example, the event text for a birthday could be: John Doe was born on this day in 1990. He is !1990! years old. Sort by time If events have a time in the event description, pal will sort these events by time. The time in the event description must be of the format h:mm or hh:mm (where hh is 0-23). If an event has more than one time in the event description, pal will sort the event by the first time. Events that do not have times in them are shown before all the events that do have times. Events without times are sorted in the order that they are loaded in pal.conf. FILE FORMATS
Unless -f or -u is used, pal looks for (or tries to create if it doesn't exist) a configuration file named ~/.pal/pal.conf. pal.conf con- tains settings for pal and a list other files that contain events to be displayed on the calendar. The file formats for pal.conf and the event files are described below. pal.conf file filename [ (color) ] Loads an event file named filename. If filename isn't found in ~/.pal, pal will look for it in /usr/share/pal. The color param- eter is optional, it will display the events in the file with the given color. Valid colors: black, red, green, yellow, blue, magenta, cyan, white file_hide filename [ (color) ] Loads an event file name filename. These events are not indicated in the calendar that is printed, but they are displayed when the -r argument is used. If filename isn't found in ~/.pal, pal will look for it in /usr/share/pal. The color parameter is optional, it will display the events in the file with the given color. Valid colors: black, red, green, yellow, blue, magenta, cyan, white event_color color The default color used for events. Valid colors: black, red, green, yellow, blue, magenta, cyan, white week_start_monday If this keyword is defined, the calendar weeks start on Monday instead of Sunday. date_fmt string Changes how dates are displayed when the -r -d or -s arguments are used. string can be a date format string that follows the format used by strftime(3). Type man strftime for more information. string is set to %a %e %b %Y by default (example: Sun 8 Aug 2010). reverse_order Display all event listings in descending order. hide_event_type Hide the event type (shown in before a ':') when listing events. The event type is defined at the top of the file that the event is found in. cal_on_bottom Display calendar at the end of the output. no_columns Display calendar in one column instead of two. compact_list List events that are shown when using -r in a more compact form. compact_date_fmt Format for the date displayed when compact_list is used. See date_fmt for more information. default_range range If you get tired of always using -r, you can set the default value for -r here. See the information on -r above to see possible values for range. Note: Remember that this will affect what is displayed when -d and -s are used too. Event Files Event files are ASCII or UTF-8 text files (usually with a .pal ending) that define events for pal to show. Example event files can be found in /usr/share/pal. The first line in these files indicate settings that apply to all of the events in the file. The first line starts with two characters that should be used in the calendar that pal displays. A longer description of the kinds of events in the file follows the two characters. This description will be displayed when the -r argument is used. All other lines in the file are in the format date event. date defines when the event occurs and event is a string that describes the event. Below is a description of the different strings that can be used with date: Events that occur only once Use the format yyyymmdd. Daily events The format DAILY can be used for an event that happens every day. Weekly events The format MON, TUE, WED, THU, FRI, SAT and SUN can be used for an event that happens every week. Monthly events Use the format 000000dd. Annual events Use the format 0000mmdd. Annual: Events that occur on the Nth day of a month. Use the format *mmnd. Where d is the day (1 = Sunday, 7 = Saturday). Example: *1023 (10=Oct; 2="second"; 3=Tuesday ==> Second Tuesday in October, every year). Monthly: Events that occur on the Nth day of a month. Use the format *00nd. Where d is the day (1 = Sunday, 7 = Saturday). Example: *0023 (2="second"; 3=Tuesday ==> Second Tuesday of every month). Annual: Events that occur on a certain last day of a month Use the format *mmLd. Example: *10L3 (10=Oct; L=Last; 3=Tuesday ==> Last Tuesday in October). This is useful for some holidays. Monthly: Events that occur on a certain last day of a month Use the format *00Ld. Example: *00L3 (3=Tuesday ==> Last Tuesday of every month). Todo events The format TODO can be used for an event that always happens on the day that you run pal. This enables you to use pal to keep track of items in your todo list(s). Easter related events Use the format Easter for Easter Sunday. Use the format Easter+nnn for events that occur nnn days after Easter. Use the format Easter-nnn for events that occur nnn days before easter. Recurring events with start and end dates If a recurring event has a starting date and an ending date, you can use the date format DATE:START:END where DATE is a recurring date format above. START and END are dates in the yyyymmdd format that specify the starting and ending date of the recurring event. START and END dates are inclusive. For example, if an event happens every Wednesday in October 2010, you could use this format: WED:20101001:20101031 Bi-weekly, Bi-annual, etc. events If a recurring event does only occurs every Nth occurence, you can add a /N to the event of the date string for that event. A start date must be specified. For example, a bi-monthly event that occurs on the first of the month can be specified as 00000001/2:20000101. INTERNATIONALIZATION AND LOCALIZATION
The calendar files that pal uses must be ASCII encoded or UTF-8 encoded text files (ASCII is a subset of UTF-8). UTF-8 enables the calen- dar files to work on any system regardless of the default encoding scheme. When pal prints text, it converts the UTF-8 characters into the local encoding for your system. If pal does not display international characters and you are using a UTF-8 calendar file, check to make sure that your locale is set correctly. You can see your locale settings by running "locale". You can see the character set that pal is using for output by running pal with "-v". If pal does not have a translation for your language and you are interested in creating a translation, see the po/README file that is distributed with the source code for pal. FILES
~/.pal/pal.conf: Contains configuration information for pal and a list of .pal text files that contain events. /etc/pal.conf: This pal.conf file is copied to ~/.pal/pal.conf when a user runs pal for the first time. /usr/share/pal: Contains several calendar files for pal. BUGS
Bugs may be reported via http://palcal.sourceforge.net/. SEE ALSO
strftime(3), cal(1), regex(7) SIMILAR PROGRAMS
pal is similar to BSD's calendar program and GNU's more complex gcal program. AUTHORS
Scott Kuhl pal(1)
All times are GMT -4. The time now is 02:38 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy