10-07-2012
Unix:Whole i/p
Code :
{2010503005|XXGfvertex|72|0|189|0|{|{30100001|XXparameter_set|@@@@{{30001002|XXparameter|mode|0x0001|3|2|$|@{0|}}
{30001002|XXparameter|key|\{retl_prod_hier_dim_id\}|3|2|$|@{0|}}
{30001002|XXparameter|Layout|$[[ record kind 85 subkind 0 parts [vector _interp_("mfile:$\{INF_RETLDATAMART_MFS\}/m_cdp2_rdm_dt_retl_prod_hier_dim_lkp.dat", "dollar_substitution")]]]|3|9||@{0|}}
{30001002|XXparameter|read_metadata|$\{INF_RETLDATAMART_DML\}/cdp2_rdm_dt_retl_prod_hier_dim.dml|3|2|f$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio 1438\\Components\\Datasets\\Input_File .mdc|3|2|Pw$|@{0|}}
}}@0|@119757|267507|145000|288000|40000|40000|37101|m_cdp2_rdm_ dt_retl_prod_hier_dim_lkp.dat|SunTrust Bank Inc.||1|10|-1||6||32769|-1|-1|}}
{2010203004|XXGoport|73|0|191|0|{@{}@184000|302000|11000|11000|read|0.0|@@@2160|0|}}
{2010503005|XXGfvertex|74|0|194|0|{Lookup files are components containing shared data. Use lookup files with the DML lookup functions to access records according to a key.|
{30100001|XXparameter_set|@@@@{{30001002|XXparameter|protection|0666|12|2|RF$||{0|}}
{30001002|XXparameter|mode|0x0200|1|2|FH$|modes of access|{0|}}
{30001002|XXparameter|condition||3|2|F$||{0|}}
{30001002|XXparameter|conditionInputPort||3|2|F$||{0|}}
{30001002|XXparameter|conditionOutputPort||3|2|F$||{0|}}
{30001002|XXparameter|condition_interpretation|Remove completely|15|1|Fl||{2|Replace with flow|Remove completely|}}
{30001002|XXparameter|condition_interpretation.display_name|condition-interpretation|3|9|P|@{0|}}
{30001002|XXparameter|key||19|2|RF$|Key specifier For Lookup File|{0|}}
{30001002|XXparameter|key.condition|mode lookup|3|15|P?|@{0|}}
{30001002|XXparameter|Layout|@28|2|RF$||{0|}}
{30001002|XXparameter|read_metadata||7|2|RF$|Record Format|{0|}}
{30001002|XXparameter|mpcmodtime|1196372206|1|1|Hl|The last modification time of this component's template|{0|}}
{30001002|XXparameter|eme_dataset_location||3|2|F$|Place in the EME to create a dataset corresponding to this file.|{0|}}
}}@0|@0|0|0|0|0|0|0|@@@1|10|-1|@6|@1|-1|-1|}}
{2010203004|XXGoport|75|0|196|0|{@{30100001|XXparameter_set|@@@@{{30001002|XXparameter|metadata||7|8|RF=|Record Format|{0|}}
{2010503005|XXGfvertex|46|0|99|0|{|{30100001|XXparameter_set|@@@@
{{30001002|XXparameter|Layout|$[[record kind 85 subkind 0 parts [vector _interp_("mfile:$\{INF_ENTRPRSDWUNFYRETLCRED_MFS\}/m_cdp2_uedw_v_cls_uld.dat", "dollar_substitution")]]]|3|9||@{0|}}
{30001002|XXparameter|read_metadata|$\{INF_ENTRPRSDWUNFYRETLCRED_DML\}/cdp2_uedw_v_cls.dml|3|2|f$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio 1438\\Components\\Datasets\\Input_File .mdc|3|2|Pw$|@{0|}}
{30001002|XXparameter|eme_dataset_location|$\{INF_ENTRPRSDWUNFYRETLCRED_MFS\}/m_cdp2_uedw_v_cls_uld.dat|3|2|$|@{0|}}
}}@0|@127358|686578|152000|707000|40000|40000|37115|m_cdp2_uedw_v_cls_uld.dat|SunTrust Bank Inc.||1|10|-1||6||32769|-1|-1|}}
{2010203004|XXGoport|47|0|101|0|{@{}@191000|721000|11000|11000|read|0.0|@@@2160|0|}}
{2010503005|XXGfvertex|48|0|104|0|{Represents one file, many files, or a multifile as an input to your graph.|{30100001|XXparameter_set|@@@@{{30001002|XXparameter|protection|0666|12|2|RF$||{0|}}
{30001002|XXparameter|mode|0x0001|1|2|FH$|modes of access|{0|}}
{30001002|XXparameter|Layout|@28|2|RF$||{0|}}
{30001002|XXparameter|read_metadata||7|1|RFl||{0|}}
{30001002|XXparameter|mpcmodtime|1196372206|1|1|Hl|The last modification time of this component's template|{0|}}
{30001002|XXparameter|eme_dataset_location|@3|9|F|Place in the EME to create a dataset corresponding to this file.|{0|}}
}}@0|@0|0|0|0|0|0|0|@@@1|10|-1|@6|@1|-1|-1|}}
{2010203004|XXGoport|49|0|106|0|{@{30100001|XXparameter_set|@@@@{{30001002|XXparameter|metadata||7|8|RF=||{0|}}
{2010600005|XXGgraph|136|0|409|0|{Repartitions data records by key values and then sorts the records within each partition.|
{30100001|XXparameter_set|@@@@{{30001002|XXparameter|Key|\{retl_sub_lob_cd\}|3|2|$|@{0|}}
{30001002|XXparameter|InputLayout|$[[record kind 85 subkind 0 parts [vector _interp_("$\{INF_RETLDATAMART_MFS\}", "dollar_substitution")]]]|3|9||@{0|}}
{30001002|XXparameter|Max_core|$AI_GRAPH_MAX_CORE|3|2|$|@{0|}}
{30001002|XXparameter|OutputLayout|$[[record kind 85 subkind 0 parts [vector _interp_("$\{INF_RETLDATAMART_MFS\}", "dollar_substitution")]]]|3|9||@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio 1438\\Components\\Sort\\Partition_by_Key_and_Sort .mp|3|2|Pw$|@{0|}}
}}@0|@298442|218821|318000|239000|481000|303000|37023|PKS1 - \{retl_sub_lob_cd\}|Ab Initio Software|Created 04/22/98 12:54:47|1|10|-1||6||32769|{0|}0|0|{0|}{0|}{0|}{0|}.4407484531402588|481000|303000|0|}}
{2010210004|XXGflow|137|0|411|0|{@{}@384|.5|.5|{8|217000|171000|237000|171000|296000|171000|316000|171000|}0|20|}}
{2010501005|XXGpvertex|138|0|413|0|{Groups data according to a collator.
A Hash Partition component is generally followed by a Local Sort component.|{30100001|XXparameter_set|@@@@{{30001002|XXparameter|in_metadata||3|8|s=|@{0|}}
{30001002|XXparameter|out_metadata||3|8|s=|@{0|}}
}}@0|@61000|118000|81000|138000|126000|68000|0|Partition by Key|Ab Initio Software|Built-in|1|10|-1||6||32769|1|{1|0|}}}
{2010203004|XXGoport|139|0|415|0|{@{}@206000|166000|11000|11000|out|0.0|@@@2322|0|}}
{2010202004|XXGiport|140|0|418|0|{@{}@71000|166000|11000|11000|in|0.0|@@@1808|0|}}
{2010501005|XXGpvertex|141|0|422|0|{Orders your data according to a collating expression.|{30100001|XXparameter_set|@@@@{{30001002|XXparameter|in_metadata||3|8|s=|@{0|}}
{30001002|XXparameter|out_metadata||3|8|s=|@{0|}}
}}@0|@307000|118504|327000|139000|104000|65000|0|Sort|Ab Initio Software|Built-in|1|10|-1||6||32769|1|{1|0|}}}
{2010203004|XXGoport|142|0|424|0|{@{}@430000|166000|11000|11000|out|0.0|@@@2448|0|}}
{2010202004|XXGiport|143|0|426|0|{@{}@317000|166000|11000|11000|in|0.0|@@@1808|0|}}
{2010203004|XXGoport|144|0|431|0|{@{}@529000|301000|11000|11000|out0|.5|@@@14736|0|}}
{2010202004|XXGiport|145|0|435|0|{@{}@308000|301000|11000|11000|in0|.5|@@@14096|0|}}
{2010600005|XXGgraph|146|0|439|0|{Repartitions data records by key values and then sorts the records within each partition.|{30100001|XXparameter_set|@@@@{{30001002|XXparameter|Key||19|2|RF$|Field to partition on|{0|}}
{30001002|XXparameter|InputLayout|@9|2|RF$||{0|}}
{30001002|XXparameter|Max_core|100663296|1|2|F$|maximum memory usage (before spilling to disk) in bytes|{0|}}
{30001002|XXparameter|OutputLayout|@9|2|RF$||{0|}}
{30001002|XXparameter|_parameters_of_Sort|@34|9|FHK|@{0|}}
{30001002|XXparameter|_parameters_of_Partition_by_Key|@34|9|FHK|@{0|}}
{30001002|XXparameter|conditionInputPort|in0|3|1|Fl||{0|}}
{30001002|XXparameter|conditionOutputPort|out0|3|1|Fl||{0|}}
{30001002|XXparameter|condition_interpretation|Replace with flow|15|1|Fl||{2|Replace with flow|Remove completely|}}
{30001002|XXparameter|condition_interpretation.display_name|condition-interpretation|3|9|P|@{0|}}
{30001002|XXparameter|mpcmodtime|1196372210|1|1|Hl|The last modification time of this component's template|{0|}}
{30001002|XXparameter|HelpID|comp_partition_by_key_and_sort|3|2|R$||{0|}}
{30001002|XXparameter|_ab_rexec_username|$USERNAME|3|1|RHl||{0|}}
}}@0|@0|0|0|0|0|0|0|@@@1|10|-1|@6|@1|{0|}0|0|{0|}{0|}{0|}{0|}1.0|0|0|7|}}
{2010210004|XXGflow|147|0|441|0|{@{}@384|.5|.5|{8|217000|171000|237000|171000|296000|171000|316000|171000|}0|20|}}
{2010501005|XXGpvertex|148|0|443|0|{Groups data according to a collator.
__________________________________________________________________________________________________________________________________________________________
A Hash Partition component is generally followed by a Local Sort component.|{30100001|XXparameter_set|@@@@{{30001002|XXparameter|Layout||3|8|=|@{0|}}
{30001002|XXparameter|key||3|8|=|@{0|}}
{30001002|XXparameter|!prototype_path|C:\\gui\\src\\mpc\\Partition\\Hash.mpc|3|2|Pw$|@{0|}}
{30001002|XXparameter|_propagate_through|metadata type: out = in
metadata type: in = out|3|9||@{0|}}
}}@0|@61000|118000|81000|138000|126000|68000|0|Partition by Key|Ab Initio Software|Built-in|1|10|-1||6||32769|1|{1|0|}}}
{2010203004|XXGoport|149|0|445|0|{@{}@206000|166000|11000|11000|out|0.0|@@@2322|0|}}
{2010202004|XXGiport|150|0|448|0|{@{}@71000|166000|11000|11000|in|0.0|@@@1808|0|}}
{2010501005|XXGpvertex|151|0|450|0|{|{30100001|XXparameter_set|@@@@{{30001002|XXparameter|mpname|hash-partition|3|2|H$|The name used on the mp command line for this component|{0|}}
{30001002|XXparameter|image__|unitool|3|2|H$|The image used if this component was a custom component|{0|}}
{30001002|XXparameter|Layout|@9|2|RF$||{0|}}
{30001002|XXparameter|key||19|2|RFO$|Field to partition on|{0|}}
{30001002|XXparameter|in_metadata||7|1|RFsl||{0|}}
{30001002|XXparameter|out_metadata||7|1|RFsl||{0|}}
{30001002|XXparameter|doc_transform||8|2|FHs$|Document your transformation for dependency analysis|{0|}}
{30001002|XXparameter|doc_operation1|out::document(in)|3|1|RHl|The custom transformation|{0|}}
{30001002|XXparameter|port_analysis|out=in|3|2|H$||{0|}}
{30001002|XXparameter|continuous_analysis||3|2|H$||{0|}}
{30001002|XXparameter|_propagate_through||3|1|FHKl|@{0|}}
}}@0|@0|0|0|0|0|0|0|@@@1|10|-1|@6|@1|1|{1|0|}}}
{2010203004|XXGoport|152|0|452|0|{@{30100001|XXparameter_set|@@@@{{30001002|XXparameter|metadata||7|8|RF=||{0|}}
}}@0|0|0|0|out|0.0|@@@2322|0|}}
{2010202004|XXGiport|153|0|455|0|{@{30100001|XXparameter_set|@@@@{{30001002|XXparameter|metadata||7|8|RF=||{0|}}
}}@0|0|0|0|in|0.0|@@@1808|0|}}
--------------------------------------------------------------------------------------------------------------------------------------------------------------
{2010501005|XXGpvertex|154|0|460|0|{Orders your data according to a collating expression.|
{30100001|XXparameter_set|@@@@{{30001002|XXparameter|Layout||3|8|=|@{0|}}
{30001002|XXparameter|key||3|8|=|@{0|}}
{30001002|XXparameter|max_core||3|8|=|@{0|}}
{30001002|XXparameter|!prototype_path|C:\\gui\\src\\mpc\\Sort-Merge\\Sort.mpc|3|2|Pw$|@{0|}}
{30001002|XXparameter|_propagate_through|metadata type: out = in
metadata type: in = out|3|9||@{0|}}
}}@0|@307000|118504|327000|139000|104000|65000|0|Sort|Ab Initio Software|Built-in|1|10|-1||6||32769|1|{1|0|}}}
{2010203004|XXGoport|155|0|462|0|{@{}@430000|166000|11000|11000|out|0.0|@@@2448|0|}}
{2010202004|XXGiport|156|0|464|0|{@{}@317000|166000|11000|11000|in|0.0|@@@1808|0|}}
{2010501005|XXGpvertex|157|0|467|0|{|{30100001|XXparameter_set|@@@@{{30001002|XXparameter|mpname|local-sort|3|2|H$|The name used on the mp command line for this component|{0|}}
{30001002|XXparameter|image__|unitool|3|2|H$|The image used if this component was a custom component|{0|}}
{30001002|XXparameter|Layout|@9|2|RF$||{0|}}
{30001002|XXparameter|key||19|2|RFO$|Field to sort on|{0|}}
{30001002|XXparameter|max_core||1|2|FK$|maximum memory usage (before spilling to disk) in bytes|{0|}}
{30001002|XXparameter|max_core.display_name|max-core|3|9|P|@{0|}}
{30001002|XXparameter|max_core.keyword|max-core|3|9|P|@{0|}}
{30001002|XXparameter|in_metadata||7|1|RFsl||{0|}}
{30001002|XXparameter|out_metadata||7|1|RFsl||{0|}}
{30001002|XXparameter|doc_transform||8|2|FHs$|Document your transformation for dependency analysis|{0|}}
{30001002|XXparameter|doc_operation1|out::document(in)|3|1|RHl|The custom transformation|{0|}}
{30001002|XXparameter|port_analysis|out=in|3|2|H$||{0|}}
{30001002|XXparameter|continuous_analysis||3|2|H$||{0|}}
{30001002|XXparameter|_propagate_through||3|1|FHKl|@{0|}}
}}@0|@0|0|0|0|0|0|0|@@@1|10|-1|@6|@1|1|{1|0|}}}
{2010203004|XXGoport|158|0|469|0|{@{30100001|XXparameter_set|@@@@{{30001002|XXparameter|metadata||7|8|RF=||{0|}}
}}@0|0|0|0|out|0.0|@@@2448|0|}}
{2010202004|XXGiport|159|0|472|0|{@{30100001|XXparameter_set|@@@@{{30001002|XXparameter|metadata||7|8|RF=||{0|}}
}}@0|0|0|0|in|0.0|@@@1808|0|}}
{2010203004|XXGoport|160|0|478|0|{@{}@0|0|0|0|out0|.5|@@@14736|0|}}
{2010202004|XXGiport|161|0|481|0|{@{}@0|0|0|0|in0|.5|@@@14096|0|}}
{2010503005|XXGfvertex|132|0|393|0|{|{30100001|XXparameter_set|@@@@
{{30001002|XXparameter|Layout|$[[record kind 85 subkind 0 parts [vector _interp_("mfile:$\{INF_RETLDATAMART_MFS\}/m_cdp2_rdm_dt_retl_prod_hier_xref.dat", "dollar_substitution")]]]|3|9||@{0|}}
{30001002|XXparameter|write_metadata|$\{INF_RETLDATAMART_DML\}/cdp2_rdm_dt_retl_prod_hier_xref.dml|3|2|f$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio 1438\\Components\\Datasets\\Output_File .mdc|3|2|Pw$|@{0|}}
}}@0|@1194511|289945|1220000|310000|40000|40000|37109|m_cdp2_rdm_ dt_retl_prod_hier_xref.dat|SunTrust Bank Inc.||1|10|0||6||32769|-1|-1|}}
{2010202004|XXGiport|133|0|395|0|{@{}@1210000|324000|11000|11000|write|0.0|@@@1776|0|}}
{2010503005|XXGfvertex|134|0|398|0|{Represents one file, many files, or a multifile as an output from your graph.|{30100001|XXparameter_set|@@@@{{30001002|XXparameter|protection|0666|12|2|RF$||{0|}}
{30001002|XXparameter|mode|0x0062|1|2|FH$|modes of access|{0|}}
{30001002|XXparameter|condition||3|2|F$||{0|}}
{30001002|XXparameter|conditionInputPort||3|2|F$||{0|}}
{30001002|XXparameter|conditionOutputPort||3|2|F$||{0|}}
{30001002|XXparameter|condition_interpretation|Remove completely|15|1|Fl||{2|Replace with flow|Remove completely|}}
{30001002|XXparameter|condition_interpretation.display_name|condition-interpretation|3|9|P|@{0|}}
{30001002|XXparameter|key||19|2|RF$|Key specifier For Lookup File|{0|}}
{30001002|XXparameter|key.condition|mode lookup|3|15|P?|@{0|}}
{30001002|XXparameter|Layout|@28|2|RF$||{0|}}
{30001002|XXparameter|write_metadata||7|1|RFl||{0|}}
{30001002|XXparameter|mpcmodtime|1196372208|1|1|Hl|The last modification time of this component's template|{0|}}
{30001002|XXparameter|eme_dataset_location||3|2|F$|Place in the EME to create a dataset corresponding to this file.|{0|}}
}}@0|@0|0|0|0|0|0|0|@@@1|10|-1|@6|@1|-1|-1|}}
{2010202004|XXGiport|135|0|400|0|{@{30100001|XXparameter_set|@@@@{{30001002|XXparameter|metadata||7|8|RF=||{0|}}
I need four files from the above data with Names
Input_File1,Input_File2 ,
Partition_by_Key_and_Sort3 and Output_File4 .
This is my requirement.Please solve it, Thanks for ur patience with me. Thanks a lot in advance
Last edited by Scrutinizer; 10-10-2012 at 03:23 AM ..
Reason: Names of files being specific; code tags (mod)
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
I need to split a file based on certain context inside the file. Is there a unix command that can do this? I have looked into split and csplit but it does not seem like those would work because I need to split this file based on certain text. The file has multiple records and I need to split this... (1 Reply)
Discussion started by: matrix1067
1 Replies
2. Shell Programming and Scripting
I need to write a shell script for below scenario
My input file has data in format:
qwerty0101TWE 12345 01022005 01022005 datainala alanfernanded 26
qwerty0101mXZ 12349 01022005 06022008 datainalb johngalilo 28
qwerty0101TWE 12342 01022005 07022009 datainalc hitalbert 43
qwerty0101CFG 12345... (19 Replies)
Discussion started by: jimmy12
19 Replies
3. Shell Programming and Scripting
I have a file as:
I/P File:
Ground Car 2009
Lib 2008
Lib 2003
Ground Car 2009
Ground Car 2003
Car 2005
Car 2003
Car 2005
Sita 2900 2006
Car 2007
I have to split the file into two: - one for names and second for years.
O/p1 (Names):
Ground Car (3 Replies)
Discussion started by: karumudi7
3 Replies
4. Shell Programming and Scripting
Hi all,
I am new to scripting and I have a requirement
we have source file as
HEADER 01.10.2010 14:32:37 NAYA
TA0022
TA0000
20000001;20060612;99991231;K4;02;3
20000008;20080624;99991231;K4;02;3
20000026;19840724;99991231;KK;01;3
20000027;19840724;99991231;KK;01;3... (6 Replies)
Discussion started by: srk409
6 Replies
5. Shell Programming and Scripting
Hi Friends,
Newbie to shell scripting. Currently i have used the below to sort data based on filenames and datestamp
$ printf '%s\n' *.dat* | sort -t. -k3,4
filename_1.dat.20120430.Z
filename_2.dat.20120430.Z
filename_3.dat.20120430.Z
filename_1.dat.20120501.Z
filename_2.dat.20120501.Z... (1 Reply)
Discussion started by: robertbrown624
1 Replies
6. Shell Programming and Scripting
I will simplify the explaination a bit, I need to parse through a 87m file -
I have a single text file in the form of :
<NAME>house........
SOMETEXT
SOMETEXT
SOMETEXT
.
.
.
.
</script>
MORETEXT
MORETEXT
.
.
. (6 Replies)
Discussion started by: sumguy
6 Replies
7. UNIX for Dummies Questions & Answers
Dear All,
I have two files but want to extract data from one based on another... can you please help me
file 1
David
Tom
Ellen
and file 2
David|0010|testnamez|resultsz
David|0004|testnamex|resultsx
Tom|0010|testnamez|resultsz
Tom|0004|testnamex|resultsx
Ellen|0010|testnamez|resultsz... (12 Replies)
Discussion started by: A-V
12 Replies
8. Shell Programming and Scripting
Hi Friends ,
Please guide me with the code to extract multiple files from one file .
The File Looks like ( Suppose a file has 2 tables list ,column length may vary )
H..- > File Header....
H....- >Table 1 Header....
D....- > Table 1 Data....
T....- >Table 1 Trailer....
H..-> Table 2... (1 Reply)
Discussion started by: AspiringD
1 Replies
9. Shell Programming and Scripting
Hi All,
I have the sales_data.csv file in the directory as below.
SDDCCR; SOM ; MD6546474777 ;05-JAN-16
ABC ; KIRAN ; CB789 ;04-JAN-16
ABC ; RAMANA; KS566767477747 ;06-JAN-16
ABC ; KAMESH; A33535335 ;04-JAN-16
SDDCCR; DINESH; GD6674474747 ;08-JAN-16... (4 Replies)
Discussion started by: ROCK_PLSQL
4 Replies
10. Shell Programming and Scripting
We have the data looks like below in a log file.
I want to generat files based on the string between two hash(#) symbol like below
Source:
#ext1#test1.tale2 drop
#ext1#test11.tale21 drop
#ext1#test123.tale21 drop
#ext2#test1.tale21 drop
#ext2#test12.tale21 drop
#ext3#test11.tale21 drop... (5 Replies)
Discussion started by: Sanjeev G
5 Replies