Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Help me pls : splitting single file in unix into different files based on data
Top Forums Shell Programming and Scripting Help me pls : splitting single file in unix into different files based on data Post 302715201 by Ravindra Swan on Sunday 14th of October 2012 04:37:12 AM
Old 10-14-2012
Next step help:
One of the inputfile contains:
Code:










Code:



{2010503005|XXGfvertex|1259|0|3869|0|{|{30100001|XXparameter_set|@@@@{{30001002|XXparameter|Layout|$[[record kind 85 subkind 0 parts [vector _interp_("mfile:$\{INF_ENTRPRSDWUNFYRETLCRED_MFS\}/m_cdp2_uedw_t_arnge_to_arnge_uld.dat", "dollar_substitution")]]]|3|9||@{0|}}
{30001002|XXparameter|read_metadata|$\{INF_ENTRPRSDWUNFYRETLCRED_DML\}/cdp2_uedw_t_arnge_to_arnge.dml|3|2|f$|@{0|}}
{30001002|XXparameter|!prototype_path|C:\\Program Files\\Ab Initio\\Ab Initio GDE 1_14_5\\Components\\Datasets\\Input_File.mdc|3|2|Pw$|@{0|}}
{30001002|XXparameter|eme_dataset_location|$\{_Projects_SunTrust_inf_inf_EntrprsDwUnfyRetlCred\}/data/mfs/main/m_cdp2_uedw_t_arnge_to_arnge_uld.dat|3|9||@{0|}}
}}@1|@151424|2797588|173000|2818000|56000|50000|39502|m_cdp2_uedw_t_arnge_to_arnge_uld.dat|M K Muralidhar||1|100|-1||6||33537|-1|-1|}}
{2010203004|XXGoport|1260|0|3871|0|{@{}@228000||11000|11000|read|0.0|@@@2160|0|}}
{2010503005|XXGfvertex|1261|0|3874|0|{Represents one file, many files, or a multifile as an input to your graph.|{30100001|XXparameter_set|@@@@{{30001002|XXparameter|protection|0666|12|2|RF$||{0|}}
{30001002|XXparameter|mode|0x0001|1|2|FH$|modes of access|{0|}}
{30001002|XXparameter|Layout|@28|2|RF$||{0|}}
{30001002|XXparameter|read_metadata||7|1|RFl||{0|}}
{30001002|XXparameter|mpcmodtime|1138303912|1|1|Hl|The last modification time of this component's template|{0|}}
{30001002|XXparameter|eme_dataset_location|@3|9|F|Place in the EME to create a dataset corresponding to this file.|{0|}}
}}@0|@0|0|0|0|0|0|0|@@@1|10|-1|@6|@1|-1|-1|}}
{2010203004|XXGoport|1262|0|3876|0|{@{30100001|XXparameter_set|@@@@{{30001002|XXparameter|metadata||7|8|RF=||{0|}}
}}@0|0|0|0|read|0.0|@@@2160|0|}}
{2010501005|XXGpvertex|1263|0|3885|0|{|{30100001|XXparameter_set|@@@@{{30001002|XXparameter|transform0|$AI_XFR/cdp2_rdm_dt_core_cnsum_arnge_dim_xfm_rfmt_orig_fico_scor_val.xfr|3|2|f$|@{0|}}
{30001002|XXparameter|out0_metadata|$AI_DML/cdp2_rdm_dt_core_cnsum_arnge_dim_xfm_rfmt_orig_fico_scor_val.dml|3|2|f$|@{0|}}
{30001002|XXparameter|error0_metadata|string('\\n')|3|1|l|@{0|}}
{30001002|XXparameter|log_metadata|record string("\|") node, timestamp, component, subcomponent, event_type; string("\|\\n") event_text; end|3|1|l|@{0|}}









 

XXGFlow:

 




Code:













Code:



{2010210004|XXGflow|35|0|69|0|{@{}@384|.5|.994955837726593|{12|207000|334000|227000|334000|2149000|334000|2149000|1069000|2159000|1069000|2180000|1069000|}39469|17|}}
{2010210004|XXGflow|36|0|71|0|{@{}@384|.5|.9550561904907227|{12|224000|1964000|244000|1964000|1939000|1964000|1939000|1429000|2019000|1429000|2180000|1429000|}39493|17|}}
{2010210004|XXGflow|37|0|73|0|{@{}@384|.5|.5|{12|225000|1524000|245000|1524000|1117000|1524000|1117000|1339000|1989000|1339000|2180000|1339000|}39487|17|}}
{2010210004|XXGflow|38|0|75|0|{@{}@384|.5|.30847951769828796|{12|229000|1409000|249000|1409000|782000|1409000|782000|1309000|1979000|1309000|2180000|1309000|}39485|17|}}
{2010210004|XXGflow|39|0|77|0|{@{}@384|.5|.5|{12|758000|784000|778000|784000|803000|784000|803000|809000|828000|809000|849000|809000|}39520|17|}}
{2010210004|XXGflow|40|0|79|0|{@{}@384|.5|.9049773812294006|{12|224000|1654000|244000|1654000|1832000|1654000|1832000|1369000|1999000|1369000|2180000|1369000|}39489|17|}}
{2010210004|XXGflow|41|0|81|0|{@{}@384|.5|1.0|{12|498000|594000|578000|594000|1019000|594000|1019000|724000|1019000|724000|1040000|724000|}23096|17|}}
{2010210004|XXGflow|42|0|83|0|{@{}@384|.5|1.0|{12|498000|644000|528000|644000|706000|644000|706000|2384000|706000|2384000|727000|2384000|}39539|529|}}
{2010210004|XXGflow|43|0|85|0|{@{}@384|.5|.9298245906829834|{12|220000|1789000|240000|1789000|1884000|1789000|1884000|1399000|2009000|1399000|2180000|1399000|}39491|17|}}
{2010210004|XXGflow|44|0|87|0|{@{}@384|.5|.5|{8|423000|934000|443000|934000|494000|934000|515000|934000|}23227|17|}}
{2010210004|XXGflow|45|0|89|0|{@{}@384|.5|.9747126698493958|{12|222000|2254000|242000|2254000|1993000|2254000|1993000|1489000|2039000|1489000|2180000|1489000|}39497|17|}}
{2010210004|XXGflow|46|0|91|0|{@{}@384|.5|.5|{12|2328000|1459000|2348000|1459000|2391000|1459000|2391000|1394000|2434000|1394000|2455000|1394000|}39467|17|}}
{2010210004|XXGflow|47|0|93|0|{@{}@384|.5|.5|{12|498000|584000|518000|584000|861000|584000|861000|574000|1205000|574000|1226000|574000|}39516|529|}}
{2010210004|XXGflow|48|0|95|0|{@{}@384|.5|.5|{12|1209000|724000|1229000|724000|1229000|1256000|904000|1256000|904000|1789000|904000|1810000|}23612|17|}}
{2010210004|XXGflow|49|0|97|0|{@{}@384|.5|.8610354065895081|{12|616000|1024000|636000|1024000|1904000|1024000|1904000|1219000|2109000|1219000|2180000|1219000|}39445|17|}}
{2010210004|XXGflow|50|0|99|0|{@{}@384|.5|.5|{12|226000|2689000|246000|2689000|465000|2689000|465000|2654000|685000|2654000|706000|2654000|}39545|17|}}
{2010210004|XXGflow|51|0|101|0|{@{}@384|.5|.47987616062164307|{12|498000|604000|568000|604000|939000|604000|939000|1474000|1342000|1474000|1363000|1474000|}39525|529|}}
{2010210004|XXGflow|52|0|103|0|{@{}@384|.5|.5|{12|1072000|2384000|1092000|2384000|1570000|2384000|1570000|1519000|2049000|1519000|2180000|1519000|}39540|17|}}
{2010210004|XXGflow|53|0|105|0|{@{}@384|.5|.5|{12|664000|1149000|684000|1149000|1391000|1149000|1391000|1249000|2099000|1249000|2180000|1249000|}39523|529|}}
{2010210004|XXGflow|54|0|107|0|{@{}@384|.5|.49438202381134033|{12|498000|654000|518000|654000|615000|654000|615000|2524000|715000|2524000|736000|2524000|}39542|17|}}
{2010210004|XXGflow|55|0|109|0|{@{}@384|.5|.5|{12|1133000|2524000|1153000|2524000|1631000|2524000|1631000|1699000|2109000|1699000|2180000|1699000|}39543|17|}}
{2010210004|XXGflow|56|0|111|0|{@{}@384|.5|.5|{12|2603000|1394000|2623000|1394000|2705000|1394000|2705000|1424000|2788000|1424000|2809000|1424000|}39513|17|}}
{2010210004|XXGflow|57|0|113|0|{@{}@384|.5|.5|{12|216000|1169000|236000|1169000|270000|1169000|270000|1149000|304000|1149000|325000|1149000|}39522|529|}}
{2010210004|XXGflow|58|0|115|0|{@{}@384|.5|.5|{12|863000|2664000|883000|2664000|1481000|2664000|1481000|1609000|2079000|1609000|2180000|1609000|}39546|17|}}
{2010210004|XXGflow|59|0|117|0|{@{}@384|.5|.5|{12|239000|2844000|259000|2844000|483000|2844000|483000|2824000|708000|2824000|729000|2824000|}39548|529|}}
{2010210004|XXGflow|60|0|119|0|{@{}@384|.5|.5|{12|1040000|2814000|1060000|2814000|1589000|2814000|1589000|1729000|2119000|1729000|2180000|1729000|}39549|17|}}
{2010210004|XXGflow|61|0|121|0|{@{}@384|.5|.5|{12|853000|3314000|873000|3314000|1084000|3314000|1084000|3269000|1296000|3269000|1327000|3269000|}39555|529|}}
{2010210004|XXGflow|62|0|123|0|{@{}@384|.5|.5|{12|224000|3139000|244000|3139000|790000|3139000|790000|3059000|1336000|3059000|1367000|3059000|}39551|17|}}
{2010210004|XXGflow|63|0|125|0|{@{}@384|.5|.5|{12|1566000|1474000|1626000|1474000|1812000|1474000|1812000|1549000|1999000|1549000|2180000|1549000|}39530|529|}}
{2010210004|XXGflow|64|0|127|0|{@{}@384|.5|.5|{12|358000|1029000|378000|1029000|416000|1029000|416000|1024000|455000|1024000|476000|1024000|}39268|17|}}
{2010210004|XXGflow|65|0|129|0|{@{}@384|.5|.5|{12|768000|2109000|788000|2109000|837000|2109000|837000|2164000|887000|2164000|918000|2164000|}39537|529|}}
{2010210004|XXGflow|66|0|131|0|{@{}@384|.5|.5|{12|358000|1039000|378000|1039000|857000|1039000|857000|3004000|1336000|3004000|1367000|3004000|}39552|17|}}
{2010210004|XXGflow|67|0|133|0|{@{}@384|.5|.5|{12|904000|1929000|904000|1949000|904000|2026000|897000|2026000|897000|2104000|918000|2104000|}23613|17|}}
{2010210004|XXGflow|68|0|135|0|{@{}@384|.5|.5|{12|205000|614000|225000|614000|315000|614000|315000|619000|405000|619000|426000|619000|}39473|17|}}
{2010210004|XXGflow|69|0|137|0|{@{}@384|.5|.5|{12|225000|2129000|245000|2129000|317000|2129000|317000|2109000|390000|2109000|411000|2109000|}39536|529|}}
{2010210004|XXGflow|70|0|139|0|{@{}@384|.5|.5|{12|1113000|2134000|1113000|2134000|1126000|2134000|1126000|2119000|1139000|2119000|1146000|2119000|}23609|17|}}
{2010210004|XXGflow|71|0|141|0|{@{}@384|.5|.5|{12|1323000|2119000|1343000|2119000|1343000|1812000|1342000|1812000|1342000|1504000|1363000|1504000|}39529|529|}}
{2010210004|XXGflow|72|0|143|0|{@{}@384|.5|.8923395276069641|{12|663000|934000|683000|934000|1964000|934000|1964000|1189000|2119000|1189000|2180000|1189000|}39444|17|}}
{2010210004|XXGflow|73|0|145|0|{@{}@384|.5|.8894736766815186|{12|1603000|574000|1623000|574000|2081000|574000|2081000|1129000|2139000|1129000|2180000|1129000|}39517|529|}}
{2010210004|XXGflow|74|0|147|0|{@{}@384|.5|.8920000195503235|{12|921000|809000|941000|809000|2000000|809000|2000000|1159000|2129000|1159000|2180000|1159000|}39443|17|}}
{2010210004|XXGflow|75|0|149|0|{@{}@384|.5|.8880000114440918|{12|1566000|1504000|1626000|1504000|1983000|1504000|1983000|1459000|2029000|1459000|2180000|1459000|}39534|529|}}
{2010210004|XXGflow|76|0|151|0|{@{}@384|.5|.42904290556907654|{12|498000|614000|558000|614000|890000|614000|890000|1534000|1332000|1534000|1363000|1534000|}39526|529|}}
{2010210004|XXGflow|77|0|153|0|{@{}@384|.5|.5|{12|1566000|1534000|1616000|1534000|1802000|1534000|1802000|1579000|1989000|1579000|2180000|1579000|}39531|529|}}
{2010210004|XXGflow|78|0|155|0|{@{}@384|.5|.47999998927116394|{12|1336000|3684000|1356000|3684000|1387000|3684000|1387000|3619000|1421000|3619000|1442000|3619000|}39560|17|}}
{2010210004|XXGflow|79|0|157|0|{@{}@384|.5|.5|{12|345000|809000|365000|809000|893000|809000|893000|3589000|1421000|3589000|1442000|3589000|}39559|17|}}
{2010210004|XXGflow|80|0|159|0|{@{}@384|.5|.5|{12|1089000|3329000|1109000|3329000|1207000|3329000|1207000|3304000|1306000|3304000|1327000|3304000|}39556|529|}}
{2010210004|XXGflow|81|0|161|0|{@{}@384|.5|.5|{8|203000|804000|223000|804000|252000|804000|273000|804000|}39475|17|}}
{2010210004|XXGflow|82|0|163|0|{@{}@384|.5|.5|{12|345000|799000|365000|799000|387000|799000|387000|784000|410000|784000|431000|784000|}39519|17|}}
{2010210004|XXGflow|83|0|165|0|{@{}@384|.5|.5|{12|1562000|3014000|1582000|3014000|1855000|3014000|1855000|1759000|2129000|1759000|2180000|1759000|}39553|17|}}
{2010210004|XXGflow|84|0|167|0|{@{}@384|.5|.5|{12|1582000|3284000|1602000|3284000|1870000|3284000|1870000|1789000|2139000|1789000|2180000|1789000|}39557|17|}}
{2010210004|XXGflow|85|0|169|0|{@{}@384|.5|.5|{12|1627000|3594000|1647000|3594000|1898000|3594000|1898000|1819000|2149000|1819000|2180000|1819000|}39561|17|}}
{2010210004|XXGflow|86|0|171|0|{@{}@384|.5|.5|{12|1690000|3839000|1710000|3839000|1934000|3839000|1934000|1849000|2159000|1849000|2180000|1849000|}39466|17|}}
{2010210004|XXGflow|87|0|173|0|{@{}@384|.5|.5|{12|1566000|1564000|1606000|1564000|1787000|1564000|1787000|1639000|1969000|1639000|2180000|1639000|}39532|529|}}
{2010210004|XXGflow|88|0|175|0|{@{}@384|.5|.5|{12|221000|1294000|241000|1294000|1105000|1294000|1105000|1279000|1969000|1279000|2180000|1279000|}39483|17|}}
{2010210004|XXGflow|89|0|177|0|{@{}@384|.5|.5|{12|1566000|1594000|1596000|1594000|1777000|1594000|1777000|1669000|1959000|1669000|2180000|1669000|}39533|529|}}
{2010210004|XXGflow|90|0|179|0|{@{}@384|.5|.39603960514068604|{12|498000|624000|548000|624000|854000|624000|854000|1564000|1322000|1564000|1363000|1564000|}39527|529|}}
{2010210004|XXGflow|91|0|181|0|{@{}@384|.5|.3564356565475464|{12|498000|634000|538000|634000|813000|634000|813000|1594000|1312000|1594000|1363000|1594000|}39528|529|}}
{2010210004|XXGflow|92|0|183|0|{@{}@384|.5|.9844961166381836|{12|210000|499000|230000|499000|2119000|499000|2119000|1099000|2149000|1099000|2180000|1099000|}39471|17|}}
{2010210004|XXGflow|93|0|185|0|{@{}@384|.5|.5|{8|1062000|3879000|1082000|3879000|1167000|3879000|1188000|3879000|}39511|17|}}
{2010210004|XXGflow|94|0|187|0|{@{}@384|.5|.5|{12|1371000|3879000|1391000|3879000|1434000|3879000|1434000|3839000|1478000|3839000|1499000|3839000|}39257|17|}}
{2010210004|XXGflow|95|0|189|0|{@{}@384|.5|.5|{12|201000|944000|221000|944000|251000|944000|251000|934000|281000|934000|302000|934000|}39477|17|}}
{2010210004|XXGflow|96|0|191|0|{@{}@384|.5|.5|{12|217000|1039000|237000|1039000|251000|1039000|251000|1034000|265000|1034000|286000|1034000|}39479|17|}}
{2010210004|XXGflow|98|0|195|0|{@{}@384|.5|.5|{12|269000|269000|289000|269000|409000|269000|409000|349000|529000|349000|550000|349000|}38427|17|}}
{2010210004|XXGflow|99|0|197|0|{@{}@384|.5|.5|{12|733000|369000|753000|369000|759000|369000|759000|354000|765000|354000|786000|354000|}38429|17|}}
{2010210004|XXGflow|100|0|199|0|{@{}@384|.5|.5|{12|199000|494000|219000|494000|374000|494000|374000|389000|529000|389000|550000|389000|}38431|17|}}
{2010210004|XXGflow|132|0|305|0|{@{}@384|.5|.5|{12|172000|194000|192000|194000|243000|194000|243000|184000|295000|184000|316000|184000|}0|20|}}
{2010210004|XXGflow|142|0|334|0|{@{}@384|.5|.5|{12|172000|194000|192000|194000|243000|194000|243000|184000|295000|184000|316000|184000|}0|20|}}
{2010210004|XXGflow|162|0|390|0|{@{}@384|.5|.5|{12|172000|194000|192000|194000|243000|194000|243000|184000|295000|184000|316000|184000|}0|20|}}
{2010210004|XXGflow|172|0|419|0|{@{}@384|.5|.5|{12|172000|194000|192000|194000|243000|194000|243000|184000|295000|184000|316000|184000|}0|20|}}
{2010210004|XXGflow|190|0|475|0|{@{}@384|.5|.5|{12|713000|294000|733000|294000|743000|294000|743000|279000|754000|279000|775000|279000|}38400|17|}}
{2010210004|XXGflow|191|0|477|0|{@{}@384|.5|.5|{12|964000|279000|984000|279000|1026000|279000|1026000|329000|1069000|329000|1090000|329000|}38402|17|}}
{2010210004|XXGflow|192|0|479|0|{@{}@384|.5|.5|{12|714000|539000|734000|539000|901000|539000|901000|369000|1069000|369000|1090000|369000|}38409|17|}}
{2010210004|XXGflow|193|0|481|0|{@{}@384|.5|.5|{12|490000|244000|510000|244000|524000|244000|524000|274000|538000|274000|559000|274000|}38397|1041|}}
{2010210004|XXGflow|194|0|483|0|{@{}@384|.5|.5|{12|479000|519000|499000|519000|518000|519000|518000|314000|538000|314000|559000|314000|}38398|1041|}}
{2010210004|XXGflow|195|0|485|0|{@{}@384|.5|.5|{12|1211000|349000|1231000|349000|1312000|349000|1312000|329000|1394000|329000|1415000|329000|}38405|17|}}
{2010210004|XXGflow|196|0|487|0|{@{}@384|.5|.5|{12|271000|254000|291000|254000|294000|254000|294000|244000|298000|244000|319000|244000|}38394|17|}}
{2010210004|XXGflow|197|0|489|0|{@{}@384|.5|.5|{12|1604000|329000|1624000|329000|1640000|329000|1640000|339000|1657000|339000|1678000|339000|}38407|17|}}
{2010210004|XXGflow|283|0|767|0|{@{}@384|.5|.5|{12|172000|194000|192000|194000|243000|194000|243000|184000|295000|184000|316000|184000|}0|20|}}
{2010210004|XXGflow|293|0|797|0|{@{}@384|.5|.5|{12|172000|194000|192000|194000|243000|194000|243000|184000|295000|184000|316000|184000|}0|20|}}
{2010210004|XXGflow|309|0|844|0|{@{}@384|.5|.5|{12|172000|194000|192000|194000|243000|194000|243000|184000|295000|184000|316000|184000|}0|20|}}
{2010210004|XXGflow|319|0|873|0|{@{}@384|.5|.5|{12|172000|194000|192000|194000|243000|194000|243000|184000|295000|184000|316000|184000|}0|20|}}
{2010210004|XXGflow|335|0|920|0|{@{}@384|.5|.5|{12|172000|194000|192000|194000|243000|194000|243000|184000|295000|184000|316000|184000|}0|20|}}
{2010210004|XXGflow|345|0|950|0|{@{}@384|.5|.5|{12|172000|194000|192000|194000|243000|194000|243000|184000|295000|184000|316000|184000|}0|20|}}
{2010210004|XXGflow|361|0|997|0|{@{}@384|.5|.5|{12|172000|194000|192000|194000|243000|194000|243000|184000|295000|184000|316000|184000|}0|20|}}
{2010210004|XXGflow|371|0|1027|0|{@{}@384|.5|.5|{12|172000|194000|192000|194000|243000|194000|243000|184000|295000|184000|316000|184000|}0|20|}}
{2010210004|XXGflow|463|0|1289|0|{@{}@384|.5|.5|{8|233000|164000|253000|164000|259000|164000|280000|164000|}23173|17|}}
{2010210004|XXGflow|481|0|1340|0|{@{}@384|.5|.5|{12|239000|104000|259000|104000|321000|104000|321000|124000|384000|124000|405000|124000|}38530|1041|}}
{2010210004|XXGflow|482|0|1342|0|{@{}@384|.5|.5|{12|294000|309000|314000|309000|349000|309000|349000|164000|384000|164000|405000|164000|}38531|1041|}}
{2010210004|XXGflow|483|0|1344|0|{@{}@384|.5|.5|{12|546000|144000|566000|144000|840000|144000|840000|139000|1114000|139000|1135000|139000|}38532|529|}}
{2010210004|XXGflow|515|0|1450|0|{@{}@384|.5|.5|{12|172000|194000|192000|194000|243000|194000|243000|184000|295000|184000|316000|184000|}0|20|}}
{2010210004|XXGflow|525|0|1479|0|{@{}@384|.5|.5|{12|172000|194000|192000|194000|243000|194000|243000|184000|295000|184000|316000|184000|}0|20|}}
{2010210004|XXGflow|541|0|1526|0|{@{}@384|.5|.5|{12|172000|194000|192000|194000|243000|194000|243000|184000|295000|184000|316000|184000|}0|20|}}
{2010210004|XXGflow|551|0|1555|0|{@{}@384|.5|.5|{12|172000|194000|192000|194000|243000|194000|243000|184000|295000|184000|316000|184000|}0|20|}}
{2010210004|XXGflow|567|0|1602|0|{@{}@384|.5|.5|{12|172000|194000|192000|194000|243000|194000|243000|184000|295000|184000|316000|184000|}0|20|}}
{2010210004|XXGflow|577|0|1631|0|{@{}@384|.5|.5|{12|172000|194000|192000|194000|243000|194000|243000|184000|295000|184000|316000|184000|}0|20|}}
{2010210004|XXGflow|612|0|1734|0|{@{}@384|.5|.5|{12|891000|184000|911000|184000|984000|184000|984000|164000|1058000|164000|1079000|164000|}38482|17|}}
{2010210004|XXGflow|613|0|1736|0|{@{}@384|.5|.5|{12|1290000|164000|1310000|164000|1371000|164000|1371000|169000|1432000|169000|1453000|169000|}38484|17|}}
{2010210004|XXGflow|614|0|1738|0|{@{}@384|.5|.5|{12|1075000|474000|1095000|474000|1263000|474000|1263000|209000|1432000|209000|1453000|209000|}38492|17|}}
{2010210004|XXGflow|615|0|1740|0|{@{}@384|.5|.5|{12|1636000|189000|1656000|189000|1740000|189000|1740000|179000|1824000|179000|1845000|179000|}38486|17|}}
{2010210004|XXGflow|616|0|1742|0|{@{}@384|.5|.5|{12|316000|359000|336000|359000|357000|359000|357000|344000|379000|344000|400000|344000|}38462|17|}}
{2010210004|XXGflow|617|0|1744|0|{@{}@384|.5|.5|{12|266000|159000|286000|159000|310000|159000|310000|144000|334000|144000|355000|144000|}38490|17|}}
{2010210004|XXGflow|618|0|1746|0|{@{}@384|.5|.5|{12|564000|144000|584000|144000|637000|144000|637000|159000|690000|159000|711000|159000|}38472|1041|}}
{2010210004|XXGflow|619|0|1748|0|{@{}@384|.5|.5|{12|609000|344000|629000|344000|659000|344000|659000|209000|690000|209000|711000|209000|}38473|1553|}}
{2010210004|XXGflow|620|0|1750|0|{@{}@384|.5|.5|{12|2024000|179000|2044000|179000|2113000|179000|2113000|189000|2182000|189000|2203000|189000|}38488|17|}}
{2010210004|XXGflow|706|0|2028|0|{@{}@384|.5|.5|{12|172000|194000|192000|194000|243000|194000|243000|184000|295000|184000|316000|184000|}0|20|}}
{2010210004|XXGflow|716|0|2058|0|{@{}@384|.5|.5|{12|172000|194000|192000|194000|243000|194000|243000|184000|295000|184000|316000|184000|}0|20|}}
{2010210004|XXGflow|732|0|2105|0|{@{}@384|.5|.5|{12|172000|194000|192000|194000|243000|194000|243000|184000|295000|184000|316000|184000|}0|20|}}
{2010210004|XXGflow|742|0|2135|0|{@{}@384|.5|.5|{12|172000|194000|192000|194000|243000|194000|243000|184000|295000|184000|316000|184000|}0|20|}}
{2010210004|XXGflow|758|0|2182|0|{@{}@384|.5|.5|{12|172000|194000|192000|194000|243000|194000|243000|184000|295000|184000|316000|184000|}0|20|}}
{2010210004|XXGflow|768|0|2212|0|{@{}@384|.5|.5|{12|172000|194000|192000|194000|243000|194000|243000|184000|295000|184000|316000|184000|}0|20|}}
{2010210004|XXGflow|784|0|2259|0|{@{}@384|.5|.5|{12|172000|194000|192000|194000|243000|194000|243000|184000|295000|184000|316000|184000|}0|20|}}
{2010210004|XXGflow|794|0|2289|0|{@{}@384|.5|.5|{12|172000|194000|192000|194000|243000|194000|243000|184000|295000|184000|316000|184000|}0|20|}}
{2010210004|XXGflow|827|0|2389|0|{@{}@384|.5|.5|{12|172000|194000|192000|194000|243000|194000|243000|184000|295000|184000|316000|184000|}0|20|}}
{2010210004|XXGflow|863|0|2511|0|{@{}@384|.5|.5|{12|172000|194000|192000|194000|243000|194000|243000|184000|295000|184000|316000|184000|}0|20|}}
{2010210004|XXGflow|879|0|2558|0|{@{}@384|.5|.5|{12|172000|194000|192000|194000|243000|194000|243000|184000|295000|184000|316000|184000|}0|20|}}
{2010210004|XXGflow|889|0|2588|0|{@{}@384|.5|.5|{12|172000|194000|192000|194000|243000|194000|243000|184000|295000|184000|316000|184000|}0|20|}}
{2010210004|XXGflow|905|0|2635|0|{@{}@384|.5|.5|{12|172000|194000|192000|194000|243000|194000|243000|184000|295000|184000|316000|184000|}0|20|}}
{2010210004|XXGflow|915|0|2665|0|{@{}@384|.5|.5|{12|172000|194000|192000|194000|243000|194000|243000|184000|295000|184000|316000|184000|}0|20|}}
{2010210004|XXGflow|1213|0|3737|0|{@{}@384|.5|.5|{12|278000|134000|298000|134000|313000|134000|313000|139000|329000|139000|350000|139000|}22209|17|}}
{2010210004|XXGflow|1226|0|3778|0|{@{}@384|.5|.5|{8|403000|164000|423000|164000|458000|164000|479000|164000|}23175|17|}}
{2010210004|XXGflow|1247|0|3838|0|{@{}@384|.5|.5|{12|1218000|129000|1238000|129000|1277000|129000|1277000|204000|1317000|204000|1338000|204000|}23390|1041|}}
{2010210004|XXGflow|1252|0|3848|0|{@{}@384|.5|.5|{12|985000|124000|1005000|124000|1013000|124000|1013000|129000|1021000|129000|1042000|129000|}23385|17|}}
{2010210004|XXGflow|1265|0|3889|0|{@{}@384|.5|.5|{8|437000|134000|457000|134000|501000|134000|522000|134000|}21961|17|}}
{2010210004|XXGflow|1293|0|3992|0|{@{}@384|.5|.5|{12|683000|94000|703000|94000|724000|94000|724000|114000|746000|114000|767000|114000|}37642|17|}}
{2010210004|XXGflow|1294|0|3994|0|{@{}@384|.5|.5|{8|506000|94000|526000|94000|590000|94000|611000|94000|}37645|19|}}
{2010210004|XXGflow|1296|0|3998|0|{@{}@384|.5|.5|{8|546000|109000|566000|109000|596000|109000|617000|109000|}37629|17|}}
{2010210004|XXGflow|1297|0|4000|0|{@{}@384|.5|.5|{12|362000|104000|382000|104000|384000|104000|384000|109000|387000|109000|408000|109000|}37620|17|}}
{2010210004|XXGflow|1298|0|4002|0|{@{}@384|.5|.5|{12|122000|114000|142000|114000|156000|114000|156000|104000|170000|104000|191000|104000|}37627|17|}}
{2010210004|XXGflow|1354|0|4155|0|{@{}@384|.5|.5|{12|1220000|254000|1240000|254000|1278000|254000|1278000|214000|1317000|214000|1338000|214000|}23391|1041|}}
{2010210004|XXGflow|1355|0|4158|0|{@{}@384|.5|.5|{12|177000|134000|197000|134000|260000|134000|260000|144000|324000|144000|345000|144000|}23368|17|}}
{2010210004|XXGflow|1356|0|4160|0|{@{}@384|.5|.5|{12|417000|139000|437000|139000|465000|139000|465000|109000|494000|109000|515000|109000|}23360|17|}}
{2010210004|XXGflow|1357|0|4162|0|{@{}@384|.5|.5|{12|417000|149000|437000|149000|465000|149000|465000|244000|494000|244000|515000|244000|}23363|17|}}
{2010210004|XXGflow|1358|0|4164|0|{@{}@384|.5|.5|{12|734000|109000|754000|109000|771000|109000|771000|124000|788000|124000|809000|124000|}23384|17|}}
{2010210004|XXGflow|1359|0|4167|0|{@{}@384|.5|.5|{12|724000|244000|744000|244000|764000|244000|764000|259000|784000|259000|805000|259000|}23365|17|}}
{2010210004|XXGflow|1360|0|4169|0|{@{}@384|.5|.5|{12|972000|259000|992000|259000|1012000|259000|1012000|254000|1032000|254000|1053000|254000|}23380|17|}}
{2010210004|XXGflow|1410|0|4314|0|{@{}@384|.5|.5|{12|172000|194000|192000|194000|243000|194000|243000|184000|295000|184000|316000|184000|}0|20|}}
{2010210004|XXGflow|1420|0|4344|0|{@{}@384|.5|.5|{12|172000|194000|192000|194000|243000|194000|243000|184000|295000|184000|316000|184000|}0|20|}}
{2010210004|XXGflow|1436|0|4391|0|{@{}@384|.5|.5|{12|172000|194000|192000|194000|243000|194000|243000|184000|295000|184000|316000|184000|}0|20|}}
{2010210004|XXGflow|1446|0|4421|0|{@{}@384|.5|.5|{12|172000|194000|192000|194000|243000|194000|243000|184000|295000|184000|316000|184000|}0|20|}}
{2010210004|XXGflow|1497|0|4578|0|{@{}@384|.5|.5|{8|197000|119000|217000|119000|427000|119000|448000|119000|}21846|17|}}
{2010210004|XXGflow|1549|0|4736|0|{@{}@384|.5|.5|{12|216000|114000|236000|114000|282000|114000|282000|124000|328000|124000|349000|124000|}21869|17|}}
{2010210004|XXGflow|1581|0|4828|0|{@{}@384|.5|.5|{12|233000|114000|253000|114000|277000|114000|277000|124000|301000|124000|322000|124000|}21851|17|}}
{2010210004|XXGflow|1597|0|4872|0|{@{}@384|.5|.5|{12|172000|194000|192000|194000|243000|194000|243000|184000|295000|184000|316000|184000|}0|20|}}
{2010210004|XXGflow|1607|0|4901|0|{@{}@384|.5|.5|{12|172000|194000|192000|194000|243000|194000|243000|184000|295000|184000|316000|184000|}0|20|}}









 

 

 

I need to travel through files.

Basically for ur understanding i am explaining this.

I am doing analysis of a graph in unix . graph contains components connected and till now we have taken a graph and divided into components(one text file splitting into different files and name of the components as file names). Now i need to get the flow from each input component to output component(form input file to output file), I need to know in which flow the particular component flows and get connected to output component. For ex:

Input1 is connected to Join and then to Partition by key and then to Reformat and then to FBE and then to Output.(u can ignore the component names , i just said so that through these files it must go).

So We need to take one Inputfile , 'cat' it and then find the output port number(The above number highlighted ) and then it needs to go to XXGFlow file(I have done scripting for getting the file and the above given file is the entire file for temp1.txt) and the need to search the line which has "2844000"(2839000(got this number frm input file)+5000), for this we have 

 




Code:













Code:



{2010210004|XXGflow|59|0|117|0|{@{}@384|.5|.5|{12|239000|2844000|259000|2844000|483000|2844000|483000|2824000|708000|2824000|729000|2824000|}39548|529|}}









 

got frm XXGFlow and the we need to take value 2819000(2824000-5000) and then again search in all files  where 2819000 has in input port , we got 

 




Code:













Code:



{2010202004|XXGiport|1480|0|4526|0|{@{}@730000|2819000|11000|11000|in0|.5|@@@14096|0|}}









 

now in this particular file again go to oport and find number 

 




Code:













Code:



{2010203004|XXGoport|1479|0|4522|0|{@{}@1029000|2809000|11000|11000|out0|.35944700460829493|@@@14736|0|}}









 

now again 2814000(2809000+5000) need to be searched in XXGFlow file......................... This process continues until the iport number of Output_File is reached in this file the number of Output_file is 1419000 , so in XXGFlow it will be 1424000(1419000+5000). I need this flow file names like 

(This is given for example purpose only)




Code:



 
File name: Input_File1_g
Input_File
Join36
PKS12
reformat
Output_File







Like this i need for every input file, Here for this particular graph , i have 26 Input_File's , so 26 files of their corresponding flows must be needed.

 

Praying GOD , so that u understand what i have written 

This is second phase and i need to do many phases after this.

Dont ask me why only u r doing +5000 and -5000 , after 1.5 month analysis i found this pattern and even i also dont know y unix is storing in such a way, If u have any idea please let me know.

 

If u think this is more confusing , i'll expalin u in subparts and we will do it part by part and then accomplish phase2.

 

Thank q very much for ur patience and if possible pls help me out in this.

 

Please get back to me for any clarifications



---------- Post updated at 02:06 PM ---------- Previous update was at 02:02 PM ----------



What i mean is:

 




Code:













Code:



{2010503005|XXGfvertex|234|0|604|0|{|{30100001|XXparameter_set|@@@@{{30001002|XXparameter|Layout|$[[record kind 85 subkind 0 parts [vector _interp_("mfile:$\{INF_ENTRPRSDWUNFYRETLCRED_MFS\}/m_cdp2_uedw_t_post_addr_uld.dat", "dollar_substitution")]]]|3|9||@{0|}}
{30001002|XXparameter|read_metadata|$\{INF_ENTRPRSDWUNFYRETLCRED_DML\}/cdp2_uedw_t_post_addr.dml|3|2|f$|@{0|}}
{30001002|XXparameter|key|\{post_addr_id\}|3|2|$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio\\Ab Initio GDE 1.15.11.1\\Components\\Datasets\\Input_File.mdc|3|2|Pw$|@{0|}}
{30001002|XXparameter|eme_dataset_location|$\{_Projects_SunTrust_inf_inf_EntrprsDwUnfyRetlCred\}/data/mfs/main/m_cdp2_uedw_t_post_addr_uld.dat|3|9||@{0|}}
}}@1|@625666|493684|648000|514000|56000|50000|38410|m_cdp2_ uedw_t_post_addr_uld.dat|a8sc20||1|10|0||6||33025|-1|-1|}}
{2010203004|XXGoport|235|0|606|0|{@{}@703000|534000|11000|11000|read|0.0|@@@2160|0|}}
{2010503005|XXGfvertex|236|0|609|0|{Represents one file, many files, or a multifile as an input to your graph.|{30100001|XXparameter_set|@@@@{{30001002|XXparameter|protection|0666|12|2|RF$||{0|}}
{30001002|XXparameter|mode|0x0001|1|2|FH$|modes of access|{0|}}
{30001002|XXparameter|Layout|@28|2|RF$||{0|}}
{30001002|XXparameter|mpcmodtime|1256159712|1|1|Hl|The last modification time of this component's template|{0|}}
{30001002|XXparameter|read_metadata||7|1|RFl||{0|}}
{30001002|XXparameter|key||19|2|RF$|Key specifier For Lookup File|{0|}}
{30001002|XXparameter|key.condition|mode lookup|3|15|P?|@{0|}}
{30001002|XXparameter|eme_dataset_location|@3|9|F|Place in the EME to create a dataset corresponding to this file.|{0|}}
}}@0|@0|0|0|0|0|0|0|@@@1|10|-1|@6|@1|-1|-1|}}
{2010203004|XXGoport|237|0|611|0|{@{30100001|XXparameter_set|@@@@{{30001002|XXparameter|metadata||7|8|RF=||{0|}}
}}@0|0|0|0|read|0.0|@@@2160|0|}}
{2010501005|XXGpvertex|238|0|614|0|{|{30100001|XXparameter_set|@@@@{{30001002|XXparameter|key|\{rsrc_itm_id\}|3|2|$|@{0|}}
{30001002|XXparameter|transform|out :: join(in0, in1) =
begin
  out.arnge_id:: in0.arnge_id;
  out.post_addr_id :: in1.post_addr_id;
end;|3|1|l|@{0|}}
{30001002|XXparameter|join_type|Explicit|3|1|l|@{0|}}
{30001002|XXparameter|record_required1|True|3|1|l|@{0|}}
{30001002|XXparameter|out_metadata|record
  decimal("\\x01") arnge_id; /* INTEGER NOT NULL*/
  decimal("\\x01") post_addr_id = NULL(""); /* INTEGER*/
  string(1) newline = "\\n";
end|3|1|l|@{0|}}
{30001002|XXparameter|error0_metadata|string('\\n')|3|1|l|@{0|}}
{30001002|XXparameter|error1_metadata|string('\\n')|3|1|l|@{0|}}
{30001002|XXparameter|log_metadata|record string("\|") node, timestamp, component, subcomponent, event_type; string("\|\\n") event_text; end|3|1|l|@{0|}}










 

This is one of the Input_File and the part highlighted in red is of another file(file name is JOIN FYI). I dont think that this a prob for further analysis.



---------- Post updated at 02:07 PM ---------- Previous update was at 02:06 PM ----------



What i mean is:

 




Code:













Code:



{2010503005|XXGfvertex|234|0|604|0|{|{30100001|XXparameter_set|@@@@{{30001002|XXparameter|Layout|$[[record kind 85 subkind 0 parts [vector _interp_("mfile:$\{INF_ENTRPRSDWUNFYRETLCRED_MFS\}/m_cdp2_uedw_t_post_addr_uld.dat", "dollar_substitution")]]]|3|9||@{0|}}
{30001002|XXparameter|read_metadata|$\{INF_ENTRPRSDWUNFYRETLCRED_DML\}/cdp2_uedw_t_post_addr.dml|3|2|f$|@{0|}}
{30001002|XXparameter|key|\{post_addr_id\}|3|2|$|@{0|}}
{30001002|XXparameter|!prototype_path|E:\\program files\\Ab Initio\\Ab Initio GDE 1.15.11.1\\Components\\Datasets\\Input_File.mdc|3|2|Pw$|@{0|}}
{30001002|XXparameter|eme_dataset_location|$\{_Projects_SunTrust_inf_inf_EntrprsDwUnfyRetlCred\}/data/mfs/main/m_cdp2_uedw_t_post_addr_uld.dat|3|9||@{0|}}
}}@1|@625666|493684|648000|514000|56000|50000|38410|m_cdp2_ uedw_t_post_addr_uld.dat|a8sc20||1|10|0||6||33025|-1|-1|}}
{2010203004|XXGoport|235|0|606|0|{@{}@703000|534000|11000|11000|read|0.0|@@@2160|0|}}
{2010503005|XXGfvertex|236|0|609|0|{Represents one file, many files, or a multifile as an input to your graph.|{30100001|XXparameter_set|@@@@{{30001002|XXparameter|protection|0666|12|2|RF$||{0|}}
{30001002|XXparameter|mode|0x0001|1|2|FH$|modes of access|{0|}}
{30001002|XXparameter|Layout|@28|2|RF$||{0|}}
{30001002|XXparameter|mpcmodtime|1256159712|1|1|Hl|The last modification time of this component's template|{0|}}
{30001002|XXparameter|read_metadata||7|1|RFl||{0|}}
{30001002|XXparameter|key||19|2|RF$|Key specifier For Lookup File|{0|}}
{30001002|XXparameter|key.condition|mode lookup|3|15|P?|@{0|}}
{30001002|XXparameter|eme_dataset_location|@3|9|F|Place in the EME to create a dataset corresponding to this file.|{0|}}
}}@0|@0|0|0|0|0|0|0|@@@1|10|-1|@6|@1|-1|-1|}}
{2010203004|XXGoport|237|0|611|0|{@{30100001|XXparameter_set|@@@@{{30001002|XXparameter|metadata||7|8|RF=||{0|}}
}}@0|0|0|0|read|0.0|@@@2160|0|}}
{2010501005|XXGpvertex|238|0|614|0|{|{30100001|XXparameter_set|@@@@{{30001002|XXparameter|key|\{rsrc_itm_id\}|3|2|$|@{0|}}
{30001002|XXparameter|transform|out :: join(in0, in1) =
begin
  out.arnge_id:: in0.arnge_id;
  out.post_addr_id :: in1.post_addr_id;
end;|3|1|l|@{0|}}
{30001002|XXparameter|join_type|Explicit|3|1|l|@{0|}}
{30001002|XXparameter|record_required1|True|3|1|l|@{0|}}
{30001002|XXparameter|out_metadata|record
  decimal("\\x01") arnge_id; /* INTEGER NOT NULL*/
  decimal("\\x01") post_addr_id = NULL(""); /* INTEGER*/
  string(1) newline = "\\n";
end|3|1|l|@{0|}}
{30001002|XXparameter|error0_metadata|string('\\n')|3|1|l|@{0|}}
{30001002|XXparameter|error1_metadata|string('\\n')|3|1|l|@{0|}}
{30001002|XXparameter|log_metadata|record string("\|") node, timestamp, component, subcomponent, event_type; string("\|\\n") event_text; end|3|1|l|@{0|}}










 

This is one of the Input_File and the part highlighted in red is of another file(file name is JOIN FYI). I dont think that this a prob for further analysis.







        

        

       



            
            
            
            


            

            
            




            
                
            

            

        



        


        


 


        

        
        

        


 

            
        




















	


		


		 
		
			












	
	









    







10 More Discussions You Might Find Interesting











1. Shell Programming and Scripting






 splitting files based on text in the file




I need to split a file based on certain context inside the file. Is there a unix command that can do this? I have looked into split and csplit but it does not seem like those would work because I need to split this file based on certain text. The file has multiple records and I need to split this... (1 Reply)





Discussion started by: matrix1067



1 Replies








2. Shell Programming and Scripting






 Splitting large file into multiple files in unix based on pattern




I need to write a shell script for below scenario 
My input file has data in format: 
qwerty0101TWE 12345 01022005 01022005 datainala alanfernanded 26 
qwerty0101mXZ 12349 01022005 06022008 datainalb johngalilo 28  
qwerty0101TWE 12342 01022005 07022009 datainalc hitalbert 43   
qwerty0101CFG 12345... (19 Replies)





Discussion started by: jimmy12



19 Replies








3. Shell Programming and Scripting






 Data Splitting into two files from one file




I have a file as: 
I/P File: 
 
Ground Car 2009 
Lib 2008 
Lib 2003 
Ground Car 2009 
Ground Car 2003 
Car 2005 
Car 2003 
Car 2005 
Sita 2900 2006 
Car 2007 
 
  
I have to split the file into two: - one for names and second for years. 
  
O/p1 (Names): 
 
Ground Car (3 Replies)





Discussion started by: karumudi7



3 Replies








4. Shell Programming and Scripting






 Splitting single file into n files




Hi all, 
  
I am new to scripting and I have a requirement 
we have source file as 
 
HEADER 01.10.2010 14:32:37 NAYA 
TA0022 
TA0000 
20000001;20060612;99991231;K4;02;3 
20000008;20080624;99991231;K4;02;3 
20000026;19840724;99991231;KK;01;3 
20000027;19840724;99991231;KK;01;3... (6 Replies)





Discussion started by: srk409



6 Replies








5. Shell Programming and Scripting






 Urgent ...pls Sorting files based on timestamp and picking the latest file




Hi Friends, 
 
Newbie to shell scripting. Currently i have used the below to sort data based on filenames and datestamp 
 
$ printf '%s\n' *.dat* | sort -t. -k3,4 
filename_1.dat.20120430.Z 
filename_2.dat.20120430.Z 
filename_3.dat.20120430.Z 
filename_1.dat.20120501.Z 
filename_2.dat.20120501.Z... (1 Reply)





Discussion started by: robertbrown624



1 Replies








6. Shell Programming and Scripting






 Sed: Splitting A large File into smaller files based on recursive Regular Expression match




I will simplify the explaination a bit, I need to parse through a 87m file -  
 
 
I have a single text file in the form of : 
 
 
<NAME>house........ 
SOMETEXT 
SOMETEXT  
SOMETEXT 
. 
. 
. 
. 
</script> 
MORETEXT 
MORETEXT 
. 
. 
. (6 Replies)





Discussion started by: sumguy



6 Replies








7. UNIX for Dummies Questions & Answers






 Extracting data from one file, based on another file (splitting)




Dear All,  
I have two files but want to extract data from one based on another... can  you please help me 
file 1  
David 
Tom 
Ellen 
and file 2 
David|0010|testnamez|resultsz 
David|0004|testnamex|resultsx 
Tom|0010|testnamez|resultsz 
Tom|0004|testnamex|resultsx 
Ellen|0010|testnamez|resultsz... (12 Replies)





Discussion started by: A-V



12 Replies








8. Shell Programming and Scripting






 Splitting a single file to multiple files




Hi Friends , 
Please guide me with the code to extract multiple files from one file . 
 
The File Looks like ( Suppose a file has 2 tables list ,column length may vary )  
 
H..- > File Header.... 
H....- >Table 1 Header.... 
D....- > Table 1 Data....  
T....- >Table 1 Trailer.... 
H..-> Table 2... (1 Reply)





Discussion started by: AspiringD



1 Replies








9. Shell Programming and Scripting






 Split a single file into multiple files based on a value.




Hi All, 
 
I have the sales_data.csv file in the directory as below. 
  
SDDCCR;  SOM   ; MD6546474777   ;05-JAN-16 
ABC   ;  KIRAN ; CB789          ;04-JAN-16 
ABC   ;  RAMANA; KS566767477747 ;06-JAN-16 
ABC   ;  KAMESH; A33535335      ;04-JAN-16 
SDDCCR;  DINESH; GD6674474747   ;08-JAN-16... (4 Replies)





Discussion started by: ROCK_PLSQL



4 Replies








10. Shell Programming and Scripting






 In PErl script: need to read the data one file and generate multiple files based on the data




We have the data looks like below in a log file. 
I want to  generat files based on the string between two hash(#) symbol like below 
 
Source: 
 
#ext1#test1.tale2 drop 
#ext1#test11.tale21 drop 
#ext1#test123.tale21 drop 
#ext2#test1.tale21 drop 
#ext2#test12.tale21 drop 
#ext3#test11.tale21 drop... (5 Replies)





Discussion started by: Sanjeev G



5 Replies












    

       

          
        

    

    

      

          
        

    






















All times are GMT -4. The time now is 07:41 AM.









	






















	
	

Unix & Linux Forums Content Copyright  1993-2022. All Rights Reserved. 

        



Privacy Policy