Splitting the records.


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Splitting the records.
# 1  
Old 07-16-2010
Splitting the records.

Hi Guys,

Hope you are doing good out there.

I got stuck in processing a log file wherein I need to spilt an application directory name from the log file. Below are the log files
Code:
TIL_MQSI_TransactAdapters.TIL_MQSI_TransactAdapters-TransactMQAdapter-3.log
TIL_MQSI_TransactAdapters.TIL_MQSI_TransactAdapters-TransactMQAdapter.log
TIL_MQSI_TransactAdapters.TIL_MQSI_TransactAdapters-TransactMQAdapter-1.log
ContactChannelManagement-SVAP-Process_Archive-1.log
ApplicationIntegration-Carisma-ApplicationIntegration-Carisma.log
TIL_MQSI_EPOSAdapters.TIL_MQSI_EPOSAdapters-RetailSystemsMQAdapter.log
SalesAndMarketing-Misc-1-SalesAndMarketing-Misc.log

.

The name of the log files consisting of the application name, like TIL_MQSI_EPOSAdapters and the engine name TIL_MQSI_EPOSAdapters-RetailSystemsMQAdapter saparated by a "." or a "-".

I need to split the application name and the engine name out of these log files. As you can see the application name and the engine name in the log file are almost same.

My desired output is
Code:
Application Name                     Engine Name
 
TIL_MQSI_TransactAdapters            TIL_MQSI_TransactAdapters
TIL_MQSI_TransactAdapters            TIL_MQSI_TransactAdapters-TransactMQAdapter
ApplicationIntegration-Carisma       Carisma-ApplicationIntegration-Carisma
TIL_MQSI_EPOSAdapters                TIL_MQSI_EPOSAdapters-RetailSystemsMQAdapter
SalesAndMarketing-Misc-1             SalesAndMarketing-Misc

Can you please let me know how this can be achieved?

Any help would be highly appereciated.

Thanks,
Chandan
# 2  
Old 07-16-2010
Code:
awk -F'.'  'BEGIN{printf("%40s %40s\n", "Application Name", "Engine Name")} 
                      { printf("%40s %40s\n", $1, $2)} inputfile.log

the %40s mean create a column 40 characters wide. You may need to play with that setting.
# 3  
Old 07-16-2010
Hi Jim,

Thanks for your prompt reply as always.

The issue is that, the Fields are always not the same due to the invariable name of log files. So even we print the col1 and col2, it will not give us the desired result always. Like in the following case
Code:
BillingAccounting-Gemini-BillingAccounting-Gemini.log
SalesAndMarketing-Misc-1-SalesAndMarketing-Misc

What I am looking for is compare the value of col1 to the rest of the cols in a log file and if it matches, then print from col1 upto one col less than the matched col,as the application name.

Like,
Code:
SalesAndMarketing-Misc-1

Here, col1=SalesAndMarketing, col2=Misc, col3=1,col4=SalesAndMarketing
and col5=Misc.
As you can see, col1 and col4 have the same value, so the desired output would be; SalesAndMarketing-Misc-1(col1, col2, col3).

Just for your information, the application name and the engine name will always be the same except that an integer in the app name sometimes.

Please let me know your thoughts.

Thanks,
Chandan

Last edited by singh.chandan18; 07-16-2010 at 01:25 PM.. Reason: corrected the mistakes.
# 4  
Old 07-20-2010
Hi Guys,

I've tried lots of tricks, but none of them is working. So, Need help from t he Experts.

Here is my problem,

I have Log files and I need to extract App Directory name from them.
Code:
SubscriptionManagement-2-SubscriptionManagement.log
SubscriptionManagement-ISAAC-2-SubscriptionManagement-ISAAC-1.log
RemoveUnpublishedMobileSubscribers-DM-Drop5-RemoveUnpublishedMobileSubscribers-DM-Drop5.log
ProvisioningAndFulfilment-Misc-ProvisioningAndFulfilment-Misc.log

The name of the log file is the combination of the App Dir name and the App Engine name.

Both the App Dir name and the Engine name are almost same. But, there is no consistiency in the naming convention of the log file which is creating the issues.
One way to solve the problem is using awk and match the first field with the rest of the fields in a record.
If the match is found, print all the fields from the first upto the field one less than the matched field.

Can you please suggest if this can be done using awk? or, is there any other way?
# 5  
Old 07-22-2010
Hi Guys,

I've found the solution for this as below:
Code:
   echo $file | awk ' {
 
       z = split($1, name, "-")
 
       DirName=name[1]
 
       for (i = 2; i <= z; i++) {
 
           if ( name[i] == name[1] ) {
 
               print DirName
 
           }
 
           else {
 
               DirName = DirName "-" name[i]
 
           }
 
       }
 
   }'


Splitting the elements of the filename into an array. Then loop through each element - if it doesn't match the first element, concatenate it to a variable holding the directory name. If the element matches the first one, print out the directory name.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Script for splitting file of records into multiple files

Hello I have a file of following format HDR 1234 abc qwerty abc def ghi jkl HDR 4567 xyz qwerty abc def ghi jkl HDR 890 mno qwerty abc def ghi jkl HDR 1234 abc qwerty abc def ghi jkl HDR 1234 abc qwerty abc def ghi jkl -Need to split this into multiple files based on tag... (8 Replies)
Discussion started by: wincrazy
8 Replies

2. UNIX for Dummies Questions & Answers

Splitting data into new records

Hi, My file is seperated with ";" delimiter, after 13 delimiter i want to put the data in new line... eg: My current file:- a;b;c;d;e;f;g;h;e;f;h;s;t;a;i;o;q;t;q;r;yu;f;sz;f;t;r........... i want o/p as:- a;b;c;d;e;f;g;h;e;f;h;s;t a;i;o;q;t;q;r;yu;f;sz;f;t;r How to achieve ths,... (2 Replies)
Discussion started by: gnnsprapa
2 Replies

3. Shell Programming and Scripting

Splitting records in a text file based on delimiter

A text file has 2 fields (Data, Filename) delimited by # as below, Data,Filename Row1 -> abc#Test1.xml Row2 -> xyz#Test2.xml Row3 -> ghi#Test3.xml The content in first field has to be written into a file where filename should be considered from second field. So from... (4 Replies)
Discussion started by: jayakkannan
4 Replies

4. Shell Programming and Scripting

Compare two files with different number of records and output only the Extra records from file1

Hi Freinds , I have 2 files . File 1 |nag|HYd|1|Che |esw|Gun|2|hyd |pra|bhe|3|hyd |omu|hei|4|bnsj |uer|oeri|5|uery File 2 |nag|HYd|1|Che |esw|Gun|2|hyd |uer|oi|3|uery output : (9 Replies)
Discussion started by: i150371485
9 Replies

5. Shell Programming and Scripting

Splitting record into multiple records by appending values from an input field (AWK)

Hello, For the input file, I am trying to split those records which have multiple values seperated by '|' in the last input field, into multiple records and each record corresponds to the common input fields + one of the value from the last field. I was trying with an example on this forum... (4 Replies)
Discussion started by: imtiaz99
4 Replies

6. Shell Programming and Scripting

awk - splitting 1 large file into multiple based on same key records

Hello gurus, I am new to "awk" and trying to break a large file having 4 million records into several output files each having half million but at the same time I want to keep the similar key records in the same output file, not to exist accross the files. e.g. my data is like: Row_Num,... (6 Replies)
Discussion started by: kam66
6 Replies

7. UNIX for Dummies Questions & Answers

Grep specific records from a file of records that are separated by an empty line

Hi everyone. I am a newbie to Linux stuff. I have this kind of problem which couldn't solve alone. I have a text file with records separated by empty lines like this: ID: 20 Name: X Age: 19 ID: 21 Name: Z ID: 22 Email: xxx@yahoo.com Name: Y Age: 19 I want to grep records that... (4 Replies)
Discussion started by: Atrisa
4 Replies

8. Shell Programming and Scripting

Based on num of records in file1 need to check records in file2 to set some condns

Hi All, I have two files say file1 and file2. I want to check the number of records in file1 and if its atleast 2 (i.e., 2 or greater than 2 ) then I have to check records in file2 .If records in file2 is atleast 1 (i.e. if its not empty ) i have to set some conditions . Could you pls... (3 Replies)
Discussion started by: mavesum
3 Replies

9. Shell Programming and Scripting

Splitting a file based on the records in another file

All, We receive a file with a large no of records (records can vary) and we have to split it into two files based on another file. e.g. File1: UHDR 2008112 "25187","00000022","00",21-APR-1991,"" ,"D",-000000519,+0000000000,"C", ,+000000000,+000000000,000000000,"2","" ... (2 Replies)
Discussion started by: er_ashu
2 Replies

10. Shell Programming and Scripting

Count No of Records in File without counting Header and Trailer Records

I have a flat file and need to count no of records in the file less the header and the trailer record. I would appreciate any and all asistance Thanks Hadi Lalani (2 Replies)
Discussion started by: guiguy
2 Replies
Login or Register to Ask a Question