Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Extracting Delimiter 'TAG' Data From log files

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Extracting Delimiter 'TAG' Data From log files
# 1  
Old 04-12-2013
Linux Extracting Delimiter 'TAG' Data From log files
Hi
I am trying to extract data from within a log file and output format to a new file for further manipulation can someone provide script to do this?

For example I have a file as below and just want to extract all delimited variances of tag 32=* up to the delimiter "|" and output to a new file so that I can calculate the total of these values.
Code:
$cat fix1.log

FIX4.4 |0=12|55=LIT.L|48=123456|32=5|38=100|10=200|
FIX4.4 |0=12|55=LIT.L|48=123456|32=15|38=100|10=200|
FIX4.4 |0=12|55=LIT.L|48=123456|32=115|38=100|10=200|
FIX4.4 |0=12|55=LIT.L|48=123456|32=500|38=100|10=200|
FIX4.4 |0=12|55=LIT.L|48=123456|32=2|38=100|10=200|110=233
FIX4.4 |0=12|55=LIT.L|48=123456|32=5|38=100|10=200|110=221
FIX4.4 |0=12|55=LIT.L|48=123456|32=5|38=100|10=200|775=1
FIX4.4 |0=12|55=LIT.L|48=123456||35=1|34=52|32=2|38=100|10=200|
FIX4.4 |0=12|55=LIT.L|48=123456|35=0|32=5|38=100|10=200|
FIX4.4 |0=12|55=LIT.L|48=123456|35=5|32=4|38=100|10=200|

As you can see the delimited tag 32 is not always in the same position within the file and ideally I would like the output to consist of just the individual numeric values after the 32=
Code:
5
15
115
500

etc etc

Assistance appreciated

Buddy
Last edited by vbe; 04-12-2013 at 05:34 AM.. Reason: code tags please...
# 2  
Old 04-12-2013
One way:
Code:
 sed 's/.*32=\([0-9]*\).*/\1/' file

Guru.
This User Gave Thanks to guruprasadpr For This Post:
Buddyluv
# 3  
Old 04-12-2013
Excellent !!

Thanks Guru
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Find a tag with data and replace its data in another tag

Hi I have one file, :16R::GENL :20C::RELA//SET//ABC123456 :22F::XYZYESR :20C::MITI//NETT/QWERTY12345 :16S::GENL :16R::GENL :20C::RELA//SET//XYZ23456 :22F::XYZYESR :16S::GENL The requirement is, if :20C::MITI// is present in any block, then replace the data of :20C::MITI// in... (8 Replies)
Discussion started by: Soumyadip Dutta
8 Replies

2. Shell Programming and Scripting

Extracting data between two tag pairs

In a huge log file (43MB, 43k lines) I am trying to extract data between two tag pairs on same line and export it to a file so I can pull it into Excel for a report. One Pair is <Text>data I need</Text> Other pair follows on same line and is <TimeStamp>more data I need</TimeStamp> I would need... (2 Replies)
Discussion started by: NanookArctic
2 Replies

3. Shell Programming and Scripting

Extracting part of data from files

Hi All, I have log files as below. log1.txt <table name="content_analyzer" primary-key="id"> <type="global" /> </table> <table name="content_analyzer2" primary-key="id"> <type="global" /> </table> Time taken: 1.008 seconds ID = gd54321bbvbvbcvb <table name="content_analyzer"... (7 Replies)
Discussion started by: ROCK_PLSQL
7 Replies

4. Shell Programming and Scripting

Extracting data from rsync log

Hi I have a daily rsync that runs and i am trying to find a easy way of extracting the start time and end time of the sync and extract the details of how much data was copied. I would like to use this information to see if i can increase the amount of rsyncs that run in a day. so the log... (3 Replies)
Discussion started by: treds
3 Replies

5. Shell Programming and Scripting

Error while extracting data from log file

I am running awk command to extract data from log file to calculate last 15 minutes log using below command and now i am getting bellow error: awk '$0>=$from' from=$(`date -u +"####<%d-%b-%Y %H:%M:%S o'clock GMT>"-15min`) test.log Error: date: 0551-402 Invalid character in date/time... (8 Replies)
Discussion started by: oberoi1403
8 Replies

6. Programming

Python script for extracting data using two files

Hello, I have two files. File 1 is a list of interested IDs Ex1 Ex2 Ex3File 2 is the original file with over 8000 columns and 20 millions rows and is a compressed file .gz Ex1 xx xx xx xx .... Ex2 xx xx xx xx .... Ex2 xx xx xx xx ....Now I need to extract the information for all the IDs of... (4 Replies)
Discussion started by: nans
4 Replies

7. Shell Programming and Scripting

awk - extracting data from a series of files

Hi, I am trying to extract data from multiple output files. I am able to extract the data from a single output file by using the following awk commands: awk '/ test-file*/{print;m=0}' out1.log > out1a.txt awk '/ test-string/{m=1;c=0}m&&++c==3{print $2 " " $3 " " $4 ;m=0}' out1.log >... (12 Replies)
Discussion started by: p_sun
12 Replies

8. Shell Programming and Scripting

Help- counting delimiter in a huge file and split data into 2 files

I’m new to Linux script and not sure how to filter out bad records from huge flat files (over 1.3GB each). The delimiter is a semi colon “;” Here is the sample of 5 lines in the file: Name1;phone1;address1;city1;state1;zipcode1 Name2;phone2;address2;city2;state2;zipcode2;comment... (7 Replies)
Discussion started by: lv99
7 Replies

9. UNIX for Dummies Questions & Answers

Extracting data from many compressed files

I have a large number (50,000) of pretty large compressed files and I need only certain lines of data from them (each relevant line contains a certain key word). Each file contains 300 such lines. The individual file names are indexed by file number (file_name.1, file_name.2, ... ,... (1 Reply)
Discussion started by: Boltzmann
1 Replies

10. Shell Programming and Scripting

extracting data from files..

frnds, I m having prob woth doing some 2-3 task simultaneously... what I want is... I have lots ( lacs ) of files in a dir... I want.. these info from arround 2-3 months files filename convention is - abc20080403sdas.xyz ( for todays files ) I want 1. total no of files for 1 dec... (1 Reply)
Discussion started by: clx
1 Replies
Login or Register to Ask a Question