Extracting Delimiter 'TAG' Data From log files


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Extracting Delimiter 'TAG' Data From log files
# 1  
Old 04-12-2013
Linux Extracting Delimiter 'TAG' Data From log files

Hi
I am trying to extract data from within a log file and output format to a new file for further manipulation can someone provide script to do this?

For example I have a file as below and just want to extract all delimited variances of tag 32=* up to the delimiter "|" and output to a new file so that I can calculate the total of these values.
Code:
$cat fix1.log

FIX4.4 |0=12|55=LIT.L|48=123456|32=5|38=100|10=200|
FIX4.4 |0=12|55=LIT.L|48=123456|32=15|38=100|10=200|
FIX4.4 |0=12|55=LIT.L|48=123456|32=115|38=100|10=200|
FIX4.4 |0=12|55=LIT.L|48=123456|32=500|38=100|10=200|
FIX4.4 |0=12|55=LIT.L|48=123456|32=2|38=100|10=200|110=233
FIX4.4 |0=12|55=LIT.L|48=123456|32=5|38=100|10=200|110=221
FIX4.4 |0=12|55=LIT.L|48=123456|32=5|38=100|10=200|775=1
FIX4.4 |0=12|55=LIT.L|48=123456||35=1|34=52|32=2|38=100|10=200|
FIX4.4 |0=12|55=LIT.L|48=123456|35=0|32=5|38=100|10=200|
FIX4.4 |0=12|55=LIT.L|48=123456|35=5|32=4|38=100|10=200|

As you can see the delimited tag 32 is not always in the same position within the file and ideally I would like the output to consist of just the individual numeric values after the 32=
Code:
5
15
115
500

etc etc

Assistance appreciated

Buddy

Last edited by vbe; 04-12-2013 at 05:34 AM.. Reason: code tags please...
# 2  
Old 04-12-2013
One way:
Code:
 sed 's/.*32=\([0-9]*\).*/\1/' file

Guru.
This User Gave Thanks to guruprasadpr For This Post:
# 3  
Old 04-12-2013
Excellent !!

Thanks Guru
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Find a tag with data and replace its data in another tag

Hi I have one file, :16R::GENL :20C::RELA//SET//ABC123456 :22F::XYZYESR :20C::MITI//NETT/QWERTY12345 :16S::GENL :16R::GENL :20C::RELA//SET//XYZ23456 :22F::XYZYESR :16S::GENL The requirement is, if :20C::MITI// is present in any block, then replace the data of :20C::MITI// in... (8 Replies)
Discussion started by: Soumyadip Dutta
8 Replies

2. Shell Programming and Scripting

Extracting data between two tag pairs

In a huge log file (43MB, 43k lines) I am trying to extract data between two tag pairs on same line and export it to a file so I can pull it into Excel for a report. One Pair is <Text>data I need</Text> Other pair follows on same line and is <TimeStamp>more data I need</TimeStamp> I would need... (2 Replies)
Discussion started by: NanookArctic
2 Replies

3. Shell Programming and Scripting

Extracting part of data from files

Hi All, I have log files as below. log1.txt <table name="content_analyzer" primary-key="id"> <type="global" /> </table> <table name="content_analyzer2" primary-key="id"> <type="global" /> </table> Time taken: 1.008 seconds ID = gd54321bbvbvbcvb <table name="content_analyzer"... (7 Replies)
Discussion started by: ROCK_PLSQL
7 Replies

4. Shell Programming and Scripting

Extracting data from rsync log

Hi I have a daily rsync that runs and i am trying to find a easy way of extracting the start time and end time of the sync and extract the details of how much data was copied. I would like to use this information to see if i can increase the amount of rsyncs that run in a day. so the log... (3 Replies)
Discussion started by: treds
3 Replies

5. Shell Programming and Scripting

Error while extracting data from log file

I am running awk command to extract data from log file to calculate last 15 minutes log using below command and now i am getting bellow error: awk '$0>=$from' from=$(`date -u +"####<%d-%b-%Y %H:%M:%S o'clock GMT>"-15min`) test.log Error: date: 0551-402 Invalid character in date/time... (8 Replies)
Discussion started by: oberoi1403
8 Replies

6. Programming

Python script for extracting data using two files

Hello, I have two files. File 1 is a list of interested IDs Ex1 Ex2 Ex3File 2 is the original file with over 8000 columns and 20 millions rows and is a compressed file .gz Ex1 xx xx xx xx .... Ex2 xx xx xx xx .... Ex2 xx xx xx xx ....Now I need to extract the information for all the IDs of... (4 Replies)
Discussion started by: nans
4 Replies

7. Shell Programming and Scripting

awk - extracting data from a series of files

Hi, I am trying to extract data from multiple output files. I am able to extract the data from a single output file by using the following awk commands: awk '/ test-file*/{print;m=0}' out1.log > out1a.txt awk '/ test-string/{m=1;c=0}m&&++c==3{print $2 " " $3 " " $4 ;m=0}' out1.log >... (12 Replies)
Discussion started by: p_sun
12 Replies

8. Shell Programming and Scripting

Help- counting delimiter in a huge file and split data into 2 files

I’m new to Linux script and not sure how to filter out bad records from huge flat files (over 1.3GB each). The delimiter is a semi colon “;” Here is the sample of 5 lines in the file: Name1;phone1;address1;city1;state1;zipcode1 Name2;phone2;address2;city2;state2;zipcode2;comment... (7 Replies)
Discussion started by: lv99
7 Replies

9. UNIX for Dummies Questions & Answers

Extracting data from many compressed files

I have a large number (50,000) of pretty large compressed files and I need only certain lines of data from them (each relevant line contains a certain key word). Each file contains 300 such lines. The individual file names are indexed by file number (file_name.1, file_name.2, ... ,... (1 Reply)
Discussion started by: Boltzmann
1 Replies

10. Shell Programming and Scripting

extracting data from files..

frnds, I m having prob woth doing some 2-3 task simultaneously... what I want is... I have lots ( lacs ) of files in a dir... I want.. these info from arround 2-3 months files filename convention is - abc20080403sdas.xyz ( for todays files ) I want 1. total no of files for 1 dec... (1 Reply)
Discussion started by: clx
1 Replies
Login or Register to Ask a Question