Sponsored Content
Top Forums Shell Programming and Scripting Extracting Delimiter 'TAG' Data From log files Post 302793313 by Buddyluv on Friday 12th of April 2013 04:05:19 AM
Old 04-12-2013
Linux Extracting Delimiter 'TAG' Data From log files

Hi
I am trying to extract data from within a log file and output format to a new file for further manipulation can someone provide script to do this?

For example I have a file as below and just want to extract all delimited variances of tag 32=* up to the delimiter "|" and output to a new file so that I can calculate the total of these values.
Code:
$cat fix1.log

FIX4.4 |0=12|55=LIT.L|48=123456|32=5|38=100|10=200|
FIX4.4 |0=12|55=LIT.L|48=123456|32=15|38=100|10=200|
FIX4.4 |0=12|55=LIT.L|48=123456|32=115|38=100|10=200|
FIX4.4 |0=12|55=LIT.L|48=123456|32=500|38=100|10=200|
FIX4.4 |0=12|55=LIT.L|48=123456|32=2|38=100|10=200|110=233
FIX4.4 |0=12|55=LIT.L|48=123456|32=5|38=100|10=200|110=221
FIX4.4 |0=12|55=LIT.L|48=123456|32=5|38=100|10=200|775=1
FIX4.4 |0=12|55=LIT.L|48=123456||35=1|34=52|32=2|38=100|10=200|
FIX4.4 |0=12|55=LIT.L|48=123456|35=0|32=5|38=100|10=200|
FIX4.4 |0=12|55=LIT.L|48=123456|35=5|32=4|38=100|10=200|

As you can see the delimited tag 32 is not always in the same position within the file and ideally I would like the output to consist of just the individual numeric values after the 32=
Code:
5
15
115
500

etc etc

Assistance appreciated

Buddy

Last edited by vbe; 04-12-2013 at 05:34 AM.. Reason: code tags please...
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

extracting data from files..

frnds, I m having prob woth doing some 2-3 task simultaneously... what I want is... I have lots ( lacs ) of files in a dir... I want.. these info from arround 2-3 months files filename convention is - abc20080403sdas.xyz ( for todays files ) I want 1. total no of files for 1 dec... (1 Reply)
Discussion started by: clx
1 Replies

2. UNIX for Dummies Questions & Answers

Extracting data from many compressed files

I have a large number (50,000) of pretty large compressed files and I need only certain lines of data from them (each relevant line contains a certain key word). Each file contains 300 such lines. The individual file names are indexed by file number (file_name.1, file_name.2, ... ,... (1 Reply)
Discussion started by: Boltzmann
1 Replies

3. Shell Programming and Scripting

Help- counting delimiter in a huge file and split data into 2 files

I’m new to Linux script and not sure how to filter out bad records from huge flat files (over 1.3GB each). The delimiter is a semi colon “;” Here is the sample of 5 lines in the file: Name1;phone1;address1;city1;state1;zipcode1 Name2;phone2;address2;city2;state2;zipcode2;comment... (7 Replies)
Discussion started by: lv99
7 Replies

4. Shell Programming and Scripting

awk - extracting data from a series of files

Hi, I am trying to extract data from multiple output files. I am able to extract the data from a single output file by using the following awk commands: awk '/ test-file*/{print;m=0}' out1.log > out1a.txt awk '/ test-string/{m=1;c=0}m&&++c==3{print $2 " " $3 " " $4 ;m=0}' out1.log >... (12 Replies)
Discussion started by: p_sun
12 Replies

5. Programming

Python script for extracting data using two files

Hello, I have two files. File 1 is a list of interested IDs Ex1 Ex2 Ex3File 2 is the original file with over 8000 columns and 20 millions rows and is a compressed file .gz Ex1 xx xx xx xx .... Ex2 xx xx xx xx .... Ex2 xx xx xx xx ....Now I need to extract the information for all the IDs of... (4 Replies)
Discussion started by: nans
4 Replies

6. Shell Programming and Scripting

Error while extracting data from log file

I am running awk command to extract data from log file to calculate last 15 minutes log using below command and now i am getting bellow error: awk '$0>=$from' from=$(`date -u +"####<%d-%b-%Y %H:%M:%S o'clock GMT>"-15min`) test.log Error: date: 0551-402 Invalid character in date/time... (8 Replies)
Discussion started by: oberoi1403
8 Replies

7. Shell Programming and Scripting

Extracting data from rsync log

Hi I have a daily rsync that runs and i am trying to find a easy way of extracting the start time and end time of the sync and extract the details of how much data was copied. I would like to use this information to see if i can increase the amount of rsyncs that run in a day. so the log... (3 Replies)
Discussion started by: treds
3 Replies

8. Shell Programming and Scripting

Extracting part of data from files

Hi All, I have log files as below. log1.txt <table name="content_analyzer" primary-key="id"> <type="global" /> </table> <table name="content_analyzer2" primary-key="id"> <type="global" /> </table> Time taken: 1.008 seconds ID = gd54321bbvbvbcvb <table name="content_analyzer"... (7 Replies)
Discussion started by: ROCK_PLSQL
7 Replies

9. Shell Programming and Scripting

Extracting data between two tag pairs

In a huge log file (43MB, 43k lines) I am trying to extract data between two tag pairs on same line and export it to a file so I can pull it into Excel for a report. One Pair is <Text>data I need</Text> Other pair follows on same line and is <TimeStamp>more data I need</TimeStamp> I would need... (2 Replies)
Discussion started by: NanookArctic
2 Replies

10. Shell Programming and Scripting

Find a tag with data and replace its data in another tag

Hi I have one file, :16R::GENL :20C::RELA//SET//ABC123456 :22F::XYZYESR :20C::MITI//NETT/QWERTY12345 :16S::GENL :16R::GENL :20C::RELA//SET//XYZ23456 :22F::XYZYESR :16S::GENL The requirement is, if :20C::MITI// is present in any block, then replace the data of :20C::MITI// in... (8 Replies)
Discussion started by: Soumyadip Dutta
8 Replies
DOVEADM-LOG(1)							      Dovecot							    DOVEADM-LOG(1)

NAME
doveadm-log - Locate, test or reopen Dovecot's log files SYNOPSIS
doveadm [-Dv] log errors [-s min_timestamp] doveadm [-Dv] log find [directory] doveadm [-Dv] log reopen doveadm [-Dv] log test DESCRIPTION
The doveadm log commands are used to locate and reopen the log files of dovecot(1). It's also possible to test the configured targets of the *log_path settings. OPTIONS
Global doveadm(1) options: -D Enables verbosity and debug messages. -v Enables verbosity, including progress counter. COMMANDS
log errors doveadm log errors [-s min_timestamp] The log errors command is used to show the last - up to 1,000 - errors and warnings. If no output is generated, no errors have occurred since the last start. -s min_timestamp An integer value, representing seconds since the epoch - also known as Unix timestamp. When a min_timestamp was given, doveadm(1) will only show errors occurred since that point in time. log find doveadm log find [directory] The log find command is used to show the location of the log files, to which dovecot(1) sends its log messages. If dovecot(1) logs its messages through syslogd(8) and doveadm(1) could not find any log files, you can specify the directory where your syslogd writes its log files. log reopen doveadm log reopen This command causes doveadm to reopen all log files, configured in the log_path, info_log_path and debug_log_path settings. These settings are configured in /etc/dovecot/conf.d/10-logging.conf. This is for example useful after manually rotating the log files. log test doveadm log test This command causes doveadm to write the message "This is Dovecot's priority log (timestamp)" to the configured log files. The used prior- ities are: debug, info, warning, error and fatal. EXAMPLE
This example shows how to locate the log files used by dovecot(1). doveadm log find Looking for log files from /var/log Debug: /var/log/dovecot.debug Info: /var/log/mail.log Warning: /var/log/mail.log Error: /var/log/mail.log Fatal: /var/log/mail.log REPORTING BUGS
Report bugs, including doveconf -n output, to the Dovecot Mailing List <dovecot@dovecot.org>. Information about reporting bugs is avail- able at: http://dovecot.org/bugreport.html SEE ALSO
doveadm(1) Dovecot v2.2 2013-11-24 DOVEADM-LOG(1)
All times are GMT -4. The time now is 11:22 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy