Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Parsing a log file to cut off some parts
Top Forums Shell Programming and Scripting Parsing a log file to cut off some parts Post 302989226 by dae on Tuesday 10th of January 2017 06:14:46 AM
Old 01-10-2017
Parsing a log file to cut off some parts
Dear all,

I would like to use SQL's log file to extract information from it.

This file can include four different types of instruction with the number of lines involved for each of them:

-> (1) "INSERT" instruction with the number of lines inserted
-> (2) "UPDATE" instruction with the number of lines updated
-> (3) "DELETE" instruction with the number of lines deleted
-> (4) "COLLECT STATISTICS" instruction with the number of lines involved.

The number of lines involved for "UPDATE" and "COLLECT STATISTICS" has exactly the same appearance in the log file:

-> for "COLLECT STATISTICS" instruction (check "Update completed" below)

Code:
COLLECT STATISTICS           COLUMN (STD_CDH_ID)
                           , COLUMN (PARTITION)
                           , COLUMN (METDVERS_OID_ID)
         -- AXE ITEMGROUP
                           , COLUMN (ITEMGROUP_OID_CD)
                           , COLUMN (STD_CDH_ID, METDVERS_OID_ID, ITEMGROUP_OID_CD)
                           -- AXE VISIT
                           , COLUMN (VISIT_OID_CD)
                           , COLUMN (STD_CDH_ID, METDVERS_OID_ID, VISIT_OID_CD)
         -- AXE FORM
                           , COLUMN (FORM_OID_CD)
                           , COLUMN (STD_CDH_ID, METDVERS_OID_ID, FORM_OID_CD)
         -- AXE SITE
                           , COLUMN (SITE_OID_CD)
                           , COLUMN (STD_CDH_ID, METDVERS_OID_ID, SITE_OID_CD)
       -- AXE SUBJCT
                           , COLUMN (SUBJCT_KEY_ID)
                           , COLUMN (STD_CDH_ID, METDVERS_OID_ID, SUBJCT_KEY_ID)

ON                           CDH_STGN_DEV1.RECRD_SHREDDED
;

 *** Update completed. 14 rows changed.
 *** Total elapsed time was 4 seconds.

-> for "UPDATE" instruction (check "Update completed" below)

Code:
UPDATE
 TGT
FROM
 CDH_REJT_DEV1.RAVE_LNKITEMGROUPITEM TGT
, CDH_STGN_DEV1.LNKITEMGROUPITEM SRC
SET
 LNKITEMGROUPITEM_REJTDELTD_TS = CURRENT_TIMESTAMP(0)
WHERE
 TGT.STD_CDH_ID   = SRC.STD_CDH_ID
AND TGT.METDVERS_OID_ID  = SRC.METDVERS_OID_ID
AND TGT.ITEM_OID_CD   = SRC.ITEM_OID_CD
AND TGT.LNKITEMGROUPITEM_REJTDELTD_TS IS NULL
;

 *** Update completed. No rows changed.
 *** Total elapsed time was 1 second.



The problem I face is the following: I am not interested by the number of lines involved in the "COLLECT STATISTICS" instruction and I am not able to recognize from the log file what kind of expression - "COLLECT STATISTICS" or "UPDATE" - produced "Update completed" in the log file !

So, I thought to cut off from the log file the multi-line record starting with "COLLECT STATISTICS" and ending with "changed." ... but I failed, I failed, I failed !

Thanks in advance for the attention you pay to my request and obviously, any suggestion over welcomed,

Didier.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Parsing a Log file

Hi All, I'm deffently not a Unix specialist so be Gentel. I need to parse a Log file that looks like that: 2006-06-12 01:00:00,463 ERROR {cleanLoggersFiles} General Error comverse.compas.shared.exceptions.SystemParametersException: Error in reading parameter FileLocation at... (4 Replies)
Discussion started by: tbirenzweig
4 Replies

2. Shell Programming and Scripting

Parsing textfile problem with cut

Hi, I have a textfile with several lines like this: text num: USER text (num) num num I need all these stuff. Problem is, how to get these stuff after ":". USER is a username and all chars are possible, even whitespace. So I cant use cut. Any ideas? (3 Replies)
Discussion started by: mcW
3 Replies

3. UNIX for Dummies Questions & Answers

parsing a log file

I need help in parsing the following log files. 10 Apr 2009 0:16:16 * name: Tuna Belly Format: Well done, Price: 999 only 10 Apr 2009 0:16:16 * name: Roast Beef Format: Raw, Price: 55 c 10 Apr 2009 0:16:16 * name: Pasta Format: Dry, Price: 88.43 only etcetc I need to parse this... (8 Replies)
Discussion started by: izuma
8 Replies

4. UNIX for Dummies Questions & Answers

How to cut a string in two parts and show the other part

hi everybody.. I have a string like : abcd:efgh xxyy:yyxx ssddf:kjlioi ghtyu:jkksk nhjkk:heuiiue please tell me how i can display only the characters after ":" in the output the output should be : efgh yyxx kjlioi jkksk heuiiue please give quick reply.. its urgent..!! (6 Replies)
Discussion started by: adityamitra
6 Replies

5. Shell Programming and Scripting

Help Parsing a Log File

Hello all, I am new to scripting and I have written a script that performs an Rsync on my NAS and then moves on to send me an email with the status etc. The problem is that I think Rsync is taking to long to complete and the IF statement is timing out, as it doesn't appear to move on. Here... (1 Reply)
Discussion started by: Mongrel
1 Replies

6. Shell Programming and Scripting

Cut the path into two parts

Hi, file=/usr/lib I need to cut and put it into two variable like string1=/usr string2=lib I made it for string2 string2=${file#/*/} How to get String1 in the same way which I have get string2. Use even more code tags ;) (4 Replies)
Discussion started by: munna_dude
4 Replies

7. Shell Programming and Scripting

Parsing Log File help

Hi, I am a newbie to scripting. I have multiple log files (saved as .gz) in a directory that looks like this 01-01-2013 10:00 pn: 123 01-01-2013 10:00 sn: 987 01-01-2013 10:00 Test1 01-01-2013 10:00 Result: Pass 01-01-2013 10:00 Time: 5:00 01-01-2013 10:00 Test2 01-01-2013 10:00... (3 Replies)
Discussion started by: linuxnew
3 Replies

8. Shell Programming and Scripting

Incrementing parts of ten digits number by parts

I have number in file which contains date and serial number: 2013101000. The last two digits are serial number (00). So maximum of serial number is 100. After reaching 100 it becomes 00 with incrementing 10 which is day with max 31. after reaching 31 it becomes 00 and increments 10... (31 Replies)
Discussion started by: Natalie
31 Replies

9. Shell Programming and Scripting

HELP on parsing this log file

Hi, I have a log file that looks like below and I am wanting to know if there is a better way of parsing it from how I am doing it right now. I am looking for when an application service is OFFLINE and ONLINE. This log file is getting written into every 30 minutes ... (1 Reply)
Discussion started by: newbie_01
1 Replies

10. Shell Programming and Scripting

Getting various parts from the log

I am dealing with some app log, see example below: 22:16:13.601 ClientSession(905)--Connection(5)--SELECT GETDATE() 22:16:13.632 ClientSession(158)--Connection(5)--SELECT 1 22:16:13.632 ClientSession(848)--Connection(6735)--SELECT 1 So far I needed to collect certain column from it, such as... (3 Replies)
Discussion started by: migurus
3 Replies

LEARN ABOUT DEBIAN

doveadm-log

DOVEADM-LOG(1)							      Dovecot							    DOVEADM-LOG(1)

NAME

       doveadm-log - Locate, test or reopen Dovecot's log files

SYNOPSIS

       doveadm [-Dv] log errors
       doveadm [-Dv] log find [directory]
       doveadm [-Dv] log reopen
       doveadm [-Dv] log test

DESCRIPTION

       The  doveadm  log commands are used to locate and reopen the log files of dovecot(1).  It's also possible to test the configured targets of
       the *log_path settings.

OPTIONS

       Global doveadm(1) options:

       -D     Enables verbosity and debug messages.

       -v     Enables verbosity, including progress counter.

COMMANDS

   log errors
       doveadm log errors

       The log errors command is used to show the last	- up to 1,000 - errors and warnings.  If no output is generated, no errors  have  occurred
       since the last start.

   log find
       doveadm log find [directory]

       The  log  find  command	is used to show the location of the log files, to which dovecot(1) sends its log messages.  If dovecot(1) logs its
       messages through syslogd(8) and doveadm(1) could not find any log files, you can specify the directory where your syslogd  writes  its  log
       files.

   log reopen
       doveadm log reopen

       This command causes doveadm to reopen all log files, configured in the log_path, info_log_path and debug_log_path settings.  These settings
       are configured in /etc/dovecot/conf.d/10-logging.conf.
       This is for example useful after manually rotating the log files.

   log test
       doveadm log test

       This command causes doveadm to write the message "This is Dovecot's priority log (timestamp)" to the configured log files.  The used prior-
       ities are: debug, info, warning, error and fatal.

EXAMPLE

       This example shows how to locate the log files used by dovecot(1).

       doveadm log find
       Looking for log files from /var/log
       Debug: /var/log/dovecot.debug
       Info: /var/log/mail.log
       Warning: /var/log/mail.log
       Error: /var/log/mail.log
       Fatal: /var/log/mail.log

REPORTING BUGS

       Report  bugs,  including doveconf -n output, to the Dovecot Mailing List <dovecot@dovecot.org>.	Information about reporting bugs is avail-
       able at: http://dovecot.org/bugreport.html

SEE ALSO

       doveadm(1)

Dovecot v2.1							    2012-02-22							    DOVEADM-LOG(1)
All times are GMT -4. The time now is 04:23 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy