Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Parsing a log file to cut off some parts
Top Forums Shell Programming and Scripting Parsing a log file to cut off some parts Post 302989226 by dae on Tuesday 10th of January 2017 06:14:46 AM
Old 01-10-2017
Parsing a log file to cut off some parts
Dear all,

I would like to use SQL's log file to extract information from it.

This file can include four different types of instruction with the number of lines involved for each of them:

-> (1) "INSERT" instruction with the number of lines inserted
-> (2) "UPDATE" instruction with the number of lines updated
-> (3) "DELETE" instruction with the number of lines deleted
-> (4) "COLLECT STATISTICS" instruction with the number of lines involved.

The number of lines involved for "UPDATE" and "COLLECT STATISTICS" has exactly the same appearance in the log file:

-> for "COLLECT STATISTICS" instruction (check "Update completed" below)

Code:
COLLECT STATISTICS           COLUMN (STD_CDH_ID)
                           , COLUMN (PARTITION)
                           , COLUMN (METDVERS_OID_ID)
         -- AXE ITEMGROUP
                           , COLUMN (ITEMGROUP_OID_CD)
                           , COLUMN (STD_CDH_ID, METDVERS_OID_ID, ITEMGROUP_OID_CD)
                           -- AXE VISIT
                           , COLUMN (VISIT_OID_CD)
                           , COLUMN (STD_CDH_ID, METDVERS_OID_ID, VISIT_OID_CD)
         -- AXE FORM
                           , COLUMN (FORM_OID_CD)
                           , COLUMN (STD_CDH_ID, METDVERS_OID_ID, FORM_OID_CD)
         -- AXE SITE
                           , COLUMN (SITE_OID_CD)
                           , COLUMN (STD_CDH_ID, METDVERS_OID_ID, SITE_OID_CD)
       -- AXE SUBJCT
                           , COLUMN (SUBJCT_KEY_ID)
                           , COLUMN (STD_CDH_ID, METDVERS_OID_ID, SUBJCT_KEY_ID)

ON                           CDH_STGN_DEV1.RECRD_SHREDDED
;

 *** Update completed. 14 rows changed.
 *** Total elapsed time was 4 seconds.

-> for "UPDATE" instruction (check "Update completed" below)

Code:
UPDATE
 TGT
FROM
 CDH_REJT_DEV1.RAVE_LNKITEMGROUPITEM TGT
, CDH_STGN_DEV1.LNKITEMGROUPITEM SRC
SET
 LNKITEMGROUPITEM_REJTDELTD_TS = CURRENT_TIMESTAMP(0)
WHERE
 TGT.STD_CDH_ID   = SRC.STD_CDH_ID
AND TGT.METDVERS_OID_ID  = SRC.METDVERS_OID_ID
AND TGT.ITEM_OID_CD   = SRC.ITEM_OID_CD
AND TGT.LNKITEMGROUPITEM_REJTDELTD_TS IS NULL
;

 *** Update completed. No rows changed.
 *** Total elapsed time was 1 second.



The problem I face is the following: I am not interested by the number of lines involved in the "COLLECT STATISTICS" instruction and I am not able to recognize from the log file what kind of expression - "COLLECT STATISTICS" or "UPDATE" - produced "Update completed" in the log file !

So, I thought to cut off from the log file the multi-line record starting with "COLLECT STATISTICS" and ending with "changed." ... but I failed, I failed, I failed !

Thanks in advance for the attention you pay to my request and obviously, any suggestion over welcomed,

Didier.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Parsing a Log file

Hi All, I'm deffently not a Unix specialist so be Gentel. I need to parse a Log file that looks like that: 2006-06-12 01:00:00,463 ERROR {cleanLoggersFiles} General Error comverse.compas.shared.exceptions.SystemParametersException: Error in reading parameter FileLocation at... (4 Replies)
Discussion started by: tbirenzweig
4 Replies

2. Shell Programming and Scripting

Parsing textfile problem with cut

Hi, I have a textfile with several lines like this: text num: USER text (num) num num I need all these stuff. Problem is, how to get these stuff after ":". USER is a username and all chars are possible, even whitespace. So I cant use cut. Any ideas? (3 Replies)
Discussion started by: mcW
3 Replies

3. UNIX for Dummies Questions & Answers

parsing a log file

I need help in parsing the following log files. 10 Apr 2009 0:16:16 * name: Tuna Belly Format: Well done, Price: 999 only 10 Apr 2009 0:16:16 * name: Roast Beef Format: Raw, Price: 55 c 10 Apr 2009 0:16:16 * name: Pasta Format: Dry, Price: 88.43 only etcetc I need to parse this... (8 Replies)
Discussion started by: izuma
8 Replies

4. UNIX for Dummies Questions & Answers

How to cut a string in two parts and show the other part

hi everybody.. I have a string like : abcd:efgh xxyy:yyxx ssddf:kjlioi ghtyu:jkksk nhjkk:heuiiue please tell me how i can display only the characters after ":" in the output the output should be : efgh yyxx kjlioi jkksk heuiiue please give quick reply.. its urgent..!! (6 Replies)
Discussion started by: adityamitra
6 Replies

5. Shell Programming and Scripting

Help Parsing a Log File

Hello all, I am new to scripting and I have written a script that performs an Rsync on my NAS and then moves on to send me an email with the status etc. The problem is that I think Rsync is taking to long to complete and the IF statement is timing out, as it doesn't appear to move on. Here... (1 Reply)
Discussion started by: Mongrel
1 Replies

6. Shell Programming and Scripting

Cut the path into two parts

Hi, file=/usr/lib I need to cut and put it into two variable like string1=/usr string2=lib I made it for string2 string2=${file#/*/} How to get String1 in the same way which I have get string2. Use even more code tags ;) (4 Replies)
Discussion started by: munna_dude
4 Replies

7. Shell Programming and Scripting

Parsing Log File help

Hi, I am a newbie to scripting. I have multiple log files (saved as .gz) in a directory that looks like this 01-01-2013 10:00 pn: 123 01-01-2013 10:00 sn: 987 01-01-2013 10:00 Test1 01-01-2013 10:00 Result: Pass 01-01-2013 10:00 Time: 5:00 01-01-2013 10:00 Test2 01-01-2013 10:00... (3 Replies)
Discussion started by: linuxnew
3 Replies

8. Shell Programming and Scripting

Incrementing parts of ten digits number by parts

I have number in file which contains date and serial number: 2013101000. The last two digits are serial number (00). So maximum of serial number is 100. After reaching 100 it becomes 00 with incrementing 10 which is day with max 31. after reaching 31 it becomes 00 and increments 10... (31 Replies)
Discussion started by: Natalie
31 Replies

9. Shell Programming and Scripting

HELP on parsing this log file

Hi, I have a log file that looks like below and I am wanting to know if there is a better way of parsing it from how I am doing it right now. I am looking for when an application service is OFFLINE and ONLINE. This log file is getting written into every 30 minutes ... (1 Reply)
Discussion started by: newbie_01
1 Replies

10. Shell Programming and Scripting

Getting various parts from the log

I am dealing with some app log, see example below: 22:16:13.601 ClientSession(905)--Connection(5)--SELECT GETDATE() 22:16:13.632 ClientSession(158)--Connection(5)--SELECT 1 22:16:13.632 ClientSession(848)--Connection(6735)--SELECT 1 So far I needed to collect certain column from it, such as... (3 Replies)
Discussion started by: migurus
3 Replies

LEARN ABOUT DEBIAN

cmannotifyd

cmannotifyd(8)						     CMAN Notification Daemon						    cmannotifyd(8)

NAME

       cmannotifyd - CMAN Notification Daemon

SYNOPSIS

       cmannotifyd [-f] [-d]

DESCRIPTION

       The cmannotifyd daemon talks to CMAN and provides a mechanism to notify external entities about cluster changes.

       CMAN dispatches 3 kind of notifications:

       - CMAN_REASON_TRY_SHUTDOWN when cman requests to all clients if it is allowed to shutdown.

       - CMAN_REASON_STATECHANGE when cman detects a node joining or leaving the cluster.

       - CMAN_REASON_CONFIG_UPDATE when a configuration change event has been detected/requested.

       These notifications are then dispatched to the shell script cman_notify in the environment variable CMAN_NOTIFICATION.

       cman_notify will then execute all the scripts in the configured notification directory (default: /etc/cluster/cman-notify.d) passing a very
       minimal set of envvars including, of course, the CMAN_NOTIFICATION= type.  The execution order is set by the filename as shown by "LC_ALL=C
       ls -las".

       cmannotifyd logs are stored in the default log file (/var/log/cluster/cmannotifyd.log).

       cman_notify  logs are stored in the default log file (/var/log/cluster/cman_notify.log). By default the output from the scripts executed by
       cman_notify is redirected to /dev/null.	Users can either set CMAN_NOTIFICATION_DEBUG=1 in their environment or set proper debug configura-
       tion in cluster.conf to redirect scripts output to the cman_notify log file.

NOTES

       cmannotifyd does not block on cman_notify nor check the exit status of the script.

       Notifications are dispatched in the same order as they arrive, one by one.

       CMAN_REASON_TRY_SHUTDOWN  is  passed  to  scripts  for information only, they can not influence cman's decsion about whether or not to shut
       down.

       CMAN_REASON_STATECHANGE also implies CMAN_NOTIFICATION_QUORUM exported in the environment. CMAN_NOTIFICATION_QUORUM will be set to 1  (when
       the node is part of a quorate cluster) or 0 (otherwise).

       A template for cman_notify scripts can be found in the doc/ directory.

OPTIONS

       -f     Run in the foreground (do not fork / daemonize).

       -d     Enable debug output.

								   November 2008						    cmannotifyd(8)
All times are GMT -4. The time now is 12:02 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy