Sponsored Content
Full Discussion: Parsing log with sed problem
Top Forums Shell Programming and Scripting Parsing log with sed problem Post 302485644 by migurus on Wednesday 5th of January 2011 09:01:24 PM
Old 01-05-2011
Parsing log with sed problem

I have huge logs that have sql statements in them and I need to parse those statements out.

Log sample:
Code:
OUT1: #STMT# [SELECT X FROM P WHERE A=1] from: [SELECT X FROM P WHERE A=?]
[E1 Fine]: 2010.12.20 14:01:16.357--DatabaseSessionImpl(17489534)--Connection(11145468)--Thread(Thread[AWT-EventQueue-0,6,main])--SELECT GETDATE()
[E2 Finer]: 2010.12.20 14:01:16.357--UnitOfWork(12167196)--Thread(Thread[AWT-EventQueue-0,6,main])--begin unit of work commit
[E3 Finer]: 2010.12.20 14:01:16.357--DatabaseSessionImpl(17489534)--Connection(11145468)--Thread(Thread[AWT-EventQueue-0,6,main])--begin transaction
[E4 Fine]: 2010.12.20 14:01:16.357--DatabaseSessionImpl(17489534)--Connection(11145468)--Thread(Thread[AWT-EventQueue-0,6,main])--UPDATE S SET CNT=CNT+1 WHERE SNAME='ABC'
[E5 Fine]: 2010.12.20 14:01:16.373--DatabaseSessionImpl(17489534)--Connection(11145468)--Thread(Thread[AWT-EventQueue-0,6,main])--SELECT CNT FROM S WHERE SNAME='ABC'
[E6 Finer]: 2010.12.20 14:01:16.373--DatabaseSessionImpl(17489534)--Connection(11145468)--Thread(Thread[AWT-EventQueue-0,6,main])--commit transaction

I'd like to get
Code:
SELECT X FROM P WHERE A=1
SELECT GETDATE()
UPDATE S SET CNT=CNT+1 WHERE SNAME='ABC'
SELECT CNT FROM S WHERE SNAME='ABC'

my script:
Code:
#!/bin/ksh
sed -n  -e '/\-\-SELECT/  s/^.*\-\-//p' \
        -e '/\-\-UPDATE/  s/^.*\-\-//p' \
        -e '/\-\-DELETE/  s/^.*\-\-//p' \
        -e '/#STMT#/    s/.*# \[//;s/\].*//p'   $1

produces this
Code:
SELECT X FROM P WHERE A=1
SELECT GETDATE()
[E2 Finer
[E3 Finer
UPDATE S SET CNT=CNT+1 WHERE SNAME='ABC'
SELECT CNT FROM S WHERE SNAME='ABC'
[E6 Finer

What would be suggestions of sed gurus to get clean sql statements. Please note: sed is much preferred in my environment.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk sed parsing

hi , i would like to parse some file with the fallowing data : data data data "unwanted data" data data "unwanted data" data data data data #unwanted data. what i want it to have any coments between "" and after # to be erased using awk or/and sed. has anyone an idea? thanks. (3 Replies)
Discussion started by: Darsh
3 Replies

2. Shell Programming and Scripting

Sed parsing error

I'm having a problem with a sed script. A programmer needs to change columns 942,943,944 to blank spaces only where it has the number 999 in every line. I didn't have a copy of the data file to test with originally so made my own up with a bunch of x's and put 999 in columns 5-7. The sed... (1 Reply)
Discussion started by: gravy26
1 Replies

3. Shell Programming and Scripting

Another parsing line awk or sed problem

Hi, After looking on different forums, I'm still in trouble to parse a parameters line received in KSH. $* is equal to "/AAA:111 /BBB:222 /CCC:333 /DDD:444" I would like to parse it and be able to access anyone from his name in my KSH after. like echo myArray => display 111 ... (1 Reply)
Discussion started by: RickTrader
1 Replies

4. Shell Programming and Scripting

Parsing with awk or sed

I want to delete corrupt records from a file through awk or sed. Can anyone help me with this Thanks Striker Change subject to a descriptive one, ty. (1 Reply)
Discussion started by: Rahul_us
1 Replies

5. Shell Programming and Scripting

Parsing cron with sed

Hello I want to convert my cron list into a csv Can you please help me with sed ? eg: Convert #06,21,36,51 * * 1,2 * (. ~/.profile ; timex /some/path/script -30 -15) >> /some/path/logfile2 2>&1 * * * * * (. ~/.profile ; timex /some/path/script2) > /some/path/logfile2 To:... (1 Reply)
Discussion started by: drbiloukos
1 Replies

6. Shell Programming and Scripting

sed (parsing value)

All, Can somebody provide me with some sed expertise on how to parse the following line. 27-MAR-2011 10:28:01 * (CONNECT_DATA=(SID=dmart)(CID=(PROGRAM=sqlplus)(HOST=mtasnprod1)(USER=mtasnord))) * (ADDRESS=(PROTOCOL=tcp)(HOST=10.197.7.47)(PORT=54881)) * establish * dmart * 0 I would like... (3 Replies)
Discussion started by: BeefStu
3 Replies

7. Shell Programming and Scripting

Sed special parsing

What is the shortest & right way to remove the string "" with a sed statement ? echo 'whateverwhatever' | sed ........ ? :) (2 Replies)
Discussion started by: ctsgnb
2 Replies

8. Shell Programming and Scripting

[SED] Parsing to get a single value

Hello guys, I guess you are fed up with sed command and parse questions, but after a while researching the forum, I could not get an answer to my doubt. I know it must be easy done with sed command, but unfortunately, I never get right syntax of this command OK, this is what I have in my... (3 Replies)
Discussion started by: manolain
3 Replies

9. UNIX for Dummies Questions & Answers

sed or Grep Parsing

I would like to parse two strings from lines in a file only when both strings appear on the same line. For example, if I have the following line: string1 string2 string3 string4 string5 string6 string7 string8 string9 I would like the output to be: string2: string7 Can someone give me... (5 Replies)
Discussion started by: ARBlue79
5 Replies

10. Shell Programming and Scripting

Parsing via sed issue

sorry I messed up the last post with too many mistakes and corrections so I closed it and opening a new one which should be clear to everyone .my apologies to the admins. I am using sun solaris and Linux , what I want is SED to print any string (or output it to a file preferably) that does... (2 Replies)
Discussion started by: boncuk
2 Replies
CREATE 
RULE(7) PostgreSQL 9.2.7 Documentation CREATE RULE(7) NAME
CREATE_RULE - define a new rewrite rule SYNOPSIS
CREATE [ OR REPLACE ] RULE name AS ON event TO table_name [ WHERE condition ] DO [ ALSO | INSTEAD ] { NOTHING | command | ( command ; command ... ) } DESCRIPTION
CREATE RULE defines a new rule applying to a specified table or view. CREATE OR REPLACE RULE will either create a new rule, or replace an existing rule of the same name for the same table. The PostgreSQL rule system allows one to define an alternative action to be performed on insertions, updates, or deletions in database tables. Roughly speaking, a rule causes additional commands to be executed when a given command on a given table is executed. Alternatively, an INSTEAD rule can replace a given command by another, or cause a command not to be executed at all. Rules are used to implement table views as well. It is important to realize that a rule is really a command transformation mechanism, or command macro. The transformation happens before the execution of the commands starts. If you actually want an operation that fires independently for each physical row, you probably want to use a trigger, not a rule. More information about the rules system is in Chapter 37, The Rule System, in the documentation. Presently, ON SELECT rules must be unconditional INSTEAD rules and must have actions that consist of a single SELECT command. Thus, an ON SELECT rule effectively turns the table into a view, whose visible contents are the rows returned by the rule's SELECT command rather than whatever had been stored in the table (if anything). It is considered better style to write a CREATE VIEW command than to create a real table and define an ON SELECT rule for it. You can create the illusion of an updatable view by defining ON INSERT, ON UPDATE, and ON DELETE rules (or any subset of those that's sufficient for your purposes) to replace update actions on the view with appropriate updates on other tables. If you want to support INSERT RETURNING and so on, then be sure to put a suitable RETURNING clause into each of these rules. Alternatively, an updatable view can be implemented using INSTEAD OF triggers (see CREATE TRIGGER (CREATE_TRIGGER(7))). There is a catch if you try to use conditional rules for view updates: there must be an unconditional INSTEAD rule for each action you wish to allow on the view. If the rule is conditional, or is not INSTEAD, then the system will still reject attempts to perform the update action, because it thinks it might end up trying to perform the action on the dummy table of the view in some cases. If you want to handle all the useful cases in conditional rules, add an unconditional DO INSTEAD NOTHING rule to ensure that the system understands it will never be called on to update the dummy table. Then make the conditional rules non-INSTEAD; in the cases where they are applied, they add to the default INSTEAD NOTHING action. (This method does not currently work to support RETURNING queries, however.) PARAMETERS
name The name of a rule to create. This must be distinct from the name of any other rule for the same table. Multiple rules on the same table and same event type are applied in alphabetical name order. event The event is one of SELECT, INSERT, UPDATE, or DELETE. table_name The name (optionally schema-qualified) of the table or view the rule applies to. condition Any SQL conditional expression (returning boolean). The condition expression cannot refer to any tables except NEW and OLD, and cannot contain aggregate functions. INSTEAD INSTEAD indicates that the commands should be executed instead of the original command. ALSO ALSO indicates that the commands should be executed in addition to the original command. If neither ALSO nor INSTEAD is specified, ALSO is the default. command The command or commands that make up the rule action. Valid commands are SELECT, INSERT, UPDATE, DELETE, or NOTIFY. Within condition and command, the special table names NEW and OLD can be used to refer to values in the referenced table. NEW is valid in ON INSERT and ON UPDATE rules to refer to the new row being inserted or updated. OLD is valid in ON UPDATE and ON DELETE rules to refer to the existing row being updated or deleted. NOTES
You must be the owner of a table to create or change rules for it. In a rule for INSERT, UPDATE, or DELETE on a view, you can add a RETURNING clause that emits the view's columns. This clause will be used to compute the outputs if the rule is triggered by an INSERT RETURNING, UPDATE RETURNING, or DELETE RETURNING command respectively. When the rule is triggered by a command without RETURNING, the rule's RETURNING clause will be ignored. The current implementation allows only unconditional INSTEAD rules to contain RETURNING; furthermore there can be at most one RETURNING clause among all the rules for the same event. (This ensures that there is only one candidate RETURNING clause to be used to compute the results.) RETURNING queries on the view will be rejected if there is no RETURNING clause in any available rule. It is very important to take care to avoid circular rules. For example, though each of the following two rule definitions are accepted by PostgreSQL, the SELECT command would cause PostgreSQL to report an error because of recursive expansion of a rule: CREATE RULE "_RETURN" AS ON SELECT TO t1 DO INSTEAD SELECT * FROM t2; CREATE RULE "_RETURN" AS ON SELECT TO t2 DO INSTEAD SELECT * FROM t1; SELECT * FROM t1; Presently, if a rule action contains a NOTIFY command, the NOTIFY command will be executed unconditionally, that is, the NOTIFY will be issued even if there are not any rows that the rule should apply to. For example, in: CREATE RULE notify_me AS ON UPDATE TO mytable DO ALSO NOTIFY mytable; UPDATE mytable SET name = 'foo' WHERE id = 42; one NOTIFY event will be sent during the UPDATE, whether or not there are any rows that match the condition id = 42. This is an implementation restriction that might be fixed in future releases. COMPATIBILITY
CREATE RULE is a PostgreSQL language extension, as is the entire query rewrite system. PostgreSQL 9.2.7 2014-02-17 CREATE RULE(7)
All times are GMT -4. The time now is 05:51 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy