MultiLine Patterns


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting MultiLine Patterns
# 1  
Old 11-08-2013
MultiLine Patterns

Experts,
I am novice unix user. At my work, most of our DBA's work on creating DDL's to create new tables in production. At every week we need to validate the scripts (do peer review) and it takes a while and also it is not effective when we have like 150 tables created in the scripts. I am working on trying to automate the validation process and need help from experts like you ! The most common problem is everyone can code the DDL in different way. For Ex:

Following SQL is expected in the script for every table:

Code:
INSERT INTO ABCXYX.PRIVILEGE_TABLE
(
 USER_NAME
,DATABASE_NAME
,TABLE_NAME
,COLLECT_STATS) 
VALUES 
( 
'ZDDW_SINK_BATCH'
,'ZDDW_SINK_TB'
,'SINK_BASEL_DFLT_TYPE'
,'Y'
);

It could also be written by some as :

Code:
INSERT INTO ABCXYX.PRIVILEGE_TABLE
(
 USER_NAME,DATABASE_NAME,TABLE_NAME,COLLECT_STATS) 
VALUES 
( 
'ZDDW_SINK_BATCH'
,'ZDDW_SINK_TB','SINK_BASEL_DFLT_TYPE'
,'Y'
);

and you can imagine there could be 100 different ways to do it.

I have reading a lot about regular expressions, where we can do a pattern search so that new lines, tabs white spaces can be compared with, but the basic problem is the code can span into multi lines, and the above was one condition to check. Every DDL can have another 100-120 conditions to check and i wanted you guide me how to proceed? Can multiline pattern search can be done in sed or awk ? or something totally different like perl?

If someone can give a piece a code which can be used to search for about statement in a file, i will customize to include all other conditions. Thank you very much in advance !
# 2  
Old 11-08-2013
Try this (for starters) to get the "INSERT" DML statements in one line:
Code:
perl -lne 'if ( $seq = (/^\s*INSERT/i .. /;\s*$/)) {
push @ddl, $_;
if ($seq =~ /E0$/) { print @ddl; @ddl = () }
}' input_file

# 3  
Old 11-09-2013
A similar awk:
Code:
awk '$1=="INSERT", $NF~/;$/{s=s $0; if($NF~/;$/){print s; s=x}}' file


Last edited by Scrutinizer; 11-09-2013 at 05:41 AM..
# 4  
Old 11-09-2013
My SQL has become a bit rusty, so I don't remember where semicolons are used to separate items, but for your above and similar cases this might do:
Code:
awk '{$1=$1}1' RS=";" ORS=";\n" FS="[ \n]*" OFS=" " file

Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Bash - Find files excluding file patterns and subfolder patterns

Hello. For a given folder, I want to select any files find $PATH1 -f \( -name "*" but omit any files like pattern name ! -iname "*.jpg" ! -iname "*.xsession*" ..... \) and also omit any subfolder like pattern name -type d \( -name "/etc/gconf/gconf.*" -o -name "*cache*" -o -name "*Cache*" -o... (2 Replies)
Discussion started by: jcdole
2 Replies

2. Shell Programming and Scripting

Multiline sed

Hi guys, I am fairly comfortable with using the sed command if the string to be replaced is all on a single line. I was wondering is it possible to use sed command in a multiline way ? Say for example I have the below string on 2 different lines: { "key": "brandNameA", ... (3 Replies)
Discussion started by: Junaid Subhani
3 Replies

3. Shell Programming and Scripting

Find matched patterns and print them with other patterns not the whole line

Hi, I am trying to extract some patterns from a line. The input file is space delimited and i could not use column to get value after "IN" or "OUT" patterns as there could be multiple white spaces before the next digits that i need to print in the output file . I need to print 3 patterns in a... (3 Replies)
Discussion started by: redse171
3 Replies

4. Shell Programming and Scripting

Need to get multiline input

As per my requirement, I need to get a multiline input. It can be stored in a file, that's not a problem. User will be prompted to enter steps. he should able to enter the steps in multiple lines by pressing enter. All I know in read command that reads the input till we press enter. Can someone... (5 Replies)
Discussion started by: annamalaikasi
5 Replies

5. UNIX for Dummies Questions & Answers

Need Multiline sed help!!

Hey everyone, I'm new to sed and I need to create a script for inserting one line of code at the beginning of every method in a Xcode project (over 6,000 methods). Each method Structure is (+ or -) (Various declarations-- could span multiple lines) ({) I've tried for days, any guidance would be... (2 Replies)
Discussion started by: jimmyz
2 Replies

6. Shell Programming and Scripting

awk multiline matching

I have a file that looks something like this with lots of text before and after. Distance method: Sum of squared size difference (RST) </data> <pairwiseDifferenceMatrix time="02/08/11 at 13:08:27"> 1 2 1 448.82151 507.94231 2 ... (7 Replies)
Discussion started by: mgray
7 Replies

7. Shell Programming and Scripting

help with multiline variable in csh

My shell is csh and it is required. I have a file like sample.txt ------------------------ a b c d e f g h i ------------------------ I want set the file to a variable and print it out in the same format. I have tried something like this, but not succed. % cat ~/tmp/sample.txt a b c d... (8 Replies)
Discussion started by: anykao
8 Replies

8. Shell Programming and Scripting

Searching patterns in 1 file and deleting all lines with those patterns in 2nd file

Hi Gurus, I have a file say for ex. file1 which has 3500 lines in it which are different account numbers and another file (file2) which has 230000 lines in it. I want to read all the lines in file1 and delete all those lines from file2 which has that same pattern as in file1. I am not quite... (4 Replies)
Discussion started by: toms
4 Replies

9. UNIX for Dummies Questions & Answers

Multiline Grep

How does one do a search for a multiline regular experssion and output the results to a file. I know this won't work since grep only searches single lines: egrep '<a>.*?</a>' source.xml > output.xml Here are some sample patterns I'd like to match and output to a single file: ... (4 Replies)
Discussion started by: tolmark
4 Replies
Login or Register to Ask a Question