Extract multiple line text from log between special words
Hi all,
I have a text file (log), it prints some SQL statements in the log. It is a bit irregular, I need to extract SQL statement into as separate file.
1. Some statements are printed as just SQL statements, in multiple lines.
2. Some SQL statements are printed line by line but after log details at the beginning of the line. So somehow I need to ignore the first part of the line but need to keep query.
What I found as a pattern is, all SQL statements have "DISTINCT" keyword in the beginning
and after each statement there are "quit;" statements.
This doesn't have to be at the beginning of the lines, it's enough for me to extract text between DISTINCT and quit.
This is where I confused and couldn't find a way to solve it with sed or awk.
Sample log structure 1:
Code:
/* clear syscc before new query */
%let syscc=0;
%let sysrc=0;
/* MAmodel_C */
SELECT DISTINCT
table0.SUBSCRIBER_KEY AS SUBSCRIBER_KEY LABEL='Maestro Subscriber Key' FORMAT=F13.0
FROM
CMD.ACCOUNTS_MAESTRO table0 Inner join CMD.PRODUCT_MAESTRO table1 on ( ( (table1.ACCOUNT_ID = table0.ACCOUNT_ID AND table1.ACCOUNT_KEY = table0.ACCOUNT_KEY) AND table1.COMPANY_NUMBER = table0.COMPANY_NUMBER) AND table1.SUBSCRIBER_KEY = table0.SUBSCRIBER_KEY)
WHERE
table1.PRODUCT_DISCOUNT_CODE IS NULL
) table2 ;
quit;
-----------------------------------------------------
2020-03-17 09:46:11,531 INFO
Structure 2: (each line is after log details)
Code:
2020-03-17 09:45:57,733 INFO [Exec-11] [02205e834:-68e0] [user1140] .execution.instructions - 2FD66078-1B0B: SELECT DISTINCT
2020-03-17 09:45:57,733 INFO [Exec-11] [02205e834:-68e0] [user1140] .execution.instructions - 2FD66078-1B0B: table0.SUBSCRIBER_KEY AS SUBSCRIBER_KEY LABEL='Maestro Subscriber Key' FORMAT=F13.0
2020-03-17 09:45:57,733 INFO [Exec-11] [02205e834:-68e0] [user1140] .execution.instructions - 2FD66078-1B0B: FROM
2020-03-17 09:45:57,733 INFO [Exec-11] [02205e834:-68e0] [user1140] .execution.instructions - 2FD66078-1B0B: CMDM.ACCOUNTS_MAESTRO table0 Inner join CMDM.PRODUCT_MAESTRO table1 on ( ( (table1.ACCOUNT_ID = table0.ACCOUNT_ID AND table1.ACCOUNT_KEY = table0.ACCOUNT_KEY) AND table1.COMPANY_NUMBER = table0.COMPANY_NUMBER) AND table1.SUBSCRIBER_KEY = table0.SUBSCRIBER_KEY)
2020-03-17 09:45:57,734 INFO [Exec-11] [02205e834:-68e0] [user1140] .execution.instructions - 2FD66078-1B0B: WHERE
2020-03-17 09:45:57,734 INFO [Exec-11] [02205e834:-68e0] [user1140] .execution.instructions - 2FD66078-1B0B: table1.PRODUCT_DISCOUNT_CODE IS NULL
2020-03-17 09:45:57,734 INFO [Exec-11] [02205e834:-68e0] [user1140] .execution.instructions - 2FD66078-1B0B: ) table2 ;
2020-03-17 09:45:57,734 INFO [Exec-11] [02205e834:-68e0] [user1140] .execution.instructions - 2FD66078-1B0B: quit;
hi I made this simple script to extract data and pretty much is a list and would like to extract data of two words separated by commas and I would like to make a new text file that would list these extracted data into a list and each in a new line.
Example that worked for me with text file... (5 Replies)
Hi All
I have to search servers name say like 1000+ "unique names" line by line in child.txt files in another file that is a master file where all server present say "master.txt",if child.txt's server name matches with master files then it print yes else no with server name. (4 Replies)
Hi
I use sed comnand to remove occurance of one workd from a line.
However I need to removed occurance of dufferent words in ne line.
Original-1 Hi this is the END of my begining
Comand sed s/"END"/"start"/g
Output-1 Hi this is the start of my beginig
But I have more... (9 Replies)
I can't decide if I should use AWK or PERL after pouring over these forums for hours today I decided I'd post something and see if I couldn't get some advice.
I've got a text file full of hundreds of events in this format:
Record Number : 1
Records in Seq : ... (3 Replies)
Hi again
I have figured out how to be able to sort through lines in a file with multiple words in any order and display them using this command:
cat file | grep -i $OPTION1 | grep -i $OPTION2 | grep -i $OPTION3 OPTION1 is 2008, OPTION2 is Mar, OPTION 3 is Tue
Result:
Tue Mar 25... (4 Replies)
Hi Guys,
Can someone help me with a way to extract text between two words on a single line.
For example if the file has below content I want to extract all text between b and f inclusive of b and f. Aparently sed does this but does it line by line and I guess it cannot read word by word.
... (11 Replies)
The file contains one line of text followed by a number. I want to take the number X at the end, take it out and display the last X words. X is the key telling me how many words from the end that I want and X will always be less than the number of words, so no problem there.
Example input and... (4 Replies)
Hi..
How to search for multiple words in a single line using grep?.
Eg: Jack and Jill went up the hill
Jack and Jill were best friends
Humpty and Dumpty were good friends too
----------
I want to extract the 2nd statement(assuming there are several statements with... (11 Replies)
hi,
I have few text templates
as a simple ex:
template 1
city Name:
zip code:
state Name:
template2:
employee Name:
Phone number:
I wish to grep on given text file and make sure the text file matches one of these templates. Please give your ideas. (6 Replies)