Sponsored Content
Top Forums Shell Programming and Scripting Extract multiple line text from log between special words Post 303045618 by sast1a on Friday 3rd of April 2020 08:33:39 PM
Old 04-03-2020
Extract multiple line text from log between special words

Hi all,

I have a text file (log), it prints some SQL statements in the log. It is a bit irregular, I need to extract SQL statement into as separate file.

1. Some statements are printed as just SQL statements, in multiple lines.
2. Some SQL statements are printed line by line but after log details at the beginning of the line. So somehow I need to ignore the first part of the line but need to keep query.

What I found as a pattern is, all SQL statements have "DISTINCT" keyword in the beginning
and after each statement there are "quit;" statements.
This doesn't have to be at the beginning of the lines, it's enough for me to extract text between DISTINCT and quit.

This is where I confused and couldn't find a way to solve it with sed or awk.

Sample log structure 1:

Code:
/* clear syscc before new query */
%let syscc=0;
%let sysrc=0;
/* MAmodel_C */
SELECT  DISTINCT
        table0.SUBSCRIBER_KEY AS SUBSCRIBER_KEY LABEL='Maestro Subscriber Key' FORMAT=F13.0
FROM
        CMD.ACCOUNTS_MAESTRO table0 Inner join CMD.PRODUCT_MAESTRO table1 on  ( ( (table1.ACCOUNT_ID = table0.ACCOUNT_ID AND table1.ACCOUNT_KEY = table0.ACCOUNT_KEY)  AND table1.COMPANY_NUMBER = table0.COMPANY_NUMBER)  AND table1.SUBSCRIBER_KEY = table0.SUBSCRIBER_KEY)
WHERE
        table1.PRODUCT_DISCOUNT_CODE IS NULL
) table2 ;
quit;
-----------------------------------------------------
2020-03-17 09:46:11,531 INFO

Structure 2: (each line is after log details)

Code:
2020-03-17 09:45:57,733 INFO  [Exec-11] [02205e834:-68e0] [user1140] .execution.instructions - 2FD66078-1B0B: SELECT  DISTINCT
2020-03-17 09:45:57,733 INFO  [Exec-11] [02205e834:-68e0] [user1140] .execution.instructions - 2FD66078-1B0B:        table0.SUBSCRIBER_KEY AS SUBSCRIBER_KEY LABEL='Maestro Subscriber Key' FORMAT=F13.0
2020-03-17 09:45:57,733 INFO  [Exec-11] [02205e834:-68e0] [user1140] .execution.instructions - 2FD66078-1B0B: FROM
2020-03-17 09:45:57,733 INFO  [Exec-11] [02205e834:-68e0] [user1140] .execution.instructions - 2FD66078-1B0B:        CMDM.ACCOUNTS_MAESTRO table0 Inner join CMDM.PRODUCT_MAESTRO table1 on  ( ( (table1.ACCOUNT_ID = table0.ACCOUNT_ID AND table1.ACCOUNT_KEY = table0.ACCOUNT_KEY)  AND table1.COMPANY_NUMBER = table0.COMPANY_NUMBER)  AND table1.SUBSCRIBER_KEY = table0.SUBSCRIBER_KEY)
2020-03-17 09:45:57,734 INFO  [Exec-11] [02205e834:-68e0] [user1140] .execution.instructions - 2FD66078-1B0B: WHERE
2020-03-17 09:45:57,734 INFO  [Exec-11] [02205e834:-68e0] [user1140] .execution.instructions - 2FD66078-1B0B:        table1.PRODUCT_DISCOUNT_CODE IS NULL
2020-03-17 09:45:57,734 INFO  [Exec-11] [02205e834:-68e0] [user1140] .execution.instructions - 2FD66078-1B0B: ) table2 ;
2020-03-17 09:45:57,734 INFO  [Exec-11] [02205e834:-68e0] [user1140] .execution.instructions - 2FD66078-1B0B: quit;

 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

grep on multiple words to match text template

hi, I have few text templates as a simple ex: template 1 city Name: zip code: state Name: template2: employee Name: Phone number: I wish to grep on given text file and make sure the text file matches one of these templates. Please give your ideas. (6 Replies)
Discussion started by: rider29
6 Replies

2. Shell Programming and Scripting

grep multiple words in a single line

Hi.. How to search for multiple words in a single line using grep?. Eg: Jack and Jill went up the hill Jack and Jill were best friends Humpty and Dumpty were good friends too ---------- I want to extract the 2nd statement(assuming there are several statements with... (11 Replies)
Discussion started by: anduzzi
11 Replies

3. Shell Programming and Scripting

Extract X words from end of line, minus last keynumber X

The file contains one line of text followed by a number. I want to take the number X at the end, take it out and display the last X words. X is the key telling me how many words from the end that I want and X will always be less than the number of words, so no problem there. Example input and... (4 Replies)
Discussion started by: fubaya
4 Replies

4. UNIX for Dummies Questions & Answers

extract text between two words on a single line

Hi Guys, Can someone help me with a way to extract text between two words on a single line. For example if the file has below content I want to extract all text between b and f inclusive of b and f. Aparently sed does this but does it line by line and I guess it cannot read word by word. ... (11 Replies)
Discussion started by: krishnaux
11 Replies

5. UNIX for Dummies Questions & Answers

Searching for multiple words on a line in any order issue

Hi again I have figured out how to be able to sort through lines in a file with multiple words in any order and display them using this command: cat file | grep -i $OPTION1 | grep -i $OPTION2 | grep -i $OPTION3 OPTION1 is 2008, OPTION2 is Mar, OPTION 3 is Tue Result: Tue Mar 25... (4 Replies)
Discussion started by: semaj
4 Replies

6. Shell Programming and Scripting

Combine multiple unique lines from event log text file into one line, use PERL or AWK?

I can't decide if I should use AWK or PERL after pouring over these forums for hours today I decided I'd post something and see if I couldn't get some advice. I've got a text file full of hundreds of events in this format: Record Number : 1 Records in Seq : ... (3 Replies)
Discussion started by: Mayday22
3 Replies

7. Shell Programming and Scripting

I need to extract uique words from text file

Hello programmers, I need to create a list of unique words from a text file using PERL...may i have the code for that please? Thank you (1 Reply)
Discussion started by: alsohari
1 Replies

8. Shell Programming and Scripting

USING sed to remove multiple strings/words from a line

Hi I use sed comnand to remove occurance of one workd from a line. However I need to removed occurance of dufferent words in ne line. Original-1 Hi this is the END of my begining Comand sed s/"END"/"start"/g Output-1 Hi this is the start of my beginig But I have more... (9 Replies)
Discussion started by: mnassiri
9 Replies

9. Shell Programming and Scripting

Search words in multiple file line by line

Hi All I have to search servers name say like 1000+ "unique names" line by line in child.txt files in another file that is a master file where all server present say "master.txt",if child.txt's server name matches with master files then it print yes else no with server name. (4 Replies)
Discussion started by: netdbaind
4 Replies

10. UNIX for Beginners Questions & Answers

Shell - Read a text file with two words and extract data

hi I made this simple script to extract data and pretty much is a list and would like to extract data of two words separated by commas and I would like to make a new text file that would list these extracted data into a list and each in a new line. Example that worked for me with text file... (5 Replies)
Discussion started by: dandaryll
5 Replies
SQL::Translator::Schema::Table(3pm)			User Contributed Perl Documentation		       SQL::Translator::Schema::Table(3pm)

NAME
SQL::Translator::Schema::Table - SQL::Translator table object SYNOPSIS
use SQL::Translator::Schema::Table; my $table = SQL::Translator::Schema::Table->new( name => 'foo' ); DESCRIPTION
"SQL::Translator::Schema::Table" is the table object. METHODS
new Object constructor. my $table = SQL::Translator::Schema::Table->new( schema => $schema, name => 'foo', ); add_constraint Add a constraint to the table. Returns the newly created "SQL::Translator::Schema::Constraint" object. my $c1 = $table->add_constraint( name => 'pk', type => PRIMARY_KEY, fields => [ 'foo_id' ], ); my $c2 = SQL::Translator::Schema::Constraint->new( name => 'uniq' ); $c2 = $table->add_constraint( $constraint ); drop_constraint Remove a constraint from the table. Returns the constraint object if the index was found and removed, an error otherwise. The single parameter can be either an index name or an "SQL::Translator::Schema::Constraint" object. $table->drop_constraint('myconstraint'); add_index Add an index to the table. Returns the newly created "SQL::Translator::Schema::Index" object. my $i1 = $table->add_index( name => 'name', fields => [ 'name' ], type => 'normal', ); my $i2 = SQL::Translator::Schema::Index->new( name => 'id' ); $i2 = $table->add_index( $index ); drop_index Remove an index from the table. Returns the index object if the index was found and removed, an error otherwise. The single parameter can be either an index name of an "SQL::Translator::Schema::Index" object. $table->drop_index('myindex'); add_field Add an field to the table. Returns the newly created "SQL::Translator::Schema::Field" object. The "name" parameter is required. If you try to create a field with the same name as an existing field, you will get an error and the field will not be created. my $f1 = $table->add_field( name => 'foo_id', data_type => 'integer', size => 11, ); my $f2 = SQL::Translator::Schema::Field->new( name => 'name', table => $table, ); $f2 = $table->add_field( $field2 ) or die $table->error; drop_field Remove a field from the table. Returns the field object if the field was found and removed, an error otherwise. The single parameter can be either a field name or an "SQL::Translator::Schema::Field" object. $table->drop_field('myfield'); comments Get or set the comments on a table. May be called several times to set and it will accumulate the comments. Called in an array context, returns each comment individually; called in a scalar context, returns all the comments joined on newlines. $table->comments('foo'); $table->comments('bar'); print join( ', ', $table->comments ); # prints "foo, bar" get_constraints Returns all the constraint objects as an array or array reference. my @constraints = $table->get_constraints; get_indices Returns all the index objects as an array or array reference. my @indices = $table->get_indices; get_field Returns a field by the name provided. my $field = $table->get_field('foo'); get_fields Returns all the field objects as an array or array reference. my @fields = $table->get_fields; is_valid Determine whether the view is valid or not. my $ok = $view->is_valid; is_trivial_link True if table has no data (non-key) fields and only uses single key joins. is_data Returns true if the table has some non-key fields. can_link Determine whether the table can link two arg tables via many-to-many. my $ok = $table->can_link($table1,$table2); name Get or set the table's name. Errors ("No table name") if you try to set a blank name. If provided an argument, checks the schema object for a table of that name and disallows the change if one exists (setting the error to "Can't use table name "%s": table exists"). my $table_name = $table->name('foo'); schema Get or set the table's schema object. my $schema = $table->schema; primary_key Gets or sets the table's primary key(s). Takes one or more field names (as a string, list or array[ref]) as an argument. If the field names are present, it will create a new PK if none exists, or it will add to the fields of an existing PK (and will unique the field names). Returns the "SQL::Translator::Schema::Constraint" object representing the primary key. These are eqivalent: $table->primary_key('id'); $table->primary_key(['name']); $table->primary_key('id','name']); $table->primary_key(['id','name']); $table->primary_key('id,name'); $table->primary_key(qw[ id name ]); my $pk = $table->primary_key; options Get or set the table's options (e.g., table types for MySQL). Returns an array or array reference. my @options = $table->options; order Get or set the table's order. my $order = $table->order(3); field_names Read-only method to return a list or array ref of the field names. Returns undef or an empty list if the table has no fields set. Useful if you want to avoid the overload magic of the Field objects returned by the get_fields method. my @names = $constraint->field_names; equals Determines if this table is the same as another my $isIdentical = $table1->equals( $table2 ); LOOKUP METHODS
The following are a set of shortcut methods for getting commonly used lists of fields and constraints. They all return lists or array refs of Field or Constraint objects. pkey_fields The primary key fields. fkey_fields All foreign key fields. nonpkey_fields All the fields except the primary key. data_fields All non key fields. unique_fields All fields with unique constraints. unique_constraints All this tables unique constraints. fkey_constraints All this tables foreign key constraints. (See primary_key method to get the primary key constraint) AUTHORS
Ken Youens-Clark <kclark@cpan.org>, Allen Day <allenday@ucla.edu>. perl v5.14.2 2012-05-01 SQL::Translator::Schema::Table(3pm)
All times are GMT -4. The time now is 07:29 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy