Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Extract strings from the file using awk s
Top Forums UNIX for Beginners Questions & Answers Extract strings from the file using awk s Post 303045844 by Mannu2525 on Wednesday 15th of April 2020 10:37:19 AM
Old 04-15-2020
Extract strings from the file using awk s
Hi,

I'm trying to make a file that gives me useful information. Format of file is below:-

Code:
--- (Tue Apr 14 09:46:43 EDT 2020): JOIN_Constraints_Schema:test_simple_joins -------------
internal optimizer errors: 0
--- (Tue Apr 14 09:48:10 EDT 2020): JOIN_Constraints_Schema:test_constraint_joins_setop_dis_oby_lmt -------------
Number of queries that causes internal optimizer errors: 0
--- (Tue Apr 14 09:49:02 EDT 2020): in External_Table_Schema:test_subquery_in_from ---------------
--- (Tue Apr 14 09:49:10 EDT 2020): EventSeries_Schema:test_Event_Series1 -------------
--- (Tue Apr 14 09:49:17 EDT 2020):  Gosalesdw_Schema:test_complex_analytics -------------
--- (Tue Apr 14 09:49:25 EDT 2020):  GBY_Schema1:test_Groupby_Rollup -------------
internal optimizer errors: 0
--- (Tue Apr 14 09:49:40 EDT 2020):  GBY_Schema1:test_Groupby_GroupingSets -------------
internal optimizer errors: 0
--- (Tue Apr 14 09:49:52 EDT 2020):  GBY_Schema1:test_Groupby_Cube -------------
internal optimizer errors: 0
--- (Tue Apr 14 09:50:05 EDT 2020):  GBY_Schema1:test_gby -------------
internal optimizer errors: 0

I need to extract the text after the timestamp and immediately next line error number having internal optimizer error.

It should be like - newout.txt
Code:
JOIN_Constraints_Schema,test_simple_joins,0
JOIN_Constraints_Schema,test_constraint_joins_setop_dis_oby_lmt,0
GBY_Schema1,test_Groupby_Rollup,0
GBY_Schema1,test_Groupby_GroupingSets,0
GBY_Schema1,test_Groupby_Cube,0
GBY_Schema1,test_gby,0

Exception - If there is no internal optimizer error attached to any field given in timestamp then no need to print

I tried using a simple script but not works for me

Code:
cat newout.txt | while read x
do
        if [ $(echo $x | grep -E '20[0-9][0-9]\):' | wc -l) == 1 ]
        then
                 schema=$(echo $x | awk '{print $(NF-1)}' | awk -F ':' '{print $1}')
                 config=$(echo $x | awk '{print $(NF-1)}' | awk -F ':' '{print $2}')
         else
                 internal=$(echo $x | awk -F ':' '{print $2}')
         fi
         echo $schema,$config,$internal
done

Please help me achieve this.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

using AWK how to extract text between two same strings

I have a file like: myfile.txt it is easy to learn awk and begin awk scripting and awk has got many features awk is a powerful text processing tool Now i want to get the text between first awk and immediate awk not the third awk . How to get it ? its urgent pls help me and file is unevenly... (2 Replies)
Discussion started by: santosh1234
2 Replies

2. Shell Programming and Scripting

using awk to extract text between two constant strings

Hi, I have a file from which i need to extract data between two constant strings. The data looks like this : Line 1 SUN> read db @cmpd unit 60 Line 2 Parameter: CMPD -> "C00071" Line 3 Line 4 SUN> generate Line 5 tabint>ERROR: (Variable data) The data i need to extract is... (11 Replies)
Discussion started by: mjoshi
11 Replies

3. UNIX for Dummies Questions & Answers

Using awk/sed to extract text between Strings

Dear Unix Gurus, I've got a data file with a few hundred lines (see truncated sample)... BEGIN_SCAN1 TASK_NAME=LA48 PDD Profiles PROGRAM=ArrayScan 1.00 21.220E+00 2.00 21.280E+00 END_DATA END_SCAN1 BEGIN_SCAN2 TASK_NAME=LA48 PDD Profiles 194.00 2.1870E+00 ... (5 Replies)
Discussion started by: tintin72
5 Replies

4. Shell Programming and Scripting

AWK: How to extract text lines between two strings

Hi. I have a text test1.txt file like:Receipt Line1 Line2 Line3 End Receipt Line4 Line5 Line6 Canceled Receipt Line7 Line8 Line9 End (9 Replies)
Discussion started by: TQ3
9 Replies

5. UNIX for Advanced & Expert Users

bash/grep/awk/sed: How to extract every appearance of text between two specific strings

I have a text wich looks like this: clid=2 cid=6 client_database_id=35 client_nickname=Peter client_type=0|clid=3 cid=22 client_database_id=57 client_nickname=Paul client_type=0|clid=5 cid=22 client_database_id=7 client_nickname=Mary client_type=0|clid=6 cid=22 client_database_id=6... (3 Replies)
Discussion started by: Pioneer1976
3 Replies

6. Shell Programming and Scripting

Extract strings within XML file between different delimiters

Good afternoon! I have an XML file from which I want to extract only certain elements contained within each line. The problem is that the format of each line is not exactly the same (though similiar). For example, oa_var will be in each line, however, there may be no value or other... (3 Replies)
Discussion started by: bab@faa
3 Replies

7. Shell Programming and Scripting

Extract strings from file - Help

Hi, I have a file say with following lines (the lines could start from any column and there can be many many create statements in the file) create table table1....table definition... insert into table1 values..... create or replace view view1....view definition.... What i want is to... (2 Replies)
Discussion started by: whoami191
2 Replies

8. Shell Programming and Scripting

Extract two strings from a file and create a new file with these strings

I have the following lines in a log file. It would be great if some one can help me to create a new file with the just entries in the below format. 66.150.161.195 HPSAC=Z05 66.150.161.196 HPSAC=A05 That is just extract the IP address and the string DPSAC=its value 66.150.161.195 -... (1 Reply)
Discussion started by: Tuxidow
1 Replies

9. Shell Programming and Scripting

awk extract strings matching multiple patterns

Hi, I wasn't quite sure how to title this one! Here goes: I have some already partially parsed log files, which I now need to extract info from. Because of the way they are originally and the fact they have been partially processed already, I can't make any assumptions on the number of... (8 Replies)
Discussion started by: chrissycc
8 Replies

10. UNIX for Beginners Questions & Answers

Use strings from nth field from one file to match strings in entire line in another file, awk

I cannot seem to get what should be a simple awk one-liner to work correctly and cannot figure out why. I would like to use patterns from a specific field in one file as regex to search for matching strings in the entire line ($0) of another file. I would like to output the lines of File2 which... (1 Reply)
Discussion started by: jvoot
1 Replies

LEARN ABOUT DEBIAN

data::ical::entry::timezone::standard

Data::ICal::Entry::TimeZone::Standard(3pm)		User Contributed Perl Documentation		Data::ICal::Entry::TimeZone::Standard(3pm)

NAME

       Data::ICal::Entry::TimeZone::Standard - Represents a Standard Time base offset from UTC for parent TimeZone

DESCRIPTION

       A time zone is unambiguously defined by the set of time measurement rules determined by the governing body for a given geographic area.
       These rules describe at a minimum the base offset from UTC for the time zone, often referred to as the Standard Time offset. Many locations
       adjust their Standard Time forward or backward by one hour, in order to accommodate seasonal changes in number of daylight hours, often
       referred to as Daylight Saving Time. Some locations adjust their time by a fraction of an hour. Standard Time is also known as Winter Time.
       Daylight Saving Time is also known as Advanced Time, Summer Time, or Legal Time in certain countries. The following table shows the changes
       in time zone rules in effect for New York City starting from 1967. Each line represents a description or rule for a particular observance.

	    Effective Observance Rule

	    Date       (Date/Time)	       Offset  Abbreviation

	    1967-*     last Sun in Oct, 02:00  -0500   EST

	    1967-1973  last Sun in Apr, 02:00  -0400   EDT

	    1974-1974  Jan 6,  02:00	       -0400   EDT

	    1975-1975  Feb 23, 02:00	       -0400   EDT

	    1976-1986  last Sun in Apr, 02:00  -0400   EDT

	    1987-*     first Sun in Apr, 02:00 -0400   EDT

       Note: The specification of a global time zone registry is not addressed by this document and is left for future study.  However,
       implementers may find the Olson time zone database [TZ] a useful reference. It is an informal, public-domain collection of time zone
       information, which is currently being maintained by volunteer Internet participants, and is used in several operating systems. This
       database contains current and historical time zone information for a wide variety of locations around the globe; it provides a time zone
       identifier for every unique time zone rule set in actual use since 1970, with historical data going back to the introduction of standard
       time.

METHODS

   ical_entry_type
       Returns "STANDARD", its iCalendar entry name.

   mandatory_unique_properties
       According to the iCalendar standard, the following properties must be specified exactly one time in a standard time declaration:

	       dtstart	tzoffsetto  tzoffsetfrom

   optional_repeatable_properties
       According to the iCalendar standard, the following properties may be specified any number of times for a standard time declaration:

	       comment	rdate  rrule  tzname

AUTHOR

       Jesse Vincent "<jesse@bestpractical.com>" with David Glasser, Simon Wistow, Alex Vandiver

LICENCE AND COPYRIGHT

       Copyright (c) 2005 - 2009, Best Practical Solutions, LLC.  All rights reserved.

       This module is free software; you can redistribute it and/or modify it under the same terms as Perl itself. See perlartistic.

DISCLAIMER OF WARRANTY

       BECAUSE THIS SOFTWARE IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY FOR THE SOFTWARE, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT
       WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES PROVIDE THE SOFTWARE "AS IS" WITHOUT WARRANTY OF ANY KIND,
       EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE.
       THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE SOFTWARE IS WITH YOU. SHOULD THE SOFTWARE PROVE DEFECTIVE, YOU ASSUME THE COST OF
       ALL NECESSARY SERVICING, REPAIR, OR CORRECTION.

       IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR
       REDISTRIBUTE THE SOFTWARE AS PERMITTED BY THE ABOVE LICENCE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL, OR
       CONSEQUENTIAL DAMAGES ARISING OUT OF THE USE OR INABILITY TO USE THE SOFTWARE (INCLUDING BUT NOT LIMITED TO LOSS OF DATA OR DATA BEING
       RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD PARTIES OR A FAILURE OF THE SOFTWARE TO OPERATE WITH ANY OTHER SOFTWARE), EVEN IF
       SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.

perl v5.14.2							    2009-07-16				Data::ICal::Entry::TimeZone::Standard(3pm)
All times are GMT -4. The time now is 03:56 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy