Sponsored Content
Top Forums UNIX for Beginners Questions & Answers Extract strings from the file using awk s Post 303045844 by Mannu2525 on Wednesday 15th of April 2020 10:37:19 AM
Old 04-15-2020
Extract strings from the file using awk s

Hi,

I'm trying to make a file that gives me useful information. Format of file is below:-

Code:
--- (Tue Apr 14 09:46:43 EDT 2020): JOIN_Constraints_Schema:test_simple_joins -------------
internal optimizer errors: 0
--- (Tue Apr 14 09:48:10 EDT 2020): JOIN_Constraints_Schema:test_constraint_joins_setop_dis_oby_lmt -------------
Number of queries that causes internal optimizer errors: 0
--- (Tue Apr 14 09:49:02 EDT 2020): in External_Table_Schema:test_subquery_in_from ---------------
--- (Tue Apr 14 09:49:10 EDT 2020): EventSeries_Schema:test_Event_Series1 -------------
--- (Tue Apr 14 09:49:17 EDT 2020):  Gosalesdw_Schema:test_complex_analytics -------------
--- (Tue Apr 14 09:49:25 EDT 2020):  GBY_Schema1:test_Groupby_Rollup -------------
internal optimizer errors: 0
--- (Tue Apr 14 09:49:40 EDT 2020):  GBY_Schema1:test_Groupby_GroupingSets -------------
internal optimizer errors: 0
--- (Tue Apr 14 09:49:52 EDT 2020):  GBY_Schema1:test_Groupby_Cube -------------
internal optimizer errors: 0
--- (Tue Apr 14 09:50:05 EDT 2020):  GBY_Schema1:test_gby -------------
internal optimizer errors: 0

I need to extract the text after the timestamp and immediately next line error number having internal optimizer error.

It should be like - newout.txt
Code:
JOIN_Constraints_Schema,test_simple_joins,0
JOIN_Constraints_Schema,test_constraint_joins_setop_dis_oby_lmt,0
GBY_Schema1,test_Groupby_Rollup,0
GBY_Schema1,test_Groupby_GroupingSets,0
GBY_Schema1,test_Groupby_Cube,0
GBY_Schema1,test_gby,0

Exception - If there is no internal optimizer error attached to any field given in timestamp then no need to print

I tried using a simple script but not works for me

Code:
cat newout.txt | while read x
do
        if [ $(echo $x | grep -E '20[0-9][0-9]\):' | wc -l) == 1 ]
        then
                 schema=$(echo $x | awk '{print $(NF-1)}' | awk -F ':' '{print $1}')
                 config=$(echo $x | awk '{print $(NF-1)}' | awk -F ':' '{print $2}')
         else
                 internal=$(echo $x | awk -F ':' '{print $2}')
         fi
         echo $schema,$config,$internal
done

Please help me achieve this.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

using AWK how to extract text between two same strings

I have a file like: myfile.txt it is easy to learn awk and begin awk scripting and awk has got many features awk is a powerful text processing tool Now i want to get the text between first awk and immediate awk not the third awk . How to get it ? its urgent pls help me and file is unevenly... (2 Replies)
Discussion started by: santosh1234
2 Replies

2. Shell Programming and Scripting

using awk to extract text between two constant strings

Hi, I have a file from which i need to extract data between two constant strings. The data looks like this : Line 1 SUN> read db @cmpd unit 60 Line 2 Parameter: CMPD -> "C00071" Line 3 Line 4 SUN> generate Line 5 tabint>ERROR: (Variable data) The data i need to extract is... (11 Replies)
Discussion started by: mjoshi
11 Replies

3. UNIX for Dummies Questions & Answers

Using awk/sed to extract text between Strings

Dear Unix Gurus, I've got a data file with a few hundred lines (see truncated sample)... BEGIN_SCAN1 TASK_NAME=LA48 PDD Profiles PROGRAM=ArrayScan 1.00 21.220E+00 2.00 21.280E+00 END_DATA END_SCAN1 BEGIN_SCAN2 TASK_NAME=LA48 PDD Profiles 194.00 2.1870E+00 ... (5 Replies)
Discussion started by: tintin72
5 Replies

4. Shell Programming and Scripting

AWK: How to extract text lines between two strings

Hi. I have a text test1.txt file like:Receipt Line1 Line2 Line3 End Receipt Line4 Line5 Line6 Canceled Receipt Line7 Line8 Line9 End (9 Replies)
Discussion started by: TQ3
9 Replies

5. UNIX for Advanced & Expert Users

bash/grep/awk/sed: How to extract every appearance of text between two specific strings

I have a text wich looks like this: clid=2 cid=6 client_database_id=35 client_nickname=Peter client_type=0|clid=3 cid=22 client_database_id=57 client_nickname=Paul client_type=0|clid=5 cid=22 client_database_id=7 client_nickname=Mary client_type=0|clid=6 cid=22 client_database_id=6... (3 Replies)
Discussion started by: Pioneer1976
3 Replies

6. Shell Programming and Scripting

Extract strings within XML file between different delimiters

Good afternoon! I have an XML file from which I want to extract only certain elements contained within each line. The problem is that the format of each line is not exactly the same (though similiar). For example, oa_var will be in each line, however, there may be no value or other... (3 Replies)
Discussion started by: bab@faa
3 Replies

7. Shell Programming and Scripting

Extract strings from file - Help

Hi, I have a file say with following lines (the lines could start from any column and there can be many many create statements in the file) create table table1....table definition... insert into table1 values..... create or replace view view1....view definition.... What i want is to... (2 Replies)
Discussion started by: whoami191
2 Replies

8. Shell Programming and Scripting

Extract two strings from a file and create a new file with these strings

I have the following lines in a log file. It would be great if some one can help me to create a new file with the just entries in the below format. 66.150.161.195 HPSAC=Z05 66.150.161.196 HPSAC=A05 That is just extract the IP address and the string DPSAC=its value 66.150.161.195 -... (1 Reply)
Discussion started by: Tuxidow
1 Replies

9. Shell Programming and Scripting

awk extract strings matching multiple patterns

Hi, I wasn't quite sure how to title this one! Here goes: I have some already partially parsed log files, which I now need to extract info from. Because of the way they are originally and the fact they have been partially processed already, I can't make any assumptions on the number of... (8 Replies)
Discussion started by: chrissycc
8 Replies

10. UNIX for Beginners Questions & Answers

Use strings from nth field from one file to match strings in entire line in another file, awk

I cannot seem to get what should be a simple awk one-liner to work correctly and cannot figure out why. I would like to use patterns from a specific field in one file as regex to search for matching strings in the entire line ($0) of another file. I would like to output the lines of File2 which... (1 Reply)
Discussion started by: jvoot
1 Replies
getdate(3C)															       getdate(3C)

NAME
getdate() - convert user format date and time SYNOPSIS
Obsolescent Interface DESCRIPTION
The function converts user definable date and/or time specifications pointed to by string into a The structure declaration is in the header file (see ctime(3C)). User-supplied templates are used to parse and interpret the input string. The templates are text files created by the user and identified via the environment variable should be set to indicate the full path name of the template file. The first line in the template that matches the input specification is used for interpretation and conversion into the internal time format. Upon successful completion, returns a pointer to a otherwise, it returns NULL and the symbol is set to indicate the error. The following field descriptors are supported: same as abbreviated weekday name full weekday name abbreviated month name full month name locale's appropriate date and time representation century number (00 through 99; leading zeros are permitted but not required) day of the month (01 through 31; the leading 0 is optional) same as date as abbreviated month name hour (00 through 23) hour (01 through 12) month number (01 through 12) minute (00 through 59) same as locale's equivalent of either AM or PM time as time as seconds (00 through 61) insert a tab time as weekday number (Sunday = 0 through Saturday = 6) locale's appropriate date representation locale's appropriate time representation year without century (00 through 99). For inputs 69-99, the 20th century(1900s) is assumed, and for inputs 00-68, the 21st century(2000s) is assumed. year as ccyy (e.g., 1986) time zone name or no characters if no time zone exists. If the time zone supplied by is not the same as the time zone expects, an invalid specification error is returned. calcu- lates the expected time zone from the environment variable. Month and weekday names may consist of any combination of uppercase and lowercase letters. The user can request that the input date or time specification be in a specific language by setting the category (see setlocale(3C)). For descriptors that allow leading zeros, leading zeros are optional. However, the number of digits used for those descriptors must not exceed two, including leading zeros. Extra whitespace in either the template file or in string is ignored. The field descriptors and are not supported if they include unsupported field descriptors. The following example shows the possible contents of a template: The following are examples of valid input specifications for the above template: If the category is set to a German locale that includes as a weekday name and as a month name, the following would be valid: This example shows how local date and time specification can be defined in the template: +---------------------------+------------------+ | Invocation | Line in Template | +---------------------------+------------------+ |getdate("11/27/86") | %m/%d/%y | |getdate("27.11.86") | %d.%m.%y | |getdate("86-11-27") | %y-%m-%d | |getdate("Friday 12:00:00") | %A %H:%M:%S | +---------------------------+------------------+ The following rules apply when converting the input specification into the internal format: o If only the weekday is given, today is assumed if the given day is equal to the current day, and next week if it is less. o If only the month is given, the current month is assumed if the given month is equal to the current month, and next year if it is less and no year is given (the first day of the month is assumed if no day is given). o If no hour, minute and second are given, the current hour, minute and second are assumed. o If no date is given, today is assumed if the given hour is greater than the current hour and tomorrow is assumed if it is less. The following examples help to illustrate the above rules assuming that the current date is and the category is set to the default locale. +-------------+----------+------------------------------+ | | Line in | | | Input | Template | Date | +-------------+----------+------------------------------+ |Mon | %a | Mon Sep 22 12:19:47 EDT 1986 | |Sun | %a | Sun Sep 28 12:19:47 EDT 1986 | |Fri | %a | Fri Sep 26 12:19:47 EDT 1986 | |September | %B | Mon Sep 1 12:19:47 EDT 1986 | |January | %B | Thu Jan 1 12:19:47 EST 1987 | |December | %B | Mon Dec 1 12:19:47 EST 1986 | |Sep Mon | %b %a | Mon Sep 1 12:19:47 EDT 1986 | |Jan Fri | %b %a | Fri Jan 2 12:19:47 EST 1987 | |Dec Mon | %b %a | Mon Dec 1 12:19:47 EST 1986 | |Jan Wed 1989 | %b %a %Y | Wed Jan 4 12:19:47 EST 1989 | |Fri 9 | %a %H | Fri Sep 26 09:00:00 EDT 1986 | |Feb 10:30 | %b %H:%S | Sun Feb 1 10:30:00 EST 1987 | |10:30 | %H:%M | Tue Sep 23 10:30:00 EDT 1986 | |13:30 | %H:%M | Mon Sep 22 13:30:00 EDT 1986 | +-------------+----------+------------------------------+ Obsolescent Interface converts user format date and time. Also see the section. ERRORS
Upon failure, returns NULL and the symbol is set to indicate the error. The following is a complete list of the settings and their interpretation: the environment variable is null or undefined, the template file cannot be opened for reading, failed to get file status information, the template file is not a regular file, an error is encountered while reading the template file, memory allocation failed (not enough memory available), there is no line in the template that matches the input, invalid input specification. For example, February 31; or the time specified cannot be represented in the data type in 32-bit HP-UX (which represents Tuesday January 19 03:14:07 UTC, 2038) or exceeds the maximum date supported in 64-bit HP-UX (which is Friday December 31 23:59:59 UTC, 9999). WARNINGS
The return value for points to data whose content is overwritten by each call by the same thread. is an obsolescent interface supported only for compatibility with existing DCE applications. New multi-threaded applications should use SEE ALSO
ctime(3C), ctype(3C), setlocale(3C), strftime(3C), thread_safety(5). getdate(3C)
All times are GMT -4. The time now is 01:54 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy