Sponsored Content
Top Forums UNIX for Beginners Questions & Answers Extract strings from the file using awk s Post 303045844 by Mannu2525 on Wednesday 15th of April 2020 10:37:19 AM
Old 04-15-2020
Extract strings from the file using awk s

Hi,

I'm trying to make a file that gives me useful information. Format of file is below:-

Code:
--- (Tue Apr 14 09:46:43 EDT 2020): JOIN_Constraints_Schema:test_simple_joins -------------
internal optimizer errors: 0
--- (Tue Apr 14 09:48:10 EDT 2020): JOIN_Constraints_Schema:test_constraint_joins_setop_dis_oby_lmt -------------
Number of queries that causes internal optimizer errors: 0
--- (Tue Apr 14 09:49:02 EDT 2020): in External_Table_Schema:test_subquery_in_from ---------------
--- (Tue Apr 14 09:49:10 EDT 2020): EventSeries_Schema:test_Event_Series1 -------------
--- (Tue Apr 14 09:49:17 EDT 2020):  Gosalesdw_Schema:test_complex_analytics -------------
--- (Tue Apr 14 09:49:25 EDT 2020):  GBY_Schema1:test_Groupby_Rollup -------------
internal optimizer errors: 0
--- (Tue Apr 14 09:49:40 EDT 2020):  GBY_Schema1:test_Groupby_GroupingSets -------------
internal optimizer errors: 0
--- (Tue Apr 14 09:49:52 EDT 2020):  GBY_Schema1:test_Groupby_Cube -------------
internal optimizer errors: 0
--- (Tue Apr 14 09:50:05 EDT 2020):  GBY_Schema1:test_gby -------------
internal optimizer errors: 0

I need to extract the text after the timestamp and immediately next line error number having internal optimizer error.

It should be like - newout.txt
Code:
JOIN_Constraints_Schema,test_simple_joins,0
JOIN_Constraints_Schema,test_constraint_joins_setop_dis_oby_lmt,0
GBY_Schema1,test_Groupby_Rollup,0
GBY_Schema1,test_Groupby_GroupingSets,0
GBY_Schema1,test_Groupby_Cube,0
GBY_Schema1,test_gby,0

Exception - If there is no internal optimizer error attached to any field given in timestamp then no need to print

I tried using a simple script but not works for me

Code:
cat newout.txt | while read x
do
        if [ $(echo $x | grep -E '20[0-9][0-9]\):' | wc -l) == 1 ]
        then
                 schema=$(echo $x | awk '{print $(NF-1)}' | awk -F ':' '{print $1}')
                 config=$(echo $x | awk '{print $(NF-1)}' | awk -F ':' '{print $2}')
         else
                 internal=$(echo $x | awk -F ':' '{print $2}')
         fi
         echo $schema,$config,$internal
done

Please help me achieve this.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

using AWK how to extract text between two same strings

I have a file like: myfile.txt it is easy to learn awk and begin awk scripting and awk has got many features awk is a powerful text processing tool Now i want to get the text between first awk and immediate awk not the third awk . How to get it ? its urgent pls help me and file is unevenly... (2 Replies)
Discussion started by: santosh1234
2 Replies

2. Shell Programming and Scripting

using awk to extract text between two constant strings

Hi, I have a file from which i need to extract data between two constant strings. The data looks like this : Line 1 SUN> read db @cmpd unit 60 Line 2 Parameter: CMPD -> "C00071" Line 3 Line 4 SUN> generate Line 5 tabint>ERROR: (Variable data) The data i need to extract is... (11 Replies)
Discussion started by: mjoshi
11 Replies

3. UNIX for Dummies Questions & Answers

Using awk/sed to extract text between Strings

Dear Unix Gurus, I've got a data file with a few hundred lines (see truncated sample)... BEGIN_SCAN1 TASK_NAME=LA48 PDD Profiles PROGRAM=ArrayScan 1.00 21.220E+00 2.00 21.280E+00 END_DATA END_SCAN1 BEGIN_SCAN2 TASK_NAME=LA48 PDD Profiles 194.00 2.1870E+00 ... (5 Replies)
Discussion started by: tintin72
5 Replies

4. Shell Programming and Scripting

AWK: How to extract text lines between two strings

Hi. I have a text test1.txt file like:Receipt Line1 Line2 Line3 End Receipt Line4 Line5 Line6 Canceled Receipt Line7 Line8 Line9 End (9 Replies)
Discussion started by: TQ3
9 Replies

5. UNIX for Advanced & Expert Users

bash/grep/awk/sed: How to extract every appearance of text between two specific strings

I have a text wich looks like this: clid=2 cid=6 client_database_id=35 client_nickname=Peter client_type=0|clid=3 cid=22 client_database_id=57 client_nickname=Paul client_type=0|clid=5 cid=22 client_database_id=7 client_nickname=Mary client_type=0|clid=6 cid=22 client_database_id=6... (3 Replies)
Discussion started by: Pioneer1976
3 Replies

6. Shell Programming and Scripting

Extract strings within XML file between different delimiters

Good afternoon! I have an XML file from which I want to extract only certain elements contained within each line. The problem is that the format of each line is not exactly the same (though similiar). For example, oa_var will be in each line, however, there may be no value or other... (3 Replies)
Discussion started by: bab@faa
3 Replies

7. Shell Programming and Scripting

Extract strings from file - Help

Hi, I have a file say with following lines (the lines could start from any column and there can be many many create statements in the file) create table table1....table definition... insert into table1 values..... create or replace view view1....view definition.... What i want is to... (2 Replies)
Discussion started by: whoami191
2 Replies

8. Shell Programming and Scripting

Extract two strings from a file and create a new file with these strings

I have the following lines in a log file. It would be great if some one can help me to create a new file with the just entries in the below format. 66.150.161.195 HPSAC=Z05 66.150.161.196 HPSAC=A05 That is just extract the IP address and the string DPSAC=its value 66.150.161.195 -... (1 Reply)
Discussion started by: Tuxidow
1 Replies

9. Shell Programming and Scripting

awk extract strings matching multiple patterns

Hi, I wasn't quite sure how to title this one! Here goes: I have some already partially parsed log files, which I now need to extract info from. Because of the way they are originally and the fact they have been partially processed already, I can't make any assumptions on the number of... (8 Replies)
Discussion started by: chrissycc
8 Replies

10. UNIX for Beginners Questions & Answers

Use strings from nth field from one file to match strings in entire line in another file, awk

I cannot seem to get what should be a simple awk one-liner to work correctly and cannot figure out why. I would like to use patterns from a specific field in one file as regex to search for matching strings in the entire line ($0) of another file. I would like to output the lines of File2 which... (1 Reply)
Discussion started by: jvoot
1 Replies
GETDATE(3)						   BSD Library Functions Manual 						GETDATE(3)

NAME
getdate, getdate_err -- convert user format date and time LIBRARY
Standard C Library (libc, -lc) SYNOPSIS
#include <time.h> struct tm * getdate(const char *str); extern int getdate_err; DESCRIPTION
The getdate() function converts a date or time character string pointed to by str into a static tm structure described in tm(3). The input string is parsed and interpreted using templates. A text file containing templates is specified by the environment variable DATEMSK. This should contain the full path to the template file. Lines in the template file represent acceptable date and/or time conver- sion specifications. These specifications are similar to those given for strptime(3). The first line in the template file that matches the input string is used to interpret and convert to internal time format. Internal Format Conversion The following rules apply to converting the input into the internal format. o If only the weekday is given, the conversion assumes today when the weekday matches today or the first future matching weekday. o If only the month and no year is given, the conversion assumes the current month when the month matches or the first future match- ing month. The first day of the month is assumed if no day is given. o If only the year is given, the values of the tm_mon, tm_mday, tm_wday, tm_yday, and tm_isdst members of the returned struct tm are unspecified. o If the century is given, but the year within the century is not given, the conversion assumes the current year. o If no hour, minute, and second are given, the conversion assumes the current hour, minute, and second. o If no date is given, the conversion assumes today when the given hour is greater than the current hour and tomorrow when the given hour is less. o If %Z is being scanned, then the broken-down time is based on the the current time of the matched timezone and not the current run- time environment timezone. RETURN VALUES
If successful, the getdate() function returns a pointer to a static tm structure containing the broken-down time. Otherwise, a null pointer is returned and getdate_err is set to indicate the error. The variable getdate_err can have the following values: 1 DATEMSK environment variable is null or undefined. 2 Cannot open the template file for reading. 3 Get file status failed for template file. 4 Template file is not a regular file. 5 Encountered an error while reading the template file. 6 Cannot allocate memory. 7 Input string does not match any line in the template file. 8 Input string is invalid (for example February 31) or could not be represented in a time_t. ENVIRONMENT
DATEMSK The full path to the text file containing the templates for acceptable date and/or time conversions. FILES
/usr/share/examples/getdate/datemsk.template An example template file that could be specified via the DATEMSK environment variable. EXAMPLES
The following example shows the possible contents of a template file: %m %A %B %d, %Y, %H:%M:%S %A %B %m/%d/%y %I %p %d,%m,%Y %H:%M at %A the %dst of %B in %Y run job at %I %p, %B %dnd %A den %d. %B %Y %H.%M Uhr The following are examples of valid input for the above template: 10/1/87 4 PM Friday Firday September 18, 1987, 10:30:30 24,9,1986 10:30 at monday the 1st of december in 1986 run job at 3 PM, december 2nd The following examples show how local data and time specification can be defined in the template. Input String Line in Template 11/27/86 %m/%d/%y 27.11.86 %d.%m/%y 86-11-27 %y-%m-%d Friday 12:00:00 %A %H:%M:%S The following examples illustrate the Internal Format Conversion rules given that the current date is Mon Sep 22 12:19:47 EDT 1986 and the LC_TIME environment variable is set to the default C locale. Input String Line in Template Date Mon %a Mon Sep 22 12:19:47 EDT 1986 Sun %a Sun Sep 28 12:19:47 EDT 1986 Fri %a Sun Sep 26 12:19:47 EDT 1986 September %B Mon Sep 1 12:19:47 EDT 1986 January %B Thu Jan 1 12:19:47 EST 1987 December %B Mon Dec 1 12:19:47 EST 1987 Sep Mon %b %a Mon Sep 1 12:19:47 EDT 1986 Jan Fri %b %a Fri Jan 2 12:19:47 EDT 1987 Dec Mon %b %a Mon Dec 1 12:19:47 EDT 1986 Jan Wed 1989 %b %a %Y Wed Jan 4 12:19:47 EST 1989 Fri 9 %a %H Fri Sep 26 09:00:00 EDT 1986 Feb 10:30 %b %H:%S Sun Feb 1 10:00:30 EST 1987 10:30 %H:%M Tue Sep 23 10:30:00 EDT 1986 13:30 %H:%M Tue Sep 22 13:30:00 EDT 1986 SEE ALSO
ctime(3), localtime(3), mktime(3), strftime(3), strptime(3), time(3) STANDARDS
The getdate() function conforms to IEEE Std 1003.1-2001 (``POSIX.1''). HISTORY
The getdate function appeared in AT&T System V Release 4 UNIX. BUGS
The getdate interface is inherently unsafe for multi-threaded programs or libraries, since it returns a pointer to a static variable and uses a global state variable. BSD
April 14, 2011 BSD
All times are GMT -4. The time now is 02:14 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy