Sponsored Content
Top Forums UNIX for Beginners Questions & Answers Extract strings from the file using awk s Post 303045844 by Mannu2525 on Wednesday 15th of April 2020 10:37:19 AM
Old 04-15-2020
Extract strings from the file using awk s

Hi,

I'm trying to make a file that gives me useful information. Format of file is below:-

Code:
--- (Tue Apr 14 09:46:43 EDT 2020): JOIN_Constraints_Schema:test_simple_joins -------------
internal optimizer errors: 0
--- (Tue Apr 14 09:48:10 EDT 2020): JOIN_Constraints_Schema:test_constraint_joins_setop_dis_oby_lmt -------------
Number of queries that causes internal optimizer errors: 0
--- (Tue Apr 14 09:49:02 EDT 2020): in External_Table_Schema:test_subquery_in_from ---------------
--- (Tue Apr 14 09:49:10 EDT 2020): EventSeries_Schema:test_Event_Series1 -------------
--- (Tue Apr 14 09:49:17 EDT 2020):  Gosalesdw_Schema:test_complex_analytics -------------
--- (Tue Apr 14 09:49:25 EDT 2020):  GBY_Schema1:test_Groupby_Rollup -------------
internal optimizer errors: 0
--- (Tue Apr 14 09:49:40 EDT 2020):  GBY_Schema1:test_Groupby_GroupingSets -------------
internal optimizer errors: 0
--- (Tue Apr 14 09:49:52 EDT 2020):  GBY_Schema1:test_Groupby_Cube -------------
internal optimizer errors: 0
--- (Tue Apr 14 09:50:05 EDT 2020):  GBY_Schema1:test_gby -------------
internal optimizer errors: 0

I need to extract the text after the timestamp and immediately next line error number having internal optimizer error.

It should be like - newout.txt
Code:
JOIN_Constraints_Schema,test_simple_joins,0
JOIN_Constraints_Schema,test_constraint_joins_setop_dis_oby_lmt,0
GBY_Schema1,test_Groupby_Rollup,0
GBY_Schema1,test_Groupby_GroupingSets,0
GBY_Schema1,test_Groupby_Cube,0
GBY_Schema1,test_gby,0

Exception - If there is no internal optimizer error attached to any field given in timestamp then no need to print

I tried using a simple script but not works for me

Code:
cat newout.txt | while read x
do
        if [ $(echo $x | grep -E '20[0-9][0-9]\):' | wc -l) == 1 ]
        then
                 schema=$(echo $x | awk '{print $(NF-1)}' | awk -F ':' '{print $1}')
                 config=$(echo $x | awk '{print $(NF-1)}' | awk -F ':' '{print $2}')
         else
                 internal=$(echo $x | awk -F ':' '{print $2}')
         fi
         echo $schema,$config,$internal
done

Please help me achieve this.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

using AWK how to extract text between two same strings

I have a file like: myfile.txt it is easy to learn awk and begin awk scripting and awk has got many features awk is a powerful text processing tool Now i want to get the text between first awk and immediate awk not the third awk . How to get it ? its urgent pls help me and file is unevenly... (2 Replies)
Discussion started by: santosh1234
2 Replies

2. Shell Programming and Scripting

using awk to extract text between two constant strings

Hi, I have a file from which i need to extract data between two constant strings. The data looks like this : Line 1 SUN> read db @cmpd unit 60 Line 2 Parameter: CMPD -> "C00071" Line 3 Line 4 SUN> generate Line 5 tabint>ERROR: (Variable data) The data i need to extract is... (11 Replies)
Discussion started by: mjoshi
11 Replies

3. UNIX for Dummies Questions & Answers

Using awk/sed to extract text between Strings

Dear Unix Gurus, I've got a data file with a few hundred lines (see truncated sample)... BEGIN_SCAN1 TASK_NAME=LA48 PDD Profiles PROGRAM=ArrayScan 1.00 21.220E+00 2.00 21.280E+00 END_DATA END_SCAN1 BEGIN_SCAN2 TASK_NAME=LA48 PDD Profiles 194.00 2.1870E+00 ... (5 Replies)
Discussion started by: tintin72
5 Replies

4. Shell Programming and Scripting

AWK: How to extract text lines between two strings

Hi. I have a text test1.txt file like:Receipt Line1 Line2 Line3 End Receipt Line4 Line5 Line6 Canceled Receipt Line7 Line8 Line9 End (9 Replies)
Discussion started by: TQ3
9 Replies

5. UNIX for Advanced & Expert Users

bash/grep/awk/sed: How to extract every appearance of text between two specific strings

I have a text wich looks like this: clid=2 cid=6 client_database_id=35 client_nickname=Peter client_type=0|clid=3 cid=22 client_database_id=57 client_nickname=Paul client_type=0|clid=5 cid=22 client_database_id=7 client_nickname=Mary client_type=0|clid=6 cid=22 client_database_id=6... (3 Replies)
Discussion started by: Pioneer1976
3 Replies

6. Shell Programming and Scripting

Extract strings within XML file between different delimiters

Good afternoon! I have an XML file from which I want to extract only certain elements contained within each line. The problem is that the format of each line is not exactly the same (though similiar). For example, oa_var will be in each line, however, there may be no value or other... (3 Replies)
Discussion started by: bab@faa
3 Replies

7. Shell Programming and Scripting

Extract strings from file - Help

Hi, I have a file say with following lines (the lines could start from any column and there can be many many create statements in the file) create table table1....table definition... insert into table1 values..... create or replace view view1....view definition.... What i want is to... (2 Replies)
Discussion started by: whoami191
2 Replies

8. Shell Programming and Scripting

Extract two strings from a file and create a new file with these strings

I have the following lines in a log file. It would be great if some one can help me to create a new file with the just entries in the below format. 66.150.161.195 HPSAC=Z05 66.150.161.196 HPSAC=A05 That is just extract the IP address and the string DPSAC=its value 66.150.161.195 -... (1 Reply)
Discussion started by: Tuxidow
1 Replies

9. Shell Programming and Scripting

awk extract strings matching multiple patterns

Hi, I wasn't quite sure how to title this one! Here goes: I have some already partially parsed log files, which I now need to extract info from. Because of the way they are originally and the fact they have been partially processed already, I can't make any assumptions on the number of... (8 Replies)
Discussion started by: chrissycc
8 Replies

10. UNIX for Beginners Questions & Answers

Use strings from nth field from one file to match strings in entire line in another file, awk

I cannot seem to get what should be a simple awk one-liner to work correctly and cannot figure out why. I would like to use patterns from a specific field in one file as regex to search for matching strings in the entire line ($0) of another file. I would like to output the lines of File2 which... (1 Reply)
Discussion started by: jvoot
1 Replies
jstatd(1)						      General Commands Manual							 jstatd(1)

NAME
jstatd - Virtual Machine jstat Daemon SYNOPSIS
jstatd [ options ] DESCRIPTION
The jstatd tool is an RMI server application that monitors for the creation and termination of instrumented HotSpot Java virtual machines (JVMs) and provides a interface to allow remote monitoring tools to attach to JVMs running on the local host. The jstatd server requires the presence of an RMI registry on the local host. The jstatd server will attempt to attach to the RMI registry on the default port, or on the port indicated by the -p port option. If an RMI registry is not found, one will be created within the jstatd application bound to the port indicated by the -p port option or to the default RMI registry port if -p port is omitted. Creation of an internal RMI registry can be inhibited by specifying the -nr option. NOTE - This utility is unsupported and may or may not be available in future versions of the J2SE SDK. It is not currently available on the Windows 98 and Windows ME platforms. PARAMETERS
options Command-line options. The options may be in any order. If there are redundant or contradictory options, the last option specified will take precedence. OPTIONS
The jstatd command supports the following options: -nr Do not attempt to create an internal RMI registry within the jstatd process when an existing RMI registry is not found. -p port Port number where the RMI registry is expected to be found, or, if not found, created if -nr is not specified. -n rminame Name to which the remote RMI object is bound in the RMI registry. The default name is JStatRemoteHost. If multiple jstatd servers are started on the same host, the name of the exported RMI object for each server can be made unique by by specify- ing this option. However, doing so will require that the unique server name be included in the monitoring client's hostid and vmid strings. -Joption Pass option to the java launcher called by javac. For example, -J-Xms48m sets the startup memory to 48 megabytes. It is a common convention for -J to pass options to the underlying VM executing applications written in Java. SECURITY
The jstatd server installs an instance of RMISecurityPolicy if no other security manager has been installed and therefore requires a secu- rity policy file to be specified. The policy file must conform to the default policy implementation's Policy File Syntax. The policy file can be specified with the -J-Djava.security.policy=file The following policy file will allow the jstatd server to run without any security exceptions. This policy is less liberal then granting all permissions to all codebases, but is more liberal than a policy that grants the minimal permissions to run the jstatd server. grant codebase "file:${java.home}/../lib/tools.jar" { permission java.security.AllPermission; }; To use this policy, copy the text into a file called jstatd.all.policy and run the jstatd server as follows: jstatd -J-Djava.security.policy=jstatd.all.policy For sites with more restrictive security practices, Sun recommends that the jstatd security policy be customized to meet your specific needs. The jstatd server can only monitor JVMs for which it has the appropriate access permissions. However, jstatd does not perform any user level authentication or authorization checking. Therefore, it opens access to the instrumentation exported by the JVMs for which the jstatd server has the appropriate access permissions, allowing arbitrary users on the network to monitor JVMs that might otherwise be inac- cessible. Such exposure may be unacceptable in your environment. Particular care should be exercised when running the jstatd server with credentials that allow wide exposure, such as running the server with root permissions on UNIX based systems. The exposure introduced by the jstatd server can be eliminated by not running the server, thus requiring all monitoring activities to be performed locally. Alternatively, the security policy file can be customized to limit access to specific trusted hosts. REMOTE INTERFACE
The interface exported by the jstatd process is proprietary and is guaranteed to change. Users and developers are discouraged from writing to this interface. EXAMPLES
Here are some examples of starting jstatd. Note that the jstatd scripts automatically start the server in the background. Using Internal RMI Registry This example demonstrates starting jstatd with an internal RMI registry. This example assumes that no other server is bound to the default RMI Registry port (port 1099). jstatd -J-Djava.security.policy=all.policy Using External RMI Registry This example demonstrates starting jstatd with a external RMI registry. rmiregistry& jstatd -J-Djava.security.policy=all.policy This example demonstrates starting jstatd with an external RMI registry server on port 2020. rmiregistry 2020& jstatd -J-Djava.security.policy=all.policy -p 2020 This example demonstrates starting jstatd with an external RMI registry on port 2020, bound to name AlternateJstatdServerName. rmiregistry 2020& jstatd -J-Djava.security.policy=all.policy -p 2020 -n AlternateJstatdServerName Inhibiting creation of an in-process RMI registry This example demonstrates starting jstatd such that it will not create a RMI registry if one is not found. This example assumes an RMI reg- istry is already running. If it is not, an appropriate error message is emitted. jstatd -nr Enabling RMI logging capabilities This example demonstrates starting jstatd with RMI logging capabilities enabled. This technique is useful as a troubleshooting aid or for monitoring server activities. jstatd -J-Djava.security.policy=all.policy -J-Djava.rmi.server.logCalls=true This example uses the Bourne Shell syntax for setting environment variables, other shells or command interpreters may require different syntax. SEE ALSO
java(1) jps(1) jstat(1) rmiregistry(1) 13 June 2004 jstatd(1)
All times are GMT -4. The time now is 07:17 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy