Sponsored Content
Top Forums Shell Programming and Scripting Help 'speeding' up this 'parsing' script - taking 24+ hours to run Post 303016895 by newbie_01 on Saturday 5th of May 2018 02:53:32 PM
Old 05-05-2018
Quote:
Originally Posted by RudiC
Not sure why the service name comes in field $4 sometimes, shoving other fields right, and in field $6 other times...
How far do you get with
Code:
awk -F\* '
BEGIN   {for (n=split("JAN*FEB*MAR*APR*MAY*JUN*JUL*AUG*SEP*OCT*NOV*DEC", T); n; n--) MTH[T[n]] = n
         "hostname" | getline HN
        }

function GETSTR(SRC, STR)       {match (SRC, STR "[^)]*")
                                 LN = length(STR) - gsub (/\(/, "&", STR)
                                 return substr (SRC, RSTART+LN, RLENGTH-LN)
                                }

        {gsub (/ *\* */, "*")
         split ($1, T, "[- ]")
         if (T[2] in MTH) $1 = sprintf ("%s-%02d-%s %s", T[3], MTH[T[2]], T[1], T[4])
         PG = GETSTR($2, "CID=\(PROGRAM=")
         US = GETSTR($2, "USER=")
         SN = GETSTR($2, "SERVICE_NAME=")
         IP = GETSTR($3, "HOST=")
         print $1, HN, "= " IP, PG, US, SN, $NF
        }
' OFS="\t" file
2018-03-24 07:59:52    RudisPC    = 66.65.60.7    JDBC Thin Client    ogre01    testapp1_app.somewhere.out.ph    12514
2018-03-24 07:59:52    RudisPC    = 66.65.60.7    JDBC Thin Client    ogre01    testapp1_app.somewhere.out.ph    12514
2018-03-12 10:04:38    RudisPC    = 66.65.60.101    sqlplus    ogre01    testapp1_app.somewhere.out.ph    12514
2018-03-12 16:23:09    RudisPC    = 66.65.60.7    JDBC Thin Client    ogre01    testapp1_app.somewhere.out.ph    12514
2018-03-12 16:23:09    RudisPC    = 66.65.60.7    JDBC Thin Client    ogre01    testapp1_app.somewhere.out.ph    12514
2018-03-12 16:23:09    RudisPC    = 66.65.60.7    JDBC Thin Client    ogre01    testapp1_app.somewhere.out.ph    12514
.
.
.


Yeah, I hate that fact too, that the service name divert from field to field. Looking at the lines, it has to do with the request being a JDBC connection or otherwise. I'll give the awk bit to work. Thanks a lot.

---------- Post updated at 01:53 PM ---------- Previous update was at 01:40 PM ----------

Sorry, I've been sick for awhile. Thanks a lot for all your advise. I will give all of the suggestion with a cut down version of the file. I will have a real long read and understand how to implement your suggestion. Wish me luck. Thanks again everyone.
 

10 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

FTP taking ages to run.

Hi every one, We have HP UX server which normally loaded as avg load of 19-21. NOw when I try and do ftp to this server it takes ages to get the FTP prompt. I have seen this server loaded as max agv load of 35-40 tht time we never had such problems of FTP sessions. Now my new Unix admin... (1 Reply)
Discussion started by: nilesrex
1 Replies

2. Shell Programming and Scripting

How to make a script run for a maximum of "x" number of hours only

How to make a script run for a maximum of "x" number of hours only (7 Replies)
Discussion started by: ScriptDummy
7 Replies

3. UNIX for Dummies Questions & Answers

Speeding up a Shell Script (find, grep and a for loop)

Hi all, I'm having some trouble with a shell script that I have put together to search our web pages for links to PDFs. The first thing I did was: ls -R | grep .pdf > /tmp/dave_pdfs.outWhich generates a list of all of the PDFs on the server. For the sake of arguement, say it looks like... (8 Replies)
Discussion started by: Dave Stockdale
8 Replies

4. HP-UX

Crontab do not run on PM hours

Hi All I have a problem, I wonder if you can help me sort it out: I have the following entry in the cron: 00 1,13 * * * /home/report/opn_amt_gestores_credito.ksh > opn_amt_gestores_credito.log But the entry only runs at 01:07 I have stopped the cron deamon, and started, but it still... (39 Replies)
Discussion started by: fretagi
39 Replies

5. Shell Programming and Scripting

Parsing log file for last 2 hours

I want to parse a log file which i am grepping root user connection but is showing whole day and previous day detail as well. First i want to see last 2 hours log file then after that i want to search particular string. Lets suppose right now its 5:00PM, So i want to see the log of 3:00PM to... (6 Replies)
Discussion started by: learnbash
6 Replies

6. Shell Programming and Scripting

Help speeding up script

This is my first experience writing unix script. I've created the following script. It does what I want it to do, but I need it to be a lot faster. Is there any way to speed it up? cat 'Tax_Provision_Sample.dat' | sort | while read p; do fn=`echo $p|cut -d~ -f2,4,3,8,9`; echo $p >> "$fn.txt";... (20 Replies)
Discussion started by: JohnN6
20 Replies

7. UNIX for Advanced & Expert Users

Zip million files taking 12 hours or more

Hi I have task to zip files based on modified time but they are in millions and it is taking lot of time more than 12 hours and also eating up high cpu is there any other / better way to handle it quickly with less cpu consumptionfind . ! -name \"*.gz\" -mtime +7 -type f | grep -v '/.*/' |... (2 Replies)
Discussion started by: reldb
2 Replies

8. Shell Programming and Scripting

Speeding up shell script with grep

HI Guys hoping some one can help I have two files on both containing uk phone numbers master is a file which has been collated over a few years ad currently contains around 4 million numbers new is a file which also contains 4 million number i need to split new nto two separate files... (4 Replies)
Discussion started by: dunryc
4 Replies

9. Shell Programming and Scripting

Run a command once in three hours

Hi All, I have a bash script which is scheduled to run for every 20 minutes. Inside the bash script, one command which I am using need to be triggered only once in two or three hours.Is there anyway to achieve this. For example, if then echo "hi" else echo "Hello" UNIX Command---once... (5 Replies)
Discussion started by: ginrkf
5 Replies

10. Shell Programming and Scripting

Help with speeding up my working script to take less time - how to use more CPU usage for a script

Hello experts, we have input files with 700K lines each (one generated for every hour). and we need to convert them as below and move them to another directory once. Sample INPUT:- # cat test1 1559205600000,8474,NormalizedPortInfo,PctDiscards,0.0,Interface,BG-CTA-AX1.test.com,Vl111... (7 Replies)
Discussion started by: prvnrk
7 Replies
RT::Client::REST::Exception(3pm)			User Contributed Perl Documentation			  RT::Client::REST::Exception(3pm)

NAME
RT::Client::REST::Exception -- exceptions thrown by RT::Client::REST methods. DESCRIPTION
These are exceptions that are thrown by various RT::Client::REST methods. EXCEPTION HIERARCHY
RT::Client::REST::Exception This exception is virtual -- it is never thrown. It is used to group all the exceptions in this category. RT::Client::REST::OddNumberOfArgumentsException This means that the method you called wants key-value pairs. RT::Client::REST::InvaildObjectTypeException Thrown when you specify an invalid type to "show()", "edit()", or "search()" methods. RT::Client::REST::RequiredAttributeUnsetException An operation failed because a required attribute was not set in the object. RT::Client::REST::MalformedRTResponseException RT server sent response that we cannot parse. This may very well mean a bug in this client, so if you get this exception, some debug information mailed to the author would be appreciated. RT::Client::REST::InvalidParameterValueException Invalid value for comments, link types, object IDs, etc. RT::Client::REST::CannotReadAttachmentException Cannot read attachment (thrown from methods "comment()" and "correspond"). RT::Client::REST::RTException This is a virtual exception and is never thrown. It is used to group exceptions thrown because RT server returns an error. RT::Client::REST::ObjectNotFoundException One or more of the specified objects was not found. RT::Client::REST::AuthenticationFailureException Incorrect username or password. RT::Client::REST::UpdateException This is a virtual exception. It is used to group exceptions thrown when RT server returns an error trying to update an object. RT::Client::REST::CouldNotSetAttributeException For one or another reason, attribute could not be updated with the new value. RT::Client::REST::InvalidEmailAddressException Invalid e-mail address specified. RT::Client::REST::AlreadyCurrentValueException The attribute you are trying to update already has this value. I do not know why RT insists on treating this as an exception, but since it does so, so should the client. You can probably safely catch and throw away this exception in your code. RT::Client::REST::ImmutableFieldException Trying to update an immutable field (such as "last_updated", for example). RT::Client::REST::IllegalValueException Illegal value for attribute was specified. RT::Client::REST::UnknownCustomFieldException Unknown custom field was specified in the request. RT::Client::REST::InvalidQueryException Server could not parse the search query. RT::Client::REST::UnauthorizedActionException You are not authorized to perform this action. RT::Client::REST::AlreadyTicketOwnerException The owner you are trying to assign to a ticket is already the owner. This exception is usually thrown by methods "take()", "untake", and "steal", if the operation is a noop. RT::Client::REST::RequestTimedOutException Request timed out. RT::Client::REST::UnknownRTException Some other RT exception that the driver cannot recognize. METHODS
_get_exception_class Figure out exception class based on content returned by RT. _rt_content_to_exception Translate error string returned by RT server into an exception object ready to be thrown. SEE ALSO
Exception::Class, RT::Client::REST. AUTHOR
Dmitri Tikhonov <dtikhonov@yahoo.com> perl v5.14.2 2011-12-27 RT::Client::REST::Exception(3pm)
All times are GMT -4. The time now is 02:08 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy