Sponsored Content
Top Forums Shell Programming and Scripting Help 'speeding' up this 'parsing' script - taking 24+ hours to run Post 303020615 by newbie_01 on Tuesday 24th of July 2018 05:33:14 AM
Old 07-24-2018
Hi Rudic

Sorry it has taken me awhile to test. It is giving some errors as below:

This is what I ran:

Code:
$ ./x.ksh
awk: cmd. line:14: warning: escape sequence `\(' treated as plain `('
awk: cmd. line:6: (FILENAME=x.txt FNR=1) fatal: Unmatched ( or \(: /CID=(PROGRAM=[^)]*/

Here's the script with the awk code:

Code:
$ cat x.ksh
#!/bin/ksh

awk -F\* '
BEGIN   {for (n=split("JAN*FEB*MAR*APR*MAY*JUN*JUL*AUG*SEP*OCT*NOV*DEC", T); n; n--) MTH[T[n]] = n
         "hostname" | getline HN
        }

function GETSTR(SRC, STR)       {match (SRC, STR "[^)]*")
                                 LN = length(STR) - gsub (/\(/, "&", STR)
                                 return substr (SRC, RSTART+LN, RLENGTH-LN)
                                }

        {gsub (/ *\* */, "*")
         split ($1, T, "[- ]")
         if (T[2] in MTH) $1 = sprintf ("%s-%02d-%s %s", T[3], MTH[T[2]], T[1], T[4])
         PG = GETSTR($2, "CID=\(PROGRAM=")
         US = GETSTR($2, "USER=")
         SN = GETSTR($2, "SERVICE_NAME=")
         IP = GETSTR($3, "HOST=")
         print $1, HN, "= " IP, PG, US, SN, $NF
        }
' OFS="\t" x.txt

Below is the input file to awk:

Code:
$ cat x.txt
12-MAR-2018 16:23:09 * (CONNECT_DATA=(CID=(PROGRAM=JDBC Thin Client)(HOST=__jdbc__)(USER=ogre01))(SERVER=DEDICATED)(SERVICE_NAME=testapp1_app.somewhere.out.ph)) * (ADDRESS=(PROTOCOL=tcp)(HOST=66.65.60.7)(PORT=11662)) * establish * testapp1_app.somewhere.out.ph * 12514
12-MAR-2018 10:04:38 *  (CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=testapp1_app.somewhere.out.ph)(CID=(PROGRAM=sqlplus)(HOST=xxx00001.somewhere.out.ph)(USER=ogre01)))  * (ADDRESS=(PROTOCOL=tcp)(HOST=66.65.60.101)(PORT=12358)) * establish *  testapp1_app.somewhere.out.ph * 12514

And here's my awk version. I tried with gawk and it gives the same error. I have no nawk.

Code:
$ awk --version
GNU Awk 3.1.7
Copyright (C) 1989, 1991-2009 Free Software Foundation.

This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 3 of the License, or
(at your option) any later version.

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

You should have received a copy of the GNU General Public License
along with this program. If not, see http://www.gnu.org/licenses/.

 

10 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

FTP taking ages to run.

Hi every one, We have HP UX server which normally loaded as avg load of 19-21. NOw when I try and do ftp to this server it takes ages to get the FTP prompt. I have seen this server loaded as max agv load of 35-40 tht time we never had such problems of FTP sessions. Now my new Unix admin... (1 Reply)
Discussion started by: nilesrex
1 Replies

2. Shell Programming and Scripting

How to make a script run for a maximum of "x" number of hours only

How to make a script run for a maximum of "x" number of hours only (7 Replies)
Discussion started by: ScriptDummy
7 Replies

3. UNIX for Dummies Questions & Answers

Speeding up a Shell Script (find, grep and a for loop)

Hi all, I'm having some trouble with a shell script that I have put together to search our web pages for links to PDFs. The first thing I did was: ls -R | grep .pdf > /tmp/dave_pdfs.outWhich generates a list of all of the PDFs on the server. For the sake of arguement, say it looks like... (8 Replies)
Discussion started by: Dave Stockdale
8 Replies

4. HP-UX

Crontab do not run on PM hours

Hi All I have a problem, I wonder if you can help me sort it out: I have the following entry in the cron: 00 1,13 * * * /home/report/opn_amt_gestores_credito.ksh > opn_amt_gestores_credito.log But the entry only runs at 01:07 I have stopped the cron deamon, and started, but it still... (39 Replies)
Discussion started by: fretagi
39 Replies

5. Shell Programming and Scripting

Parsing log file for last 2 hours

I want to parse a log file which i am grepping root user connection but is showing whole day and previous day detail as well. First i want to see last 2 hours log file then after that i want to search particular string. Lets suppose right now its 5:00PM, So i want to see the log of 3:00PM to... (6 Replies)
Discussion started by: learnbash
6 Replies

6. Shell Programming and Scripting

Help speeding up script

This is my first experience writing unix script. I've created the following script. It does what I want it to do, but I need it to be a lot faster. Is there any way to speed it up? cat 'Tax_Provision_Sample.dat' | sort | while read p; do fn=`echo $p|cut -d~ -f2,4,3,8,9`; echo $p >> "$fn.txt";... (20 Replies)
Discussion started by: JohnN6
20 Replies

7. UNIX for Advanced & Expert Users

Zip million files taking 12 hours or more

Hi I have task to zip files based on modified time but they are in millions and it is taking lot of time more than 12 hours and also eating up high cpu is there any other / better way to handle it quickly with less cpu consumptionfind . ! -name \"*.gz\" -mtime +7 -type f | grep -v '/.*/' |... (2 Replies)
Discussion started by: reldb
2 Replies

8. Shell Programming and Scripting

Speeding up shell script with grep

HI Guys hoping some one can help I have two files on both containing uk phone numbers master is a file which has been collated over a few years ad currently contains around 4 million numbers new is a file which also contains 4 million number i need to split new nto two separate files... (4 Replies)
Discussion started by: dunryc
4 Replies

9. Shell Programming and Scripting

Run a command once in three hours

Hi All, I have a bash script which is scheduled to run for every 20 minutes. Inside the bash script, one command which I am using need to be triggered only once in two or three hours.Is there anyway to achieve this. For example, if then echo "hi" else echo "Hello" UNIX Command---once... (5 Replies)
Discussion started by: ginrkf
5 Replies

10. Shell Programming and Scripting

Help with speeding up my working script to take less time - how to use more CPU usage for a script

Hello experts, we have input files with 700K lines each (one generated for every hour). and we need to convert them as below and move them to another directory once. Sample INPUT:- # cat test1 1559205600000,8474,NormalizedPortInfo,PctDiscards,0.0,Interface,BG-CTA-AX1.test.com,Vl111... (7 Replies)
Discussion started by: prvnrk
7 Replies
getopt(1)						      General Commands Manual							 getopt(1)

NAME
getopt - Parses command line flags and arguments SYNOPSIS
getopt format tokens DESCRIPTION
The getopt command is used to parse a list of tokens using a format that specifies expected flags and arguments. A flag is a single ASCII letter and, when followed by a : (colon), is expected to take a modifying argument that may or may not be separated from it by one or more tabs or spaces. (You can include multi-byte characters in arguments, but not as flag letters.) The getopt command completes processing when it has read all tokens or when it encounters the special token -- (double dash). It then out- puts the processed flags, a --, and any remaining tokens. If a token fails to match a flag, getopt writes a message to standard error. NOTES
In the csh, use the following command to run getopt: set argv=`getopt flag_string $*` EXAMPLES
The following is an example of the use of getopt in a skeleton shell script to parse options: #!/bin/sh # parse command line into arguments set -- `getopt a:bc $*` # check result of parsing if [ $? != 0 ] then exit 1 fi while [ $1 != -- ] do case $1 in -a) # set up the -a flag AFLG=1 AARG=$2 shift;; -b) # set up the -b flag BFLG=1;; -c) # set up the -c flag CFLG=1;; esac shift # next flag done shift # skip double dash # now do the work . . . The following are all equivalent arguments to the script: -a ARG -b -c -- A B C -a ARG -bc -- A B C -aARG -b -c -- A B C -b -c -a ARG -- A B C SEE ALSO
Commands: sh(1) Functions: getopt(3) getopt(1)
All times are GMT -4. The time now is 05:04 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy