Optimize shell code Post: 302434970

Sponsored Content

Top Forums Shell Programming and Scripting Optimize shell code Post 302434970 by sandy1028 on Tuesday 6th of July 2010 03:26:02 AM

07-06-2010

Registered User

Optimize shell code

Code:

#!/usr/bin/perl
use strict;
use warnings;

use Date::Manip;
my $date_converted = UnixDate(ParseDate("3 days ago"),"%e/%h/%Y");
open FILE,">$ARGV[0]";

while(<DATA>){

my @tab_delimited_array = split(/\t/,$_);
  $tab_delimited_array[3] =~ s/^\[//;
  $tab_delimited_array[3] =~ s/^\-//;
        my $converted_date = Date_ConvTZ( UnixDate($tab_delimited_array[3],"%Y%m%d%H:%M:%S"),'GMT','PST');
        my $pst_converted_date = UnixDate($converted_date,"%e/%h/%Y:%H:%M:%S");
        $pst_converted_date =~ s/^\s//g;
        my $extracted_YMD=UnixDate($converted_date,"%e/%h/%Y");
if($extracted_YMD =~ m/$date_converted/){
print FILE $_;
}
}

close FILE;
close DATA;


__DATA__
66.249.65.43    www.cabc.com    -       3/Jul/2010:19:00:01     +0000   "GET http://www.abc.com HTTP/1.1"       200     0.730   7362    0       0       -       -       Mozilla/5.0+(compatible;+Googl/2.1;+\+http://www.google.com/bot.html)   TCP_MISS
66.249.65.43    www.cabc.com    -       4/Jul/2010:19:00:01     +0000   "GET http://www.abc.com HTTP/1.1"       200     0.730   7362    0       0       -       -       Mozilla/5.0+(compatible;+Googl/2.1;+\+http://www.google.com/bot.html)   TCP_MISS
66.249.65.43    www.cabc.com    -       2/Jul/2010:09:00:01     +0000   "GET http://www.abc.com HTTP/1.1"       200     0.730   7362    0       0       -       -       Mozilla/5.0+(compatible;+Googl/2.1;+\+http://www.google.com/bot.html)   TCP_MISS
66.249.65.43    www.cabc.com    -       3/Jul/2010:09:00:01     +0000   "GET http://www.abc.com HTTP/1.1"       200     0.730   7362    0       0       -       -       Mozilla/5.0+(compatible;+Googl/2.1;+\+http://www.google.com/bot.html)   TCP_MISS

---------- Post updated at 02:26 AM ---------- Previous update was at 02:25 AM ----------

The file is very large and it takes long time to execute.
The input is tab delimited file

sandy1028

View Public Profile for sandy1028

Find all posts by sandy1028

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

optimize shell script (snapshots)

I've a script to do some snapshots but the time it does so is very different... once i got a snapshot under 1 sec, on the other hand it took 3 sec, but nothing else changed, i didnt even move the cursor or something. I put the script on a ramdisk and its faster, but still swing from under 1...

2. Shell Programming and Scripting

pl help me to Optimize the given code

Pl help to me to write the below code in a simple way ... i suupose to use this code 3 to 4 places in my makefile(gnu) .. **************************************** @for i in $(LIST_A); do \ for j in $(LIST_B); do\ if ;then\ echo "Need to sign"\ echo "List A = $$i , List B =$$j"\ ...

3. Shell Programming and Scripting

Optimize the nested IF

Hi, I have to assign a value for a varaiable based on a Input. I have written the below code: if then nf=65 elif then nf=46 elif then nf=164 elif then nf=545 elif then nf=56 elif then

4. Shell Programming and Scripting

Can someone please help me optimize my code (script searches subdirectories)?

Here is my code. What it does is it reads an input file (input.txt which contains roughly 2,000 search phrases) and searches a directory for files that contains the search phrase. The directory contains roughly 1900 files and 84 subdirectories. The output is a file (output.txt) that shows only the...

5. UNIX for Dummies Questions & Answers

optimize if block : shell script

Hi, I need a shell script to determine if a no. is either even, greater than 4, less than 8 SHELL : ksh OS : RHEL 6 this is the if block of the script mod=`expr $num % 2` if || || then echo "No. is either even or greater than 4 or less than 8" fi this code works...

6. Shell Programming and Scripting

Optimize my mv script

Hello, I'm wondering if there is a quicker way of doing this. Here is my mv script. d=/conversion/program/out cd $d ls $d > /home/tempuser/$$tmp while read line ; do a=`echo $line|cut -c1-5|sed "s/_//g"` b=`echo $line|cut -c16-21` if ;then mkdir...

7. Shell Programming and Scripting

Optimize shell script to run faster

data.file: contact { contact_name=royce-rolls modified_attributes=0 modified_host_attributes=0 modified_service_attributes=0 host_notification_period=24x7 service_notification_period=24x7 last_host_notification=0 last_service_notification=0 host_notifications_enabled=1...

8. Shell Programming and Scripting

Looking to optimize code

Hi guys, I feel a bit comfortable now doing bash scripting but I am worried that the way I do it is not optimized and I can do much better as to how I code. e.g. I have a whole line in a file from which I want to extract some values. Right now what I am doing is : STATE=`cat...

9. Shell Programming and Scripting

Optimize awk code

sample data.file: 0,mfrh_green_screen,1454687485,383934,/PROD/G/cicsmrch/sys/unikixmain.log,37M,mfrh_green_screen,28961345,0,382962--383934 0,mfrh_green_screen,1454687785,386190,/PROD/G/cicsmrch/sys/unikixmain.log,37M,mfrh_green_screen,29139568,0,383934--386190...

10. Shell Programming and Scripting

Help Optimize the Script Further

Hi All, I have written a new script to check for DB space and size of dump log file before it can be imported into a Oracle DB. I'm relatively new to shell scripting. Please help me optimize this script further.

LEARN ABOUT DEBIAN

modgzip2dlf

MODGZIP2DLF.IN(1)					  LogReport's Lire Documentation					 MODGZIP2DLF.IN(1)

NAME

       modgzip2dlf - convert Common Log Format with mod_gzip extensions web server log files to DLF

SYNOPSIS

       modgzip2dlf file

DESCRIPTION

       modgzip2dlf converts web server log file in the common log format with extensions for mod_gzip to www DLF.

       mod_gzip is an Apache extension developed by RemoteCommunications, Inc. Sourcecode is free available from
       http://www.RemoteCommunications.com/apache/mod_gzip/mod_gzip. More informations can be found at
       http://www.RemoteCommunications.com/apache/mod_gzip/mod_gzip_faq.htm.

       mod_gzip can log informations about the compression of pages by configuring Apache to log using the 'gzip' format which can be defined as
       follows:

	LogFormat "%h %l %u %t "%r" %>s %b %{mod_gzip_result}n %{mod_gzip_compression_ratio}n" gzip

MOD_GZIP EXTENSIONS
       The mod_gzip extensions adds two fields to the Common Log Format; see common2dlf(1):

	 gzip_result compression_ratio

       gzip_result
	   The gzip's result code. Usually OK.

       compressio_ratio
	   The ratio by which the content was compressed. A number from 0 to 100.

       Log example :

	 127.0.01 - - [11/03/2001 12:12:01 -0400] "GET / HTTP/1.0" 200 252 "-" "Mozilla/5.0" OK 49
	 dsl1.myprovider.com - francis [11/03/2001 12:14:01 -0400] "GET /secret/ HTTP/1.0" 200 300 "http://www.example.com/index.html" "Mozilla/5.0" OK 74

EXAMPLES

       To process a log as produced in the modgzip logformat:

	$ modgzip2dlf < common.log

       modgzip2dlf will be rarely used on its own, but is more likely called by lr_log2report:

	$ lr_log2report modgzip < /var/log/httpd/common.log

SEE ALSO

       common2dlf(1), combined(2), referer2dlf(1)

AUTHORS

       Joost van Baal <joostvb@logreport.org>, Francis J. Lacoste <flacoste@logreport.org> and Egon Willighagen <egonw@logreport.org>, based on an
       idea by Edwin Groothuis

VERSION

       $Id: modgzip2dlf.in,v 1.10 2006/07/23 13:16:36 vanbaal Exp $

COPYRIGHT

       Copyright (C) 2000, 2001 Stichting LogReport Foundation LogReport@LogReport.org

       This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by
       the Free Software Foundation; either version 2 of the License, or (at your option) any later version.

       This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
       MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for more details.

       You should have received a copy of the GNU General Public License along with this program (see COPYING); if not, check with
       http://www.gnu.org/copyleft/gpl.html.

Lire 2.1.1							    2006-07-23							 MODGZIP2DLF.IN(1)