Can someone please help me optimize my code (script searches subdirectories)? Post: 302607625

Sponsored Content

Top Forums Shell Programming and Scripting Can someone please help me optimize my code (script searches subdirectories)? Post 302607625 by Chubler_XL on Thursday 15th of March 2012 02:08:15 AM

03-15-2012

Moderator

How about this using awk:

Code:

find /path/to/files -type f -print | awk '
NR==FNR{for(i=1;i<=NR;i++) w[tolower($i)]++ ; next }
{ FILE=$0
  while(getline< FILE) {
     for(i=1;i<=NR;i++) {
         if($i && tolower($i) in w) print tolower($i)" is found in: "FILE
      }
  }
  close(FILE)
}' input.txt - >> output.txt

---------- Post updated at 04:08 PM ---------- Previous update was at 03:50 PM ----------

Sorry didn't pick up that the requirement was to find phrases not individual words, this should work but not quite as blazing fast:

Edit: also avoids printing result more than once if phrase appears multiple times in file.

Code:

find /path/to/files -type f -print | awk '
NR==FNR{w[tolower($0)" "]++ ; next}
{ FILE=$0
  delete h
  while(getline< FILE) {
     $0=" "tolower($0)" "
     for(L in w)
           if(!(L in h) && match($0, " "L)) {
              print L "is found in: "FILE
              h[L]++
           }
  }
  close(FILE)
}' input.txt - >> output.txt

Last edited by Chubler_XL; 03-15-2012 at 03:21 AM..

Chubler_XL

View Public Profile for Chubler_XL

Find all posts by Chubler_XL

10 More Discussions You Might Find Interesting

1. News, Links, Events and Announcements

New Tool Searches and Replaces SCO Code

See this article: http://story.news.yahoo.com/news?tmpl=story&cid=74&ncid=738&e=9&u=/cmp/20030809/tc_cmp/13000487

2. Shell Programming and Scripting

optimize the script

Hi, I have this following script below. Its searching a log file for 2 string and if found then write the strings to success.txt and If not found write strings to failed.txt . if one found and not other...then write found to success.txt and not found to failed.txt. I want to optimize this...

3. UNIX for Dummies Questions & Answers

Can we optimize this simple script ?

Hi All , I am just a new bie in Unix/Linux . With help of tips from 'here and there' , I just created a simple script to 1. declare one array and some global variables 2. read the schema names from user (user input) and want2proceed flag 3. if user want to proceed , keep reading user...

4. Shell Programming and Scripting

Optimize and Speedup the script

Hi All, There is a script (test.sh) which is taking more CPU usage. I am attaching the script in this thread. Could anybody please help me out to optimize the script in a better way. Thanks, Gobinath

5. Shell Programming and Scripting

Optimize shell code

#!/usr/bin/perl use strict; use warnings; use Date::Manip; my $date_converted = UnixDate(ParseDate("3 days ago"),"%e/%h/%Y"); open FILE,">$ARGV"; while(<DATA>){ my @tab_delimited_array = split(/\t/,$_); $tab_delimited_array =~ s/^\ =~ s/^\-//; my $converted_date =...

6. Shell Programming and Scripting

pl help me to Optimize the given code

Pl help to me to write the below code in a simple way ... i suupose to use this code 3 to 4 places in my makefile(gnu) .. **************************************** @for i in $(LIST_A); do \ for j in $(LIST_B); do\ if ;then\ echo "Need to sign"\ echo "List A = $$i , List B =$$j"\ ...

7. Shell Programming and Scripting

Optimize my mv script

Hello, I'm wondering if there is a quicker way of doing this. Here is my mv script. d=/conversion/program/out cd $d ls $d > /home/tempuser/$$tmp while read line ; do a=`echo $line|cut -c1-5|sed "s/_//g"` b=`echo $line|cut -c16-21` if ;then mkdir...

8. Shell Programming and Scripting

Looking to optimize code

Hi guys, I feel a bit comfortable now doing bash scripting but I am worried that the way I do it is not optimized and I can do much better as to how I code. e.g. I have a whole line in a file from which I want to extract some values. Right now what I am doing is : STATE=`cat...

9. Shell Programming and Scripting

Optimize awk code

sample data.file: 0,mfrh_green_screen,1454687485,383934,/PROD/G/cicsmrch/sys/unikixmain.log,37M,mfrh_green_screen,28961345,0,382962--383934 0,mfrh_green_screen,1454687785,386190,/PROD/G/cicsmrch/sys/unikixmain.log,37M,mfrh_green_screen,29139568,0,383934--386190...

10. Shell Programming and Scripting

Help Optimize the Script Further

Hi All, I have written a new script to check for DB space and size of dump log file before it can be imported into a Oracle DB. I'm relatively new to shell scripting. Please help me optimize this script further.

LEARN ABOUT REDHAT

zgrep

ZGREP(1)						      General Commands Manual							  ZGREP(1)

NAME

       zgrep - search possibly compressed files for a regular expression

SYNOPSIS

       zgrep [ grep_options ] [ -e ] pattern filename...

DESCRIPTION

       Zgrep  is used to invoke the grep on compress'ed or gzip'ed files. All options specified are passed directly to grep.  If no file is speci-
       fied, then the standard input is decompressed if necessary and fed to grep.  Otherwise the given files are uncompressed	if  necessary  and
       fed to grep.

       If  zgrep  is invoked as zegrep or zfgrep then egrep or fgrep is used instead of grep.  If the GREP environment variable is set, zgrep uses
       it as the grep program to be invoked. For example:

	   for sh:  GREP=fgrep	zgrep string files
	   for csh: (setenv GREP fgrep; zgrep string files)

AUTHOR

       Charles Levert (charles@comm.polymtl.ca)

SEE ALSO

       grep(1), egrep(1), fgrep(1), zdiff(1), zmore(1), znew(1), zforce(1), gzip(1), gzexe(1)

																	  ZGREP(1)

10 More Discussions You Might Find Interesting

1. News, Links, Events and Announcements

New Tool Searches and Replaces SCO Code

Discussion started by: Neo

2. Shell Programming and Scripting

optimize the script

Discussion started by: amitrajvarma

3. UNIX for Dummies Questions & Answers

Can we optimize this simple script ?

Discussion started by: rajavu

4. Shell Programming and Scripting

Optimize and Speedup the script

Discussion started by: ntgobinath