Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: need to process for loop faster
Top Forums Shell Programming and Scripting need to process for loop faster Post 302642515 by SkySmart on Thursday 17th of May 2012 02:14:25 PM
Old 05-17-2012
need to process for loop faster
I have the following code running against a file. The file can have upwards of 10000 lines.

problem is, the for loop takes a while to go through all those lines. is there a faster way to go about it?


Code:
for line in `grep -P "${MONTH} ${DAY}," file | ${AWK} -F" " '{print $4}' | awk -F":" '{print $1":"$2}' | sort | uniq`

do
.....
.....
done

file is:

Code:
<Mar 31, 2012 4:40:40 PM UTC> <Warning> <Socket> <BEA-00670450> <Socket 211 internal data record unavailable (probable closure due idle timeout), event received 17>
<Mar 31, 2012 5:40:40 PM UTC> <Warning> <Socket> <BEA-00670450> <Socket 211 internal data record unavailable (probable closure due idle timeout), event received 17>
<Mar 31, 2012 6:40:40 PM UTC> <Warning> <Socket> <BEA-00670450> <Socket 211 internal data record unavailable (probable closure due idle timeout), event received 17>
<Mar 31, 2012 7:40:40 PM UTC> <Warning> <Socket> <BEA-00670450> <Socket 211 internal data record unavailable (probable closure due idle timeout), event received 17>
<Mar 31, 2012 8:40:40 PM UTC> <Warning> <Socket> <BEA-00670450> <Socket 211 internal data record unavailable (probable closure due idle timeout), event received 17>
<Mar 31, 2012 9:40:40 PM UTC> <Warning> <Socket> <BEA-00670450> <Socket 211 internal data record unavailable (probable closure due idle timeout), event received 17>
<Mar 31, 2012 10:40:40 PM UTC> <Warning> <Socket> <BEA-00670450> <Socket 211 internal data record unavailable (probable closure due idle timeout), event received 17>
<Mar 31, 2012 11:40:40 PM UTC> <Warning> <Socket> <BEA-00670450> <Socket 211 internal data record unavailable (probable closure due idle timeout), event received 17>

 

10 More Discussions You Might Find Interesting

1. Programming

mmap vs shared memory - which is faster for reading data between multiple process

Between mmap and shared memory which is the best method of sharing data between multiple applications, interms of speed? (1 Reply)
Discussion started by: nmds
1 Replies

2. UNIX for Advanced & Expert Users

faster way to loop?

Sample Log file IP.address Date&TimeStamp GET/POST URL ETC 123.45.67.89 MMDDYYYYHHMM GET myURL http://ABC.com 123.45.67.90 MMDDYYYYHHMM GET myURL http://XYZ.com I have a very huge web server log file (about 1.3GB) that contains entries like the one above. I need to get the last entries of... (9 Replies)
Discussion started by: tads98
9 Replies

3. Shell Programming and Scripting

loop when process running

Hi Gurus, Could you please help me to create a shell script that will be started by a cron job once every night at 24.00 h (that should bee easy:)) The shell script should control every 30 seconds the name of a process, and when the process doesn't run anymore it should execute a few further... (12 Replies)
Discussion started by: blackwire
12 Replies

4. Shell Programming and Scripting

How to loop this process?

for two txt files, f1 and f2, I like to do the following grep "abcde" f1 > abcde$f1 grep "xyz" f1 > xyz$f1 can I use a loop to get this done? Thanks for i in f1 f2 do grep "abcde" $i > abcde$i grep "xyz" $i > xyz$i ... done (11 Replies)
Discussion started by: ksgreen
11 Replies

5. Shell Programming and Scripting

Process checking loop

Hi, I want to create a script who will check if the java process is running & if it finds the process is still there it continues to execute & when the process completes it exit from the script. I have written a code to check & notify the process existence but i am not getting how to write... (4 Replies)
Discussion started by: d8011
4 Replies

6. Shell Programming and Scripting

[SOLVED] for loop to process files

I need to process a dirtree containing ms office files such that each file is stored as a variable and also, just the file file stem. Why? They will be using as input and output parameters for another script. For example /path/to/second_script -i filename.docx -o filename Here's what I... (1 Reply)
Discussion started by: graysky
1 Replies

7. UNIX for Dummies Questions & Answers

For loop in process each file

Hi I have following codecd /tmp/test/ for vfile in `ls -1` do for vlink in `ls -l /tmp/testfile/*|bin/grep "local/init\.d/$vfile$"|bin/awk -F"->" '{print($1)}'|bin/awk -F"/" '{print($NF)}'` I know `ls -1` list only file, but I don't... (3 Replies)
Discussion started by: stew
3 Replies

8. Shell Programming and Scripting

Script to loop process

As I would like to test the open files usage , I would like to have a process that use the open files up to a certain amount eg. 1000 . If I want to have a script ( may be run in a loop ) that could repeatly use open files resource , so that the usage of open files increases , may I know how to... (10 Replies)
Discussion started by: ust
10 Replies

9. Shell Programming and Scripting

Process files in loop which have spaces in name

I have a folder with files and I have to process them in a loop. However the filenames have space characters, so the list get split. $ touch "File Number_1" $ touch "File Number_2" $ ls "/tmp/File Number"_* /tmp/File Number_1 /tmp/File Number_2 I tried following (sorry for using the... (3 Replies)
Discussion started by: Wernfried
3 Replies

10. Shell Programming and Scripting

How to make faster loop in multiple directories?

Hello, I am under Ubuntu 18.04 Bionic. I have one shell script run.sh (which is out of my topic) to run files under multiple directories and one file to control all processes running under those directories (control.sh). I set a cronjob task to check each of them with two minutes of intervals.... (3 Replies)
Discussion started by: baris35
3 Replies

LEARN ABOUT DEBIAN

continue

continue(3tcl)						       Tcl Built-In Commands						    continue(3tcl)

__________________________________________________________________________________________________________________________________________________

NAME

       continue - Skip to the next iteration of a loop

SYNOPSIS

       continue
_________________________________________________________________

DESCRIPTION

       This  command  is  typically invoked inside the body of a looping command such as for or foreach or while.  It returns a TCL_CONTINUE code,
       which causes a continue exception to occur.  The exception causes the current script to be aborted out to  the  innermost  containing  loop
       command,  which	then  continues with the next iteration of the loop.  Catch exceptions are also handled in a few other situations, such as
       the catch command and the outermost scripts of procedure bodies.

EXAMPLE

       Print a line for each of the integers from 0 to 10 except 5:
	      for {set x 0} {$x<10} {incr x} {
		 if {$x == 5} {
		    continue
		 }
		 puts "x is $x"
	      }

SEE ALSO

       break(3tcl), for(3tcl), foreach(3tcl), return(3tcl), while(3tcl)

KEYWORDS

       continue, iteration, loop

Tcl																    continue(3tcl)
All times are GMT -4. The time now is 06:20 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy