Shell Programming and Scripting

i stored a lot of lines in a variable. and i want to be able to scan that variable quickly. sounds simple enough. but really, it isnt. scanning a variable containing a lot of lines actually is very slow.

i'd rather not have to write the contents of the variable into a file.

but here's an example of what i'm doing that is taking too long:

Code:

echo "$VarLines" | egrep "error"

I tried something like the following but its not working:

Code:

egrep "error" "$VarLines"

Any ideas?

Perhaps this?

Code:

egrep "error" << EOF
$VarLines
EOF

--
How many lines is "lots of lines" ?

it could be 200,000 lines or more.

what problems can you envision this causing? and by the way, i tested your command. it works, but i get this error:

Code:

line 7: warning: here-document at line 5 delimited by end-of-file (wanted `EOF')

@SkySmart
This command is a complete error unless $VarLines is the name of a file.

Please post what Operating System you are running and what Shell you are using. If you do not know how to do this, please ask.

Environment Variables are not files. IMHO Any Script which stores more than one line in an Environment Variable is faulty.

Please post the script you are testing complete with sample input, expected output and actual output. If you have timing issues, please post real time values.

If you do not understand these basic requirements, please post in "Unix for Dummies" and ask lots of relevant questions.

The operating system I intend to run this own will be linux (any variant of it) and sun solaris.

the shell is always bash.

sample of the script is:

Code:

#!/bin/sh

CONTENT=$(tail -200000 /var/logs/catalina.out)

egrep "Accepted order" << EOF | tail -1 $CONTENT

The problem is that the catalina.out file is a huge file. over 1 GB. i have it seen it go up to 3 or 4 GB. I have no control over that.

I'm sure by looking at the sample script i provided, you can maybe come up with much more smarter way of doing this...without having to write to a file.

as far as the output of the log file. lets just imagine that each line of the log file i'm interested in must contain "Accepted Order".

thank you so much

This script fragment is syntactically incorrect, contains erroneous assumptions about the "tail" command and quite honestly does not make sense. You cannot store multiple lines in one variable sensibly (let alone 200,000 lines !).
The script contains too many fundamental script errors to guess the intention. What (in words) are you try to achive?



Make you mind up ! Under Solaris the Shell /bin/sh is the basic Bourne Shell and definitely not bash. The bash Shell may not be available unless installed as a non-standard add-on to the Solaris system.



Footnote: Bit of a guess based on the sketchy information posted, but did you mean?:

Code:

grep "Accepted order" /var/logs/catalina.out | tail -1

so here's the problem.

as i pointed out in my previous post, i have a file that can be as huge as 4GB is size.

grepping out anything from a file that big is bound to take forever. which means a command like the one below is no good:

Code:

grep "Accepted order" /var/logs/catalina.out | tail -1

so, this script is meant to choose the number of lines to grep from instead of having to grep through a file that is huge. this will pose the least resource usage on the system.

in other words, what the script is doing is something to the effect of:

if the file is too big, just pull out the last 200000 lines from it and work on those lines.