Hi rovf,
On every system I've seen, for a large amount of data (which one might assume from a variable named MASSIVETEXT), there is a noticeable difference in performance between grep -F (fastest), grep without -E and without -F (slower), and grep -E (slower still).
Interesting point. I tried it with a 300MB file, using Cygwin grep, and searching for a fixed string with various settings. I repeated each run 3 times. Here the times
with -F:
without option:
with -E:
Of course, it could be that the amount of data is not massive enough to show a significant difference; or that the I/O of the Cygwin layer is so heavy, that it shadows performance differences. Or that gnu grep is clever enough to recognize, that the pattern does not contain any regexp characters and does internally a '-F' always.
To evaluate at the last hypothesis, I replaced the pattern by one which really contained an extended regexp instead of a fixed string (using a .+ regexp operation). In this case, the times were systematically higher, but not very much:
Perhaps such a test should really be repeated with a file which is several GB in size.
Interestingly, in the OP's problem, the whole content of the file was stored in a shell variable, and this makes me wonder, where the practical limit is for the content of a variable in, say, bash or ksh...
Last edited by rbatte1; 01-08-2018 at 10:49 AM..
Reason: Added CODE tags for output
Hi Gurus,
I had a question regarding avoiding duplicates.i have a file abc.txt
abc.txt
-------
READER_1_1_1> HIER_28056 XML Reader: Error occurred while parsing:; line number ; column number
READER_1_3_1> Sun Mar 23 23:52:48 2008
READER_1_3_1> HIER_28056 XML Reader: Error occurred while... (0 Replies)
Hi Experts,
Now the problem is when I run the install script inside .app folder, it opens a GUI and asks for user input. I want to avoid these GUI. I want to provide input when i run install script e.g.
$ ./install < inputfile
I used to redirect input from input file to install script. But... (4 Replies)
I am no Unix administrator...I live in windows land.
I wrote a script to find files of certain names and process them but was later advised to avoid checking sparse files since it would use up a lot of resources and the files I was looking for were not there.
How do I avoid doing the find on... (3 Replies)
Hi all,
I want to put a check in my script to check if the same instance is already running and not finished and if not then does not allow it to run! in which part of my script I should put this? and any idea how I should write it? tx (4 Replies)
Hi,
I have a script which has the below line:
ps -ef | grep ${SCRIPT_NAME} | grep ksh | grep -v grep >> /tmp/instance.tmp
When the script is invoked through CRON, I get 2 lines in instance.tmp when actually only one instance is running:
cdrd 17790 17789 0 15:14:01 ? 0:00 /bin/ksh... (8 Replies)
In bash shell, how we can avoid the commands getting recorded in history file.
One way i can think of is :
export HISTSIZE=0
Is there any other way to achieve this?
Thanks (1 Reply)
In my script I need to loop around some files like below
example files are
fa.info.abcd
fa.info.bxde
fa.info.cdas
------
for test_data in fa.info.*
do
# Some text appending logic applied
# Copy to another directory
done
Now I need to discard some files while looping around
... (9 Replies)
Hi all,
I have a list which I want to search in another file.
I can do that using
grep -f
but the search is failing due to special characters, how do I solve this?
One row in that list is
amino-acid permease inda1 gb|EDU41782.1| amino-acid permease inda1 Input file to be searched... (2 Replies)
Hi Team,
Am getting the below output but need the count of records to be displayed in same line but currently count alone moves to next line. Please let me know how we can still keep the count in the same line.
######code #####
while read YEAR; do
for i in TEST_*PGYR${YEAR}_${DT}.csv; do... (3 Replies)