Can I speed up my grep command?


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Can I speed up my grep command?
# 43  
Old 10-10-2013
OK. Build a sandbox, create dummy xml files with "Status" in their first line, and then run the respective commands. Experiment with parameters. Use commands with whole absolute paths. Come back with results and details. If your problems exists only with your sensitive data, and you cannot disclose those, you're doomed. I am / we are not interested in your data, but to analyse that strange behaviour, we need more than an uttered "It doesn't work." As long as there's no data to work on, I'm giving up.
# 44  
Old 10-10-2013
Thanks I will do!
# 45  
Old 10-11-2013
Might I suggest an alternate?

You are finding files ending .xml and for each of them in the order that they are found, you are running your grep The issue you might have is that the grep command working on a single file on some OS version may produce no output if you use the -l flag.

Try adding /dev/null into the grep command so that you always have two files and grep may behave differently.
Code:
find . -type f -name "*.xml" -exec grep -l "Status" {} /dev/null \;

Note how the /dev/null is part of the grep command, i.e. it's before the \; and so it is taken as an input file. It is not redirecting output to as it is normally used for or redirecting standard input to be reading 'nothing' from.


I hope that this helps.


Robin
# 46  
Old 10-11-2013
Quote:
Originally Posted by emc^24sho
I need the help! Do to the nature of my job there's only information I can disclose. I do truly need the help! I just tried the find command that just worked from the command line and this is what I got.

Code:
D:/home/dv/daa/ae/pt(104) find . -name "*.xml" -exec -exec ls -al {} \; -exec grep -l "Status" {} \;
find: cannot execute "-exec": The system cannot find the file specified.
D:/home/dv/daa/ae/pt(105)

That's what I'm talking about....

---------- Post updated at 10:38 AM ---------- Previous update was at 10:37 AM ----------

The difference is the wild card! Why would that matter?
The difference is not the wild card; the difference is the -exec -exec. Try changing:
Code:
find . -name "*.xml" -exec -exec ls -al {} \; -exec grep -l "Status" {} \;

to:
Code:
find . -name "*.xml" -exec ls -al {} \; -exec grep -l "Status" {} \;

and see what happens.

I never heard the results of you trying the suggestion I made in message #16 in this thread. What were the results when you tried that suggested command line?
# 47  
Old 10-14-2013
It show's the files correct files along with all of the details... I don't understand why it won't grab them without the
Code:
ls -al part

When I remove this section. The commands act like they run then just return with a prompt but no filenames... It takes like 45 minutes for that to happen...

---------- Post updated at 01:29 PM ---------- Previous update was at 01:28 PM ----------

I'm trying to read every MKS toolkit & Kornshell pdf that I can find trying to solve this.....

---------- Post updated at 01:30 PM ---------- Previous update was at 01:29 PM ----------

I really do appreciate all of the help that everybody is giving me! Big THANK YOU ALL!

---------- Post updated at 01:32 PM ---------- Previous update was at 01:30 PM ----------

This command line takes 45 minutes and returns nothing!
Code:
find . -name "*.xml" -exec grep -l "Status" {} \;

While this one comes right back showing all of xmls plus their details!
Code:
find . -name "*.xml" -exec ls -al {} \; -exec grep -l "Status" {} \;

Does anybody have a clue why?

---------- Post updated at 02:26 PM ---------- Previous update was at 01:32 PM ----------

It's not running the second half of those commands. (The grep -l part)....

---------- Post updated at 02:28 PM ---------- Previous update was at 02:26 PM ----------

I put in this code per status and got back all statuses..
Code:
find . -name "*.xml" -exec ls -al {} \; -exec grep -l "AStatus" {} \;

Where A is one of (A,B,C Status) And the command returned all xml files....
# 48  
Old 10-16-2013
The commands are simple. Idk what happened? I installed MKS toolkit on my workstation & ran the code again & it happened on there too. I unstalled & reinstalled MKS & the error was still there...
# 49  
Old 10-16-2013
Have you considered trying cygwin
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Grep command giving different result for different users for same command

Hello, I am running below command as root user #nodetool cfstats tests | grep "Memtable switch count" Memtable switch count: 12 Where as when I try to run same command as another user it gives different result. #su -l zabbix -s /bin/bash -c "nodetool cfstats tests | grep "Memtable switch... (10 Replies)
Discussion started by: Pushpraj
10 Replies

2. Shell Programming and Scripting

Speed : awk command to count the occurrences of fields from one file present in the other file

Hi, file1.txt AAA BBB CCC DDD file2.txt abc|AAA|AAAabcbcs|fnwufnq bca|nwruqf|AAA|fwfwwefwef fmimwe|BBB|fnqwufw|wufbqw wcdbi|CCC|wefnwin|wfwwf DDD|wabvfav|wqef|fwbwqfwfe i need the count of rows of file1.txt present in the file2.txt required output: AAA 2 (10 Replies)
Discussion started by: mdkm
10 Replies

3. Shell Programming and Scripting

Speed Up Grep

Hi, I have to grep string from 20 - 30 files each carries 200 - 300 MB size and append to the file. How to speed the grepping time. cat catalina.out_2012_01_01 | grep "xxxxx" >> backup.txt PLZ, Suggest me, Regards, Nanthagopal A (5 Replies)
Discussion started by: nanthagopal
5 Replies

4. Shell Programming and Scripting

How to speed up grep?

hi i'm greping the files with sepefic keyword, where the file is of too big. Assume there are 10 days log file each of more than 200mb. i've to grep all those files with a specific keywords. for example, 1. i'll grep for error message 2. after the i'll do one more grep for keyword... (4 Replies)
Discussion started by: vij_krr
4 Replies

5. Filesystems, Disks and Memory

data from blktrace: read speed V.S. write speed

I analysed disk performance with blktrace and get some data: read: 8,3 4 2141 2.882115217 3342 Q R 195732187 + 32 8,3 4 2142 2.882116411 3342 G R 195732187 + 32 8,3 4 2144 2.882117647 3342 I R 195732187 + 32 8,3 4 2145 ... (1 Reply)
Discussion started by: W.C.C
1 Replies

6. Shell Programming and Scripting

can anyone help with shell script command about searching word with grep command?

i want to search in the current directory all the files that contain one word for example "hello" i want to achieve it with the grep command but not with the grep * (2 Replies)
Discussion started by: aintour
2 Replies

7. Shell Programming and Scripting

Is there a command to measure compile speed?

Hello Ive written 2 programs in shell and I need to compare their speed (Compile) against one another. what methods could I go about doing this? Is there a feature in shell do accommodate this? (2 Replies)
Discussion started by: Darklight
2 Replies

8. UNIX for Advanced & Expert Users

how to exclude the GREP command from GREP

I am doing "ps -f" to see my process. but I get lines that one of it represents the ps command itself. I want to grep it out using -v flag, but than I get another process that belongs to the GREP itself : I would like to exclude # ps -f UID PID PPID C STIME TTY TIME CMD... (2 Replies)
Discussion started by: yamsin789
2 Replies

9. Filesystems, Disks and Memory

dmidecode, RAM speed = "Current Speed: Unknown"

Hello, I have a Supermicro server with a P4SCI mother board running Debian Sarge 3.1. This is the "dmidecode" output related to RAM info: RAM speed information is incomplete.. "Current Speed: Unknown", is there anyway/soft to get the speed of installed RAM modules? thanks!! Regards :)... (0 Replies)
Discussion started by: Santi
0 Replies

10. UNIX for Dummies Questions & Answers

grep - speed of search

I grepped for a string from a directory of very large files. This took quite a long time (not a problem). When I grepped for a different string from the same files immediately after, the output was MUCH quicker. My question is, does anybody know why the second grep was so much quicker than the... (1 Reply)
Discussion started by: davirime
1 Replies
Login or Register to Ask a Question