Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Can I speed up my grep command?
Top Forums Shell Programming and Scripting Can I speed up my grep command? Post 302861897 by Don Cragun on Wednesday 9th of October 2013 08:47:07 PM
Old 10-09-2013
I assume that by now you have figured out how to find where the MKS toolkit utilities are on your ssytem again... If not, we probably can't help you.

With a Windows operating system that old and an MKS toolkit that is even older, you probably have a 16-bit shell (which leaves you with a relatively limited address space). With 30,000+ files to process, there is a fair chance that you'll overflow the shell size limit on the contents of a shell variable even if less than 10% of your xml files contain the string you're looking for. Therefore, I'd strongly suggest saving the list of files found in a file instead of in a shell variable.
Code:
find . -name "*.xml" -print | xargs fgrep -l Status > StatusXF.txt

and show us any diagnostic output produced. With any luck, StatusXF.txt will contain the list of files you want and won't take forever to run.
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

grep - speed of search

I grepped for a string from a directory of very large files. This took quite a long time (not a problem). When I grepped for a different string from the same files immediately after, the output was MUCH quicker. My question is, does anybody know why the second grep was so much quicker than the... (1 Reply)
Discussion started by: davirime
1 Replies

2. Filesystems, Disks and Memory

dmidecode, RAM speed = "Current Speed: Unknown"

Hello, I have a Supermicro server with a P4SCI mother board running Debian Sarge 3.1. This is the "dmidecode" output related to RAM info: RAM speed information is incomplete.. "Current Speed: Unknown", is there anyway/soft to get the speed of installed RAM modules? thanks!! Regards :)... (0 Replies)
Discussion started by: Santi
0 Replies

3. UNIX for Advanced & Expert Users

how to exclude the GREP command from GREP

I am doing "ps -f" to see my process. but I get lines that one of it represents the ps command itself. I want to grep it out using -v flag, but than I get another process that belongs to the GREP itself : I would like to exclude # ps -f UID PID PPID C STIME TTY TIME CMD... (2 Replies)
Discussion started by: yamsin789
2 Replies

4. Shell Programming and Scripting

Is there a command to measure compile speed?

Hello Ive written 2 programs in shell and I need to compare their speed (Compile) against one another. what methods could I go about doing this? Is there a feature in shell do accommodate this? (2 Replies)
Discussion started by: Darklight
2 Replies

5. Shell Programming and Scripting

can anyone help with shell script command about searching word with grep command?

i want to search in the current directory all the files that contain one word for example "hello" i want to achieve it with the grep command but not with the grep * (2 Replies)
Discussion started by: aintour
2 Replies

6. Filesystems, Disks and Memory

data from blktrace: read speed V.S. write speed

I analysed disk performance with blktrace and get some data: read: 8,3 4 2141 2.882115217 3342 Q R 195732187 + 32 8,3 4 2142 2.882116411 3342 G R 195732187 + 32 8,3 4 2144 2.882117647 3342 I R 195732187 + 32 8,3 4 2145 ... (1 Reply)
Discussion started by: W.C.C
1 Replies

7. Shell Programming and Scripting

How to speed up grep?

hi i'm greping the files with sepefic keyword, where the file is of too big. Assume there are 10 days log file each of more than 200mb. i've to grep all those files with a specific keywords. for example, 1. i'll grep for error message 2. after the i'll do one more grep for keyword... (4 Replies)
Discussion started by: vij_krr
4 Replies

8. Shell Programming and Scripting

Speed Up Grep

Hi, I have to grep string from 20 - 30 files each carries 200 - 300 MB size and append to the file. How to speed the grepping time. cat catalina.out_2012_01_01 | grep "xxxxx" >> backup.txt PLZ, Suggest me, Regards, Nanthagopal A (5 Replies)
Discussion started by: nanthagopal
5 Replies

9. Shell Programming and Scripting

Speed : awk command to count the occurrences of fields from one file present in the other file

Hi, file1.txt AAA BBB CCC DDD file2.txt abc|AAA|AAAabcbcs|fnwufnq bca|nwruqf|AAA|fwfwwefwef fmimwe|BBB|fnqwufw|wufbqw wcdbi|CCC|wefnwin|wfwwf DDD|wabvfav|wqef|fwbwqfwfe i need the count of rows of file1.txt present in the file2.txt required output: AAA 2 (10 Replies)
Discussion started by: mdkm
10 Replies

10. Shell Programming and Scripting

Grep command giving different result for different users for same command

Hello, I am running below command as root user #nodetool cfstats tests | grep "Memtable switch count" Memtable switch count: 12 Where as when I try to run same command as another user it gives different result. #su -l zabbix -s /bin/bash -c "nodetool cfstats tests | grep "Memtable switch... (10 Replies)
Discussion started by: Pushpraj
10 Replies

LEARN ABOUT OSX

platform::shell

platform::shell(n)					       Tcl Bundled Packages						platform::shell(n)

__________________________________________________________________________________________________________________________________________________

NAME

       platform::shell - System identification support code and utilities

SYNOPSIS

       package require platform::shell ?1.1.4?

       platform::shell::generic shell
       platform::shell::identify shell
       platform::shell::platform shell
_________________________________________________________________

DESCRIPTION

       The platform::shell package provides several utility commands useful for the identification of the architecture of a specific Tcl shell.

       This  package  allows the identification of the architecture of a specific Tcl shell different from the shell running the package. The only
       requirement is that the other shell (identified by its path), is actually executable on the current machine.

       While for most platform this means that the architecture of the interrogated shell is identical to the architecture of  the  running  shell
       this  is  not generally true. A counter example are all platforms which have 32 and 64 bit variants and where a 64bit system is able to run
       32bit code. For these running and interrogated shell may have different 32/64 bit settings and thus different identifiers.

       For applications like a code repository it is important to identify the architecture of the shell which will  actually  run  the  installed
       packages, versus the architecture of the shell running the repository software.

COMMANDS

       platform::shell::identify shell
	      This command does the same identification as platform::identify, for the specified Tcl shell, in contrast to the running shell.

       platform::shell::generic shell
	      This command does the same identification as platform::generic, for the specified Tcl shell, in contrast to the running shell.

       platform::shell::platform shell
	      This command returns the contents of tcl_platform(platform) for the specified Tcl shell.

KEYWORDS

       operating system, cpu architecture, platform, architecture

platform::shell 						       1.1.4							platform::shell(n)
All times are GMT -4. The time now is 05:07 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy