Sponsored Content
Top Forums Shell Programming and Scripting Find files and seach limit with grep Post 302657379 by alister on Sunday 17th of June 2012 11:29:04 AM
Old 06-17-2012
Code:
find "$2" -mindepth 2 -type f | xargs -P $max_parallel awk '/topic/ {print FILENAME} FNR==2 {nextfile}' | sort -u

Untested.

nextfile is not part of posix awk, but it's available on the most popular implementations.

AWK uses extended regular expression, instead of grep's basic flavor. You may need to tweak your regular expressions if they're more complicated than a simple word like "topic".

Since this find prints file, and since you almost certainly don't want to spawn awk once per file, I dropped -n 1 in xargs. Further, sometimes parallelizing i/o bound tasks can degrade performance if the parallel invocations are not backed by different disks. Something to keep in mind. You might actually improve performance by lowering the value of -P (if not by eliminating it altogether).

Regards,
Alister

Last edited by alister; 06-17-2012 at 12:58 PM..
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

grep something to find which files, lines contain something

in aix, i use grep something * to get the files, lines that contain something. but i couldnt get the same result from a sun box. what is the equivalent? thanks (4 Replies)
Discussion started by: yls177
4 Replies

2. Shell Programming and Scripting

grep line length limit

Hi Friends, I am having a funny problem with grep. When I run grep 'expr' file.txt things work fine. But when try to get the line number using the -n option, i.e, grep -n 'expr' file.txt I get a message, "grep: 0652-226 Maximum line length of 2048 exceeded." If the line has more than... (3 Replies)
Discussion started by: hnhegde
3 Replies

3. AIX

Help Using Grep command to Find the string in the files in the root directory

I want to serch for a string in all the files in the root directory. i want the search to be limited to only the files in the directory i.e the search to be done only in the files not in the sub directory. the approaches tried are 1)grep "pattern string" this command was serching the... (3 Replies)
Discussion started by: Subbu_Angeline
3 Replies

4. Shell Programming and Scripting

Is it better to grep and pipe to awk, or to seach with awk itself

This may just be a lack of experience talking, but I always assumed that when possible it was better to use a commands built in abilities rather than to pipe to a bunch of commands. I wrote a (very simple) script a while back that was meant to pull out a certain error code, and report back what... (4 Replies)
Discussion started by: DeCoTwc
4 Replies

5. UNIX for Dummies Questions & Answers

Shell to seach all .mov files in a drive

I want a shell script which does: -> search all .mov files in a drive (say e: drive) which are above 800 mb of size -> Delete the searched .mov files which are above 800mb Thanks, Manoj (4 Replies)
Discussion started by: manoj.b
4 Replies

6. Shell Programming and Scripting

Any limit on files

I am doing an ftp of around 1010 files and I am using mput for this. For some reason its only transferring 10 or 20 files and the rest are not getting transferred. There is some socket error in the log. is there an issue if we have more than 50 or so files for mput. here is the o/p in the log... (2 Replies)
Discussion started by: dsravan
2 Replies

7. UNIX for Dummies Questions & Answers

Using grep to find files that don't contain a string

Hi all, I am still learning my way around unix commands and I have the following question. I have a website and I want to search for all the html pages that don't contain a certain js file. The file I am searching for is located under /topfolder/js/rules.js . So I assume in my grep search I... (5 Replies)
Discussion started by: SyphaX
5 Replies

8. Shell Programming and Scripting

grep by limit search

Hi I am in new in unix,can any one tell how to grep the data by limit. suppose I have below data:- is :mSecs is :mSecs is :mSecs is :mSecs requirement is how to grep the data which is having count greater than 1000 msecs only. thanks in adnavce. (2 Replies)
Discussion started by: abhigrkist
2 Replies

9. UNIX for Dummies Questions & Answers

Limit Number of files

hi guys how can i limit number of files in a disk or partition ? or how can i make a limit to inode number for a disk or partition ? ext3 or ext4 file system (1 Reply)
Discussion started by: mhs
1 Replies

10. UNIX for Dummies Questions & Answers

General find /grep question: files with lines ending in r

I am trying to find files that have lines in them that end in an r. I have been able to locate files by using the following command: find . -type f -name "*RECORDS"| xargs grep -l r$ However, I now want to find files that don't end in r anywhere. That means that no sentences or lines in... (9 Replies)
Discussion started by: newbie2010
9 Replies
IGAWK(1)							 Utility Commands							  IGAWK(1)

NAME
igawk - gawk with include files SYNOPSIS
igawk [ all gawk options ] -f program-file [ -- ] file ... igawk [ all gawk options ] [ -- ] program-text file ... DESCRIPTION
Igawk is a simple shell script that adds the ability to have ``include files'' to gawk(1). AWK programs for igawk are the same as for gawk, except that, in addition, you may have lines like @include getopt.awk in your program to include the file getopt.awk from either the current directory or one of the other directories in the search path. OPTIONS
See gawk(1) for a full description of the AWK language and the options that gawk supports. EXAMPLES
cat << EOF > test.awk @include getopt.awk BEGIN { while (getopt(ARGC, ARGV, "am:q") != -1) ... } EOF igawk -f test.awk SEE ALSO
gawk(1) Effective AWK Programming, Edition 1.0, published by the Free Software Foundation, 1995. AUTHOR
Arnold Robbins (arnold@skeeve.com). ATTRIBUTES
See attributes(5) for descriptions of the following attributes: +--------------------+-----------------+ | ATTRIBUTE TYPE | ATTRIBUTE VALUE | +--------------------+-----------------+ |Availability | SUNWgawk | +--------------------+-----------------+ |Interface Stability | Volatile | +--------------------+-----------------+ NOTES
Source for gawk is available on http://opensolaris.org. Free Software Foundation Nov 3 1999 IGAWK(1)
All times are GMT -4. The time now is 12:59 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy