search all files and sub directory


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting search all files and sub directory
# 1  
Old 06-17-2004
search all files and sub directory

I wanted to search in all the sub directories under /vob/project (recurse) in everything inside /vob/project.

search.run

for x in `cat search.strings`
do
find /vob/project -type f -print | xargs grep -i $x > ~/$x.txt
done


search.string
hello
whoami

I am getting the error
Maximum line length of 2048 exceeded.

Thanks for any help
# 2  
Old 06-17-2004
The message is giving you a clue here. Is there any line in any file under /vob/project that contains a line longer than 2048 chars, which is a limit for some Unix commands?

Try this to count the length of lines in your files
Code:
#!/bin/sh

for file in "$@"; do
  echo -e "\nFILE: $file"
  awk '{printf( "%5d: %s\n", length( $0 ), $0 ) }' $file
done

Save this as linelength.sh, make it executable, and run it as
$ linelength /vob/project/*

Cheers
ZB
# 3  
Old 06-17-2004
Thank you Bob.

Yes, it looks like there are some files, it returned a whole bunch of results.

Is there any way i can still search by ignoring these.

i ma new to unix.

thank you
siva
# 4  
Old 06-18-2004
The script will post the length of all lines in all files, regardless of length. To find the lines longer than 2047 chars, try

Code:
#!/bin/sh

for file in "$@"; do
  awk -v f=$file '{ if ( length( $0 ) > 2047 ) {
         printf( "%s %5d: %s\n", f, length( $0 ), $0 );
         }
       }' $file

done

and invoke it as above.

If you have a newer version of wc you can try wc -L /vob/project/* (captial L) to show the length of the longest line in each file.

So, assuming your wc supports this,
Code:
#!/bin/sh

MAX_LINE=2047
TEMPFILE=/tmp/out.$$.txt

# first, create temp file with list of files + max line lengths
find . -type f -print | xargs wc -L | grep -v total >> ${TEMPFILE}

while read term
do
   while read line
   do
      max_line=$(echo $line | awk '{print $1}')
      filename=$(echo $line | awk '{print $2}')
      if [ "$max_line" -lt "$MAX_LINE" ]; then
         echo "$filename:" >> ~/$term.txt
         grep -i "$term" $filename >> ~/$term.txt
      fi
   done < ${TEMPFILE}
done < search.strings

rm -f ${TEMPFILE}

Will do what you want, excluding any file from the search that contains a line longer than 2047 chars.

Cheers
ZB
# 5  
Old 06-21-2004
Thanks a lot Bpb.

Looks like the wc -L (-L) option is not there in this version.

just wondering how should i do.

thank you
siva
# 6  
Old 06-21-2004
Substitute
Code:
find . -type f -print | xargs wc -L | grep -v total >> ${TEMPFILE}

with

Code:
for file in $(find . -type f -print)
do
linecount=`awk 'BEGIN { f=0 } {if ( length( $0 ) > f ) { f = length( $0 ); }} END { print f}' $file`
string="$linecount $file"
echo "$string" >> ${TEMPFILE}
done

Obviously, replace the "." in the find command with the directory you are reading, /vob/project, in your case

Cheers
ZB

Last edited by zazzybob; 06-21-2004 at 07:42 PM..
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to search for a string in all the files irrespective of directory.?

Hi All, How to search for a string in all the files irrespective of directory. If I use grep -i 'hello' *.* It will search for the string hello in the files of current directory. But I want to check for the string hello in the files of all the directories. Thanks (4 Replies)
Discussion started by: ROCK_PLSQL
4 Replies

2. Shell Programming and Scripting

Search files in directory for keywords using bash

I have ~100 text files in a directory that I am trying to parse and output to a new file. I am looking for the words chr,start,stop,ref,alt in each of the files. Those fields should appear somewhere in those files. The first two fields of each new set of rows is also printed. Since this is on a... (7 Replies)
Discussion started by: cmccabe
7 Replies

3. Shell Programming and Scripting

How to search files by time in this directory

Hello dears, In my home directory (msc-loader) there are some files like this: # ls -l -rwxrwxrwx 1 user dba 1680318 Jan 20 13:32 103386_s.c -rwxrwxrwx 1 user dba 256743 Jan 20 13:37 103387_o.c -rwxrwxrwx 1 user dba 1635363 Jan 20 13:39 103387_s.c -rwxrwxrwx 1 user dba 264735 Jan 20... (3 Replies)
Discussion started by: Mohannad
3 Replies

4. Shell Programming and Scripting

Search a directory for files that contain pathnames

Hello, I really don't know if this is possible, but I figured the wonderful users of unix.com may be able to provide me with some help/suggestions. I want to create a (bash) script that would go will accept an input file. That input file contains important information as well as pathnames. For... (4 Replies)
Discussion started by: tester213
4 Replies

5. UNIX for Dummies Questions & Answers

How to search all the files in a directory for a specific string

Hi Guys, I want to search the content of all the files (of a particular type like .txt) in a directory for a specific string pattern. Can anyone help me? Thanks (7 Replies)
Discussion started by: mwrg
7 Replies

6. UNIX for Dummies Questions & Answers

Search for a text within files in a directory

I need to search for a particular string. This string might be present in many files. The directory in which I am present has more than one subdirectories. Hence, the search should check in all the subdirectories and all the corresponding files and give a list of files which have the particular... (5 Replies)
Discussion started by: pakspan
5 Replies

7. Shell Programming and Scripting

how to do search and replace on text files in directory

I was google searching and found Perl as a command line utility tool This almost solves my problem: find . | xargs perl -p -i.old -e 's/oldstring/newstring/g' I think this would create a new file for every file in my directory tree. Most of my files will not contain oldstring and I... (1 Reply)
Discussion started by: siegfried
1 Replies

8. Shell Programming and Scripting

search files in a directory and its subdirectories

Hello my friends, I need to write a simple shell bad file :D that search and delete a file it's name 'Microsoft.txt' in the current directory and its subdirectories? So can you help to guide me how i can write this shell, Just give me the beginning :o thank you. (1 Reply)
Discussion started by: Net-Man
1 Replies

9. UNIX for Dummies Questions & Answers

search for certain word in a files from directory

hi can i know how to get the file name of any files containing for example "abc" from a number of files in a directory? i tried "ls -ltrc | grep abc" but no resulted generated. thanks in advance. (1 Reply)
Discussion started by: legato
1 Replies
Login or Register to Ask a Question