Reading compressed files during a grep search


 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers Reading compressed files during a grep search
# 1  
Old 07-09-2010
Reading compressed files during a grep search

All,

The bottom line is that im reading a file, storing it as variables, recursively grep searching it, and then piping it to allow word counts as well. I am unsure on how to open any .zip .tar and .gzip, search for keywords and return results.

Any help would be much appreciated!

Thanks
# 2  
Old 07-09-2010
What about zgrep or gzgrep?
# 3  
Old 07-09-2010
I think i've tried zgrep in the past, but I can't remember why i didn't use it. I just tried it with my script, but it returns a message saying that it doesn't recognize my options. The message reads: "/usr/bin/zgrep: -iro: option not supported" and "/usr/bin/zgrep: -iroln: option not supported". I tried separating them out (e.g. -i -r -o) and it only returned a message saying the "-r" option was invalid. Does zgrep search recursively?
# 4  
Old 07-09-2010
On Linux it's a shell script, so you may try to tweak it, if you wish (not sure what you mean by recursively here).
May be if you try to describe your task giving more details and examples, helping you would be easier.
# 5  
Old 07-09-2010
here is a copy of my script
Code:
#!/bin/bash

echo "Enter file for search criteria"
read fname

echo "Enter directory for recursive search"
read dname

echo -e "\n"

FILE14=$fname
i=0

while read line; do
    dwords[$i]=$line
      i=$((i+1))
done < "$FILE14"

for    (( i=0; i<${#dwords[@]}; i++ )); do
        current_result="$(zgrep -i -r -o ${dwords[i]} $dname | wc -w)"
        resultstr+="$(zgrep -i -r -o -l -n ${dwords[i]} $dname)"
        resultstr+="\n"

        if [ $current_result -eq 0 ]; then    #if no matches found, ends run.
                echo "${dwords[i]}:"        
                echo -e "0 matches found. \n"
                
        else
                echo "${dwords[i]}:"                
                if [ $current_result -eq 0 ]; then
                        echo "$current_result match found"
                else
                        echo "$current_result matches found"
                fi
                echo -e $resultstr
        fi
done

echo "Search complete"

The script is meant to take in a file containing a list of words, recursively search through a directory excluding hidden files, and finally printing results of the words found, as well as where it was found.

Thanks for your help so far!

p.s. I am new to posting on forums and therefore might have messed up the table containing my script

---------- Post updated at 11:07 AM ---------- Previous update was at 10:52 AM ----------

Im sorry, i posted the wrong script. The most current one is bassically the same. Instead of using zgrep, I used grep, and i havent added the --exclude string.

---------- Post updated at 11:12 AM ---------- Previous update was at 11:07 AM ----------

and also, when I mentioned recursively I intended to have it search from a directory down, through all the subdirectories and files.

Last edited by radoulov; 07-09-2010 at 11:53 AM.. Reason: Fixed code tags!
# 6  
Old 07-13-2010
You could try something like this (untested):

Code:
#!/bin/bash

shopt -s nullglob

read -p 'Enter file for search criteria: ' fname

[[ -z "$fname" ]] && {
  echo File name is mandatory!
  exit 1
  }

read -p 'Enter directory for recursive search' dname

[[ -z "$dname" ]] && echo You didn\'t provide search directory, I\'ll search the current one.

FILE14=$fname

while IFS= read -r; do
    dwords+=( "$REPLY" )
done < "$FILE14"

find "${dname:-.}" -type d ! -name '.*' |
  while IFS= read -r; do
    fnames=( "$REPLY"/*.gz "$REPLY"/*.zip ) # adjust to taste
    if (( ${#fnames[@]} )); then
      for word in "${dwords[@]}"; do
        echo "$REPLY contains $(zcat "${fnames[@]}" | grep -iwoc "$word") occurences of the word $word"
      done
    else
      echo "No compressed files in $REPLY"
    fi
  done

echo "Search complete"

Consider that the script contains various non standard features (shell and GNU specific).
# 7  
Old 07-13-2010
Thanks! That looks sound. I'll have to test it out.
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Search & Replace in Multiple Files by reading a input file

I have a environment property file which contains: Input file: value1 = url1 value2 = url2 value3 = url3 and so on. I need to search all *.xml files under directory for value1 and replace it with url1. Same thing I have to do for all values mentioned in input file. I need script in unix bash... (7 Replies)
Discussion started by: Shamkamde
7 Replies

2. Shell Programming and Scripting

Search compressed files with awk and get FILENAME

I have many compressed files I want to search using awk and want to print some file contents along with the filename it came from on each output record (I simplified awk command). Here are the results with the files uncompressed: awk '{print FILENAME, $0}' test*.txt test1.txt from test1... (3 Replies)
Discussion started by: mjf
3 Replies

3. Shell Programming and Scripting

Using grep with multiple loops in reading files

I am trying to read a file line by line and then search that line in another file and get a particular column from the second file. I have written this code.but its not working properly #!/bin/sh while read keyword in duplicate.txt do echo $keyword while read line do ... (7 Replies)
Discussion started by: Prachi Gupta
7 Replies

4. Red Hat

RAMDISK: EOF while reading compressed data ...Kernel panic - Unable to mount root

I was following this tutorial on How install the rpmfusion nvidia drivers in Fedora 13: F13, F12 & F11 Nvidia driver guides - FedoraForum.org Here's the tutorial: And this is what I did: First I executed the following commands: su rpm -Uvh ... (0 Replies)
Discussion started by: skyxn3t
0 Replies

5. Shell Programming and Scripting

Search & Replace in Multiple Files by reading a input file

Hi, I have a folder which contains multiple config.xml files and one input file, Please see the below format. Config Files format looks like :- Code: <application name="SAMPLE-ARCHIVE"> <NVPairs name="Global Variables"> <NameValuePair> ... (0 Replies)
Discussion started by: haiksuresh
0 Replies

6. UNIX for Advanced & Expert Users

how to grep/read a file inside compressed tgz without extract?

Hi all, I would like to ask whether in Unix shell/perl have any functions or command to allow grep/cat/read a file inside compressed .tgz without extract it? I know we can tar tvf a compressed tgz but this only allow we read the path/filename contained inside the tarball. If we want to read... (3 Replies)
Discussion started by: mayshy
3 Replies

7. UNIX for Dummies Questions & Answers

Using grep to search within files

Hi, At my company, we have custom web sites that we create for different clients. The folder structure is something like: <project name>/html/web/custom/ The custom folder contains a file called "category.html" Every project has the same folder structure, and same file names but, the data... (2 Replies)
Discussion started by: miklo
2 Replies

8. Shell Programming and Scripting

Reading files using grep/sed/awk

After pouring over my LTKS and Unix in a nutshell, I'm stuck! I have a large (BMC Report File) that has breaks on DM (district managers). After a report header, there is a DM header like: DM: DBP AARON ROBERTS At the end of each DM break, there is: ** END OF REPORT ** ... (6 Replies)
Discussion started by: Jodyman
6 Replies

9. UNIX for Advanced & Expert Users

How to search for text within compressed file

I was wondering if there's a way to search within a file that's been compressed. i.e. if file a is inside file a.zip or a.gz, is there a a command that will retrieve the string of data I'm looking for in file a, and list which compressed file it found it in? Please help! Thanks. (8 Replies)
Discussion started by: HLee1981
8 Replies

10. UNIX for Advanced & Expert Users

Search first line of compressed file

I want to read a directory full of compressed files and move the file to another directory if it meets certain criteria. I only want to look at the first line of the compressed file and if I find the string, do the move. I am currently using the following: zgrep -R -L... (1 Reply)
Discussion started by: cbreiner
1 Replies
Login or Register to Ask a Question