Problem with egrep and cat


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Problem with egrep and cat
# 1  
Old 04-26-2013
Problem with egrep and cat

Hello,
I want to find some keywords in a dd image.

I have created a keyword file (1.txt) and search the dd image using,
Code:
cat /media/sdb1/test/c.dd.001 | strings | egrep -i --color -f 1.txt

It works,

But how can I get the file name and path?

Many thanks.
# 2  
Old 04-27-2013
Assuming that you want to add the pathname because you want to process multiple files, you could try something like:
Code:
#!/bin/ksh
# Usage: testscript pathname...
for file in "$@"
do      strings "$file" | while IFS="" read -r line
        do      printf "%s: %s\n" "$file" "$line"
        done
done | egrep -i --color -f 1.txt

However, this may give you a lot of extraneous output if any of the EREs in 1.txt match any of the pathnames of the files being processed.

Note that I got rid of the cat you were using. The strings utility accepts a pathname operand and adding extra commands in a pipeline wastes system resources. Also note that if you try to feed multiple files into a singe invocation of strings (whether by giving strings multiple file operands or by using cat to feed data to strings) there is no way to add a filename to the line being processed after string has processed the input if it is given multiple input files in a single invocation.

If the filenames are matched by something in 1.txt, you could create a similar script that adds the filename to the start of each line output by the egrep command, but it will take more system resources to get the results.

If is also possible to write this script as a 1-liner, but I much prefer readable over producing the minimum number of characters/lines needed to get a job done.

PS I use the Korn shell, but this script will work with any shell that recognizes basic Bourne shell syntax.

Last edited by Don Cragun; 04-27-2013 at 12:51 AM.. Reason: add comment on shell choice
# 3  
Old 04-27-2013
Not sure what you mean by "dd image". Is this a binary file created by dding some partition? Is it mounted somewhere? Then the file system is accessible, and it should be easy to find out the matching files' names. Or do you search the entire binary file with strings? Then I think it's utterly difficult to get at the file names...
# 4  
Old 04-27-2013
Quote:
Originally Posted by RudiC
Not sure what you mean by "dd image". Is this a binary file created by dding some partition? Is it mounted somewhere? Then the file system is accessible, and it should be easy to find out the matching files' names. Or do you search the entire binary file with strings? Then I think it's utterly difficult to get at the file names...
It's a full disk raw image, I am searching the entire binary file with the stings.

I am a new Linux learner, is there any other ways to search them and out put the path? I will print the byte off set number using egrep and try to find its inode number.

---------- Post updated at 08:12 AM ---------- Previous update was at 07:47 AM ----------

Quote:
Originally Posted by Don Cragun
Assuming that you want to add the pathname because you want to process multiple files, you could try something like:
Code:
#!/bin/ksh
# Usage: testscript pathname...
for file in "$@"
do      strings "$file" | while IFS="" read -r line
        do      printf "%s: %s\n" "$file" "$line"
        done
done | egrep -i --color -f 1.txt

However, this may give you a lot of extraneous output if any of the EREs in 1.txt match any of the pathnames of the files being processed.

Note that I got rid of the cat you were using. The strings utility accepts a pathname operand and adding extra commands in a pipeline wastes system resources. Also note that if you try to feed multiple files into a singe invocation of strings (whether by giving strings multiple file operands or by using cat to feed data to strings) there is no way to add a filename to the line being processed after string has processed the input if it is given multiple input files in a single invocation.

If the filenames are matched by something in 1.txt, you could create a similar script that adds the filename to the start of each line output by the egrep command, but it will take more system resources to get the results.

If is also possible to write this script as a 1-liner, but I much prefer readable over producing the minimum number of characters/lines needed to get a job done.

PS I use the Korn shell, but this script will work with any shell that recognizes basic Bourne shell syntax.
Thank you so much, but I don't understand how to use shell....also this search is searching the binary content not the file name.

If the keywords are matched and how to translate the disk off set to the partition inode number? The partition block size is 512 and partition off set is 2048, NTFS file system.
# 5  
Old 04-27-2013
Is the image corrupt or the partition table mangled in some way? Why do you not simply mount the filesystem(s) and traverse them with find or grep -R? That's much simpler than taking the byte offset of a string in an image, analyzing the partition table (MBR or GPT) to determine in which partition the byte is located, then jumping to the beginning of that partition and searching for the relevant filesystem data structures.

Regards,
Alister

Last edited by alister; 04-27-2013 at 11:36 AM..
This User Gave Thanks to alister For This Post:
# 6  
Old 04-27-2013
Quote:
Originally Posted by alister
Is the image corrupt or the partition table mangled in some way? Why do you not simply mount the filesystem(s) and traverse them with find or grep -R? That's much simpler than taking the byte offset of a string in an image, analyzing the partition table (MBR or GPT) to determine in which partition the byte is located, then jumping to the beginning of that partition and searching for the relevant filesystem data structures.

Regards,
Alister
Thank you very much. After mount the partition, I found the file names now using grep -iR

Code:
grep -iR -f 1.txt /mnt

Smilie
# 7  
Old 04-27-2013
Happy to help. Welcome to the forums.

Regards,
Alister
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Egrep problem

Hi everyone, Can someone look this over? find /oracle/diag/rdbms/*/*/trace -type f -name '*d00*.trc' -mtime 0 \ -exec egrep -c 'TNS-12535: TNS:operation timed out' '{}' '+' | awk -F: '{print $1}' | egrep -c '2015-01-22' usidp/trace/abcdef_d001_21751.trc:9 \... (1 Reply)
Discussion started by: bdby
1 Replies

2. UNIX for Dummies Questions & Answers

Problem pattern redundancy with egrep

%%%%% (2 Replies)
Discussion started by: lucasvs
2 Replies

3. Shell Programming and Scripting

Problem with cat

I am trying to use the following code: cat *.txt > OutPutFile.txt Using files with a expression in one single line that always end with " ; " something like this: Block1.txt Block2.txt Block3.txt The expected result should be something like this: OutPutFile.txt: My code works OK in this... (4 Replies)
Discussion started by: Xterra
4 Replies

4. Shell Programming and Scripting

Formatting problem with cat, egrep and perl

Hi guys I'm using the following script to change input file format to another format. some where I'm getting the error. Could you please let me know if you find out? cat input.txt|egrep -v ‘^#'|\ perl -ane ‘if (@F>3){$_=~/(chr.+):(\d+)\ s()/;print $1,”\t”,$2,”\t”,($2+35),”\n”}'\ > output.bed ... (1 Reply)
Discussion started by: repinementer
1 Replies

5. Shell Programming and Scripting

Problem using egrep: spaces and newline

Hello: I am working in bash and am a newbie. I want to eliminate spaces from strings. Since this is a basic operation, I searched online and implemented the suggestions; however, I am facing a problem here. I have an input file which looks like this: abc defghi jklmno pqrs tuvw xyzabcd... (8 Replies)
Discussion started by: andyu11
8 Replies

6. Shell Programming and Scripting

egrep problem

Hello Experts, I am trying to parse a gz file like this gzip -cd filename | xargs egrep -h -e '.*somepattern</TAG>' | grep -c '<TAG2>`date '+%Y-%m-%d'`</TAG2>' But I am getting an error : egrep cant open. Any ideas fellas? (1 Reply)
Discussion started by: King Nothing
1 Replies

7. Shell Programming and Scripting

egrep problem | printing next line?????

Hi !!! Dear People, Please help me with the following problem. consider this output: Top 5 Timed Events ~~~~~~~~~~~~ Total Event Waits Time (s) Ela Time ---------------------------- ------------ ----------- ----- CPU time ... (3 Replies)
Discussion started by: user__user3110
3 Replies

8. Shell Programming and Scripting

egrep understanding problem

Hi, Can anyone please let me know the meaning of this line,i am not able to understand the egrep part(egrep '^{1,2}).This will search for this combination in beginning but what does the values in {}signifies here. /bin/echo $WhenToRun | egrep '^{1,2}:$' >/dev/null (1 Reply)
Discussion started by: namishtiwari
1 Replies

9. Shell Programming and Scripting

cat problem

Hello again; I have a file in this format ./this is/first/1 ./this is/second/2 ./this is/third/3 and i am using this file in a for loop with cat command like this for i in `cat directory.txt` do .......... done Bu there is a problem because my directory is "this is" but... (7 Replies)
Discussion started by: redbeard_06
7 Replies

10. UNIX for Advanced & Expert Users

Problem with cat

On Solaris 5.8 in ksh, I have a sample.txt with contents A 105 305 B 205 405 C 100 198 ....................... when I do a cat sample.txt the O/P is exactly as above but when I do a echo `cat sample.txt` the O/P changes to A 105 305 B 205 405 C 100 198........... Everything is... (3 Replies)
Discussion started by: macrulez
3 Replies
Login or Register to Ask a Question