KSH: Opening Files based on a file list


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting KSH: Opening Files based on a file list
# 1  
Old 08-08-2010
Question KSH: Opening Files based on a file list

I'd like to grep files for key words using korn shell, and compile the actual contents (not just file name) of those files that contain a combination of those grepped key words into one repository file for reference. However, I'm stuck at the combining part. Here's what I have thus far:

egrep key * | egrep word

This locates all files that contain both the words key and word. Now I want to combine the contents of the files that were found. Does anybody have a notion of how this is accomplished? I'm sure it involves cat, but am having a hard time thinking outside of the box on this.

Thanks in advance!
# 2  
Old 08-08-2010
Hi
Not sure if I understood you correctly.

Code:
egrep -h '(key.*word|word.*key)' * > result

where result is the file which will contain the compiled result.

Guru.
# 3  
Old 08-08-2010
Quote:
Originally Posted by guruprasadpr
Hi
Not sure if I understood you correctly.

Code:
egrep -h '(key.*word|word.*key)' * > result

where result is the file which will contain the compiled result.

Guru.
I read it just a bit differently.... OP wants to have the contents of any file that contains any combination of a set of keywords, not just the lines that contain the keywords. If that is correct, then something like this will work depending on whether all keywords must appear on the same line, or if the file is considered 'matched' if any of the keywords appears in the file.

Code:
egrep -l "key|word" * | xargs cat >result  

egrep -l "key.*word|word.*key" | xargs cat >result

This User Gave Thanks to agama For This Post:
# 4  
Old 08-08-2010
Thanks so much! The egrep -l "system.*log|log.*system" * | xargs cat > result
is exactly what I'm looking for. I'm trying to understand xargs, though. What does that do for me here? I only ask b/c I've never used before.
# 5  
Old 08-08-2010
Quote:
Originally Posted by drumminfool91
I'm trying to understand xargs, though. What does that do for me here? I only ask b/c I've never used before.
It is possible to have written the command like this:

Code:
cat $(egrep -l "pattern" *)

except that the result of the egrep might be more arguments than can be handled on a single command line. By piping the output to xargs, xargs will execute the command that you give it (cat in this case) with n number of arguments such that n does not exceed the limit from a command line perspective. It may invoke the command multiple times to accomplish its task.

With this said, I realise that my original command would be flawed as the '*' given to egrep might expand into too many arguments, causing an error, and thus the xargs in this case would do no good. This would probably be better:

Code:
ls | xargs egrep -l "system.*log|log.*system" | xargs cat > result

The ls command generates a list of filenames which is given to egrep via the first xargs. The result of the searches is then given to cat via the second xargs.

When writing a script it is usually best to assume the worst case and use xargs in this situation. If you're willing to live with the errors, and are pretty sure that there will not be more files in a directory than the max allowable args on a command line, then the first form (without xargs) is easiest and straightforward.
# 6  
Old 08-08-2010
Code:
 cat $(grep -l "word1*word2|word2*word1"  *)

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to find list of missing files based on the file format?

Hi All, In the file names we have dates. Based on the file format given by the user, if any file is not existed for a particular date with in a given interval we should consider that file is missing. I have the below files in the directory /bin/daily/voda_files. ... (9 Replies)
Discussion started by: nalu
9 Replies

2. Shell Programming and Scripting

List the files after sorting based on file content

Hi, I have two pipe separated files as below: head -3 file1.txt "HD"|"Nov 11 2016 4:08AM"|"0000000018" "DT"|"240350264"|"56432" "DT"|"240350264"|"56432" head -3 file2.txt "HD"|"Nov 15 2016 2:18AM"|"0000000019" "DT"|"240350264"|"56432" "DT"|"240350264"|"56432" I want to list the... (6 Replies)
Discussion started by: Prasannag87
6 Replies

3. Shell Programming and Scripting

I have this list of files . Now I will have to pick the latest file based on some condition

3679 Jul 21 23:59 belk_rpo_error_**po9324892**_07212014.log 0 Jul 22 23:59 belk_rpo_error_**po9324892**_07222014.log 3679 Jul 23 23:59 belk_rpo_error_**po9324892**_07232014.log 22 Jul 22 06:30 belk_rpo_error_**po9324267**_07012014.log 0 Jul 20 05:50... (5 Replies)
Discussion started by: LoneRanger
5 Replies

4. Shell Programming and Scripting

Help with ksh-to read ip file & append lines to another file based on pattern match

Hi, I need help with this- input.txt : L B white X Y white A B brown M Y black Read this input file and if 3rd column is "white", then add specific lines to another file insert.txt. If 3rd column is brown, add different set of lines to insert.txt, and so on. For example, the given... (6 Replies)
Discussion started by: prashob123
6 Replies

5. UNIX for Dummies Questions & Answers

Help with ksh script to list files, cp it to another UNIX server

Hi, I'm quite new to ksh scripting, can someone help me with this. Requirements: I need to create a script that list the files from a user input date range. e. g. format of file: *c1*log.2012-12-22-14-00* *c1*log.2012-12-22-14-00* *c1*log.2012-12-22-14-00*... (7 Replies)
Discussion started by: chococrunch6
7 Replies

6. Shell Programming and Scripting

scp list of files using FOR LOOP in ksh

hi i want to scp files from remote server B to my local server A... and i have a file containing list of all files to be scped from remote server B there is a passwordless connectivity set between remote server and my local server. need a ksh script.. with a for loop that goes through... (2 Replies)
Discussion started by: billpeter3010
2 Replies

7. Shell Programming and Scripting

Moving log files based on month - help with ksh shell script

Hello, I'm trying to move the log files from the parent directory to respective monthly folder and I would be running this script on a weekly basis through cron. I'm new to this scripting and here is what i could come up and it runs without really doing anything. I even tried placing echo... (2 Replies)
Discussion started by: jusblues
2 Replies

8. Shell Programming and Scripting

Perl-opening a file then copying files

Good morning guys!! Im still practicing with Perl and now Im trying to open a file, and copy its contents to another file. Them I want to remeove the information out of the orginal file after it is copied over. The flow should be messages-->messages1-->messages2. Kind of like a log... (1 Reply)
Discussion started by: bigben1220
1 Replies

9. Shell Programming and Scripting

create diffrent files based on other file and parameters list

I would like ot create shell script/ bash to create diffrent files based on a file and parameters list. Here is the detail example: I have a textfile and four static parameter files (having ‘?'). mainfile.txt has below records (this count may be more than 50) A200001 A200101 B200001... (9 Replies)
Discussion started by: raghav525
9 Replies

10. Shell Programming and Scripting

ksh: How to get latest file from a list of files in a directory

Hi, I need to get the latest file from a list of files in a particular directory. Please could anyone help me out to get the file. Thank you, - Jay. (1 Reply)
Discussion started by: Jayathirtha
1 Replies
Login or Register to Ask a Question