Indexing or Filtering code- Pattern Search by comparing two files Post: 302220659

Sponsored Content

Top Forums Shell Programming and Scripting Indexing or Filtering code- Pattern Search by comparing two files Post 302220659 by aavam on Friday 1st of August 2008 10:41:16 AM

08-01-2008

Registered User

True Coding Challeng...Indexing or Filter code- Pattern Search by comparing two files

So here is goes to the Gurus of shell programming......I have tried a lot of different ways and its a very challenging code to write but i am enjoying it as i troubleshoot and hopefully someone can provide me a better option....Thank you in advance for your time and support....Much appreciated...

i have two files
Log file
Keyword file

Logfiles varies in page numbers and has a lot of logs. This logs files on the bottom of each page contains a page number field like (page 10) etc.

I need to write a script (filter) where i would like to read both files and get an output files........

I like to update keyword file at any time to provide search criteria for log file..

So logs file containing thousands of words and i am extracting some words which are listed in my keyword file and placing those words in my output files alphabetically as well as providing the page number where they have occurrences in log file.

I would also like to have my first line of the output file as "Filtered Output Results".....as a heading for this output file...(so something similar to the index page that we have at the end of the book).

Like to get Sed, Awk and grep script examples as i am very interested how can i make this happen in all three.....its really cool and exciting when you get your desired result.....

I have serach several example in this forum and tried to write a script but i am not successful yet......tried grep , sed and awk as follows

Some of my tries......

#!/bin/bash
Keyword='/home/aavam/keyword'
Data='/home/aavam/data'
Output='C:/cygwin/home/aavam/output'
grep -f $Keyword $Data > $Output

#/usr/xpg4/bin/grep -f $Keyword $Data > $Output
#page=`echo $LINE | awk -F= '{print $NF}'`
#/usr/xpg4/bin/grep -Ff $Keyword $Data > $Output | page=`echo $LINE | awk -F= '{print $NF}'`
#/usr/xpg4/bin/grep -f $Keyword $Data > $Output
#/usr/xpg4/bin/grep -Ff $Keyword $Data > $Output | page=`echo $LINE | awk -F= '{print $NF}'`

#sed -f keyword.sed /export/home/aavam/shell-prog/data ---->gives error that keyword.sed not known

/export/home/aavam/outputv3

#for name in 'cat keyword'
#do
#grep $name data
#done > output
#awk -f compare.awk keyword data ---->gives error compare.awk not known etc

Thank you again

Last edited by aavam; 08-01-2008 at 12:09 PM..

aavam

View Public Profile for aavam

Find all posts by aavam

9 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

search all files for a pattern

Hi there, I am looking for a shell script which recursively searches all the files under all the directories for a pattern specified in the script. For e.g., i am looking forward to search for the file names which contains numbers of the form 001*****. Thanks in advance, Naik

2. UNIX for Advanced & Expert Users

pattern search between 2 files

Afternoon guys, I have 2 files, 1.txt and 2.txt containing employee numbers. the 1st file (1.txt) is an extract from sybase with active employee numbers, the 2nd (2.txt) is a scan from the sybase log containing successfull logins *** which i have already mined and now contains only employee...

3. Shell Programming and Scripting

awk script issue : comparing two files with a pattern

File 1 ################################################################# pma.zcal.iop_pma_zcal_cntl (2710.080 115.200) pma.lanea23.rx0.cntl (696.960 844.800) pma.lanea67.rx0.cntl (1733.760 844.800) pma.zcal.iop_pma_zcal_cust (2280.960 115.200)...

4. Shell Programming and Scripting

[ask]filtering file to indexing...

dear all, i have file with format like this file_master.txt 20110212|231213|rio|apri|23112|222222 20110212|312311|jaka|dino|31223|543234 20110301|343322|alfan|budi|32131|333311 ... i want filter with output like this index_nm.txt rio|apri jaka|dino ... index_years.txt 20110212...

5. Shell Programming and Scripting

Indexing each repeating pattern of rows in a column using awk/sed

Hello All, I have data like this in a column. 0 1 2 3 0 3 4 5 6 0 1 2 3 etc. where 0 identifies the start of a pattern in my data. So I need the output like below using either awk/sed. 0 1

6. Shell Programming and Scripting

To search the pattern on the basis of date and exit code

Hi, I am getting scheduler log file on daily basis from windows box which contains job status and corresponding date, date is in windows format. I wanted to write one script which will search the pattern (Exit code) for the today's date and if code is Zero then Job Success message should be...

7. Shell Programming and Scripting

Search pattern in today's files only

Hi Friends, I am in search of unix command which can search a particular pattern in all files which are created/modified today ONLY. Which is the best way to achieve this? Thanks in advance.

8. Shell Programming and Scripting

Pattern search multiple files

#!/usr/bin/ksh a="Run successfully" cd $APPS ls -l *.txt | while read $txt do if then cp $APPS/$txt cp $hist/$txt else rm $APPS/$txt echo "Files has been removed" fi done New in shell script please help me out Around 100 txt files in $APPS dir i want to search pattern from...

9. UNIX for Beginners Questions & Answers

Grep/awk using a begin search pattern and end search pattern

I have this fileA TEST FILE ABC this file contains ABC; TEST FILE DGHT this file contains DGHT; TEST FILE 123 this file contains ABC, this file contains DEF, this file contains XYZ, this file contains KLM ; I want to have a fileZ that has only (begin search pattern for will be...

LEARN ABOUT NETBSD

dh_installdeb

DH_INSTALLDEB(1)						     Debhelper							  DH_INSTALLDEB(1)

NAME

       dh_installdeb - install files into the DEBIAN directory

SYNOPSIS

       dh_installdeb [debhelperoptions]

DESCRIPTION

       dh_installdeb is a debhelper program that is responsible for installing files into the DEBIAN directories in package build directories with
       the correct permissions.

FILES

       package.postinst
       package.preinst
       package.postrm
       package.prerm
	   These maintainer scripts are installed into the DEBIAN directory.

	   Inside the scripts, the token #DEBHELPER# is replaced with shell script snippets generated by other debhelper commands.

       package.triggers
       package.shlibs
	   These control files are installed into the DEBIAN directory.

	   Note that package.shlibs is only installed in compat level 9 and earlier.  In compat 10, please use dh_makeshlibs(1).

       package.conffiles
	   This control file will be installed into the DEBIAN directory.

	   In v3 compatibility mode and higher, all files in the etc/ directory in a package will automatically be flagged as conffiles by this
	   program, so there is no need to list them manually here.

       package.maintscript
	   Lines in this file correspond to dpkg-maintscript-helper(1) commands and parameters.  However, the "maint-script-parameters" should not
	   be included as debhelper will add those automatically.

	   Example:

	       # Correct
	       rm_conffile /etc/obsolete.conf 0.2~ foo
	       # INCORRECT
	       rm_conffile /etc/obsolete.conf 0.2~ foo -- "$@"

	   In compat 10 or later, any shell metacharacters will be escaped, so arbitrary shell code cannot be inserted here.  For example, a line
	   such as "mv_conffile /etc/oldconffile /etc/newconffile" will insert maintainer script snippets into all maintainer scripts sufficient
	   to move that conffile.

	   It was also the intention to escape shell metacharacters in previous compat levels.	However, it did not work properly and as such it
	   was possible to embed arbitrary shell code in earlier compat levels.

	   The dh_installdeb tool will do some basic validation of some of the commands listed in this file to catch common mistakes.  The
	   validation is enabled as a warning since compat 10 and as a hard error in compat 12.

SEE ALSO

       debhelper(7)

       This program is a part of debhelper.

AUTHOR

       Joey Hess <joeyh@debian.org>

11.1.6ubuntu2							    2018-05-10							  DH_INSTALLDEB(1)