Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Counting number of files that contain words stored in another file
Top Forums Shell Programming and Scripting Counting number of files that contain words stored in another file Post 302491735 by shoaibjameel123 on Friday 28th of January 2011 06:43:37 AM
Old 01-28-2011
Counting number of files that contain words stored in another file
Hi All,

I have written a script on this but it does not do the requisite job. My requirement is this:

1. I have two kinds of files each with different extensions. One set of files are *.dat (6000 unique DAT files all in one directory) and another set *.dic files (6000 unique DIC files in all in the same directory where DAT files are located)

2. The files only contain words all in new lines. For example:
1.dat contains something like this:
Code:
computer
red
apple
orange

1.dic looks like this:
Code:
computer
apple
red
blue

3. For every corresponding DAT file there is a DIC file. For 1.dat, I have 1.dic, 2.dat and 2.dic .......6000.dat and 6000.dic

4. What I want to do is to read every word from DIC files and search in all DAT files and find the number of DAT files that contain that word from the DIC file and store the result in FIL files. This means I have to only count once in the DAT files even if that word appears several times in that DAT file. For example:
1.dic contains 10 words, I read every word from 1.dic line by line and search in all DAT files as to how many DAT files contain that word from 1.dic. Then I write the result (i.e. count values) in every line in 1.fil. Similarly, I read every word in 2.dic line by line, search words in all DAT files and write the count values in 2.fil. My 2.fil should look something like this:
Code:
2
3
1
3

i.e word in the first line (of 2.dic) appears 2 times in all the DAT files (counting that word only once in all DAT files even if one DAT file contains that word several times). Same thing has to be done with all the 6000 DIC files.

What I have done so far:

Code:
for DAT in *.dat
do
for DIC in *.dic
do
while read word
CNT=$(basename "$DAT" .dat).fil
DIC=$(basename "$DAT" .dat).dic
grep -il "$word" | find . | wc -l $DIC $DAT > $FIL
done
done

 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Counting words in a file

I'm trying to figure out a way to count the number of words in the follwing file: cal 2002 > file1 Is there anyway to do this without using wc but instead using the cut command? (1 Reply)
Discussion started by: r0mulus
1 Replies

2. UNIX for Dummies Questions & Answers

Counting number of files in a directory

Some simple questions from a simple man. If i wanted to count the number of files contained within a directory, say /tmp would ls -l /tmp ¦ wc -l suffice and will it be accurate? second one: How would i check the number of files with a certain string in the filename, in the same directory. ... (2 Replies)
Discussion started by: iamalex
2 Replies

3. UNIX for Dummies Questions & Answers

count the number of files which have a search string, but counting the file only once

I need to count the number of files which have a search string, but counting the file only once if search string is found. eg: File1: Please note that there are 2 occurances of "aaa" aaa bbb ccc aaa File2: Please note that there are 3 occurances of "aaa" aaa bbb ccc... (1 Reply)
Discussion started by: sudheshnaiyer
1 Replies

4. UNIX for Dummies Questions & Answers

counting words then amending to a file

i want to count the number of words in a file and then redirect this to a file echo 'total number of words=' wc -users>file THis isnt working, anyone any ideas. (1 Reply)
Discussion started by: iago
1 Replies

5. Programming

Counting the words in a file

Please find the below program. It contains the purpose of the program itself. /* Program : Write a program to count the number of words in a given text file */ /* Date : 12-June-2010 */ # include <stdio.h> # include <stdlib.h> # include <string.h> int main( int argc, char *argv ) {... (6 Replies)
Discussion started by: ramkrix
6 Replies

6. Shell Programming and Scripting

Help in counting the no of repeated words with count in a file

Hi Pls help in solving my doubt.Iam having file like below file1.txt priya jenny jenny priya raj radhika priya bharti bharti Output required: I need a output like count of repeated words with name for ex: priya 3 jenny 2 (4 Replies)
Discussion started by: bha148
4 Replies

7. Shell Programming and Scripting

Counting occurrence of all words in a file

Hi, Given below is the input file: http://i53.tinypic.com/2vmvzb8.png Given below is what the output file should look like: http://i53.tinypic.com/1e6lfq.png I know how to count the occurrence of 1 word from a file, but not all of them. Can someone help please? An explanation on the... (1 Reply)
Discussion started by: r4v3n
1 Replies

8. Shell Programming and Scripting

Counting occurrences of all words in multiple files

Hey Unix gurus, I would like to count the number occurrences of all the words (regardless of case) across multiple files, preferably outputting them in descending order of occurrence. This is well beyond my paltry shell scripting ability. Researching, I can find many scripts/commands that... (4 Replies)
Discussion started by: twjolson
4 Replies

9. Shell Programming and Scripting

Eliminating words from a file through ngrams stored in another file

Hello, I have a large data file which contains a huge amount of garbage i.e. words which do not exist in the language. An example will make this clear: kpaware nlupset rrrbring In other words these words are invalid in English and constitute garbage in the data. I have identified such... (2 Replies)
Discussion started by: gimley
2 Replies

10. Shell Programming and Scripting

How count the number of two words associated with the two words occurring in the file?

Hi , I need to count the number of errors associated with the two words occurring in the file. It's about counting the occurrences of the word "error" for where is the word "index.js". As such the command should look like. Please kindly help. I was trying: grep "error" log.txt | wc -l (1 Reply)
Discussion started by: jmarx
1 Replies

LEARN ABOUT SUNOS

datadm

datadm(1M)																datadm(1M)

NAME

       datadm - maintain DAT static registry file

SYNOPSIS

       /usr/bin/datadm [-v] [-u] [-a service_provider.conf] [-r service_provider.conf]

       The datadm utility maintains the DAT static registry file, dat.conf(4).

       This administrative configuration program allows uDAPL service providers to add and remove themselves to the dat.conf file.

       You  can  add  or  remove  interface  adapters that a service provider supports from a system after its installation. You can use datadm to
       update the dat.conf file to reflect the current state of the system. A new set of interface adapters for all  the  service  providers  cur-
       rently installed is regenerated.

       The following options are supported:

       -a service_provider.conf

	   Enumerate  each  device  entry  in the service_provider.conf(4) file into a list of interface adapters, that is, interfaces to external
	   network that are available to uDAPL consumers.

       -r service_provider.conf

	   Remove the list of interface adapters that corresponds to the device entry in the service_provider.conf(4) file.

       -u

	   Update the dat.conf to reflect the current state of the system with an up to date set of interface adapters for the	service  providers
	   that are currently listed in the DAT static registry.

       -v

	   Display the DAT static registry file, dat.conf.

       Example 1: Enumerating a Device Entry

       The following example enumerates a device entry in the service_provider.conf(4) file into interface adapters in the dat.conf(4) file.

       Assume  that  SUNW  has	a service provider library that supports the device tavor. It has a service_provider.conf(4) file installed in the
       directory /usr/share/dat/SUNWudaplt.conf with a single entry as follows:

       driver_name=tavor u1.2 nonthreadsafe default udapl_tavor.so.1 SUNW.1.0 ""

       tavor is an Infiniband Host Channel Adapter with two ports. Both IB ports exist in a single IB partition, 0x8001. If  an  IP  interface	is
       plumbed to each port, there are two IPoIB device instances, ibd0 and ibd1:

       # ls -l /dev/ibd*
       /dev/ibd0 -> /devices/pci@1/pci15b3,5a44@0/ibport@1,8001,ipib:ibd0
       /dev/ibd1 -> /devices/pci@1/pci15b3,5a44@0/ibport@2,8001,ipib:ibd1

       Running	the  command,  datadm  -a  /usr/share/dat/SUNWudaplt.conf  appends  two  new  entries  (if  they  do  not  already exists]) in the
       /etc/dat/dat.conffile:

       ibd0 u1.2 nonthreadsafe default udapl_tavor.so.1 SUNW.1.0 ""
       "driver_name=tavor"
       ibd1 u1.2 nonthreadsafe default udapl_tavor.so.1 SUNW.1.0 ""
       "driver_name=tavor"

       Example 2: Updating the dat.conf to Reflect the Current State of the System

       A new IB partition, 0x8002 is added to the above example covering port 1 of the Host Channel Adapter. If a new IP interface is  plumbed	to
       port 1/partition 0x8002, there is a third IPoIB device instance: ibd2.

       # ls -l /dev/ibd*
	/dev/ibd0 -> /devices/pci@1/pci15b3,5a44@0/ibport@1,8001,ipib:ibd0
	/dev/ibd1 -> /devices/pci@1/pci15b3,5a44@0/ibport@2,8001,ipib:ibd1
	/dev/ibd2 -> /devices/pci@1/pci15b3,5a44@0/ibport@1,8002,ipib:ibd2

       Running datadm -u command, updates the /etc/dat/dat.conf file with a new entry added reflecting the current state of the system.

       datadm -v shows that there are now three entries in the /etc/dat/dat.conf file:

       ibd0 u1.2 nonthreadsafe default udapl_tavor.so.1 SUNW.1.0 ""
       "driver_name=tavor"
       ibd1 u1.2 nonthreadsafe default udapl_tavor.so.1 SUNW.1.0 ""
       "driver_name=tavor"
       ibd2 u1.2 nonthreadsafe default udapl_tavor.so.1 SUNW.1.0 ""
       "driver_name=tavor"

       /etc/dat/dat.conf       DAT static registry file

       See attributes(5) for descriptions of the following attributes:

       +-----------------------------+-----------------------------+
       |      ATTRIBUTE TYPE	     |	    ATTRIBUTE VALUE	   |
       +-----------------------------+-----------------------------+
       |Availability		     |SUNWudaplu		   |
       +-----------------------------+-----------------------------+
       |Interface Stability	     |Evolving			   |
       +-----------------------------+-----------------------------+

       pkgadd(1M), pkgrm(1M), libdat(3LIB), dat.conf(4), service_provider.conf(4). attributes(5)

								    18 Jun 2004 							datadm(1M)
All times are GMT -4. The time now is 09:45 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy