Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

How to grep a number in a file to find them in another file

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting How to grep a number in a file to find them in another file
# 1  
Old 12-11-2007
How to grep a number in a file to find them in another file
Hi,

I tried to simulate some example in Shell particularly finding item and matching in another file.

A very simple example:-
Basically I have 2 files, file A and file B.

In file A, I have columns of fields such that:-

aaa 107
bbb 108
ccc 109

In file B, I have columns of fields such that:-
101 2 1
102 3 1
107 2 1
108 3 1
109 2 1

I would like to know, if I would like to extract let say first element of file A and compare with file B elements.

If found, I would like to have the position of element 107 in file B in this case it is on 3rd line. From the elements found in file B, I would like to perform some computation on their fields. And the same goes for other elements in file A which in the end will pipe to another output file.

Currently, I tried to do below:-

#!/bin/sh

cat a.txt | awk '
{

for $2 in a.txt

do

grep -i $2 b.txt
# printf("%d", ); #to print the position of the string found in b.txt
}'

cat b.txt| awk '
{
count[$i]=$2+$3;

}
END{
printf("%d",count[$i]);
}'
done

I could not get it working because of several reason:-

i)rom the grep option -i it match any numbers entries start with let say "1"
rather than the exact number.

ii)I could not figure out how to open two files similarly using AWK so I can extract fields.

Please advise. Thanks.


-Jason
# 2  
Old 12-11-2007
Hi.

Some of the initial complexity can be removed by using command join:
Code:
#!/usr/bin/env sh

# @(#) s1       Demonstrate a join, using a field different from 1.

set -o nounset
echo

debug=":"
debug="echo"

## Use local command version for the commands in this demonstration.

echo "(Versions displayed with local utility \"version\")"
version >/dev/null 2>&1 && version bash my-nl join

echo
echo " Input files:"
cat data1
echo
cat data2

echo
echo " Result of join:"
join -1 2 data1 data2

exit 0

Producing:
Code:
% ./s1

(Versions displayed with local utility "version")
GNU bash 2.05b.0
my-nl (local) 296
join (coreutils) 5.2.1

 Input files:
aaa 107
bbb 108
ccc 109

101 2 1
102 3 1
107 2 1
108 3 1
109 2 1

 Result of join:
107 aaa 2 1
108 bbb 3 1
109 ccc 2 1

You may still wish to use awk for the post-join processing, because that makes it easy to use the resulting fields. See man pages for details ... cheers, drl
# 3  
Old 12-11-2007
Hi,

Thanks for the below input.

However if we are using Join command, the position of the element in the matching file could not be retrived as similar elements are already grouped together.

I assume that the join results would be pipe into a new output file.

So far I have:-

cat a.txt| while read LINE
do

char=`echo "${LINE}"| awk '{print $2}'`

echo $char

cat b.txt| awk '{

#grep -n $char b.txt| tr ":" " "|awk '{print $1}'

done

However, I am stuck in how to retrieve the first field of b.txt.
Next, I am stuck in getting the current line found in b.txt

Please advise.Thanks.


-Jason
# 4  
Old 12-11-2007
Hi, Jason.

That is why I asked you to provide the desired results -- to let us see what you need. An example is better than 1000 words, yes?
Code:
#!/usr/bin/env sh

# @(#) s2       Demonstrate a join, with file position added.

set -o nounset
echo

debug=":"
debug="echo"

## Use local command version for the commands in this demonstration.

echo "(Versions displayed with local utility \"version\")"
version >/dev/null 2>&1 && version bash my-nl join

echo
echo " Input files:"
cat data1
echo
cat data2
echo
cat -n data2 >data3
cat data3

echo
echo " Result of join:"
join -1 2 -2 2 data1 data3

exit 0

Code:
% ./s2

(Versions displayed with local utility "version")
GNU bash 2.05b.0
my-nl (local) 296
join (coreutils) 5.2.1

 Input files:
aaa 107
bbb 108
ccc 109

101 2 1
102 3 1
107 2 1
108 3 1
109 2 1

     1  101 2 1
     2  102 3 1
     3  107 2 1
     4  108 3 1
     5  109 2 1

 Result of join:
107 aaa 3 2 1
108 bbb 4 3 1
109 ccc 5 2 1

cheers, drl
# 5  
Old 12-11-2007
Hi,

I miss ed out the output in earlier post.

Currently, my input file of A.txt

aaa 1
bbb 2
ccc 3
ddd 100

B.txt

2 3 4
3 4 5

My code so far is:-

#!/bin/bash

cat a.txt| while read LINE
do

#to get the second field of a.txt
char=`echo "${LINE}"| awk '{print $2}'`

echo $char

#to grep the line position of b.txt which match the element in a.txt
grep -n $char b.txt | tr ":" " "| awk '{print $1}'

done

The output is:-
1

2
1

3
1
2

100

The problem here is i) grep does not want to recoqnize only the first field of b.txt (which you can see for 3 in a.txt there is 1, 2 returned because there is "3" in b.txt for two entries.

ii)And there is problem if I applied grep -m 1 $char b.txt|awk '{print $1}'
for printing the matching element for first field of b.txt. The output would be

1

2
2

3
2

100


Ultimately, what I would like to have the desired otuput is:-

1 Not Found
2 Position 1 3+4
3 Position 2 4+5
100 Not Found


Thanks.
-Jason
# 6  
Old 12-11-2007
Hi.

I am still using your original 2 data sets, which I call data1 and data2. The final output consists of a first set of matching lines, with all the fields from both files. The second set is the set of lines from data2 that is unmatched with the text string " Not found" appended.

The line position is given, as you can see, because the file was augmented with the line numbers by "cat -n".

The output from the first join should be easily read by awk, and the fields will be available to you, as usual, in variables "$1", "$2", etc.

If you need the reports to be different, you can change the join options -- see man join.
Code:
#!/usr/bin/env sh

# @(#) s3       Demonstrate a join, file position, unmatched lines.

set -o nounset
echo

debug=":"
debug="echo"

## Use local command version for the commands in this demonstration.

echo "(Versions displayed with local utility \"version\")"
version >/dev/null 2>&1 && version bash my-nl join

echo
echo " Input files:"
cat data1
echo
cat data2
echo
cat -n data2 >data3
cat data3

echo
echo " Result of join for matched lines:"
join -j 2 data1 data3

echo
echo " Result of join unmatchable lines from data2"
join -v 2 -j 2 data1 data3 |
sed -e 's/$/ Not found/'

exit 0

Producing:
Code:
% ./s3

(Versions displayed with local utility "version")
GNU bash 2.05b.0
my-nl (local) 296
join (coreutils) 5.2.1

 Input files:
aaa 107
bbb 108
ccc 109

101 2 1
102 3 1
107 2 1
108 3 1
109 2 1

     1  101 2 1
     2  102 3 1
     3  107 2 1
     4  108 3 1
     5  109 2 1

 Result of join for matched lines:
107 aaa 3 2 1
108 bbb 4 3 1
109 ccc 5 2 1

 Result of join unmatchable lines from data2
101 1 2 1 Not found
102 2 3 1 Not found

Best wishes ... cheers, drl
# 7  
Old 12-12-2007
Hi drl,

Thanks for the reply.

Now i see this as easier implementation compared to others.
Unfortunately,I just realized that the results expected is not what I expected at the first place. Sorry for thatSmilie I just get confused.

Instead it is supposed to be in a.txt;each element is compared separated to b.txt at one time (b.txt is multiple text file which varies across different directories).

To give a bigger picture, a.txt contain all the elements to compare with different file of b.txt.

Can we still use join in this case?

Example:-

Input files:
a.txt

aaa 107
bbb 108
ccc 109

b.txt
101 2 1
102 3 1
107 2 1
108 3 1
109 2 1

b.txt
101 2 1
102 3 1
107 2 1
108 4 2
109 2 1

b.txt
101 2 1
102 3 1
107 2 1
108 4 2
109 6 1

At one time, each line of a.txt is read starting with "aaa 107" is match with elements in b.txt. Next, second line of a.txt which is "bbb 108" is match with elements in another file of b.txt which can be similar to earlier b.txt.

Ultimately, I would think your idea of join is great where it would be nice if
every element of a.txt could be joined with matching element in different b.txt file.

Desired output would be :-
aaa 107 2 1
bbb 108 4 2
ccc 109 6 1


I am so sorry for the earlier confusion of the intended otuput i would liek to see.Please advise.

Thanks.


-Jason
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Grep: Retrieve two strings from one file to find them anyone on line in another file

I am having trouble matching *two* strings from one file anywhere in a line of a second file, and could use some help getting this figured out. My preference would be to use grep for this because I would like to take advantage of its -A option. The latter is due to the fact that I would like both... (2 Replies)
Discussion started by: jvoot
2 Replies

2. Shell Programming and Scripting

Grep pattern after specific line number in a file

Hi guys, I am running a while loop in a script ro read a file line by line. Now I want to run a grep only on the lines below the line I am that is being read by the while loop. Eg: If my while loop is on line 4 of the file, the grep only runs below line 4 and does not include line 1,2... (3 Replies)
Discussion started by: Junaid Subhani
3 Replies

3. UNIX for Dummies Questions & Answers

Grep SQL output file for greater than number.

Hi, This is my first post. I have a korn shell script which outputs a select statment to a file. There is only one column and one row which contains a record count of the select statement. The select statement looks something like this: SELECT COUNT(some_field) AS "count_value" ... (2 Replies)
Discussion started by: MurdocUK
2 Replies

4. Programming

Find the number in the file

In my system , there are text files will be generated monthly , the file name begins with xxx , then year , month ( for example xxxxx201310.txt means Oct 2013 ) I have below command to count how many abc in the month , but it only count the number in this month . NUMBER=$(cat xxxxx201310.txt... (2 Replies)
Discussion started by: ust
2 Replies

5. Shell Programming and Scripting

AWK-grep from line number to the end of file

Does anyone know how to use awk to act like grep from a particular line number to the end of file? I am using Solaris 10 and I don't have any GNU products installed. Say I want to print all occurrences of red starting at line 3 to the end of file. EXAMPLE FILE: red green red red... (1 Reply)
Discussion started by: thibodc
1 Replies

6. Shell Programming and Scripting

How to use grep & find command to find references to a particular file

Hi all , I'm new to unix I have a checked project , there exists a file called xxx.config . now my task is to find all the files in the checked out project which references to this xxx.config file. how do i use grep or find command . (2 Replies)
Discussion started by: Gangam
2 Replies

7. Shell Programming and Scripting

File find | xargs grep for pattern file

Folks I've been struggling this with for far too liong now and need your help! I've been happily using grep for a search of a directory, to list the files which contain a string: find . -type f -mtime -5 -print | xargs grep -l 'invoiceID=\"12345\"' Now the list of 'invoiceID' I am... (4 Replies)
Discussion started by: daveaasmith
4 Replies

8. UNIX for Dummies Questions & Answers

Find number of columns in a file

Hi all, may seem a very stupid question.but me stuck up in it for long.... How to find the number of columns in a ASCII file. EX:-Demo.lst songs 1 34 45 67 Number of columns should be 5. Regards, Anindya ;) (13 Replies)
Discussion started by: rahul26
13 Replies

9. UNIX for Dummies Questions & Answers

find file grep it and print file name

i am trying to search a few hundred release note text files for a certain word. however when i use the below command i can find a file that contains it but i dont know the file name. how can i change this command to output the name of the file that grep was successful in? find builds -name... (4 Replies)
Discussion started by: borderblaster
4 Replies

10. Shell Programming and Scripting

How to grep a number in a file name

Hi, I have multiple files where it starts with test1.c, test2.c,test3.c and so on. I would like to get each file separately to perform abstraction from these files. I tried something like:- for t in ./* filenumber=${t:4} # to cut the "test" in order to get the number cat... (3 Replies)
Discussion started by: ahjiefreak
3 Replies
Login or Register to Ask a Question