Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Search data from one file to another

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Search data from one file to another
# 1  
Old 07-09-2009
Search data from one file to another
Hello Friends,

I have
File1.txt has 300,000 records
File2.txt has 90,000 records

I need to verify that all 90,000 records are exist in File1.txt.

I have following query but it takes for ever.

cat File2.txt|while read line; do cat /tmp/File1.txt|grep $line >> res.tmp; done

Any better suggestion please?

Thanks,
Prashant
# 2  
Old 07-09-2009
How about?:
Code:
while read line; do fgrep "$line" File1.txt >> res.tmp; done < File2.txt

# 3  
Old 07-10-2009
See if this works. This gives the missing records in file1.

Code:
 
diff file1 file2 | grep '>' | sed 's/^.//'

Last edited by lathavim; 07-10-2009 at 12:17 AM..
# 4  
Old 07-10-2009
Code:
egrep -v -f file1 file2 > temp
if [ $? -eq 0 ];then
 echo "missed content"
 cat temp
else
 echo "all exist"
fi
rm temp

# 5  
Old 07-17-2009
Friends,

If I have file1 with ONE column and file2 has TWO columns.
I need to print the entire line from file2 where file1 column matches.

File1
ABC
123
678

File2
QWE,ABC
DFR,123
D34,678
PP3,888

Thanks,
Prashant
# 6  
Old 07-18-2009
With awk (nawk in Solaris), we can take advantage of NR and FNR (see man page) to store the mapping for file1 and check on the existence in the mapping for file2
Code:
awk -F"," '
NR==FNR {
    f1[$1]=1
    next
}
{
    if ($2 in f1) { 
        print $1
    }
}' file1 file2

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Script to search log file for last 15 mins data

Hi All, I have an issue which I'm trying to understand a way of doing, I have several nodes which contain syslog events which I want to force trigger an email initially (eventually leading to another method of alerting but to start with an email). Basically the syslog file will have hours worth... (6 Replies)
Discussion started by: mutley2202
6 Replies

2. Shell Programming and Scripting

How to input data from a file into a web browser search engine?

Hi, I have a list of strings in a file on my terminal. I would like to input each line into a search engine and press the search button within a web browser. How can this be done? I am using firefox Thanks (1 Reply)
Discussion started by: cyberfrog
1 Replies

3. Shell Programming and Scripting

Want to read data from a file name.txt and search it in another file and then matching...

Hi Frnds... I have an input file name.txt and another file named as source.. name.txt is having only one column and source is having around 25 columns...i need to read from name.txt line by line and search it in source file and then save the result in results file.. I have a rough idea about the... (15 Replies)
Discussion started by: ektubbe
15 Replies

4. UNIX for Advanced & Expert Users

search data in a file

there is file abc.txt. I have to find out whether the below values is present in a file or not. 20110101 20110415 20110611 if present, add 1 to the date and store in a variable. (2 Replies)
Discussion started by: md.kashif20
2 Replies

5. Shell Programming and Scripting

search data from text file in different folders

I am fairly new to unix scripting, the problem is i can understand the unix script. but i fail to write. I do not know where to start and how to end. I am sure this forum will help to achive my dream scriptings in unix. Thanks in adv for your help. Here I need.. I have list of columns in one... (2 Replies)
Discussion started by: dsnrhdy
2 Replies

6. Shell Programming and Scripting

Search for a specific data in a file based on a date range

Hi, Currently I am working on a script to automate the process of converting the log file from binary into text format. To achieve this, partly I am depending on my application’s utility for this conversion and the rest I am relying on shell commands to search for directory, locate the file and... (5 Replies)
Discussion started by: svajhala
5 Replies

7. Shell Programming and Scripting

search and replace with data from another file

Can someone help me in solving the problem below. I have the following two files template_file ------------ ...other data.. ...other data.. FILE_NAME= ...other data.. ...other data.. list_file ---------- <file_name1> <file_name2> <file_name3> I need to produce another output... (3 Replies)
Discussion started by: paruthiveeran
3 Replies

8. UNIX for Dummies Questions & Answers

awk and grep to search a data file

Hi everyone, I cannot figure out how I can do a search in a file that has Names, Surnames, Addresses and telephone number of a number of people. Here is an example of the data file Daisy:Hunter:490 London Road:07313196347 Richard:Murphy:983 Main Road:07002625997 Isobel:Magnusson:133 London... (1 Reply)
Discussion started by: philipisaia
1 Replies

9. Shell Programming and Scripting

Extracting data between tags based on search string from unix file

Input file is on Linux box and the input file has data in just one line with 1699741696 characters. Sample Input: <xxx><document coll="uspatfull" version="0"><CMSdoc>xxxantivirus</CMSdoc><tag1>1</tag1></document><document coll="uspatfull"... (5 Replies)
Discussion started by: gaya
5 Replies

10. UNIX for Dummies Questions & Answers

search and grab data from a huge file

folks, In my working directory, there a multiple large files which only contain one line in the file. The line is too long to use "grep", so any help? For example, if I want to find if these files contain a string like "93849", what command I should use? Also, there is oder_id number... (1 Reply)
Discussion started by: ting123
1 Replies
Login or Register to Ask a Question