Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

How to I compare 100s of files?

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting How to I compare 100s of files?
# 1  
Old 03-14-2013
How to I compare 100s of files?
Dear All !

I would like to compare my database ref file with other files

my ref file looks like this

Code:
15-12-1996   21-2-2024 12 34,000 888 976  1
15-12-1994   21-2-2026 12 35,000 885 977  2
15-12-1993   21-2-2025 12 37,000 887 976  3
15-12-1995   21-2-2029 12 39,000 878 976  4
15-12-1999   21-2-2025 12 74,000 868 986  5

other file contains user deatils is like this
Code:
15-12-1996   21-2-2024 12 35,000 888 976
15-12-1996   21-2-2024 12 34,000 888 976
15-12-1996   21-2-2024 12 54,000 888 976
15-12-1996   21-2-2024 12 54,000 888 976
15-12-1996   21-2-2024 12 74,000 888 976
15-12-1996   21-2-2024 12 9,000 888  976
15-12-1996   21-2-2024 12 9,000 888  976
15-12-1996   21-2-2024 12 37,000 888 976

I need to write last column number 1 corresponding to 888 976 to user file
output expected is like this

Code:
15-12-1996   21-2-2024 12 35,000 888 976 1
15-12-1996   21-2-2024 12 34,000 888 976 1
15-12-1996   21-2-2024 12 54,000 888 976 1
15-12-1996   21-2-2024 12 54,000 888 976 1
15-12-1996   21-2-2024 12 74,000 888 976 1
15-12-1996   21-2-2024 12 9,000 888  976 1
15-12-1996   21-2-2024 12 9,000 888  976 1
15-12-1996   21-2-2024 12 37,000 888 976 1

Code:
15-12-1994   21-2-2026 12 35,000 885 977  2
15-12-1994   21-2-2026 12 45,000 885 977  2
15-12-1994   21-2-2026 12 55,000 885 977  2
15-12-1994   21-2-2026 12 75,000 885 977  2
15-12-1994   21-2-2026 12 95,000 885 977  2
15-12-1994   21-2-2026 12 55,000 885 977  2
15-12-1994   21-2-2026 12 75,000 885 977  2


what I mean to say is I want to compare 4th and 5th column record of user file with ref file
if user file's 4th and 5th columns 1st record(because other records are same as 1st record ) is avilable in ref file, I want to write number in 6th column of ref file corresponding to that record to user file
Last edited by Dona Clara; 03-14-2013 at 12:34 PM..
# 2  
Old 03-14-2013
You can 'sort' each on the key columns and then 'join' the files to find matches.
# 3  
Old 03-14-2013
OK assumption is that user files start with "user" (eg user001) and exist somewhere below starting directory. reffile is your reference file.

This script writes *.new files (eg user001.new) with updated info:

Code:
find . -name 'user*' -print | xargs awk '
    FNR==NR {R[$5,$6]=$7; next}
    FNR==1{close(F);C=R[$5,$6];F=FILENAME".new"}
    {print $0 OFS C > F}' reffile

# 4  
Old 03-15-2013
no not helping me my ref file is not in order..its creating new file but not writing number to userfile
# 5  
Old 03-15-2013
If you want good answers, please supply all of the information. Before someone gives you another answer you cannot use or understand.

1. Is there some reason why the ref file cannot be copied and sorted?
2. how often are records added to the ref file, or is it static?

We are here to help you, not to provide a coding service.
# 6  
Old 03-15-2013
No "Bring me another rock!" requirements analysis? Smilie Bring me a rock
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Ssh to 100s of hosts with password in a shell script or text file

I have about 500 hosts where I need to ssh by sending the password on the command line or in a text file in a clear text . However I am not able to download "sshpass" or other tools . Any other ways to pass the password in a script ? (3 Replies)
Discussion started by: gubbu
3 Replies

2. Shell Programming and Scripting

Compare files and share output from both files

hi all, Thanks to all for your great help... I have a scenario that I have two files (file1 & file2). I need to compare two files entire row by row and share the output if any discrepancies within two files. File1: DB1|TB1|C1,C3 DB2|TB2|C1,C2 DB3|TB3|C1,C2,C3,C4 File2: ... (2 Replies)
Discussion started by: Selva_2507
2 Replies

3. Shell Programming and Scripting

Compare multiple files, and extract items that are common to ALL files only

I have this code awk 'NR==FNR{a=$1;next} a' file1 file2 which does what I need it to do, but for only two files. I want to make it so that I can have multiple files (for example 30) and the code will return only the items that are in every single one of those files and ignore the ones... (7 Replies)
Discussion started by: castrojc
7 Replies

4. Shell Programming and Scripting

Compare two files, then overwrite first file with only that in both files

I want to compare two files, and search for items that are in both. Then override the first file with that containing only elements which were in both files. I imagine something with diff, but not sure. File 1 One Two Three Four Five File 2 One Three Four Six Eight (2 Replies)
Discussion started by: castrojc
2 Replies

5. Shell Programming and Scripting

Require compare command to compare 4 files

I have four files, I need to compare these files together. As such i know "sdiff and comm" commands but these commands compare 2 files together. If I use sdiff command then i have to compare each file with other which will increase the codes. Please suggest if you know some commands whcih can... (6 Replies)
Discussion started by: nehashine
6 Replies

6. Shell Programming and Scripting

Compare 2 folders to find several missing files among huge amounts of files.

Hi, all: I've got two folders, say, "folder1" and "folder2". Under each, there are thousands of files. It's quite obvious that there are some files missing in each. I just would like to find them. I believe this can be done by "diff" command. However, if I change the above question a... (1 Reply)
Discussion started by: jiapei100
1 Replies

7. Shell Programming and Scripting

How to compare 2 files & get only few columns based on a condition related to both files?

Hiiiii friends I have 2 files which contains huge data & few lines of it are as shown below File1: b.dat(which has 21 columns) SSR 1976 8 12 13 10 44.00 39.0700 70.7800 7.0 0 0.00 0 2.78 0.00 0.00 0 0.00 2.78 0 NULL ISC 1976 8 12 22 32 37.39 36.2942 70.7338... (6 Replies)
Discussion started by: reva
6 Replies

8. Shell Programming and Scripting

compare files in two directories and output changed files to third directory

I have searched about 30 threads, a load of Google pages and cannot find what I am looking for. I have some of the parts but not the whole. I cannot seem to get the puzzle fit together. I have three folders, two of which contain different versions of multiple files, dist/file1.php dist/file2.php... (4 Replies)
Discussion started by: bkeep
4 Replies

9. Shell Programming and Scripting

compare two files and to remove the matching lines on both the files

I have two files and need to compare the two files and to remove the matching lines from both the files (4 Replies)
Discussion started by: shellscripter
4 Replies

10. Shell Programming and Scripting

How to compare 2 files.

Hi, I want to compare 2 files as below file1: -------- yellow:sure blue:567 red:75843kjsad file2: -------- yellow:sujji blue:summi orange:girri red:akash white:alias Output: --------- (6 Replies)
Discussion started by: ashoka123
6 Replies
Login or Register to Ask a Question