Comparing multiple files


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Comparing multiple files
# 1  
Old 10-17-2017
Hammer & Screwdriver Comparing multiple files

I want to develop one unix script that will first match the multiple files on one server say A with multiple files on another server say B and copy those to server A. After that need to compare the contents of these 2 set of multiple files on different location on same server and generate the mismatch report.


Any help one this would be more helpful. Thanks.
# 2  
Old 10-17-2017
Have you considered using rsync? It can probably do what you want with a lot less fuss.



Kind regards,
Robin
# 3  
Old 10-17-2017
Lightbulb

No, this is just for copying files from another server to local server. Secondly, it will ask for password everytime. I dont want this.
# 4  
Old 10-17-2017
So that is that not a good job for rsync? How do you propose to authenticate with the remote server anyway?

I might be missing the point, but if you copy the files from ServerB to ServerA then they should be the same. What is the report to highlight?

Please can you clarify your requirement a little more.


Thanks, in advance,
Robin
# 5  
Old 10-17-2017
Quote:
Originally Posted by Charnjeet Singh
No, this is just for copying files from another server to local server.
This is exactly what rsync was made for. You have gotten very good advice from my colleague rbatte1!

Quote:
Originally Posted by Charnjeet Singh
Secondly, it will ask for password everytime.
From the man page of rsync (which, i suggest, you should read instead of disregarding it):

Quote:
Some modules on the remote daemon may require authentication. If so,
you will receive a password prompt when you connect. You can avoid the
password prompt by setting the environment variable RSYNC_PASSWORD to
the password you want to use or using the --password-file option. This
may be useful when scripting rsync.
And, a few lines later:
Quote:
You may also establish a daemon connection using a program as a proxy
by setting the environment variable RSYNC_CONNECT_PROG to the commands
you wish to run in place of making a direct socket connection. The
string may contain the escape "%H" to represent the hostname specified
in the rsync command (so use "%%" if you need a single "%" in your
string). For example:

export RSYNC_CONNECT_PROG='ssh proxyhost nc %H 873'
rsync -av targethost1::module/src/ /dest/
rsync -av rsync:://targethost2/module/src/ /dest/

The command specified above uses ssh to run nc (netcat) on a proxyhost,
which forwards all data to port 873 (the rsync daemon) on the targeth-
ost (%H).
Notice that ssh may use exchanged keys to allow passwordless connections. Find details about this in the man page of ssh.

I hope this helps.

bakunin
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Comparing multiple columns using awk

Hello All; I have two files with below conditions: 1. Entries in file A is missing in file B (primary is field 1) 2. Entries in file B is missing in file A (primary is field 1) 3. Field 1 is present in both files but Field 2 is different. Example Content: File A ... (4 Replies)
Discussion started by: mystition
4 Replies

2. Shell Programming and Scripting

Comparing multiple network files (edge lists)

I want to compare 4 edge-lists to basically see if an edge is present in all 4 networks. The issue is that an edge A-B in one file can be present as B-A in another file. Example: Input 1: net1.txt A B 0.1 C D 0.65 D E 0.9 E A 0.7 Input 2: net2.txt A Z 0.1 C D 0.65 E D 0.9 E A... (1 Reply)
Discussion started by: Sanchari
1 Replies

3. UNIX for Advanced & Expert Users

Need help in comparing multiple columns from two files.

Hi all, I have two files as below. I need to compare field 2 of file 1 against field 1 of file 2 and field 5 of file 1 against filed 2 of file 2. If both matches , then create a result file 1 with first file data and if not matches , then create file with first fie data. Please help me in... (12 Replies)
Discussion started by: sivarajb
12 Replies

4. Shell Programming and Scripting

awk arrays comparing multiple columns across two files.

Hi, I'm trying to use awk arrays to compare values across two files based on multiple columns. I've attempted to load file 2 into an array and compare with values in file 1, but success has been absent. If anyone has any suggestions (and I'm not even sure if my script so far is on the right lines)... (4 Replies)
Discussion started by: hubleo
4 Replies

5. UNIX for Dummies Questions & Answers

Comparing multiple fields from 2 files uing awk

Hi I have 2 files as below File 1 Chr Start End chr1 120 130 chr1 140 150 chr2 130 140 File2 Chr Start End Value chr1 121 128 ABC chr1 144 149 XYZ chr2 120 129 PQR I would like to compare these files using awk; specifically if column 1 of file1 is equal to column 1 of file2... (7 Replies)
Discussion started by: sshetty
7 Replies

6. Shell Programming and Scripting

Comparing multiple variable in if statement

Hi there this script is an atempt to define which instances of Jboss relate to its PID by the date and timestamp I am using calc to test with. On our system the only way you can tell which instance relates to a particular instance is by looking at the start up time and date in a log. The... (9 Replies)
Discussion started by: nathan.harris
9 Replies

7. Shell Programming and Scripting

comparing multiple variables by 'if then'

Hi, I am a noob at shell scripting. basically I am trying to compare row counts from 8 tables in different databases. I have managed to get the row counts using awk from the spool files for both databases. now I have 16 variables with me for database 1 : $A $B $C $D $E $F $G... (3 Replies)
Discussion started by: smallville
3 Replies

8. Shell Programming and Scripting

Comparing multiple variables

Hi! I've come up with a ksh-script that produces one or more lists of hosts. At the and of the script, I would like to print only those hosts that exists in all the lists. Ex. HOSTS="host1 host2 host3 host11" HOSTS="host1 host2 host4" HOSTS="host2 host11" HOSTS="host2 host5 host6 host7... (1 Reply)
Discussion started by: Bugenhagen
1 Replies

9. Shell Programming and Scripting

comparing multiple files

hi, quick question i have one file which join one file with reference one Looks like this: KB0000 KB207418 KB0001 KB244904 KB0002 KB215027 KB0003 KB215027 KB0004 KB215027 KB0005 KB204320 KB0006 KB207074 KB0007 KB215204 KB0008 KB223809 KB0009 KB236640 KB0010 KB244506 ....... (2 Replies)
Discussion started by: karla
2 Replies

10. Shell Programming and Scripting

comparing multiple files in multiple subfolders

Hello, I am having a bit of hard time to get my head around this one. I really hope someone is out there to help me out! Background of my code: I am doing some automation where I am verifying multiple files in multiple sub folders and if they are all identical, I would echo a line with my test... (0 Replies)
Discussion started by: Riz
0 Replies
Login or Register to Ask a Question