Sponsored Content
Full Discussion: Comparing multiple files
Top Forums Shell Programming and Scripting Comparing multiple files Post 303005383 by bakunin on Tuesday 17th of October 2017 01:55:49 PM
Old 10-17-2017
Quote:
Originally Posted by Charnjeet Singh
No, this is just for copying files from another server to local server.
This is exactly what rsync was made for. You have gotten very good advice from my colleague rbatte1!

Quote:
Originally Posted by Charnjeet Singh
Secondly, it will ask for password everytime.
From the man page of rsync (which, i suggest, you should read instead of disregarding it):

Quote:
Some modules on the remote daemon may require authentication. If so,
you will receive a password prompt when you connect. You can avoid the
password prompt by setting the environment variable RSYNC_PASSWORD to
the password you want to use or using the --password-file option. This
may be useful when scripting rsync.
And, a few lines later:
Quote:
You may also establish a daemon connection using a program as a proxy
by setting the environment variable RSYNC_CONNECT_PROG to the commands
you wish to run in place of making a direct socket connection. The
string may contain the escape "%H" to represent the hostname specified
in the rsync command (so use "%%" if you need a single "%" in your
string). For example:

export RSYNC_CONNECT_PROG='ssh proxyhost nc %H 873'
rsync -av targethost1::module/src/ /dest/
rsync -av rsync:://targethost2/module/src/ /dest/

The command specified above uses ssh to run nc (netcat) on a proxyhost,
which forwards all data to port 873 (the rsync daemon) on the targeth-
ost (%H).
Notice that ssh may use exchanged keys to allow passwordless connections. Find details about this in the man page of ssh.

I hope this helps.

bakunin
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

comparing multiple files in multiple subfolders

Hello, I am having a bit of hard time to get my head around this one. I really hope someone is out there to help me out! Background of my code: I am doing some automation where I am verifying multiple files in multiple sub folders and if they are all identical, I would echo a line with my test... (0 Replies)
Discussion started by: Riz
0 Replies

2. Shell Programming and Scripting

comparing multiple files

hi, quick question i have one file which join one file with reference one Looks like this: KB0000 KB207418 KB0001 KB244904 KB0002 KB215027 KB0003 KB215027 KB0004 KB215027 KB0005 KB204320 KB0006 KB207074 KB0007 KB215204 KB0008 KB223809 KB0009 KB236640 KB0010 KB244506 ....... (2 Replies)
Discussion started by: karla
2 Replies

3. Shell Programming and Scripting

Comparing multiple variables

Hi! I've come up with a ksh-script that produces one or more lists of hosts. At the and of the script, I would like to print only those hosts that exists in all the lists. Ex. HOSTS="host1 host2 host3 host11" HOSTS="host1 host2 host4" HOSTS="host2 host11" HOSTS="host2 host5 host6 host7... (1 Reply)
Discussion started by: Bugenhagen
1 Replies

4. Shell Programming and Scripting

comparing multiple variables by 'if then'

Hi, I am a noob at shell scripting. basically I am trying to compare row counts from 8 tables in different databases. I have managed to get the row counts using awk from the spool files for both databases. now I have 16 variables with me for database 1 : $A $B $C $D $E $F $G... (3 Replies)
Discussion started by: smallville
3 Replies

5. Shell Programming and Scripting

Comparing multiple variable in if statement

Hi there this script is an atempt to define which instances of Jboss relate to its PID by the date and timestamp I am using calc to test with. On our system the only way you can tell which instance relates to a particular instance is by looking at the start up time and date in a log. The... (9 Replies)
Discussion started by: nathan.harris
9 Replies

6. UNIX for Dummies Questions & Answers

Comparing multiple fields from 2 files uing awk

Hi I have 2 files as below File 1 Chr Start End chr1 120 130 chr1 140 150 chr2 130 140 File2 Chr Start End Value chr1 121 128 ABC chr1 144 149 XYZ chr2 120 129 PQR I would like to compare these files using awk; specifically if column 1 of file1 is equal to column 1 of file2... (7 Replies)
Discussion started by: sshetty
7 Replies

7. Shell Programming and Scripting

awk arrays comparing multiple columns across two files.

Hi, I'm trying to use awk arrays to compare values across two files based on multiple columns. I've attempted to load file 2 into an array and compare with values in file 1, but success has been absent. If anyone has any suggestions (and I'm not even sure if my script so far is on the right lines)... (4 Replies)
Discussion started by: hubleo
4 Replies

8. UNIX for Advanced & Expert Users

Need help in comparing multiple columns from two files.

Hi all, I have two files as below. I need to compare field 2 of file 1 against field 1 of file 2 and field 5 of file 1 against filed 2 of file 2. If both matches , then create a result file 1 with first file data and if not matches , then create file with first fie data. Please help me in... (12 Replies)
Discussion started by: sivarajb
12 Replies

9. Shell Programming and Scripting

Comparing multiple network files (edge lists)

I want to compare 4 edge-lists to basically see if an edge is present in all 4 networks. The issue is that an edge A-B in one file can be present as B-A in another file. Example: Input 1: net1.txt A B 0.1 C D 0.65 D E 0.9 E A 0.7 Input 2: net2.txt A Z 0.1 C D 0.65 E D 0.9 E A... (1 Reply)
Discussion started by: Sanchari
1 Replies

10. Shell Programming and Scripting

Comparing multiple columns using awk

Hello All; I have two files with below conditions: 1. Entries in file A is missing in file B (primary is field 1) 2. Entries in file B is missing in file A (primary is field 1) 3. Field 1 is present in both files but Field 2 is different. Example Content: File A ... (4 Replies)
Discussion started by: mystition
4 Replies
SHELL-QUOTE(1p) 					User Contributed Perl Documentation					   SHELL-QUOTE(1p)

NAME
shell-quote - quote arguments for safe use, unmodified in a shell command SYNOPSIS
shell-quote [switch]... arg... DESCRIPTION
shell-quote lets you pass arbitrary strings through the shell so that they won't be changed by the shell. This lets you process commands or files with embedded white space or shell globbing characters safely. Here are a few examples. EXAMPLES
ssh preserving args When running a remote command with ssh, ssh doesn't preserve the separate arguments it receives. It just joins them with spaces and passes them to "$SHELL -c". This doesn't work as intended: ssh host touch 'hi there' # fails It creates 2 files, hi and there. Instead, do this: cmd=`shell-quote touch 'hi there'` ssh host "$cmd" This gives you just 1 file, hi there. process find output It's not ordinarily possible to process an arbitrary list of files output by find with a shell script. Anything you put in $IFS to split up the output could legitimately be in a file's name. Here's how you can do it using shell-quote: eval set -- `find -type f -print0 | xargs -0 shell-quote --` debug shell scripts shell-quote is better than echo for debugging shell scripts. debug() { [ -z "$debug" ] || shell-quote "debug:" "$@" } With echo you can't tell the difference between "debug 'foo bar'" and "debug foo bar", but with shell-quote you can. save a command for later shell-quote can be used to build up a shell command to run later. Say you want the user to be able to give you switches for a command you're going to run. If you don't want the switches to be re-evaluated by the shell (which is usually a good idea, else there are things the user can't pass through), you can do something like this: user_switches= while [ $# != 0 ] do case x$1 in x--pass-through) [ $# -gt 1 ] || die "need an argument for $1" user_switches="$user_switches "`shell-quote -- "$2"` shift;; # process other switches esac shift done # later eval "shell-quote some-command $user_switches my args" OPTIONS
--debug Turn debugging on. --help Show the usage message and die. --version Show the version number and exit. AVAILABILITY
The code is licensed under the GNU GPL. Check http://www.argon.org/~roderick/ or CPAN for updated versions. AUTHOR
Roderick Schertler <roderick@argon.org> perl v5.8.4 2005-05-03 SHELL-QUOTE(1p)
All times are GMT -4. The time now is 09:27 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy