Shell Programming and Scripting

I need to pull down a good bit of files for another support team for an upgrade project. I have a server.list with all of the server names.

I need to do two parts:

FIRST:
I have this example, but it does not list the server name in front of each line.

Code:

#! /bin/bash
for server in $(< server.list);
do
echo "********* $server *********"
ssh $server "sudo find /u01/ -name \"filename\""
done

I need to get an output file that will look similar to this:
SERVER1:/path/path/filename
SERVER2:/path/path/filename
SERVER3:/path/path/filename
SERVER3:/path/path/filename
so on (and I might have multi files on one server)...

SECOND:
Once I have the "new" list of all servers and files I need to be able to take that output and copy the files down to a remote server and rename the file to include the server name and full path.

EX: SERVER1/path/path/filename
Script will use the above to ssh to SERVER1 and copy the file from the full path then rename it to SERVER1_path_path_filename

This is needed so that the support team will know which file belongs to which server and the full path as one server could have multi files with the same name.

This solution uses sed to prepend the server name to filelist and scp to copy the files locally. Most servers that support ssh also support scp so it should work for you.

Code:

#!/bin/bash
 
for server in $(< server.list)
do
   ssh $server "find /u01/ -name \"filename\"" | sed "s/^/$server:/"
done > new.list
 
OLDIFS=$IFS
IFS=":"
while read server filepath
do
   scp $server:$filepath $server${filepath//\//_}
done < new.list
IFS=$OLDIFS

THANK YOU VERY MUCH Chubler_XL

I had to add " " around the $server:$filepath for some reason.

Code:

OLDIFS=$IFS
IFS=":"
while read server filepath
do
   scp "$server:$filepath" $server${filepath//\//_}
done < new.list
IFS=$OLDIFS

Without the " " I was getting scp SERVER /path/path/file (No such file or directory)

Once I updated it to "$server:$filepath" I would get scp SERVER:/path/path/file

Thanks again for your help

---------- Post updated at 03:19 PM ---------- Previous update was at 11:02 AM ----------

I was getting the following error on some of the servers because I was using sudo find:

sudo: sorry, you must have a tty to run sudo

I just added ssh -t and that appears to have fixed it

Very odd... When I tested this out on Jan 15th it worked just fine... I try to run it today and I get the following error:

scp: ambiguous target

When I check the cmd it appears to be adding a : to the end of the line / filename
I am getting the following:
/path/filename: No such file or directory

My current script is :

Code:

#!/bin/bash
set -x

OLDIFS=$IFS
IFS=":"
while read server filepath
do
  scp -t "$server:$filepath" $server${filepath//\//_}
done < new.list
IFS=$OLDIFS

Any ideas ??

You probably have a blank line in the new.list file try:

Code:

#!/bin/bash
set -x

OLDIFS=$IFS
IFS=":"
while read server filepath
do
  [ -n "$filepath" ] && scp -t "$server:$filepath" $server${filepath//\//_}
done < new.list
IFS=$OLDIFS

I double checked and saved the original new.list to new.list-backup. I removed all but 15 lines (no blank lines or spaces):

server1.domain.net:/path/path/path/filename
server2.domain.net:/path/path/path/filename
server3.domain.net:/path/path/path/filename
... on to server 15

I tried adding your update

Code:

[ -n "$filepath" ] &&

and I am still receiving the "scp: ambiguous target" for each line...

Like I said this worked fine on the Jan 15th. I did not update the script and tried to run it today to check again and am now getting the "scp: ambiguous target" error.

When I run the script with set -x I am getting the following:

Looks like your data file is in DOS format (perhaps you edited it with windows notepad or something) try $ dos2unix new.list to clean it up.