Problem to match a path in a file and put it into a variable


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Problem to match a path in a file and put it into a variable
# 1  
Old 12-08-2016
Problem to match a path in a file and put it into a variable

Hello,Smilie

I made a script which do a backup on remote servers with a rsync command. I have a config.cfg with the IPs and the paths where it will copy the directory. The problem is that it doesn't match the paths, So, here my script and its output with the debug :
Code:
#!/bin/bash
# PATHS
HOME_BACKUP="/home/backup"
HOME_SCRIPT="/home/scripts/test/backup_server"
# DATE
DATE_Ymd=$(date +%Y-%m-%d)
# DEBUG
set -x

# SCRIPT
for IP in `egrep -o '([0-9]{1,3}\.){3}[0-9]{1,3}' config_ip.cfg`
do
        ssh "$IP" 'hostname' > hostname.txt
        HOSTNAME=$(cat hostname.txt)
                cd "$HOME_BACKUP"
                mkdir -p "$HOSTNAME"
                cd "$HOSTNAME"
                mkdir "$DATE_Ymd"

        TARGET=$(grep "$IP" config_ip.cfg |awk -F "PATH: " '{print $2}')
        rsync -azvtP rsync_user@"$IP":"$TARGET" "$HOME_BACKUP"/"$HOSTNAME"/"$DATE_Ymd"
                rm -rf "$DATE_Ymd"
                cd "$HOME_SCRIPT"
                rm "$HOME_SCRIPT"/hostname.txt

done

Config.cfg :
Code:
#########################################
# Enter the remote servers IP as follow:#
# IP CLIENT: XXX.XXX.XXX.XXX            #
# Source: backup_script.sh              #
#########################################

IP CLIENT: 192.168.1.93   -   PATH: /var/www
IP CLIENT: 192.168.1.30   -   PATH: /var/www

Output :
Code:
rsync_user@SERVER:/home/scripts/test/backup_server$ ./backup_script.sh
++ egrep -o '([0-9]{1,3}\.){3}[0-9]{1,3}' config_ip.cfg
+ for IP in '`egrep -o '\''([0-9]{1,3}\.){3}[0-9]{1,3}'\'' config_ip.cfg`'
+ ssh 192.168.1.93 hostname
++ cat hostname.txt
+ HOSTNAME=vm1
+ cd /home/backup
+ mkdir -p vm1
+ cd vm1
+ mkdir 2016-12-07
++ awk -F 'PATH: ' '{print $2}'
++ grep 192.168.1.93 config_ip.cfg
grep: config_ip.cfg: No such file or directory
+ TARGET=
+ rsync -azvtP rsync_user@192.168.1.93: /home/backup/vm1/2016-12-07

receiving incremental file list
sent 161 bytes  received 3,109 bytes  934.29 bytes/sec
total size is 4,928  speedup is 1.51

+ rm -rf 2016-12-07
+ cd /home/scripts/test/backup_server
+ rm /home/scripts/test/backup_server/hostname.txt

...

If anyone have an idea ? Smilie I hope have been understandable

Last edited by Arnaudh78; 12-08-2016 at 06:44 PM..
# 2  
Old 12-08-2016
Hi,

You do cd twice so your directory is not same as when your first line of
Code:
 for IP in `egrep -o '([0-9]{1,3}\.){3}[0-9]{1,3}' config_ip.cfg`

Hence this grep says no such file or directory :
Code:
 $(grep "$IP" config_ip.cfg |awk -F "PATH: " '{print $2}')

Modify the code to cd the directory where you have config_ip.cfg

Minor comment:
Code:
TARGET=$(grep "$IP" config_ip.cfg |awk -F "PATH: " '{print $2}')

can be written as
Code:
TARGET=$(awk -F "PATH: " -vx=$IP ' $0 ~ x {print $2}' config_ip.cfg)


Last edited by greet_sed; 12-08-2016 at 06:51 PM.. Reason: Add text
This User Gave Thanks to greet_sed For This Post:
# 3  
Old 12-08-2016
Quote:
Originally Posted by greet_sed
Hi,

You do cd twice so your directory is not same as when your first line of
Code:
 for IP in `egrep -o '([0-9]{1,3}\.){3}[0-9]{1,3}' config_ip.cfg`

Hence this grep says no such file or directory :
Code:
 $(grep "$IP" config_ip.cfg |awk -F "PATH: " '{print $2}')

Modify the code to cd the directory where you have config_ip.cfg
Thank you very much ! I created a variable for the path and I modified, your awk is a very good way SmilieSmilie :
Code:
TARGET=$(awk -F "PATH: " -vx=$IP ' $0 ~ x {print $2}' "$HOME_SCRIPT")

---------- Post updated at 06:16 PM ---------- Previous update was at 06:15 PM ----------

can you describe the command ? Without asking too much

Last edited by Arnaudh78; 12-08-2016 at 07:11 PM..
# 4  
Old 12-08-2016
Quote:
Originally Posted by Arnaudh78
Thank you very much ! I created a variable for the path and I modified, your awk is a very good way SmilieSmilie :
Code:
TARGET=$(awk -F "PATH: " -vx=$IP ' $0 ~ x {print $2}' "$HOME_SCRIPT")

---------- Post updated at 06:16 PM ---------- Previous update was at 06:15 PM ----------
can you describe the command ? Without asking too much
Hello Arnaush78,

Could you please go through the following and let me know if this helps you.
Code:
TARGET=$(awk -F "PATH: " -vx=$IP ' $0 ~ x {print $2}' config_ip.cfg)
awk -F "PATH: "   #### -F in awk defines the field seprator whuch we want to makein any Input_file, so here we are mentioning it should be string "PATH: "
-vx=$IP           #### -v in awk is used for initializing a vatiable, so here we are creating a variable named named x whose value is equal to value f shell variable named $IP. Because in awk we can't mention variable's
                       value like shell so like this we could assign any shell's variable value to awk's variabe to make use of it in awk program.
' $0 ~ x          #### Mentioning a condition here if $0(current line) equals to value of variable named x(which we defines with -v explained as above), if this condition is TRUE then execute the following statements.
{print $2}'       #### printing the field 2nd of the current line in case above condition is TRUE.
config_ip.cfg     #### Mentioning the Input_file here which awk has to read and process whose name is config_ip.cfg here.

Thanks,
R. Singh
These 2 Users Gave Thanks to RavinderSingh13 For This Post:
# 5  
Old 12-09-2016
Looking at the script in post#1 I feel inclined to comment on a few points:

- Don't cd to and fro ( as already pointed out by greet_sed) but use absolut paths.
- Don't use temp files if not necessary.
- Don't use "command substitution" excessively - use available shell tools where appropriate, avoiding surplus process creation.
- Looks like you're removing the freshly created backup with rm -rf "$DATE_Ymd". Why?

Look into this little proposal; you might want to consider this or that hint in it:
Code:
while read V1  X  IP  X X  TARGET  X
  do    [ ! "$V1" = "IP" ] && continue
#       test the IP if necessary
        echo $IP, $TARGET
#       further processing, like
        HOSTNAME=$(ssh "$IP" 'hostname')
        mkdir -p "$HOME_BACKUP"/"$HOSTNAME"/"$DATE_Ymd"

        rsync -azvtP rsync_user@"$IP":"$TARGET" "$HOME_BACKUP"/"$HOSTNAME"/"$DATE_Ymd"

  done < cfg_file

This User Gave Thanks to RudiC For This Post:
# 6  
Old 12-09-2016
Thanks for your all replies !SmilieSmilie

Last edited by Arnaudh78; 12-09-2016 at 12:28 PM..
# 7  
Old 12-09-2016
You have DOS <carriage return> line terminators (\r, 0x0D, ^M) in your config file. Use tr -d $'\r' <file to remove them upfront, or do it in the awk when assigning TARGET.


It's difficult to believe that it should work flawlessly from the command line with the same config file.
This User Gave Thanks to RudiC For This Post:
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Problem setting path to external hard drive as a variable

Hello all, I am EXTREMELY new to using bash and I have a bit of a problem: I'm trying to set up a shell script that can 1.) take one of several giant files off of an external hard drive 2.) use it as input for scripts on my laptop's hard drive ... (3 Replies)
Discussion started by: machine_spirit
3 Replies

2. Shell Programming and Scripting

I must use first sentence from a file to put into variable

i am having some bash script which must use first sentence of the file. For example i have file which content is: test 213 So I must use word test into my bash script, and put it into variable. I am using a one variable named value value=$(</home/rusher/test.txt) so instead using test.txt... (1 Reply)
Discussion started by: tomislav91
1 Replies

3. Shell Programming and Scripting

Pattern match a path anywhere in the line and replace it with new path

I want to pattern match only path part from below and replace them with new path string. LoadModule jk_module /fldrA/fldrBaf/fldrCaa/modules/mod_jk.so JkWorkersFile /fldrA/fldrBaf/fldrCaa/config/OHS/ohs1/workers.properties JkLogFile... (4 Replies)
Discussion started by: kchinnam
4 Replies

4. Shell Programming and Scripting

Read value of a file, remove first 2 chars and put this value in a variable

Hi i have need of read a file value with cat command and remove first 2character for example cat /sys/class/rtc/day 0x12 Remove char 12 And put this value in a variable is possible with a script thanks for help (6 Replies)
Discussion started by: enaud
6 Replies

5. Shell Programming and Scripting

How to put content of file into a variable?

For example, I have a simple text file note: this a note a simple note a very very simple notewhen I use this command, temp=$(cat "note.txt")then I echo temp, the result is in one line. echo $temp note: this a note a simple note a very very simple noteMy variable doesn't have newline. How... (7 Replies)
Discussion started by: 14th
7 Replies

6. Shell Programming and Scripting

put the contents of this file into a variable with multiple lines

I have a file that contains the following lines the brown quick fox jumped over the white laze dog 0123456789 I wanted to put the contents of this file into a variable so I used this code: VAR_LIST=`cat $2` where $2 is the file name passed as an argument to the script If I... (3 Replies)
Discussion started by: Nomaad
3 Replies

7. AIX

Put one ligne from a file a variable

Hi everybody I looking the put the result of a commane to a Variable i explain here is my command: FJTS_UK:root:common@ukaix3:/> cat sortie | grep "^"| awk '{ print $1}' 15 FJTS_UK:root:common@ukaix3:/> sortie is a texte file I want to put the result of commande in a... (1 Reply)
Discussion started by: kykyboss
1 Replies

8. Shell Programming and Scripting

Using a variable in File Path

How can i add a varying filename at the end of a constant file path? Eg: Variable containing file name is varfile (say varfile=newfile) File path is C:\Test\ I want to access newfile which can be done as C:/\Test/\newfile But instead of directly accesing the newfile i want to access it... (2 Replies)
Discussion started by: sandeep_hi
2 Replies

9. Shell Programming and Scripting

problem in getting the path of environment variable set in bashrc in my shell script

hi all i have joined new to the group. i have set an variable in my bashrc file. .bashrc PROGHOME=/home/braf/braf/prog export PROGHOME but while using it in my shell script its path is not taken and i had to explicitly give the export command to set the path. in my script... (8 Replies)
Discussion started by: krithika
8 Replies

10. Shell Programming and Scripting

PATH variable problem

root->echo $PATH /usr/ccs/bin:/opt/sfw/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin root->find / -name gcc /usr/local/bin/gcc /usr/local/doc/gcc root->which gcc no gcc in /usr/sbin /usr/bin Why cant it find gcc - the path looks okay doesnt it? I am running solaris 9 - and... (7 Replies)
Discussion started by: frustrated1
7 Replies
Login or Register to Ask a Question