Script to automate recovery process


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Script to automate recovery process
# 1  
Old 05-19-2015
Script to automate recovery process

Hello All!
First post...

I am working on a script that is used to recover a crashed drive from an rsync backup.

I'm down to the place where I need to create all of the directories in /mnt where I will then mount each of the volumes and begin the restore process to each volume...

I have scratched out my basic thoughts about what I need to happen, but am at a loss as to HOW to do it.

Any help would be greatly appreciated!!

Roger
***************************************************

/mnt/USBDrive/lvminfo/fstab_info was saved during the backup process...

Code:
vgname=$(grep ^/dev /mnt/USBDrive/lvminfo/fstab_info | cut -d'/' -f4 | cut -d'-' -f1 | sed '2,99d')

This results in vgname=fedora

Code:
read f1 f2 f3 f4 f5 f6 f7 f8 f9 f10 f11 f12 <<< $(grep $vgname /mnt/USBDrive/lvminfo/fstab_info | cut -d'-' -f2 | cut -d'/' -f1 | cut -d" " -f1 | sort -d -t" " | awk '{printf("%s%s",$0,NR%9?" ":"\n")}')

This results in a sorted string in which each word is assigned to a variable:

Code:
 echo $f1 $f2 $f3 $f4 $f5 $f6 $f7 $f8 $f9
backup home opt root swap tmp usr usr_local var

So, from that I need:

Code:
for ($f1-$f9)
    mkdir /mnt/$vgname/($f1-$f9)
    mount /dev/mapper/$vgname-($f1-$f9)
    increment to next varname ($f1-$f9)
repeat until all filled variables have created required directories and mounted lvms
exit

Thank you again for any assistance!! Smilie

---------- Post updated at 03:50 PM ---------- Previous update was at 02:40 PM ----------

For those interested, this is the script so far...
Any elegance suggestions are greatly appreciated! Smilie
Code:
#!/bin/bash
#
# This scipt is to be run from a LiveCD environment in a root terminal
#
clear
echo
echo
echo "Mounting USB Hard Drive to mount point /mnt/USBDrive..."
# We'll create a mount point and mount the main directory structure of our backup drive to this point.
mkdir -p /mnt/USBDrive
mount -o defaults /dev/sdb2 /mnt/USBDrive
echo "Done!"
echo 
# We are going to need the physical volume name and UUID along with the Volume Group name later in this process
# We also want to determine the starting and ending sector (along with the partition number) of our luks partition
echo "Setting a few variables for later..."
pvname=$(grep Name /mnt/USBDrive/lvminfo/pvparse | cut -d'/' -f4)
pvuuid=$(grep UUID /mnt/USBDrive/lvminfo/pvparse | cut -b 25-62)
vgname=$(grep ^/dev /mnt/USBDrive/lvminfo/fstab_info | cut -d'/' -f4 | cut -d'-' -f1 | sed '2,99d')
startsec=$(grep ^/dev /mnt/USBDrive/info/fdisk_info | grep G | cut -d" " -f3)
endsect=$(grep ^/dev /mnt/USBDrive/info/fdisk_info | grep G | cut -d" " -f4)
partnum=$(grep $startsec /mnt/USBDrive/info/sgdisk_info | cut -b4)
# The following line initializes a series of variables from a string of alphabetically sorted lvnames created from fstab_info (with modification you can initialize as many variables as you desire -- just add more fX to the list) Here I know that I only have 9 lvm's so I chose to add only 9 f variables.
read f1 f2 f3 f4 f5 f6 f7 f8 f9 <<< $(grep $vgname /mnt/USBDrive/lvminfo/fstab_info | cut -d'-' -f2 | cut -d'/' -f1 | cut -d" " -f1 | sort -d -t" " | awk '{printf("%s%s",$0,NR%9?" ":"\n")}')
# Once we have gotten the volume group name from fstab we can create a mount point for the restore process 
mkdir -p /mnt/$vgname
echo
# We start the recovery process by writing our saved critical disk areas to the new drive
# This will give us a mountable drive with the partition table, EFI and boot partitions as they were on the original drive
echo "Restoring the first 2048 sectors of /dev/sda..."
dd if=/mnt/USBDrive/dd/partition-restore of=/dev/sdc bs=1 count=2047
echo
echo "Restoring the EFI partition as /dev/sda1..."
dd if=/mnt/USBDrive/dd/sda1EFI-restore of=/dev/sdc1 bs=8192
echo
echo "Restoring the Boot partition as /dev/sda2..."
dd if=/mnt/USBDrive/dd/sda2Boot-restore of=/dev/sdc2 bs=8192
echo
echo "Prepping to restore the luks partition, and VG/LVM structures..."
echo
# Just to ensure that there isn't any old luks data on our new drive that will present errors during the restortion of our saved luks partition header image, we need to wipe out any signs of any previous lvm info that might be there
echo "First we will ensure that no old crypt/lvm info remains on our recovery disk"
cryptsetup -v isLuks /dev/sdc$partnum
head -c 3145728 /dev/zero > /dev/sdc$partnum; sync
# Next, we will recreate the partition from scratch and format it to ext4
echo "Then we will create a new /dev/sda3 and restore our backup img"
sgdisk -d $partnum -n $partnum:$startsec:$endsec -t $partnum:8300 /dev/sdc
# We need to inform the kernel about the changes
partprobe
# And format the partition we just created
mkfs -t ext4 /dev/sdc$partnum
echo
echo "Restoring Cryptsetup header and key slots..."
cryptsetup -q luksHeaderRestore /dev/sdc$partnum --header-backup-file /mnt/USBDrive/lvminfo/cryptbackup.img
echo
echo "Opening crypt partition for writing..."
cryptsetup -vv open /dev/sdc$partnum --type luks $pvname
echo
echo "Restoring Physical Volume..."
pvcreate -f --uuid $pvuuid --restorefile /mnt/USBDrive/lvminfo/$vgname /dev/mapper/$pvname
echo
echo "Restoring & Checking Volume Group and Logical Volumes structures..."
vgcfgrestore $vgname
lvs -a -o +devices
vgchange -ay $vgname
pvck -vv
vgck -vv
mkfs -t ext4 /dev/mapper/$vgname-*
mkswap /dev/mapper/$vgname-swap
echo

---------- Post updated at 03:53 PM ---------- Previous update was at 03:50 PM ----------

All of the /dev/sdX's above are set to another external USB test drive -- wouldn't want to slam my /dev/sda! Smilie

Last edited by Scott; 05-19-2015 at 07:28 PM.. Reason: Code tags, please...
# 2  
Old 05-19-2015
Code:
for fs in $f{1..12}
do
    mkdir /mnt/$vgname/$fs
    mount /dev/mapper/${vgname}-${fs}
done

However, I would consider using an array instead of f1 .. f12 variables as you are not limited to fixed number of filesystems:

Code:
read -a farray <<<< $(grep $vgname ....)

for fs in ${farray[@]}
do
    mkdir /mnt/$vgname/$fs
    mount /dev/mapper/${vgname}-${fs}
done

And if you don't need access to these values again you avoid any array or variables all together simply like this:

Code:
for fs in $(grep $vgname ... )
do
    mkdir /mnt/$vgname/$fs
    ...
done


Last edited by Chubler_XL; 05-19-2015 at 06:03 PM..
This User Gave Thanks to Chubler_XL For This Post:
# 3  
Old 05-19-2015
Thank you very much for your reply.
I have been reading so much information in the last couple of days that when I got to doing a for loop, by brain just shut off! I feel embarrassed that it was such an "easy" piece of script!

The last piece of this will be to run rsync against the backups to the newly mounted VG/LVM's.

Thank you very much!!

Roger

---------- Post updated at 05:42 PM ---------- Previous update was at 05:19 PM ----------

Quote:
Originally Posted by Chubler_XL
Code:
read -a farray <<<< $(grep $vgname ....)

for fs in ${farray[@]}
do
    mkdir /mnt/$vgname/$fs
    mount /dev/mapper/${vgname}-${fs}
done

Sorry to bother you again, but I forgot that I need a list of exclusions when it comes to restoring the root...

Is there a way to exclude root in the first section and then do just root in the second part?
Also, can we "variable-ize" the rsync line in the second part???

Thank you very much in advance!!
Roger

Code:
echo "Creating Mount Points, mounting individual LVM's from Volume Group and restoring data..."
for fs in ${farray[@]}
do
    mkdir /mnt/$vgname/$fs
    mount /dev/mapper/${vgname}-${fs} /mnt/$vgname/$fs
    rsync -aHvu /mnt/USBDrive/Recover/$fs /mnt/$vgname/$fs
done

# Restoring root requires all of the other lvm's as excludes...
# rsync -aHvu --exclude={/backup/*,/home/*,/opt/*,/tmp/*,/usr/*,/usr/local/*,/var/*}

---------- Post updated at 08:13 PM ---------- Previous update was at 05:42 PM ----------

Ok. I may have come up with a workaround...

By not sorting the list prior to reading it into the array, root will always be ${farry[0]}. Therefore, I can start my mkdirs and mounts with ${farry[1]} and then come back and do ${farry[0]} at the end.

How do we create and increment a variable to read in everything from 1 through an empty offset and perform our actions using their values in the commands?

---------- Post updated at 08:25 PM ---------- Previous update was at 08:13 PM ----------

Answer:
Code:
for fs in ${farray[@]:1}
do
    mkdir /mnt/$vgname/$fs
    mount /dev/mapper/${vgname}-${fs} /mnt/$vgname/$fs
    rsync -aHvu /mnt/USBDrive/Recover/$fs /mnt/$vgname/$fs
done

After that we can come back with
Code:
    mkdir /mnt/$vgname/${farray[0]}
    mount /dev/mapper/${vgname}-${farray[0]} /mnt/$vgname/${farray[0]}
    rsync -aHvu --exclude={/backup/*,/home/*,/opt/*,/tmp/*,/usr/*,/usr/local/*,/var/*} /mnt/USBDrive/Recover/${farry[0]} /mnt/$vgname/${farry[0]}

Last step:
To "auto-variable-ize" the rsync exclusions for / restoration...

Last edited by RogerBaran; 05-21-2015 at 11:18 AM.. Reason: minor corrections to text
# 4  
Old 05-21-2015
UPDATE:
As it turns out, I am able to get the results I want by using this code:
Code:
for fs in ${farry[@]}
do
    mkdir -p /mnt/$vgname/$fs
    mount /dev/mapper/${vgname}-$fs /mnt/$vgname/$fs
    rsync -aHvu /mnt/USBDrive/Recover/$fs /mnt/$vgname/$fs
done
rsync -aHvu --ignore-existing /mnt/USBDrive/Recover/ /mnt/$vgname/

However, I discovered that since my drive geometry is different between the two drives (cylinder values are not the same) a simple dd of the first 2047 sectors does not effectively copy the MBR, GPT and Partition info correctly. So, I need to find a way to recreate the drive from scratch (using sgdisk?), format and copy over the EFI and Boot partition files.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Open Source

Help with writing Shell Script to automate process using multiple commands

Hello! Need help to write a Linux script that can be run from windows using command/Cygwin/any other way. I am new to scripting, actually i am trying to automate server health check like free disk space, memory along with few services status, if any services is not running then start services ,... (7 Replies)
Discussion started by: Sayed Ibrahim
7 Replies

2. UNIX for Dummies Questions & Answers

Automate sftp process using script

Hi, guys, I am trying to automate a sftp process using "expect" method (since the key authentication method is disabled in my company network, there is no helping it). In order to try, I type in the command manually: sftp @ > << EOF >cd >ls -l >EOF >Connecting to @servername password: ... (3 Replies)
Discussion started by: warmboy610
3 Replies

3. Shell Programming and Scripting

Need help in creating a shell script to automate svnstats process

Hi, I am trying to create a shell script to automate the following process of getting svn stats:- Step1:- cd to checkout location. Note that the checked code have multiple modules in respective folders Step2:- Execute this command inside each module:- svn log -v --xml >... (0 Replies)
Discussion started by: d8011
0 Replies

4. Shell Programming and Scripting

Bash script to Automate the Virtual Host creation process!!

Hi all, This is my sample code in /etc/httpd/conf.d/applications.conf file currently we are creating subdomain mannually for every new subdomain. I want to automate this process througs bash script , how its possible. <VirtualHost *:80> ServerName google.com ServerAlias google.com... (5 Replies)
Discussion started by: anishkumarv
5 Replies

5. UNIX for Dummies Questions & Answers

To automate a process

CAN ANYONE HELP TO SOLVE i wann write a script to automate a process .i.e, to search files in the FTP server and and if files are there and we hav to bring that files to our system. After copying the files in our system we have to upload the data in the tables. I have scripts to load the... (2 Replies)
Discussion started by: nani1984
2 Replies

6. UNIX for Advanced & Expert Users

TO Automate the FTP process

Plzz Some One Help in this matter I have scripts to load the data into tables and to copy files from ftp to our system. we use to run the scripts every day.... we hav the files in the FTP server and we hav to bring the files to our system and we hav to load the data into the tables. We... (0 Replies)
Discussion started by: nani1984
0 Replies

7. Shell Programming and Scripting

To automate a process

CAN ANYONE HELP TO SOLVE i wann write a script to automate a process .i.e, to search files in the FTP server and and if files are there and we hav to bring that files to our system. After copying the files in our system we have to upload the data in the tables. I have scripts to load the... (0 Replies)
Discussion started by: nani1984
0 Replies

8. Shell Programming and Scripting

Error in script to automate the daily monitoring process of UNIX server and it's proc

hi friends, I am trying to automate the daily monitoring process of UNIX server and it's processes. the script are below i executed the above script using ksh -x monitortest1.sh in root login . It shows error at some lines . 1. i logged in using root ,but it... (8 Replies)
Discussion started by: rdhaprakasam
8 Replies

9. Shell Programming and Scripting

Automate Log Monitoring Process

I am a new member of this forum and am also new to unix shell scripting. I joined the forum to seek for help to achieve my task as this forum helps people. here's what i do manually on daily basis 1)Loginto different unix box 2)Ftp the log files (morethan 50 each dir) to windows 3)use text pad... (3 Replies)
Discussion started by: sharugan
3 Replies

10. Shell Programming and Scripting

How to automate an FTP process?

Hello script experts, I am newbie to shell script. But I have to write a shell script (ASAP) where I need to ftp a file on daily basis to a remote server, and send an email with final status. I I should have a properties file with hostname, Userid, and pwd. And a shall script file should read... (1 Reply)
Discussion started by: ksak
1 Replies
Login or Register to Ask a Question