AIX break rootvg mirror from system down


 
Thread Tools Search this Thread
Operating Systems AIX AIX break rootvg mirror from system down
# 1  
Old 04-23-2015
AIX break rootvg mirror from system down

Hello,
aix 5.2, mirrored rootvg on hdisk0 and hdisk1. hdisk0 is dead. I can boot to cd, into sms, into maintenance mode. I can fsck all the various partitions on hdisk1 (the hd4 hd2 hd3, etc...) all is fine. But without the hdisk0 part of the mirror I cannot get the system to boot. ystem hangs on boot with a LED code of 517 (or is it 518?).

None of the lvm commands for dealing with mirrors are in the cd /usr.

I tried editing the image.data file to remove references to hdisk0, and changed all the copies from 2 to 1, fixed the PP entries to match, and hoped that would break the mirrors, but alas.

So for kicks I mounted the "real" usr and now the commands are available, but now they won't work because rootvg cannot be varyonvg'd.
Code:
# lsvg -l rootvg
0516-013 : The volume group cannot be varied on because
        there are no good copies of the descriptor area.

Kind of stuck. suggestions?

Moderator's Comments:
Mod Comment Use code tags, thanks.

Last edited by zaxxon; 04-24-2015 at 06:17 AM..
# 2  
Old 04-24-2015
It would suggest that when the server was first installed that someone used a single disk (that would be labelled hdisk0) and the when complete they ran a mirrorvg rootvg only. There are more steps required to make it a usable mirror, namely running bosboot and setting the bootlist

Within SMS, can you choose your boot disk and enquire what is on each disk? I can't remember how right now and all my servers are running, so I can't just bounce one. I think you need to ask it to scan all devices, and it might take a while depending on your configuration.

If it sees your disk has a bootable AIX, try selecting it. That would suggest that just the bootlist is incorrect.

If SMS doesn't think that your disk has, all may not be lost, but it might be a little difficult. When in single user off media (tape / CD / DVD / or NIM) maybe you could try running the bosboot command against the correct disk? You may need to get this from the serial number.

It still may not work, but nothing else springs to mind at the moment without restoring from mksysb which I presume you don't want to do.




Robin
# 3  
Old 04-24-2015
Thanks - I am able to see the (second) disk which is how I'm able to fsck the partitions.
Code:
SMS sees both disks:
 Select Device
 Device  Current  Device
 Number  Position  Name
 1.        1      SCSI 146814 MB Harddisk, part=2 (AIX 5.2.0)
                  ( loc=U787B.001.DNWD543-P1-T14-L8-L0 )
 2.        -      SCSI 146814 MB Harddisk, part=4 ()
                  ( loc=U787B.001.DNWD543-P1-T14-L8-L0 )

if I chose #1 and the get information on it:
It does say it is bootable:
Code:
 Device Information
    /pci@800000020000003/pci@2/pci1069,b166@1/scsi@0/sd@8,0:2
                   :  (Bootable)       
DEVICE             : SCSI 146814 MB Harddisk, part=2 (AIX 5.2.0)
    ( loc=U787B.001.DNWD543-P1-T14-L8-L0 )
NAME               : sd
DEVICE-TYPE        : block

So when I service mode boot that disk, I get:
518 Remote mount of the root (/) and /usr file systems failed during network boot.
Now the normal way to fix that, as I understand it, is to reboot to cd and follow a tech note, which basically says fsck the filesystems. Which I am able to do.

But that doesn't seem to allow the darn thing to boot. And I'm guessing that the problem is that the rootvg is mirrored and one of the disks is dead. Surely that is something that should be able to be worked around? Or am I just screwed and will need to do a re-install?

Last edited by rbatte1; 04-27-2015 at 08:35 AM.. Reason: Added CODE tags for output.
# 4  
Old 04-24-2015
That same technote:

IBM Repairing File Systems with fsck in AIX (LED 517 or 518) - United States

Says you are basically screwed...

Quote:
If you followed all of the preceding steps and the system still stops at an LED 517 or 518 during a reboot in Normal mode, you may want to consider reinstalling your system from a recent backup. Isolating the cause of the hang could be excessively time-consuming and may not be cost-effective in your operating environment. To isolate the possible cause of the hang, would require a debug boot of the system. Instructions for doing this are included in the document "Capturing Boot Debug", available at Support Search for IBM Power Servers. It is still possible, in the end, that isolation of the problem may indicate a restore or reinstall of AIX is necessary to correct it.
# 5  
Old 04-24-2015
That's what I was just coming to grips with. Ok - first time for everything. I've got a mksysb from a couple of weeks ago. I've burned it (volumes 1 of 3) to a DVD and am trying to do the (re)install. interestingly enough I happen to have a new disk which I've swapped into as a replacement for the bad disk.

ok so far. Boot to that cd which works nicely thank you very much.

Trying to do the restore to the new disk and it is circling around a
image.data has invalid file system data.
Can not continue.

I suppose I could try to extract the image.data and look at it, but even if I could, I doubt I'd be able to "put it back" into the mksysb.

I see two options. (1) try to restore to what I think is a good disk and hope for the best, or (2) do a fresh install, then restore from backups, etc. Kind like not to do that one, but I may not have a choice.
# 6  
Old 04-25-2015
I suspect you are in for a lot of late nights....
Restore a mksysb from a mirrored system to a single disk....hmmm
A mksysb that takes 3 DVD's...hmmm
Clean install and a preservation restore might be your best bet.
LVM redbook is you next step I think:
Code:
http://www.redbooks.ibm.com/abstracts/sg245433.html

And this is worth a read:
Code:
http://www-01.ibm.com/support/docview.wss?uid=isg3T1011782

HTH
# 7  
Old 04-27-2015
I've never managed to span DVD media for a mksysb. How big was your rootvg? Did you have stuff other than just the base OS? (and perhaps your backup software)

I have restored a mirrored rootvg to a single disk, but that was planned in advance and we adjust the backup process.



Robin
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Linux

LVM - break mirror taking out orignal copy

Hi Guys, I have a mirrored LVM volume, and wish to break the mirror leaving only copy1 (effectively removing copy 0). Here's the config: # lvs -a -o +devices lvdata vg01 mwi-ao 2.19T lvdata_mlog 100.00 lvdata_mimage_0(0),lvdata_mimage_1(0) vg01... (0 Replies)
Discussion started by: badoshi
0 Replies

2. AIX

How to restore rootvg archive after AIX 6.1 crash

Hello, Few days ago I created both mksysb and savevg archives of rootvg. How can I restore this rootvg now because the AIX crashed during some software tests. There is no way to start from hdisk0 because most of the system files are deleted (this includes libc.a). I tough it will be trivial... (6 Replies)
Discussion started by: +Yan
6 Replies

3. Solaris

Solaris Volume Manager: Break Mirror and use to make a 2nd machine

Hello: I have a machine built on 2 drives and mirrors have been created to a second set of 2 drives (4 total) on the same platform. SVM. Might anyone provide guidance to create a second machine by breaking the mirrored set, moving the two mirrors (2) to another machine (same platform type),... (1 Reply)
Discussion started by: 4dailyrunner
1 Replies

4. AIX

Erase AIX rootvg system disk

Hello all, I need help for erasing IBM P5 machine running AIX 5.3. I have 2 system disks (Hardware mirror) and i'm looking for a way like a LIVE CD bootable CD so I can boot with the cd and then format the system disks with a "secure erase" software. (7 Replies)
Discussion started by: kobi gabai
7 Replies

5. AIX

AIX Rootvg mirror and sysdumplv

Guys, In my AIX 6.1 box the rootvg was on hdisk2, I tried to migrated it to hdisk0 Added hisk0 to rootvg , mirrored rootvg and changed bootlist and and sucessfully rebooted from hdisk0 Now I tried to remove the hdisk2 from rootvg so breaked mirror -bash-3.00# unmirrorvg rootvg hdisk2... (3 Replies)
Discussion started by: kkeng808
3 Replies

6. AIX

AIX 5.2/5.3 - rootvg on SAN disk - pros and cons

We are considering a DR strategy of booting AIX 5.3 and 5.3 logical partitions from EMC Symmetrix SAN disks, so that we can replicate via SRDF to a recovery site. Has anyone tried configuring AIX 5.x systems to boot from SAN disk? If so, can you provide any information on the pros and cons of... (6 Replies)
Discussion started by: jjgarrot
6 Replies

7. AIX

mirror rootvg

Hello I have a question I have a box with Aix 5.3 with rootvg on mirror. I deleted a filesystem and I create a new one to install some software but when I type lsvg -l rootvg all my filesystems has mirror unless the new one. My question is I can mirror this fs only ??? or I have to... (9 Replies)
Discussion started by: lo-lp-kl
9 Replies

8. Solaris

break solaris mirror

Hi All, How do I break the solaris 8 mirror? And how do I make sure I will point the first hd mirror on reboot. If the first break mirror won't boot. How do I make the copy mirror to boot? Thanks in advance, itik (2 Replies)
Discussion started by: itik
2 Replies

9. Solaris

patch update and break mirror

Hi All, How do you know if you need to break the mirror on solaris 8 patch update? Here's the patch example, 117350-54. On the site, it's just a low type update and not a medium or critical. Is breaking a mirror a best practice only or really a requirement to some patch? Thanks, itik (1 Reply)
Discussion started by: itik
1 Replies

10. HP-UX

Break mirror so that it can be used later

Hello again, We need to install patches to HP-UX B.11.11 but would like to break the mirror it has (with out damaging it) so that in case of failure we can use this a meassure procedure. Any ideas on how to do this Thank you! (2 Replies)
Discussion started by: AQG
2 Replies
Login or Register to Ask a Question