AIX

Hello,

Our NIM system becomes unresponsive while working. Look like first time that the root disk was missing. I cannot use many commands like lspv, lspath, ... ssh and other services are not operating.

I have rebooted the machine, and I can boot it to AIX, but it's an older AIX version 6.1 and not the current one 7.1

I also see that the OS is booted from hdisk0, while the current OS is booted from hdisk9. But now hdisk9 belongs to altinst_rootvg and varied off.

Look like the boot volume on hdisk9 was corrupted and system turns to the old one on hdisk0 to boot from it.

How can we re-create the boot volume on hdisk9? I tried to boot the system to single user mode on hdisk0 (old OS version), and try the bosboot command on hdisk9, but not success. It says the hd5 is already available in use.

I tried to use synclvodm and it changed my LVs on altinst_rootvg to new ones. Not sure that it just only renames the LV name, I hope it not overwrite the data on the altinst_rootvg.

Code:

# synclvodm -v altinst_rootvg
synclvodm: Physical volume data updated.
0516-530 synclvodm: Logical volume name hd6 changed to pagelv00.
synclvodm: Logical volume pagelv00 updated.
0516-530 synclvodm: Logical volume name hd8 changed to loglv02.
0516-712 synclvodm: The chlv succeeded, however chfs must now be
        run on every filesystem which references the old log name hd8.
synclvodm: Logical volume loglv02 updated.
0516-530 synclvodm: Logical volume name hd4 changed to fslv01.
synclvodm: Logical volume fslv01 updated.
0516-530 synclvodm: Logical volume name hd2 changed to fslv02.
synclvodm: Logical volume fslv02 updated.
0516-530 synclvodm: Logical volume name hd9var changed to fslv03.
synclvodm: Logical volume fslv03 updated.
0516-530 synclvodm: Logical volume name hd3 changed to fslv04.
synclvodm: Logical volume fslv04 updated.
0516-530 synclvodm: Logical volume name hd1 changed to fslv05.
synclvodm: Logical volume fslv05 updated.
0516-530 synclvodm: Logical volume name hd10opt changed to fslv06.
synclvodm: Logical volume fslv06 updated.
0516-530 synclvodm: Logical volume name hd11admin changed to fslv07.
synclvodm: Logical volume fslv07 updated.
0516-530 synclvodm: Logical volume name loglv00 changed to loglv03.
0516-712 synclvodm: The chlv succeeded, however chfs must now be
        run on every filesystem which references the old log name loglv00.
synclvodm: Logical volume loglv03 updated.
0516-530 synclvodm: Logical volume name livedump changed to fslv08.
synclvodm: Logical volume fslv08 updated.
0516-530 synclvodm: Logical volume name sec_dumplv changed to lv00.
synclvodm: Logical volume lv00 updated.
0516-530 synclvodm: Logical volume name pri_dumpdev changed to lv01.
synclvodm: Logical volume lv01 updated.

Now I'm downloading the ISO file and can boot into maintenance mode. Can we do the followings:

The thing I'm not very clear is, what is altinst_rootvg? If I can create the boot volume and rename the LV successfully as above, I can boot from it?

With my research about altinst_rootvg, it's a technique to replicate the current OS, this is done by backup purpose. Still reading it for more info ...

Please send the link to the man page for this command you are talking about:

Code:

altinst_rootvg

Sorry, I cannot find an AIX reference for it.

Here is the AIX reference for alt_rootvg_op :

alt_rootvg_op Command

altinst_rootvg is the name to the alternate rootvg you can boot from, as backup don't make boot disks... So not sure what results you can expect from what you done so far.

Neo gave you the man pages to the commands you need, and the reason you do altinst_rootvg is when you decide to apply patches or upgrade of TL etc... you have an altinst_rootvg to reboot from if things go wrong, so I never heard of anyone trying to sync like you did as for that you create a mirror that you un-mirror after only as most are using virtual devices we have this new option...

ADDENDUM
Sorry for the late posting I went off to work and forgot to click send...

Thank Neo and vbe.

Because I vary on the altinst_rootvg and try to use bosboot to create the boot info again, but it complains that hd5 already exists. So I try to sync it but finally it renamed all the LVs.

I have downloaded the AIX iso 7.1 which SP is higher the problematic AIX box. Can the higher version of AIX can create the lower version boot partition?

IMHO it would use what it has so 7.1...
You do have a mksysb somewhere I hope? Thats would be the best deal to put all at the level of that image... then you could start all again... When you say it renamed all the LV are you talking of rootvg only ? because those have just numbers to what I remember so I dont see how you could see anything change, but if the others that is because of their definition you find in /etc/filesystems which is found on rootvg, in other words using a mksysb from a previous configuration may show trouble mounting disks if not the same or for unfound filesystem and will not mount what it doeesnt know of like new LVs etc...
The same is true for passwords, reason why I include root passwd somewhere with it, I did before that make mksysb removing root passwd but found it was a terrible threat as too many engineers around, one could do silly things by mistake...
I have learned maybe the hard way, but I did too because of an HACMP failure long ago, that when anything system has changed like adding removing disks, new net or HBA cards etc... to update a text file YOU manage with vi and nothing else, somewhere you can always find if the system can boot (single user or maintenance level) where you can see what you know of and compare in order to get things right and not drown into more trouble such as the system seeing "new disks" will quite surely scan and find LVs and so ...
AIX is like HP-UX very robust, both have their pros and cons, but both have first class system administration tools which explains they are still on the market, one strength is that very very rarely you need to re-install after a serious issue on the hundred I had to administrate I never had once to reinstall a box except and AIX 4.3 as I never found out what a previous sysadm did to wreck the box , I had to format the root/boot disk twice to get it looking sane again but we are talking pre Y2K ...

Hi all,

I have used the following methods to restore the server from its mksysb

1. I create virtual DVD on VIOS and map it to the LPAR
2. I create DVD iso from mksysb, transfer it to VIOS and load it.
3. Booting LPAR to the DVD, and perform the restore.

I think, first an explanation what this "altinst_rootvg" is is in order:

When you are updating AIX there is always a non-zero chance of something going wrong so that you may want to roll back. With AIX 5.2 (IIRC - not sure about that) IBM introduced the "alternate disk installation". First, here is the principle:

- you start with a mirrored rootvg, where you have two disks. Each holding one mirror.
- you break up the mirror so that you have two identical copies
- then you update one with the new version while retaining the other as it is
- if everything goes right, the second (not updated) mirror is remirrored from the updated one so that you have a mirrored rootvg again (new version)
- if something goes wrong, reboot from the not-updated original, remirror the updated one from this so that you have a mirrored rootvg again (old version)

OK, with this in mind: notice that you first need to remove any installed emergency fixes before you update. Second, you do NOT do what i described above by hand! You use the adequate commands: alt_disk_install, alt_disk_copy, etc.. Read up on them before you attempt any update using them! There are options to alt_disk_install to create/remove the rootvg copies. Use only these, do NOT doctor with what the commands have created. Here is a link with the procedure.

Also notice that your NIM-server should ALWAYS be higher than or at least at the same level as your highest system. A NIM server with AIX 6.1 can only serve clients up to 6.1 and todays this means it is almost useless (AIX 6.1 is out of support in a few weeks). Your NIM-server should right now be at AIX 7.2 latest TL, even if the rest of your environment is 7.1 (which, btw., i think is a good idea). You can always serve clients below the NIMs level but never above.

I hope this helps.

bakunin