AIX

Hello Everyone,

Hope you all are doing great!

As you can see by the title on top, we are in the process of migrating alot of our servers from Power5 (physical) to Power8 (Virtual). Now it's turn for servers with HACMP Cluster on it. Let me lay out the environment like:

OLD ENVIRONMENT:

The Primary and secondary nodes resides on a Power5 host. Both are physical servers. The rootvg is on internal disks and the data vg's are on SAN Attached storage. Both primary and secondary nodes are on AIX 7.1 TL3 SP4. For HACMP we have a Active-Passive configuration.

NEW ENVIRONMENT:

Everything is virtualized. We have a Dual VIO setup on the Power8s. The boot disk are coming through a Storage cluster on VIO (vSCSI) and Data disks from SAN (NPIV).
As part of the new environment we are also moving on to the new network i.e. different IP subnet. (old=172.XXX) (new=10.XXX)

The way we have been migrating our previous (non-HA/test/dev) servers is that we clone the mksysb on to the vscsi disk on the new lpar before the day of the cutover and configure the new network on the server. On the day of cutover we bring down apps/DB on the old server (p5) then umount FS, varyoff and export VGs and remove the SAN disks. The storage team unmaps the LUNs and maps them back on the NEW WWPNs on the Power8 (NPIV).

On the new server we configure and import the disks and VGs and mount all FS inclulding NFS. Make DNS changes with the new IP and bring down all apps and DBs.

Now once we bring HACMP into the picture things become a little more complicated, can some of the experts here give me sound advice on how can we make the process as smooth as possible as we are moving on to the new server hardware as well as new network.


Thanks in Advance!

1. make an HACMP snapshot before migration/mksysb.
2. I personally would do server and IP migration separately. Either first IP migration and cluster reconfiguration and then server migration or vice versa, but not all at the same time.
Many changes at once - too many points of failures and too much troubleshooting in case of a problem.

I thought about that, but here is the issue;

Both networks have restrictions. I cannot put old IP (172.X) on the new hardware as it connect to the new network and vice versa, that is what makes it so complicated.

In this case you will have to have a downtime of some sorts. Alas, there seems to be no way around that because you need to (newly) configure HACMP on your new system and this means some downtime when you make the transition. Fortunately this downtime can be minimised pretty much by good planning.

Furthermore you haven't said anything about involved OS and HACMP versions. I suppose you will have to update at least one of them (most probably both) too.

I'd investigate the following procedure (you might have to add some things, this is just a first idea):

- Take an mksyb from the running systems, create the appropriate LPARs on the new system and try to install it from the mksysb image without any data (just the rootvg).

- Now do all the updates (start with OS, then the HACMP software) on the new systems

- get a single LUN from your SAN and configure your new cluster with the same topology as the old one, just with one test-VG on the single LUN. Use this to test your zoning, (basic) cluster setup and other details.

- Finally, after preparing your new cluster with new addresses on the new hardware you need the downtime: get the SAN people to zone the old data-LUNs to your new hardware so that the disks are seen from your cluster nodes, create the new cluster config for the imported VGs and start. You might want to add the DNS names of the old cluster service IPs as aliases to the new addresses so that the transition becomes smoother for the clients.

I hope this helps.

bakunin

Thanks for the reply!!

AIX version running is 7.1 TL3 SP4 and HA version is 6.1. Both are supported on the Power8s.
As far as the outage goes, we do have a 2.5 hour window to re-configure the cluster.
I have already included the mksysb and restore part into my plan. Doing my research I found this on the IBM website:

To change a service IP label/address definition:

Do you think, if after I move everything on the new Power8 (OS and SAN). I can change the service and bootip addresses using the above method and then try to start the cluster? Do you think it'll work?

Current version for AIX 7.1 is TL3 SP6 and there is already 7.2 out there. I'd suggest at least the former because it fixes some problems with HACMP: on 7.1.3.4 one of the rsct-daemons is running wild and cluttering up the /var. You have either to install efixes (generally generating as many problems as it is solving) or update to the latest level.

Current version for HACMP is 7.1.3 and 6.1 is already EOL by September. Do yourself a favour and use this maintenance window to update to the as latest version as possible. DO NOT use any version below 7.1.3 if updating to 7.1! Many things (like the repo disks via NPIV with non-IBM storage) worked only in theory, not in practice. 7.1.3 is more or less stable. I have some 40-50 clusters running here and could go on for pages and pages about the workarounds and quick-fixes we had to use to get working clusters with the earlier versions of 7.1.

You do not need to: as i said, create your LPARs when your old cluster is still working, from teh mksysbs (plus necessary updates, see above), create a NEW 7.1-cluster and test that until you are ready to make the move. You can pre-create the complete cluster-configuration into a series of commands now, because FINALLY the clmgr-command really works and it is possible to do a cluster-config via commandline! This (not having to navigate all these SMITTY-menus all the time) is by far the biggest relief since i work with HACMP.


You said you needed to use new IP-addresses anyway, so don't bother. Create your new cluster with the new addresses and test thoroughly, then make the transition basically by moving the data disks (they are NPIV, no?) to the new LPARs.

It might work, but again: you don't need that. I can give you a complete procedure for setting up a (7.1-)-cluster and in fact it is 10 minutes work now, only a few commands. Far better and far easier than to navigate these endless SMIT-menus.

I hope this helps.

bakunin

/PS: don't get me wrong: SMITty is fine if you don't know exactly what you want to do and what format a certain command is. But for what i do daily and know exactly what and how to do SMITty is more a hindrance than a tool.

You're definitely a life saver! Updating to HA 7.1.3 makes sense. We have been using smitty which definitely take more than 10 mins lol.

If it's possible can you give me a complete procedure for setting up a (7.1-)-cluster that will help me a lot.