First off, your "inter-policy" is set to "maximum", which is nonsense. "inter-policy=maximum" means to use as many PVs as possible when creating or extending a LV, which results in a "poor mans striping". The basic idea is to distribute I/O over as many disks as possible. This might make sense with very small PP sizes (2-16MB) and many very old disks. With todays disks and todays typical PP sizes it is utter nonsense and only makes administration overly complicated. My suggestion is to get rid of it. Use an inter-policy of "minimum" only.

Next: your VG is 1.5 TB in size and has a laughable amount of 128M in contingency! I suggest you get some more disk space to allow at least for reorganisations, which should take place regularly when you deal with physical disks. Here, like in any area, it doesn't pay off to be greedy and it certainly lowers efficiency when using a resource to its abolute limit. You do not use your processor ressources at a permanent 100%, you do not tune your system to death so that it constantly has zero free RAM and for the same reason you do not fill your disks to absolute capacity - there is no room left to react if something unforeseen comes up. Plus some VG operations need unused space to be undertaken. You should have at least 100-200GB free space once you have created all the filesystems, better even more.

My suggestion is to recreate the VG completely anew: backup your data using the savevg utility, delete the VG (you can use exportvg to remove it completely from the system), then recreate it using the restvg. Select sensible values for the PP size when recreating it, because your system seems to use pretty small disks: you have 22 (now 21) PVs in your VG and a net capacity of 1.5TB, the single average disk is about 70G in size. No disk you can get today has as few as 70GB but with your PP size of 128M you can't use any disk larger than ~128G. On the other hand you cannot use a factor as i suggested above, because even the smallest factor of 2 would limit the VG to 16 PVs which you already exceed. Therefore, when recreating the VG use a PP size of 512MB or even bigger, which will allow you to replace the (probably quite old) disks with new, bigger ones. Be warned that this operation will take some time and you need downtime to do it.

If you can get additional disks to hold the data you could just create a new VG and copy the data from the old one, then delete the old one, this way you save the time for creating the backup and you always have a working copy as a fallback solution. A healthy leayout would be 4 PVs of 500GB each, which gives you 2TB of space, 1.5 TB taken and 500GB as a reserve to draw on. This (or the solution above) is the recommended way to do it.

The next best is to reorganize the existing VG on the existing disks. Some problems described above will stay, but at least the problems internal to the VG could be repaired.

Reorganisation means: remove all the mirrors from all the LVs in the VG. The command is unmirrorvg. This way you get some free space to work with. Then change the LVs using the chlv command to set inter-policy to "minimum". Once you have done this use the reorgvg command to bring the new LV layout into place. Be prepared to wait for quite some time (probably some hours) before this is finished, it will physically relocate the LPs of all the LVs. You can do all this online without downtime, but i suggest to do it in times of least activity or even take a downtime anyway. It will certainly be slowed down by much system stress and users will notice the degraded performance when this operation runs. Finally remirror using the mklvcopy command or - if you can get additional disks to add to the VG - a mirrorvg which does a mklvcopy for all LVs.

If you do a mirrorvg here is a tip: switch off the syncing, because it uses only 1 PP at a time. Do it this way (deferred syncing):

Code:

# mirrorvg -s myvg
# syncvg -P <NN> -v myvg

For "NN" set a number 1-32, which is the number of PPs be synced in parallel. Which number is good depends on your RAM, load, PP size and other things, but i suggest 10-20 for PP sizes between 64M and 512M.

I hope this helps.

bakunin

/PS:

This might be because a VGDA (volume group descriptor area) was found on the disk. Maybe it was already in use in this or another system and already held data. In any way, when you forced it into the VG it was cleaned of these data and reformatted, so speculations about its former content are moot at this point.

bakunin