Solaris

According to the metadb man page:

Now based on italicized part, if a mirror only has a single replica and never crashes, it should still function.

Same with booting. I thought three replicas were necessary for the system to be able to come up but the comment about crashing tells me that as long as the mirrors are stable (iow never crash), the system should come up.

The reason I ask this is that I've started a new position and discovered 8 disks in need of maintenance (metareplace), one disk out of sync (all partitions) and one disk with the status of Unknown.

In addition, I found several replicas that have failed (W - write errors and M - replica had problem with master blocks) and quite a few with less than 3 replicas on a disk (I prefer to configure a system with a mininum of 3 replicas per disk since disks are so large). At least one system had only a single replica database and 2 or 3 had two replica databases.

Per the comment in the man page, you need a majority (technically one is a majority) for the state information to be validated.

The problem is that systems are functioning correctly with one or two replicas and the one with a single replica was rebooted four days ago. Since it was booted, I have to believe that three replicas aren't required. In fact, as long as the replica db is stable, you can boot with a single replica (based on evidence).

So:

How does a system with only two replicas still manage to operate correctly? I can logically see how one will work (majority) but only two should fail.

Are my thoughts on a single replica and the system not crashing accurate? Or at least logical


My next steps appear to be to get the bad disks replaced (easy enough) but also to get the replicas in order. For the systems with 1 or 2 replicas, I think I'm going to have to break the mirrors and rebuild with what I think is the correct number of replicas (some of the disks have 5 replicas per disk).

Thoughts? Pointers to more technical information that what's on docs.sun.com?

Thanks.

Carl

Carl,

as long as a system has a majority of its total number of replicas available, it will boot. It doesn't matter where these are, how many are on any one particular disk, etc., but obviously you want to spread these evenly around to make the configuration as resilient to disk failure as possible.

All replicas on a system are the same, they are not associated with the specific volumes/mirrors or anything like that.

If you are going to replace disks, I would first delete the state database replicas from that disk (medadb -d /dev/dsk/cXtXdXsX) so that you don't end up with less than a majority available on boot.

Exactly. I was more concerned with taking how I thought it was supposed to work vs how it was actually working. Seeing a single replica total not only raised a flag, it had the guy waving the flag jumping up and down and screaming

I have one system with a single replica database. Per the man page:

As long as the replica is stable, the system will continue to function. If this replica bails, the system won't boot. However if there are two replicas and one goes south then we have a majority and the system will continue to run but it won't boot unless we delete the bad replica (assuming we can determine it's bad before rebooting).

The other problem, per the man page:

It appears that if the system crashes and only has two replicas and there's an inconsistancy, the system will not boot (needs one more than half to boot).

So if the system has three replicas and two of them become unstable, the system won't boot. Based on the section I quoted, three is a minimum if there's any inconsistancy and more would be better, in odd numbers. So 5 sounds like it'd be the absolute best without getting silly. And if you have two disks, five replicas on each disk would be the best as well. They only take about 512k per replica and the minimum size partition (on the 73 gig drive) is 10 megs.

I would determine based on the man pages that three would be the minimum and that five would be preferred.

man page from Sun

To answer my original questions, a system will continue to function with 1 or 2 replicas, however it's set up to fail as soon as there's an inconsistancy.

Yep, I knew that. It was the number that most bothered me, plus trying to remove the "required" from "suggested" and "highly suggested" in the man pages.

Technically if a disk is dead, you can't remove the replicas from that disk (it won't respond). Each replica also knows where the other replicas are. So when you run metadb, it's pulling the info from one of the available replicas. When you run metadb -f -d c1t1d0s7 (for example), you're removing the pointers from one of the replicas and the info is then disbursed to the others.

Lastly, to clear errored entries from the replicas databases, you have to delete the bad entry. Because you can't just add a single one back, the only way to replace it is to delete all the replicas from that slice and then recreate them.

Thanks Phil. I appreciate the answer.

Carl