FAULTY DISK replacement HP rx4640

gjk · #1 05-20-2012

Hello,

I'm new to this forum and as you will see from my question I'm new to UNIX as well.
One of our costumers has HP rx4640 running on UNIX with two 300GB hot-swappable disks that are mirrored. They reported to us that one of the disks is faulty and they want us to take care of it. Below is the only log they sent to us.

Code:

Fri May 18 17:50:11 2012    STCHK 122 sd_procchk sd_procchk 1 Logical volume 
    /dev/vg00/lvol1 is mirrored but has some stale blocks. Data loss on 
    hardware failure could occur. 
Fri May 18 17:50:12 2012    STCHK 122 sd_procchk sd_procchk 1 Logical volume 
    /dev/vg00/lvol3 is mirrored but has some stale blocks. Data loss on 
    hardware failure could occur. 
Fri May 18 17:50:12 2012    STCHK 122 sd_procchk sd_procchk 1 Logical volume 
    /dev/vg00/lvol4 is mirrored but has some stale blocks. Data loss on 
    hardware failure could occur. 
Fri May 18 17:50:12 2012    STCHK 122 sd_procchk sd_procchk 1 Logical volume 
    /dev/vg00/lvol5 is mirrored but has some stale blocks. Data loss on 
    hardware failure could occur. 
Fri May 18 17:50:12 2012    STCHK 122 sd_procchk sd_procchk 1 Logical volume 
    /dev/vg00/lvol6 is mirrored but has some stale blocks. Data loss on 
    hardware failure could occur. 
Fri May 18 17:50:12 2012    STCHK 122 sd_procchk sd_procchk 1 Logical volume 
    /dev/vg00/lvol7 is mirrored but has some stale blocks. Data loss on 
    hardware failure could occur. 
Fri May 18 17:50:12 2012    STCHK 122 sd_procchk sd_procchk 1 Logical volume 
    /dev/vg00/lvol8 is mirrored but has some stale blocks. Data loss on 
    hardware failure could occur. 
Fri May 18 17:50:12 2012    STCHK 122 sd_procchk sd_procchk 1 Logical volume 
    /dev/vg00/SwapVol2 is mirrored but has some stale blocks. Data loss on 
    hardware failure could occur. 
Fri May 18 17:56:06 2012    STCHK 122 sd_procchk sd_procchk 1 Logical volume 
    /dev/vg00/lvol1 is mirrored but has some stale blocks. Data loss on 
    hardware failure could occur. 
Fri May 18 17:56:06 2012    STCHK 122 sd_procchk sd_procchk 1 Logical volume 
    /dev/vg00/lvol3 is mirrored but has some stale blocks. Data loss on 
    hardware failure could occur. 
Fri May 18 17:56:06 2012    STCHK 122 sd_procchk sd_procchk 1 Logical volume 
    /dev/vg00/lvol4 is mirrored but has some stale blocks. Data loss on 
    hardware failure could occur. 
Fri May 18 17:56:06 2012    STCHK 122 sd_procchk sd_procchk 1 Logical volume 
    /dev/vg00/lvol5 is mirrored but has some stale blocks. Data loss on 
    hardware failure could occur. 
Fri May 18 17:56:06 2012    STCHK 122 sd_procchk sd_procchk 1 Logical volume 
    /dev/vg00/lvol6 is mirrored but has some stale blocks. Data loss on 
    hardware failure could occur. 
Fri May 18 17:56:06 2012    STCHK 122 sd_procchk sd_procchk 1 Logical volume 
    /dev/vg00/lvol7 is mirrored but has some stale blocks. Data loss on 
    hardware failure could occur. 
Fri May 18 17:56:06 2012    STCHK 122 sd_procchk sd_procchk 1 Logical volume 
    /dev/vg00/lvol8 is mirrored but has some stale blocks. Data loss on 
    hardware failure could occur. 
Fri May 18 17:56:06 2012    STCHK 122 sd_procchk sd_procchk 1 Logical volume 
    /dev/vg00/SwapVol2 is mirrored but has some stale blocks. Data loss on 
    hardware failure could occur.

Scrutinizer · #2 05-20-2012

I would not touch this if I were you. Ask an experienced HP-UX admin in your company to have a look...

gjk · #3 05-21-2012

That was my first thought, to not touch it. But at the moment our UNIX admin is not available and costumer is expecting a solution tonight. If there is a straight-forward procedure, for at least how to identify which physical disk is, please help.
What i got so far is this command:

Code:

 # ioscan –m lun /dev/disk/disk13

vbe · #4 05-21-2012

I dont know if I can help because I have VERY little experience on RX boxes now I managed to find one for home someone removed the disks before I got hold of it...
You did not say (or Im blind...) your OS version!
vg00 is usually the OS (and so root / boot disks...) even more true when you see no lvol2... for its the swap...
I would suggest before going further to get 2 new disks, get them ordered you may not need them or perhaps one but if things go wrong or the situation is not cool you may be glad to have 2 with you (believe my experience...).
Next once you have them try to make a recovery tape or an "Golden Image" somehow, if the system manages then you are perhaps in the way to solve your issue online, if not try to find the last bootable backup of the system (you may need it...)
Lest say you managed a make_recovery, you know can go and try stm or xstm (X GUI ) and see what the tool diagnose about your disks

here give us what you found, and we will do a bit of brainstorming...

Good luck!

gjk · #5 05-21-2012

Thank you for your help VBE and you are not blind but I don't know the OS version. As i have very limited access to the costumer site only tonight i will be able to find out the OS version and more about the disk. What i have confirmed is that it is a boot disk. With what i know from costumer the system is connected to msl6030 tape library and they do backups of configuration every day. Unfortunately I'm not familiar with diagnose tool but I'll try my best to get as much info as possible and post.
Thank you very very much

vbe · #6 05-22-2012

Type

Code:

 vgdisplay -v vg00 | grep dsk

gjk · #7 05-23-2012

Hello,

I was able last night to gather some more info about the OS and the condition of the disk. I think it needs to be replaced. As you will see from below is the alternate disk and not primary. I'm trying to scramble a procedure for replacement from "When Good Disks Go Bad" but i have some doubts about it. If you can help it will be real helpfully.

Code:

# uname -a
HP-UX - B.11.31 U ia64 2801820572 unlimited-user license

# model
ia64 hp server rx4640

# ioscan -funC disk
Class     I  H/W Path       Driver S/W State   H/W Type     Description
=======================================================================
disk      5  0/0/3/0.0.0.0  sdisk   CLAIMED     DEVICE       TEAC    DV-28E-N
                           /dev/dsk/c0t0d0   /dev/rdsk/c0t0d0
disk      0  0/1/1/0.1.0    sdisk   CLAIMED     DEVICE       HP 146 GST3146855LC
                           /dev/dsk/c2t1d0     /dev/dsk/c2t1d0s2   /dev/rdsk/c2t1d0    /dev/rdsk/c2t1d0s2
                           /dev/dsk/c2t1d0s1   /dev/dsk/c2t1d0s3   /dev/rdsk/c2t1d0s1  /dev/rdsk/c2t1d0s3
disk      4  0/1/1/1.0.0    sdisk   NO_HW       DEVICE       HP 146 GST3146855LC
                           /dev/dsk/c3t0d0     /dev/dsk/c3t0d0s2   /dev/rdsk/c3t0d0    /dev/rdsk/c3t0d0s2
                           /dev/dsk/c3t0d0s1   /dev/dsk/c3t0d0s3   /dev/rdsk/c3t0d0s1  /dev/rdsk/c3t0d0s3

# lvlnboot -v
Boot Definitions for Volume Group /dev/vg00:
Physical Volumes belonging in Root Volume Group:
    /dev/disk/disk9_p2 -- Boot Disk
    /dev/disk/disk13_p2 
Boot: lvol1    on:     /dev/disk/disk9_p2
            /dev/disk/disk13_p2
Root: lvol3    on:     /dev/disk/disk9_p2
            /dev/disk/disk13_p2
Swap: lvol2    on:     /dev/disk/disk9_p2
            /dev/disk/disk13_p2
Dump: lvol2    on:     /dev/disk/disk9_p2, 0

# pvdisplay –v /dev/dsk/c3t0d0 | more

# lvdisplay –v /dev/vg00/lvol1 | grep “Mirror copies”
Mirror copies               1  
# lvdisplay –v /dev/vg00/lvol2 | grep “Mirror copies”
Mirror copies               1  
# lvdisplay –v /dev/vg00/lvol3 | grep “Mirror copies”
Mirror copies               1  
# lvdisplay –v /dev/vg00/lvol4 | grep “Mirror copies”
Mirror copies               1  
# lvdisplay –v /dev/vg00/lvol5 | grep “Mirror copies”
Mirror copies               1  
# lvdisplay –v /dev/vg00/lvol6 | grep “Mirror copies”
Mirror copies               1  
# lvdisplay –v /dev/vg00/lvol7 | grep “Mirror copies”
Mirror copies               1  
# lvdisplay –v /dev/vg00/lvol8 | grep “Mirror copies”
Mirror copies  
             1              
# vgdisplay -v /dev/vg00
--- Volume groups ---
VG Name                     /dev/vg00
VG Write Access             read/write     
VG Status                   available                 
Max LV                      255    
Cur LV                      9      
Open LV                     9      
Max PV                      16     
Cur PV                      2      
Act PV                      2      
Max PE per PV               4357         
VGDA                        4   
PE Size (Mbytes)            32              
Total PE                    8694    
Alloc PE                    1994    
Free PE                     6700    
Total PVG                   0        
Total Spare PVs             0              
Total Spare PVs in use      0                     
VG Version                  1.0       
VG Max Size                 2230784m   
VG Max Extents              69712         

   --- Logical volumes ---
   LV Name                     /dev/vg00/lvol1
   LV Status                   available/stale           
   LV Size (Mbytes)            1824            
   Current LE                  57        
   Allocated PE                114         
   Used PV                     2       

   LV Name                     /dev/vg00/lvol2
   LV Status                   available/syncd           
   LV Size (Mbytes)            1024            
   Current LE                  32        
   Allocated PE                64          
   Used PV                     2       

   LV Name                     /dev/vg00/lvol3
   LV Status                   available/stale           
   LV Size (Mbytes)            1024            
   Current LE                  32        
   Allocated PE                64          
   Used PV                     2       

   LV Name                     /dev/vg00/lvol4
   LV Status                   available/stale           
   LV Size (Mbytes)            32              
   Current LE                  1         
   Allocated PE                2           
   Used PV                     2       

   LV Name                     /dev/vg00/lvol5
   LV Status                   available/stale           
   LV Size (Mbytes)            5024            
   Current LE                  157       
   Allocated PE                314         
   Used PV                     2       

   LV Name                     /dev/vg00/lvol6
   LV Status                   available/stale           
   LV Size (Mbytes)            544             
   Current LE                  17        
   Allocated PE                34          
   Used PV                     2       

   LV Name                     /dev/vg00/lvol7
   LV Status                   available/stale           
   LV Size (Mbytes)            3808            
   Current LE                  119       
   Allocated PE                238         
   Used PV                     2       

   LV Name                     /dev/vg00/lvol8
   LV Status                   available/stale           
   LV Size (Mbytes)            2624            
   Current LE                  82        
   Allocated PE                164         
   Used PV                     2       

   LV Name                     /dev/vg00/SwapVol2
   LV Status                   available/stale           
   LV Size (Mbytes)            16000           
   Current LE                  500       
   Allocated PE                1000        
   Used PV                     2       


   --- Physical volumes ---
   PV Name                     /dev/disk/disk9_p2
   PV Status                   available                
   Total PE                    4347    
   Free PE                     3350    
   Autoswitch                  On        
   Proactive Polling           On               

   PV Name                     /dev/disk/disk13_p2
   PV Status                   unavailable              
   Total PE                    4347    
   Free PE                     3350    
   Autoswitch                  On        
   Proactive Polling           On               

# cat bootconf 
l  /dev/disk/disk9_p2
l  /dev/disk/disk13_p2

# setboot
Primary bootpath : 0/1/1/0.0x1.0x0 (/dev/rdisk/disk9)
HA Alternate bootpath : 0/1/1/1.0x0.0x0 (/dev/rdisk/disk13)
Alternate bootpath : 0/1/2/0 (LAN Interface)

Autoboot is ON (enabled)
Hyperthreading : OFF
               : OFF (next boot)


# ioscan –m lun /dev/disk/disk13
Class     I  Lun H/W Path  Driver  S/W State   H/W Type     Health    Description
========================================================================
disk     13  64000/0xfa00/0x5   esdisk  NO_HW       DEVICE       disabled  HP 146 GST3146855LC       
             0/1/1/1.0x0.0x0
                      /dev/disk/disk13      /dev/disk/disk13_p2   /dev/rdisk/disk13     /dev/rdisk/disk13_p2
                      /dev/disk/disk13_p1   /dev/disk/disk13_p3   /dev/rdisk/disk13_p1  /dev/rdisk/disk13_p3

Then from the below i was able to identify it physicaly
 # dd if=/dev/dsk/c3t0d0 of=/dev/null bs=1024 
  /dev/dsk/c3t0d0: No such device or address
  dd: cannot open /dev/dsk/c3t0d0

# dd if=/dev/dsk/c2t1d0 of=/dev/null bs=1024
  2339888+0 records in
  2339888+0 records out

Thread Tools	Search this Thread
Show Printable Version Email this Page Subscribe to this Thread	Search this Thread: Advanced Search
Display Modes
Linear Mode Switch to Hybrid Mode Switch to Threaded Mode

More UNIX and Linux Forum Topics You Might Find Helpful
Thread	Thread Starter	Forum	Replies	Last Post
Remove faulty disk LV from VG	Shirishlnx	HP-UX	5	04-24-2012 10:27 PM
Remove Faulty disk from HP-UX LVM VG	Shirishlnx	HP-UX	9	04-13-2012 10:05 AM
Help with faulty Disk on Sun OS	phanidhar6039	Solaris	3	03-30-2012 03:12 PM
Removing Faulty Disk SSA	vuppala360	AIX	2	09-08-2008 11:59 AM
Disk Replacement SVM	Actuator	Solaris	4	05-21-2008 01:14 AM