Recovering corrupted LVM data: No readable superblocks


 
Thread Tools Search this Thread
Operating Systems Linux Recovering corrupted LVM data: No readable superblocks
# 1  
Old 05-16-2014
Recovering corrupted LVM data: No readable superblocks

Hi all. Not sure where to post this, so figured I'd start here. I have a LVM2 partition that has become unreadable. I've scoured dozens of threads about the topic and have hit a wall, so any advice is appreciated. Below is what I think shows what my major problem is:

First, a simple mount attempt gives me:

Code:
$ sudo mount /dev/VolGroup00/LogVol01 /fluffy_main/
mount: /dev/mapper/VolGroup00-LogVol01 is write-protected, mounting read-only
mount: /dev/mapper/VolGroup00-LogVol01: can't read superblock
$ sudo dmesg | tail -n18
[4396115.450550] Buffer I/O error on device dm-1, logical block 153599984
[4396115.450553] Buffer I/O error on device dm-1, logical block 153599984
[4396115.450557] Buffer I/O error on device dm-1, logical block 153599998
[4396115.450559] Buffer I/O error on device dm-1, logical block 153599998
[4396115.450563] Buffer I/O error on device dm-1, logical block 0
[4396115.450565] Buffer I/O error on device dm-1, logical block 0
[4396115.450569] Buffer I/O error on device dm-1, logical block 1
[4396115.450573] Buffer I/O error on device dm-1, logical block 153599999
[4396115.450577] Buffer I/O error on device dm-1, logical block 153599999
[4396115.450579] Buffer I/O error on device dm-1, logical block 153599999
[4396115.451072] EXT4-fs (dm-1): unable to read superblock
[4396115.451109] EXT4-fs (dm-1): unable to read superblock
[4396115.451136] EXT4-fs (dm-1): unable to read superblock
[4396115.451256] EXT4-fs (dm-1): unable to read superblock
[4396115.451284] EXT4-fs (dm-1): unable to read superblock
[4396115.451309] EXT4-fs (dm-1): unable to read superblock
[4396115.451337] isofs_fill_super: bread failed, dev=dm-1, iso_blknum=16, block=32
[4396115.451365] FAT-fs (dm-1): unable to read boot sector

I then tried checking the LVM through fsck:
Code:
$ sudo e2fsck -fv /dev/dm-1
e2fsck 1.42.7 (21-Jan-2013)
e2fsck: Attempt to read block from filesystem resulted in short read while trying to open /dev/dm-1
Could this be a zero-length partition?

with a similar dmesg output to above (Buffer I/O error...).

I found other possible superblocks to try and rebuild the filesystem:
Code:
$ sudo mke2fs -n /dev/sde2
mke2fs 1.42.7 (21-Jan-2013)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=0 blocks, Stripe width=0 blocks
43376640 inodes, 173505280 blocks
8675264 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=4294967296
5295 block groups
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Superblock backups stored on blocks: 
    32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208, 
    4096000, 7962624, 11239424, 20480000, 23887872, 71663616, 78675968, 
    102400000

But none of these are able to be read using e2fsck.

Low number superblocks give me this error:
Code:
$ sudo e2fsck -f -b 32768 /dev/VolGroup00/LogVol01 
e2fsck 1.42.7 (21-Jan-2013)
e2fsck: Attempt to read block from filesystem resulted in short read while trying to open /dev/VolGroup00/LogVol01
Could this be a zero-length partition?
$ sudo dmesg | tail
[4396348.465272] Buffer I/O error on device dm-1, logical block 8192
[4396348.465275] Buffer I/O error on device dm-1, logical block 8192
[4396348.465388] Buffer I/O error on device dm-1, logical block 16384
[4396348.465391] Buffer I/O error on device dm-1, logical block 16384
[4396348.465412] Buffer I/O error on device dm-1, logical block 32768
[4396348.465414] Buffer I/O error on device dm-1, logical block 32768
[4396348.465434] Buffer I/O error on device dm-1, logical block 65536
[4396348.465436] Buffer I/O error on device dm-1, logical block 65536
[4396348.465467] Buffer I/O error on device dm-1, logical block 131072
[4396348.465471] Buffer I/O error on device dm-1, logical block 131072

While higher numbered superblocks give me this error:
Code:
$ sudo e2fsck -f -b 20480000 /dev/VolGroup00/LogVol01
e2fsck 1.42.7 (21-Jan-2013)
e2fsck: Invalid argument while trying to open /dev/VolGroup00/LogVol01

The superblock could not be read or does not describe a correct ext2
filesystem.  If the device is valid and it really contains an ext2
filesystem (and not swap or ufs or something else), then the superblock
is corrupt, and you might try running e2fsck with an alternate superblock:
    e2fsck -b 8193 <device>
$ sudo dmesg | tail 
[4396444.356681] quiet_error: 318 callbacks suppressed
[4396444.356684] Buffer I/O error on device dm-1, logical block 5120000
[4396444.356687] Buffer I/O error on device dm-1, logical block 5120000
[4396444.356860] Buffer I/O error on device dm-1, logical block 10240000
[4396444.356864] Buffer I/O error on device dm-1, logical block 10240000
[4396444.356896] Buffer I/O error on device dm-1, logical block 20480000
[4396444.356900] Buffer I/O error on device dm-1, logical block 20480000
[4396444.356929] Buffer I/O error on device dm-1, logical block 40960000
[4396444.356933] Buffer I/O error on device dm-1, logical block 40960000
[4396444.356966] Buffer I/O error on device dm-1, logical block 81920000
[4396444.356970] Buffer I/O error on device dm-1, logical block 81920000

Anyways, I am stuck as to what I can possibly do from here or what else I should look for. If anyone has any other ideas or can point me towards a resource that can help me along, I would be most grateful!
# 2  
Old 05-19-2014
This sounds more like an experts problem, so i move it to the appropriate forum. It would have helped, though, would you have told us which system you are on. I suppose it is some Linux flavour, but kernel version, etc.. might be helpful in tracking the error.

bakunin
# 3  
Old 05-19-2014
Quote:
Originally Posted by dargason
Code:
$ sudo mke2fs -n /dev/sde2
mke2fs 1.42.7 (21-Jan-2013)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=0 blocks, Stripe width=0 blocks
43376640 inodes, 173505280 blocks
8675264 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=4294967296
5295 block groups
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Superblock backups stored on blocks: 
    32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208, 
    4096000, 7962624, 11239424, 20480000, 23887872, 71663616, 78675968, 
    102400000


Quote:
$ sudo mke2fs -n /dev/sde2
That should have not given you any superblock output if it were the partition for the logical volume /dev/VolGroup00/LogVol01

The output would had been more in the order of:
/dev/sde2 is apparently in use by the system, blah, blah
even if it is broken

Question? Are you sure that /dev/sde2 is the partition of the LVM?
Code:
dumpe2fs /dev/dm-1 | grep -i superblock
dumpe2fs /dev/mapper/VolGroup00-LogVol01 | grep -i superblock

Either of those two would output the proper superblocks if available, assuming for sure that /dev/dm-1 maps to the LVM.

What's the output of lsblk if available?

What's the output of pvdisplay?

Last edited by Aia; 05-19-2014 at 11:43 PM.. Reason: asking for pvdisplay
# 4  
Old 05-20-2014
Hi Aia, thanks a bunch for your response.

/dev/sde2 is the LVM partition I am interested in. Running mke2fs on /dev/VolGroup00/LogVol01 explicitly gives me the same thing.

Code:
$ sudo mke2fs -n /dev/VolGroup00/LogVol01
[sudo] password for ds229: 
mke2fs 1.42.7 (21-Jan-2013)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=0 blocks, Stripe width=0 blocks
38404096 inodes, 153600000 blocks
7680000 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=4294967296
4688 block groups
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Superblock backups stored on blocks: 
    32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208, 
    4096000, 7962624, 11239424, 20480000, 23887872, 71663616, 78675968, 
    102400000

Both dumpe2fs commands are not able to read anything
Code:
$ sudo dumpe2fs /dev/dm-1 | grep -i superblock
dumpe2fs 1.42.7 (21-Jan-2013)
dumpe2fs: Attempt to read block from filesystem resulted in short read while trying to open /dev/dm-1
Couldn't find valid filesystem superblock.

$ sudo dumpe2fs /dev/mapper/VolGroup00-LogVol01 | grep -i superblock
dumpe2fs 1.42.7 (21-Jan-2013)
dumpe2fs: Attempt to read block from filesystem resulted in short read while trying to open /dev/mapper/VolGroup00-LogVol01
Couldn't find valid filesystem superblock.

And here is the output for lsblk and pvdisplay

Code:
$ sudo lsblk
NAME   MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
sda      8:0    0 596.2G  0 disk 
├─sda1   8:1    0 196.1M  0 part 
└─sda2   8:2    0   596G  0 part 
sdb      8:16   0 931.5G  0 disk 
├─sdb1   8:17   0   500M  0 part /boot
├─sdb2   8:18   0 742.2G  0 part /home
├─sdb3   8:19   0  58.6G  0 part /
├─sdb4   8:20   0     1K  0 part 
├─sdb5   8:21   0  48.8G  0 part /usr/swr
├─sdb6   8:22   0  19.5G  0 part /var
├─sdb7   8:23   0  11.7G  0 part [SWAP]
└─sdb8   8:24   0  29.3G  0 part 
sde      8:64   0 698.7G  0 disk 
├─sde1   8:65   0     1G  0 part /run/media/marsluo/e08527a9-c875-4cc4-b83c-1158e07640c6
├─sde2   8:66   0 661.9G  0 part 
├─sde3   8:67   0    16G  0 part 
├─sde4   8:68   0     1K  0 part 
└─sde5   8:69   0    10G  0 part /run/media/marsluo/9c758ac5-3a6d-4181-a401-9dce5175144c
sr0     11:0    1  1024M  0 rom  
sr1     11:1    1  1024M  0 rom  
$ sudo pvdisplay
  --- Physical volume ---
  PV Name               /dev/sde2
  VG Name               VolGroup00
  PV Size               661.87 GiB / not usable 27.00 MiB
  Allocatable           yes 
  PE Size               32.00 MiB
  Total PE              21179
  Free PE               554
  Allocated PE          20625
  PV UUID               sqJHbZ-Udp4-OtHq-GQ38-Rkff-TSHQ-unS59g

FYI, the drive is currently sitting in a FC19 machine and was partitioned/written in a different Fedora box (18 or 19, I forget).
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. Solaris

Recovering DATA from sparc classic

I have a Sun sparc classic that I am trying to recover data off. The main CPU part just clicks or beeps when powered up, but does not come on (nothing on screen, and LED in front not lighting up). There is also an external SCSI drive, and I have verified there is a drive inside the CPU. ... (8 Replies)
Discussion started by: mackconsult
8 Replies

2. Red Hat

Configure LVM to gain access to the data on disks

Hello friends, We had to upgrade RHEL 4.x to 5.7 so we took complete backup. Since DIRECT UPGRADE not possible, we rebuilt 5.7 from scratch. We had lvm configuration on DATA disks in previous OS and now we need to configure such that we should be able to have same old LVM configuration (like... (1 Reply)
Discussion started by: prvnrk
1 Replies

3. UNIX for Dummies Questions & Answers

How to convert non LVM root partition to LVM?

Hi Guys, I m using redhat 6, I have installed root partition as non-LVM . Is there any way i can convert it to LVM? (1 Reply)
Discussion started by: pinga123
1 Replies

4. UNIX for Dummies Questions & Answers

Flash drive recovering data.

I have a flash drive which contained very important docs. But somebidy accidently dleted those files. I want to recover these files anyhow. I have listened the Linux have best possible chances of recovering it. Can anybody tell me how to recover that? (1 Reply)
Discussion started by: nixhead
1 Replies

5. HP-UX

How to reduce LVM to create another LVM

Hi, I'm new to HP-UX. I have LVM on /var with 92Gig. I would like to reduce it to create another LVM for Oracle client with 800 meg or so. How to do it. I'm running 11.iv3 Thanks (4 Replies)
Discussion started by: lamoul
4 Replies

6. UNIX for Advanced & Expert Users

Corrupted journal in a Linux LVM How to recover

Not real Linux expert but here is the problem. Shutdown this machine and then it would not reboot. From everything I can tell it looks like the journal file in the logical volume is corrupted. I have tried everything I can think of to get the volume mounted. Anybody have any ideas on how... (19 Replies)
Discussion started by: ccj4467
19 Replies

7. UNIX for Advanced & Expert Users

Data Recovery from file system overwritten with LVM.

Hey peeps, Here is somethin u might find interestin.... Is it possible to recover data from a partition which used to be an ext3 file sytem with some nice forgotten backups, which now is an lvm partion containg root partition of another OS. :) I couldn't create any mess better than this, can... (2 Replies)
Discussion started by: squid04
2 Replies

8. UNIX for Dummies Questions & Answers

Recovering lost folders/files data

Hello, Is there a way to recover data from a SCO UNIXWARE 7.4 operating system without using a tape backup device? We believe there is some data in some directories that was there once; but not anymore, we don't have a backup on tape. So, is there any other solution to recover? Hope... (0 Replies)
Discussion started by: Yorgy
0 Replies

9. News, Links, Events and Announcements

Filesystems: Hiding and recovering data

Here is a very interesting article on not only hiding data on filesystems (the article deals mainly with the ext2 filesystem, which should also work with etx3), but also recovering, including from slack space on raw blocks, and even deleted data! ... (0 Replies)
Discussion started by: LivinFree
0 Replies
Login or Register to Ask a Question