cp to copy only non-corrupt files


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting cp to copy only non-corrupt files
# 8  
Old 12-27-2011
As already explained, dd_rescue doesn't care at all about filesystems, it makes a raw bit-for-bit copy. This is the entire point of using it, because it will not seize up and start foaming at the mouth when the filesystem isn't completely valid. You can copy filesystems from any OS. So if there's actual bad sectors onthe drive, you can use dd_rescue to copy it from the bad driveto the good then commence data recovery on the new drive.

Code:
ddrescue inputdevice outputdevice

Try fdisk -l to see which device is which.
# 9  
Old 12-27-2011
Alright thanks, I will try in a few minutes. I assume I want a log file.

What kinds of things should I do to check the new disk after I make the data dump?

LMHmedchem
# 10  
Old 12-27-2011
Quote:
Originally Posted by LMHmedchem
I assume I want a log file.
No point keeping one unless you have something to save it on. You can't save it on the old disk, you can't save it on the new disk, what exactly would you save it on?

Look at the value of errxfer during/after transfer. If it's ANYTHING but zero, there were bad sectors on the source disk during transfer.
Quote:
What kinds of things should I do to check the new disk after I make the data dump?
Depends what it is and what's on it. My expertise is with UNIX systems, not Windows ones. chkdsk may be able to successfully run on the new drive if it couldn't on the old one. Of course, my advice would be to hook it to a system which can mount the parittion and then just recover your data before you get too carried away...

Last edited by Corona688; 12-27-2011 at 10:48 PM..
# 11  
Old 12-28-2011
Quote:
Originally Posted by Corona688
No point keeping one unless you have something to save it on. You can't save it on the old disk, you can't save it on the new disk, what exactly would you save it on?
I wrote the logfile to /home/ubuntu/desktop/logfile. Since I am running ubuntu off of a flash drive, I presume that is a writeable location, otherwise it is in memory, but the file does appear on the desktop.

Quote:
Originally Posted by Corona688
Look at the value of errxfer during/after transfer. If it's ANYTHING but zero, there were bad sectors on the source disk during transfer. Depends what it is and what's on it.
So far it has recorded 24 errors. I guess it is unclear to me what the data on the new disk will look like when there were errors and how I would find files that are now unreadable, etc.

Quote:
Originally Posted by Corona688
My expertise is with UNIX systems, not Windows ones. chkdsk may be able to successfully run on the new drive if it couldn't on the old one. Of course, my advice would be to hook it to a system which can mount the partition and then just recover your data before you get too carried away...
Once the ddrescue is finished, I can hook up an external drive and copy the data from inside ubuntu. Will it be ok to just use cp, or should I use some other method to move the data to the external drive? What will cp do if it runs into corrupt files.

Once I have made a backup of the new drive, I will boot windows on the new computer and see what it makes of the drive. If it is fine, or if checkdisk runs and completes, I will put the drive back into the original computer and see if there are still issues or if the problems have resolved.

LMHmedchem
# 12  
Old 12-28-2011
The ddrescue run is finished, I have attached the log file if anyone is interested.

This is the summary,
Code:
Summary for /dev/sda -> /dev/sdc:
dd_rescue: (info): ipos: 976762560.0k, opos: 976762560.0k, xferd: 976762560.0k
                   errs:     24, errxfer:        12.0k, succxfer: 976762560.0k
             +curr.rate:     3064kB/s, avg.rate:    37374kB/s, avg.load: 10.9%

I'm going to copy the data from the new drive to an external backup drive now and see how that goes. Is there anything I can do to check the new drive to look for files that are incomplete or damaged?

LMHmedchem
# 13  
Old 12-28-2011
I'm running cp -Rfp (in Ubuntu) to copy the data from the new drive to an external backup drive.

There is one error,
cp: cannot access `Data_Old/Data_Applications/seamonkey2_profiles/yahoo/newstuf/Cache.Trash/Trash/Cache': Input/output error

I presume this means that the file in that path is corrupted. What can I do to get rid of that bad file entry? That is not a file that I need, if it was, I would go to my third backup and get a good copy of it.

LMHmedchem
# 14  
Old 12-28-2011
Quote:
Originally Posted by LMHmedchem
The ddrescue run is finished, I have attached the log file if anyone is interested.

This is the summary,
Code:
Summary for /dev/sda -> /dev/sdc:
dd_rescue: (info): ipos: 976762560.0k, opos: 976762560.0k, xferd: 976762560.0k
                   errs:     24, errxfer:        12.0k, succxfer: 976762560.0k
             +curr.rate:     3064kB/s, avg.rate:    37374kB/s, avg.load: 10.9%

Good. The drive had bad sectors and needed replacing, then -- and extremely few sectors were actually bad. That's fairly decent odds that next to none of them landed on anything important -- though that's just down to luck.
Quote:
I'm going to copy the data from the new drive to an external backup drive now and see how that goes. Is there anything I can do to check the new drive to look for files that are incomplete or damaged?
Well, copy them and see.

It's possible a block of zeroes might have landed right in the middle of a file of course. How to check for that depends on how you'd normally verify your data.

Also possible is the filesystem structure itself being disrupted.

---------- Post updated at 12:13 PM ---------- Previous update was at 12:10 PM ----------

Quote:
Originally Posted by LMHmedchem
I presume this means that the file in that path is corrupted.
No, it means the filesystem is corrupted. A bad file would just have chunks of zeroes inside it. There's no magic combination of zeroes and ones inside a file which causes it to be unable to be read.

Probably something in the filesystem metadata got filled with a chunk of zeroes when it couldn't be read. But now, instead of retrying and retrying for minutes at a time until it gives up on a bad sector, it's actually a good sector filled with nulls where the filesystem was expecting directory entries or inodes or something. So it gives up instantly.
Quote:
What can I do to get rid of that bad file entry?
Depending on what the partition is and how badly its damaged, fsck, chkdsk, or reformat. Same as any other partition.
This User Gave Thanks to Corona688 For This Post:
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Copy files from one drive to another, keeping most recently modified files

Hi all, I am a bit of a beginner with shell scripting.. What I want to do is merge two drives, for example moving all data from X to Y. If a file in X doesn't exist in Y, it will be moved there. If a file in X also exists in Y, the most recently modified file will be moved to (or kept) in... (5 Replies)
Discussion started by: apocolapse
5 Replies

2. Solaris

Solaris 10 GIF files corrupt during unzip

Problem occurs on one Solaris build. Every time we unzip the Jan CPU, there are several patches that error out (appears to be related to the GIF files). When we unzip the CPU on another Solaris build to a network storage area, we can execute without issue on the original machine. Any ideas? ... (1 Reply)
Discussion started by: grahamr72
1 Replies

3. Red Hat

Unable to copy files due to many files in directory

I have directory that has some billion file inside , i tried copy some files for specific date but it's always did not respond for long time and did not give any result.. i tried everything with find command and also with xargs.. even this command find . -mtime -2 -print | xargs ls -d did not... (2 Replies)
Discussion started by: before4
2 Replies

4. HP-UX

corrupt disk

Hallo Friends, I have application X running on hpux 11.11 and oracle 9i release 2. I recently had a hardware failure on disk /dev/dsk/c2t0d0 Below is the systemlog file : root@a7dmc:/var/adm/syslog > /opt/resmon/bin/resdata -R 155713541 -r /storage/events/enclosures/gazemon/0_1_1_0.0.0... (11 Replies)
Discussion started by: kekanap
11 Replies

5. Shell Programming and Scripting

how to copy files followed by list of names of all the files in /etc?

....... (2 Replies)
Discussion started by: pcbuilder
2 Replies

6. Solaris

How to safely copy full filesystems with large files (10Gb files)

Hello everyone. Need some help copying a filesystem. The situation is this: I have an oracle DB mounted on /u01 and need to copy it to /u02. /u01 is 500 Gb and /u02 is 300 Gb. The size used on /u01 is 187 Gb. This is running on solaris 9 and both filesystems are UFS. I have tried to do it using:... (14 Replies)
Discussion started by: dragonov7
14 Replies

7. Solaris

How to corrupt a superblock?

I need to corrupt a superblock of a mounted device in a soalris m/c and check recovery from an alternate superblock. How can this be done? (2 Replies)
Discussion started by: sujathan
2 Replies

8. HP-UX

Corrupt Member File

I have been fine adding/removing printers up until this week. Now when I go to add a new remote printer I get "corrupted member file". I go to /etc/lp/member and the byte count on the new printer name is 0. I VI the file and put /dev/null in to make it the correct size and it all looks fine and... (2 Replies)
Discussion started by: astout
2 Replies

9. UNIX for Dummies Questions & Answers

I got a corrupt /etc/inittab file....what next?

Hi guys, For some reason a client has given us a Sun Netra T1 with Solaris 8 to administer for them. That's always good business. However, the other day we rebooted the machine and to our amazement, after doing the preliminary hardware tests, we got an error messgae saying that /etc/inittab was... (3 Replies)
Discussion started by: Ivo
3 Replies

10. UNIX for Dummies Questions & Answers

corrupt or lost data

Thank you livin Free for all your help. We removed a lot of spool files and report files. Which should have freed up some space. But now I think a major problem we have is we have lost or corrupt files which are preventing us from coming up correctly. Can we load or can you copy us a directory... (1 Reply)
Discussion started by: NOT A CLUE
1 Replies
Login or Register to Ask a Question