Filesystems, Disks and Memory

Hi All

I was wondering how the copying of vast amounts of data affected the overall lifespan of an HDD.

In my example, I'm copying approx 120GB (250,000) of files, once per hour from disk to another.

Is this likely to have a detrimental effect on the disk in terms of reads/writes etc?

Thanks.

From an engineering standpoint, yes. It would have to.

Most of the MTBF rates are based on some standard percentage of use, such as 30%. If you are keeping the drive busier than that, then you can expect the lifetime to be decreased.

Heavy usage over a long period is more likely to result in a thermal failure than anything else. The magnetic surface is good for X number of write operations and the servo motors are good for X number of direction reverses and distance traveled.

I would suggest that you have multiple spare drives ready to go (most shops will do this just out of a need for short-as-possible downtime). You could of course use some form of RAID that spreads the usage out over multiple drives. Using a form of RAID that provides redundancy also means that a drive failure doesn't immediately impact your uptime too.

Another option is to use rsync or similar technology that only copies the new files or those parts of the files that have changed, reducing the number of overall writes to the drive. The problem with rsync is that it has to read the file first and calculate block checksums which will have an impact on the overall access to the drives if they are configured for concurrent use.

Thanks Azhrei
We're using a JBOD at the moment so it's a raid configuration of sorts.

My colleague is using a program called robocopy (similar to rsync) to get this ever changing data over to another disk (as a kind of resilience option). I've pointed out to them that there's a purge function in robocopy which will copy all the files that have changed, and also delete files that need deleting.

However, they are convinced that this particular feature is somehow cursed and we'd be at risk of losing all the data!

Therefore, I need to prove to them that, over time, their method could well be degrading the disks.

Convining them to use the 'few files at a time' is extremely difficult and they're set in their ways at the moment!