UNIX for Dummies Questions & Answers

Is there a reasonable maximum limit for tar file sizes? I want to transfer a pile of files from one server to another but have restricted means, so tarring them first will probably be best... but how big can I go - both for the file format itself and for the operating system (linux) to handle? We're talking thousands upon thousands of files, and many many gigabytes. Should I keep things under 4 gigs per file, or can they safely exceed ten or twenty or thrity gigs each?

Thanks.

Depends entirely on your filesystem, tar should happily just keep on spooling the files together without worrying about maximum sizes.

Find your max file size for the filesystem you are using and work with that.

As a suggestion, after you finish tar'ing, consider using gzip or compress afterwards.

Individual files do have maximum file length due to the encoding, ie each files header tells you how long the contained file is.

Tar (file format) - Wikipedia, the free encyclopedia

Twelve bytes are used to encode the length in octal, so that is 12x3 bits, so 36 bits, hence 2^36 hence 68719476736-1 bytes.

A ~68Gig tar file - crikey, you'd definately want to compress that sucker!

Presumably a DVD of all M*A*S*H episodes.

> Find your max file size for the filesystem

How would one go about doing that?


> A ~68Gig tar file - crikey, you'd definately want to compress that sucker!

Unless the files are already compressed. Eg mp3, zip


A secondary concern is getting a good transfer. The larger the file, the more risk of corruption. I've used md5sum to check file integrity, but not on anything over 10 gigs yet.



Thanks everybody for the quick replies.