Programming

I want to efficiently read parts of a file, in the case of sparse files, i'd like to detect what is sparse beforehand and not read those parts. Is there a way to do this? i imagine this will be quite filesystem dependant, but in most cases, this would be a considerable win in startup time.

Call stat() on the file - then: if st_blocks * st_blocksize != st_size the file has holes

i don't really want to know if the file has holes, but where EXACTLY the holes are located.

suppose i have a sparse file of about 1.5GB, in reality it's disk usage is about 300MB, i need to read the file, but for each read, i need to do a bunch of stuff, which i don't have to do, if it's sparse, in all actuality, i don't even have to read it, if it's sparse. thus:

lets say i have a function that's called that has to read from a certain offset a certain size. if part is sparse, i'd like only to read the non-sparse part.

if the whole part of it is sparse, i'd like to skip the whole read function.

thus: i have an offset and size and need to find out (without reading) if it's sparse and (possibly even which part of it is sparse).

is this possible?

any ideas?

The only way to find a hole is to read until you hit ascii nul characters. There is no system call that says 'here is a hole in this file'. Unix files without special helper libraries (ISAM for example) are purely sequential access - meaning there is no metadata to say what is in the next logical record. You have to read it to find out.

Plus, if the file is a sparse file that has the blocksize mismatch thing I mentioned earlier, it means the 'holes' are not on disk, so there is no disk I/O overhead to read thru a hole.
You have to read using fread() or read() and read a block at a time - probably not fgets().

hmm, so there is no specialized way? syscalls, ext3 libraries to get that information?

No - none.

What application is creating so many sparse files that you need this kind of thing?