Sponsored Content
Full Discussion: Rsync size descrepancies
Top Forums UNIX for Dummies Questions & Answers Rsync size descrepancies Post 302771037 by Corona688 on Tuesday 19th of February 2013 11:18:14 AM
Old 02-19-2013
du does not measure the strict size of the files. du measures disk used, at the sector and cluster level.

Perhaps one of your files was a sparse file -- a file with a big hole of empty space in the middle. Many UNIX filesystems support these. You can make one by opening a file, seeking forward a whole bunch, then writing; the operating system will just mark the space before it as empty instead of actually storing all the 0's. du's results will be skewed for it, since it measures what's stored on disk, not the length of the file itself.

It's difficult to replicate sparse files with the sparse section intact. They usually get converted to real binary 0's instead of an actual hole. This would make them bigger when copied by any traditional means.
 

10 More Discussions You Might Find Interesting

1. Solaris

command to find out total size of a specific file size (spread over the server)

hi all, in my server there are some specific application files which are spread through out the server... these are spread in folders..sub-folders..chid folders... please help me, how can i find the total size of these specific files in the server... (3 Replies)
Discussion started by: abhinov
3 Replies

2. Solaris

Rsync Size ????

Hi I have been using rsync for the past few days and would vouch for it anytime.However i am unable to find the total size of files being transferred. The output of rsync looks something like this: sent 2.92M bytes received 90.75K bytes 6.78K bytes/sec total size is 6.27G speedup... (2 Replies)
Discussion started by: Hari_Ganesh
2 Replies

3. Solaris

Directory size larger than file system size?

Hi, We currently have an Oracle database running and it is creating lots of processes in the /proc directory that are 1000M in size. The size of the /proc directory is now reading 26T. How can this be if the root file system is only 13GB? I have seen this before we an Oracle temp file... (6 Replies)
Discussion started by: sparcman
6 Replies

4. Shell Programming and Scripting

The scripts not able to make the file to size 0, every times it go back to its original size

#!/bin/sh ########################################################################################################## #This script is being used for AOK application for cleaning up the .out files and zip it under logs directory. # IBM # Created #For pdocap201/pdoca202 .out files for AOK #1.... (0 Replies)
Discussion started by: mridul10_crj
0 Replies

5. Shell Programming and Scripting

Script to read file size and send email only if size > 0.

Hi Experts, I have a script like $ORACLE_HOME/bin/sqlplus username/password # << ENDSQL set pagesize 0 trim on feedback off verify off echo off newp none timing off set serveroutput on set heading off spool Schemaerrtmp.txt select ' TIMESTAMP COMPUTER NAME ... (5 Replies)
Discussion started by: welldone
5 Replies

6. UNIX for Advanced & Expert Users

Limiting size of rsync batch output

Anyone know if there's a way to limit the size of rsync batch output blob? I need each batch to fix on a 64GB USB key. Using syntax like: rsync -av --only-write-batch=/Volumes/usb/batch --stats /Users/dfbadmin/sandbox/ /Users/dfbadmin/archives/ (7 Replies)
Discussion started by: dfbills
7 Replies

7. UNIX for Advanced & Expert Users

Physical disk IO size smaller than fragment block filesystem size ?

Hello, in one default UFS filesystem we have 8K block size (bsize) and 1K fragmentsize (fsize). At this scenary I thought all "FileSytem IO" will be 8K (or greater) but never smaller than the fragment size (1K). If a UFS fragment/blocksize is allwasy several ADJACENTS sectors on disk (in a ... (4 Replies)
Discussion started by: rarino2
4 Replies

8. Solaris

/tmp size is less whereas size allocated to swap is more

Hi, the /tmp size is less whereas the size allocated to swap is quite big. how to increase the size of /tmp - #: swap -l swapfile dev swaplo blocks free /dev/md/dsk/d20 85,20 8 273096 273096 #: swap -s total: 46875128k bytes allocated + 2347188k reserved =... (2 Replies)
Discussion started by: psb74
2 Replies

9. UNIX for Dummies Questions & Answers

Ls directory size reporting byte size instead of file count

I have been searching both on Unix.com and Google and have not been able to find the answer to my question. I think it is partly because I can't come up with the right search terms. Recently, my virtual server switched storage devices and I think the problem may be related to that change.... (2 Replies)
Discussion started by: jmgibby
2 Replies

10. Shell Programming and Scripting

Rsync Error: rsync: link_stat failed: No such file or directory (2)

I wish to copy all the files & folder under /web/Transfer_Files/data/ on mymac1 (Linux) to remote server mybank.intra.com (Solaris 10) /tmp/ location I am using Ansible tool synchronize module which triggers the unix rsync command as below:rsync --delay-updates -F --compress --archive --rsh=ssh... (2 Replies)
Discussion started by: mohtashims
2 Replies
FALLOCATE(1)							   User Commands						      FALLOCATE(1)

NAME
fallocate - preallocate or deallocate space to a file SYNOPSIS
fallocate [-c|-p|-z] [-o offset] -l length [-n] filename fallocate -d [-o offset] [-l length] filename fallocate -x [-o offset] -l length filename DESCRIPTION
fallocate is used to manipulate the allocated disk space for a file, either to deallocate or preallocate it. For filesystems which support the fallocate system call, preallocation is done quickly by allocating blocks and marking them as uninitialized, requiring no IO to the data blocks. This is much faster than creating a file by filling it with zeroes. The exit code returned by fallocate is 0 on success and 1 on failure. OPTIONS
The length and offset arguments may be followed by the multiplicative suffixes KiB (=1024), MiB (=1024*1024), and so on for GiB, TiB, PiB, EiB, ZiB and YiB (the "iB" is optional, e.g., "K" has the same meaning as "KiB") or the suffixes KB (=1000), MB (=1000*1000), and so on for GB, TB, PB, EB, ZB and YB. The options --collapse-range, --dig-holes, --punch-hole and --zero-range are mutually exclusive. -c, --collapse-range Removes a byte range from a file, without leaving a hole. The byte range to be collapsed starts at offset and continues for length bytes. At the completion of the operation, the contents of the file starting at the location offset+length will be appended at the location offset, and the file will be length bytes smaller. The option --keep-size may not be specified for the collapse-range operation. Available since Linux 3.15 for ext4 (only for extent-based files) and XFS. -d, --dig-holes Detect and dig holes. This makes the file sparse in-place, without using extra disk space. The minimum size of the hole depends on filesystem I/O block size (usually 4096 bytes). Also, when using this option, --keep-size is implied. If no range is specified by --offset and --length, then the entire file is analyzed for holes. You can think of this option as doing a "cp --sparse" and then renaming the destination file to the original, without the need for extra disk space. See --punch-hole for a list of supported filesystems. -i, --insert-range Insert a hole of length bytes from offset, shifting existing data. -l, --length length Specifies the length of the range, in bytes. -n, --keep-size Do not modify the apparent length of the file. This may effectively allocate blocks past EOF, which can be removed with a truncate. -o, --offset offset Specifies the beginning offset of the range, in bytes. -p, --punch-hole Deallocates space (i.e., creates a hole) in the byte range starting at offset and continuing for length bytes. Within the specified range, partial filesystem blocks are zeroed, and whole filesystem blocks are removed from the file. After a successful call, subse- quent reads from this range will return zeroes. This option may not be specified at the same time as the --zero-range option. Also, when using this option, --keep-size is implied. Supported for XFS (since Linux 2.6.38), ext4 (since Linux 3.0), Btrfs (since Linux 3.7) and tmpfs (since Linux 3.5). -v, --verbose Enable verbose mode. -x, --posix Enable POSIX operation mode. In that mode allocation operation always completes, but it may take longer time when fast allocation is not supported by the underlying filesystem. -z, --zero-range Zeroes space in the byte range starting at offset and continuing for length bytes. Within the specified range, blocks are preallo- cated for the regions that span the holes in the file. After a successful call, subsequent reads from this range will return zeroes. Zeroing is done within the filesystem preferably by converting the range into unwritten extents. This approach means that the spec- ified range will not be physically zeroed out on the device (except for partial blocks at the either end of the range), and I/O is (otherwise) required only to update metadata. Option --keep-size can be specified to prevent file length modification. Available since Linux 3.14 for ext4 (only for extent-based files) and XFS. -V, --version Display version information and exit. -h, --help Display help text and exit. AUTHORS
Eric Sandeen <sandeen@redhat.com> Karel Zak <kzak@redhat.com> SEE ALSO
truncate(1), fallocate(2), posix_fallocate(3) AVAILABILITY
The fallocate command is part of the util-linux package and is available from Linux Kernel Archive <https://www.kernel.org/pub/linux/utils /util-linux/>. util-linux April 2014 FALLOCATE(1)
All times are GMT -4. The time now is 05:07 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy