Login or Register to Ask a Question and Join Our Community


Solaris

Inaccurate file size report with du?

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Operating Systems Solaris Inaccurate file size report with du?
# 1  
Old 05-16-2013
Inaccurate file size report with du?
This is really strange... my system's du is reporting file size smaller than it really is... and it is only happening on one of Solaris 10 I have...

Code:
#ls -lrt tracks.txt
-rw-r--r--   1 oracle   oinstall 2166525362 May 16 03:32 tracks.txt

# du -k tracks.txt
953805  tracks.txt

# du -h tracks.txt
 931M   tracks.txt

Reports the same from global to non-global zones of this box

Code:
#ls -lrt tracks.txt
-rw-r--r--   1 user      user      2166525362 May 16 16:34 tracks.txt

#du -k tracks.txt
2126577 tracks.txt

#du -h tracks.txt
 2.0G   tracks.txt

Reports correct on a different box running same version of Solaris and from what I can tell.. same "du" binary


Code:
#ls -lrt tracks.txt
-rw-r--r-- 1 backup backup 2166525362 May 16 16:16 tracks.txt

#du -k tracks.txt
2115752 tracks.txt

#du -h tracks.txt
2.1G    tracks.txt

Copied it to a ubuntu 12.04.. reports correctly at about 2.0G


Any insight?? Thanks!!!!

---------- Post updated at 07:29 PM ---------- Previous update was at 06:51 PM ----------

It's the zfs.... not sure why it is reporting inaccurate file size though....

On the problematic box... global zone:
Code:
# ls -lrt tracks.txt
-rw-r--r--   1 105      101      2166525362 May 16 03:32 tracks.txt

# ls -lrt /export/home/tracks.txt
-rw-r--r--   1 root     root     2166525362 May 16 17:24 /export/home/tracks.txt

# du -h tracks.txt
 931M   tracks.txt

# du -h /export/home/tracks.txt
 2.0G   /export/home/tracks.txt

file in the local directory and file in "/export/home" is the same size..
but du reports different sizes...
local directory is a ZFS mount

# 2  
Old 05-17-2013
Quote:
Originally Posted by JT-KGY
It's the zfs.... not sure why it is reporting inaccurate file size though....
Did you check the file contents are identical ?

If true, there are a couple of explanations for this apparent discrepancy:

- the file is sparse
- compression is enabled on the file system.
This User Gave Thanks to jlliagre For This Post:
JT-KGY
# 3  
Old 05-17-2013
Sparse files have to be created deliberately right?
Seems like all files on the ZFS system behave the same way.

Code:
#ls -lh tracks.txt
-rw-r--r--   1 oracle   oinstall    2.0G May 16 03:32 tracks.txt

I used ls and it is returning the correct size on the file when du only returns 931M


So maybe the whole ZFS mount is compression enabled??

How does this affect performance? I'm running a Oracle DB on this filesystem.
Would I be able to turn off compression?

Thanks!!

---------- Post updated at 03:31 AM ---------- Previous update was at 03:27 AM ----------

DAMN!!! Compression is on....

here's the zpool history....

Code:
zpool create rz2pool raidz2 c2t0d0 c2t1d0 c2t2d0 c2t3d0 c2t4d0 c2t5d0 c2t6d0 c2t7d0 c2t8d0 c2t9d0 c2t10d0

zfs set compression=on rz2pool

---------- Post updated at 03:39 AM ---------- Previous update was at 03:31 AM ----------

https://blogs.oracle.com/observatory...sion_a_win_win

Found this post as well...

So I guess there's no compression-aware "du" out there?
# 4  
Old 05-17-2013
Quote:
Originally Posted by JT-KGY
Sparse files have to be created deliberately right?
Generally yes.
Code:
I used ls and it is returning the correct size on the file when du only returns 931M

931M is the correct disk usage.
Quote:
How does this affect performance?
Compression usually affect performance positively. Disks are generally much slower to process data than CPUs nowadays.
Quote:
Would I be able to turn off compression?
Sure.
Quote:
So I guess there's no compression-aware "du" out there?
That would make du a misnomer. du means disk usage, i.e. how much disk space a file is using. Solaris du correctly reports that.
GNU du implements the --apparent-size option though.
This User Gave Thanks to jlliagre For This Post:
JT-KGY
# 5  
Old 05-18-2013
I have several observation which you should consider (not so much related to question) :

Creating a zpool out of 11 disks is not good, maximum recommended is 9, you have 11.
You should make 2 raid arrays rather.

Of course, depending what you need, raidz2 is not great for performance setup. Mirror is recommended for database, unless you are using SSD disks.

Compression is fine if you have CPU available for it, although oracle suggests you should use it only for archive logs (remember you can always compress internally in database).

Hope that helps.
Regards
Peasant.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Shell script to report file size, pid and also kill the process

Hi All, Looking for a quick LINUX shell script which can continuously monitors the flle size, report the process which is creating a file greater than certain limit and also kill that process. Can someone please help me on this? (4 Replies)
Discussion started by: vasavimacherla
4 Replies

2. UNIX for Beginners Questions & Answers

Inaccurate scanning of Bash array elements

username=cogiz #!/bin/bash shuffle() #@ USAGE: shuffle { #@ TODO: add options for multiple or partial decks Deck=$( printf "%s\n" {2,3,4,5,6,7,8,9,T,J,Q,K,A}{H,S,D,C} | awk '## Seed the random number generator BEGIN { srand() } ## Put a random number in front... (4 Replies)
Discussion started by: cogiz
4 Replies

3. Shell Programming and Scripting

Inaccurate awk response

can someone help me figure out what's wrong with this awk code? awk '/CRON/ && /./ {i++;} { if((i>=4) && (i<8)) {print | "TIMO=$(date +%s) ; TIMO2=$(uname;who) ; echo $TIMO ---- 1 ---- Yes_Recoded,$TIMO2 ---- i >> /tmp/gona.txt" } else if (i>=8) {print | "TIMO=$(date +%s) ;... (2 Replies)
Discussion started by: SkySmart
2 Replies

4. UNIX for Dummies Questions & Answers

Ls directory size reporting byte size instead of file count

I have been searching both on Unix.com and Google and have not been able to find the answer to my question. I think it is partly because I can't come up with the right search terms. Recently, my virtual server switched storage devices and I think the problem may be related to that change.... (2 Replies)
Discussion started by: jmgibby
2 Replies

5. Shell Programming and Scripting

Help in Shell script to report size of files=0

Hello, I am trying to write a shell script in unix which looks for the last two files in a folder, checks their size and reports(via mail) if either of their file size is =0. cd /tmp list last two files: ls -ltr | tail -2 check if size of any of these two files is 0 if true, then send a... (8 Replies)
Discussion started by: kiran1112
8 Replies

6. Shell Programming and Scripting

Script to read file size and send email only if size > 0.

Hi Experts, I have a script like $ORACLE_HOME/bin/sqlplus username/password # << ENDSQL set pagesize 0 trim on feedback off verify off echo off newp none timing off set serveroutput on set heading off spool Schemaerrtmp.txt select ' TIMESTAMP COMPUTER NAME ... (5 Replies)
Discussion started by: welldone
5 Replies

7. Infrastructure Monitoring

SNMP disk space - inaccurate numbers

on the remote server that im running the snmp command against, below is the information about the specific directory i'm concerned about: SIZE USED AVAIL 673G 483G 157G can someone explain to me why snmp is telling me the size of this filesystem is 176399584? ... (5 Replies)
Discussion started by: SkySmart
5 Replies

8. Shell Programming and Scripting

The scripts not able to make the file to size 0, every times it go back to its original size

#!/bin/sh ########################################################################################################## #This script is being used for AOK application for cleaning up the .out files and zip it under logs directory. # IBM # Created #For pdocap201/pdoca202 .out files for AOK #1.... (0 Replies)
Discussion started by: mridul10_crj
0 Replies

9. Solaris

Directory size larger than file system size?

Hi, We currently have an Oracle database running and it is creating lots of processes in the /proc directory that are 1000M in size. The size of the /proc directory is now reading 26T. How can this be if the root file system is only 13GB? I have seen this before we an Oracle temp file... (6 Replies)
Discussion started by: sparcman
6 Replies

10. Solaris

command to find out total size of a specific file size (spread over the server)

hi all, in my server there are some specific application files which are spread through out the server... these are spread in folders..sub-folders..chid folders... please help me, how can i find the total size of these specific files in the server... (3 Replies)
Discussion started by: abhinov
3 Replies
Login or Register to Ask a Question