UNIX for Beginners Questions & Answers

Good evening.

because the folder has thousand of files it takes too long and have some trouble to get the largest files and then compress files or delete it, for instance

Code:

find . -size +10000000c -exec ls -ld {} \; |sort -k5n | grep -v .gz

The above commad took an hour and i have to cancel it

Code:

du -sk * | sort -nr | more

secondly The above command took up some much time and got me root: args too long so it filed


Tried to compress some files but it took so long

Code:

nohup gzip log_HistoricoRecargas*201703*.log &

are there more efficient methods or commands to do this because i failed tried with these 3 choices
The operating system is SunOS

I appreciate your help in advanced

If there are many many files, then any search trough them will take time, potentially a lot of time. If this is on a filesystems that is mounted over the network, then this time will be much greater. Far better to do the work when the disk really is.

Perhaps this may be slightly more efficient for you though:-

Code:

find . -type f -size +10000000c ! -name "*.gz" -ls | sort -bnk 7

Can you tell us more about what hardware you have?



Kind regards,
Robin

Yes you are right, this a Filesyem mounted over a network

Code:

df -k 
10.80.1.83:/SCEL/logs1
                     37748736 29987224 7742352    88%    /SCEL/logs1

I dont have much infor about hardware just this;

Code:

SCEL /SCEL/logs1 # uname -a
SunOS prosclxxc 5.10 Generic_144488-01 sun4u sparc SUNW,SPARC-Enterprise

But if i run the sugested command there is a risk to compress logs from today i just want to compress logs from some day ago but not today.

I'd appreciate your help in advanced

Hi,

You can modify rbatte1's command to:-

Code:

find . -type f -mtime +2 -size +10000000c ! -name "*.gz" -ls | sort -bnk 7

This will only deal with files older than two days.

Regards

Gull04

You may create a list of filenames and sizes prior to having a closer look with different criteria. So the long running part - the reading of all file sizes is only being done once.

Example

1) Read the sizes

Code:

find / -type f -exec stat -c "%n %s" "{}" + >$HOME/files_sizes.txt

2) find the big files

Code:

awk '$2 > 1000000 { print $0 }' $HOME/file_sizes.txt

I've no knowledge with SunOS. The above commands may have slightly different syntax there.

Can you sign on to the server at address 10.80.1.83 ? If you can, running your code there will be significantly faster that running over the network. This will be not just the searching, but the actual compression too. If you compress over the network, then you have to read the file across the network to your local memory, compress it and then write the resultant file back across the network to the server disk.

It really could be a massive difference in performance.


Robin

Stomp - there is no stat command on vanilla SunOS. AFAIK. Since Oracle took over the Solaris freeware site died as well.

I think the OP also has another problem - Solaris 10 file systems (not ZFS) and earlier
all had a problem. If there are large numbers of files in a single directory, some file-related commands, notable is ls, bog down. A lot.

We had a directory with >30K small files in it. I fixed the performance problems by
moving files off the primary directory every day in a cron job. But still kept on the same file system. With about 5000 files performance was acceptable.

alexcol - please post the the output of a command the gives the physical size in bytes of the exact directory with the problem.

Since I do not know the name of the directory, here is an example, note the lowercase "d" in the command:

Code:

ls -ld  /path/to/directory

Please post the result so we can help

And if you happen to have too many individual files, regardless of size, on a file system then you can run out of inodes as well. This is pretty hard to do, but if the filesystem was created with unusual parameters this happens.
To see used inodes, try:

Code:

df -i /path/to/mountpoint

where mountpoint is the place in the file system where you interesting directory is mounted