Compression - Exclude huge files Post: 302526924

9 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

du (exclude files)

Hi, I want to get the disk usage of a directory. But I want it to ignore a particular directory within it. Lets say I want disk usage of all files/dirs within dir1 except those that are named .snapshot Does du have the option of excluding a particular directory.

2. UNIX for Advanced & Expert Users

Best compression for log files?

I have been doing some investigation into a log file from one of my systems, and the means which I currently use to compress and rotate it. I am looking for something smarter than gzip, faster than bzip2, and that can match or beat "my script" (which is slow as heck, but WAY better compression...

3. UNIX for Dummies Questions & Answers

Difference between two huge files

Hi, As per my requirement, I need to take difference between two big files(around 6.5 GB) and get the difference to a output file without any line numbers or '<' or '>' in front of each new line. As DIFF command wont work for big files, i tried to use BDIFF instead. I am getting incorrect...

4. UNIX for Dummies Questions & Answers

list the files but exclude the files in subdirectories

If I execute the command "ls -l /export/home/abcde/dev/proj/code/* | awk -F' ' '{print $9}' | cut -d'/' -f6-8" it will list all the files in /export/home/abcde/dev/proj/code/ directory as well as the files in subdirectories also proj/code/test.sh proj/code/test1.c proj/code/unix...

5. UNIX for Advanced & Expert Users

Huge files manipulation

Hi , i need a fast way to delete duplicates entrys from very huge files ( >2 Gbs ) , these files are in plain text. I tried all the usual methods ( awk / sort /uniq / sed /grep .. ) but it always ended with the same result (memory core dump) In using HP-UX large servers. Any advice will...

6. Shell Programming and Scripting

Compare 2 folders to find several missing files among huge amounts of files.

Hi, all: I've got two folders, say, "folder1" and "folder2". Under each, there are thousands of files. It's quite obvious that there are some files missing in each. I just would like to find them. I believe this can be done by "diff" command. However, if I change the above question a...

7. Shell Programming and Scripting

Need to exclude .NFSxxx files in clear old files batch script

I am new to Shell Scripting and need some help. The following batch job has been failing for me due to the .nfsxxx files in use. I need to know how to modify the following script to exclude the .nfsxxx files so this batch job will not fail on me. I have done lots of googling and keep coming back...

8. Shell Programming and Scripting

Modification of MySQLDump-files before compression needed

Hi @all! In my MySQL-backup-script I backup and compress every single table with this command: /usr/bin/mysqldump --opt database_x table_y | /usr/bin/pbzip2 -c > "/media/BackUpDrive/Backup/table_x.gz"Unfortunately these files need modification - they have to start with the following line(s):...

9. Shell Programming and Scripting

Exclude files in ls

Hi, I need to exlucde the files which are present in exclude.txt from a directory exlcude.txt AUZ.txt AUZ.chk NZ.txt NZ.chk tried with below code but not working ls -ltr | grep -v `cat exclude.lst`

LEARN ABOUT REDHAT

sendfiles

SENDFILES(1)                                                         [nmh-1.5]                                                        SENDFILES(1)

NAME

       sendfiles - send multiple files via a MIME message

SYNOPSIS

       sendfiles [delay] mailpath subject file1 [file2 ...]

DESCRIPTION

       The shell script sendfiles, is used to send a collection of files and directories via electronic mail.

            sendfiles mailpath "subject" files ...

       sendfiles will archive the files and directories you name with the tar command, and then mail the compressed archive to the "mailpath" with
       the given "subject".  The archive will be automatically split up into as many messages as necessary in order to get past most mailers.

       Sometimes you want sendfiles to pause after posting a partial message.  This is usually the case when you are running sendmail  and  expect
       to  generate  a lot of partial messages.  If the first argument given to sendfiles starts with a dash, then it is interpreted as the number
       of seconds to pause in between postings, e.g.,

            sendfiles -30 mailpath "subject" files ...

       will pause 30 seconds in between each posting.

   Extracting the Received Files
       When these messages are received, invoke mhstore once for the list of messages.  The default is for mhstore to store the combined parts  as
       a  new  message  in the current folder, although this can be changed using storage formatting strings.  You can then use mhlist to find out
       what's inside; possibly followed by mhstore again to write the archive to a file where you can subsequently uncompress and untar  it.   For
       instance:

            % mhlist 5-8
             msg part  type/subtype             size description
               5       message/partial           47K part 1 of 4
               6       message/partial           47K part 2 of 4
               7       message/partial           47K part 3 of 4
               8       message/partial           18K part 4 of 4
            % mhstore 5-8
            reassembling partials 5,6,7,8 to folder inbox as message 9
            % mhlist -verbose 9
             msg part  type/subtype             size description
               9       application/octet-stream 118K
                         (extract with uncompress | tar xvpf -)
                         type=tar
                         conversions=compress
            % mhstore 9
            % uncompress < 9.tar.Z | tar xvpf -

       Alternately, by using the -auto switch, mhstore will automatically do the extraction for you:

            % mhlist 5-8
             msg part  type/subtype             size description
               5       message/partial           47K part 1 of 4
               6       message/partial           47K part 2 of 4
               7       message/partial           47K part 3 of 4
               8       message/partial           18K part 4 of 4
            % mhstore 5-8
            reassembling partials 5,6,7,8 to folder inbox as message 9
            % mhlist -verbose 9
             msg part  type/subtype             size description
               9       application/octet-stream 118K
                         (extract with uncompress | tar xvpf -)
                         type=tar
                         conversions=compress
            % mhstore -auto 9
            -- tar listing appears here as files are extracted

       As the second tar listing is generated, the files are extracted.  A prudent user will never put -auto in the .mh_profile file.  The correct
       procedure is to first use mhlist to find out what will be extracted.  Then mhstore can be invoked with -auto to perform the extraction.

FILES

       $HOME/.mh_profile          The user profile

PROFILE COMPONENTS

       Path:                To determine the user's nmh directory
       Current-Folder:      To find the default current folder

SEE ALSO

       mhbuild(1), mhlist(1), mhshow(1), mhstore(1).  Proposed Standard for Message Encapsulation (RFC-934)

DEFAULTS

       `-noverbose'

CONTEXT

       None

MH.6.8                                                             11 June 2012                                                       SENDFILES(1)