Sponsored Content
Top Forums Shell Programming and Scripting Compression - Exclude huge files Post 302527051 by ciupinet on Thursday 2nd of June 2011 07:38:25 AM
Old 06-02-2011
If you have your log files names following a pattern (for example they all are ending in .log), then you can use this command:

Code:
tar zcf DB_Folder.tar.gz DB_Folder/ --exclude=*.log

If you cannot have a pattern that matches all but you have a list with all your log files, you can save that list to a file (say exclude.list) or you can even add patterns to that file and run the following command:

Code:
tar zcf DB_Folder.tar.gz DB_Folder/ --exclude-from=exclude.list

The contents of exclude.list could be:

Code:
*.log
some_unintuitive_log_file.ext
mylogs

etc
 

9 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

du (exclude files)

Hi, I want to get the disk usage of a directory. But I want it to ignore a particular directory within it. Lets say I want disk usage of all files/dirs within dir1 except those that are named .snapshot Does du have the option of excluding a particular directory. (1 Reply)
Discussion started by: the_learner
1 Replies

2. UNIX for Advanced & Expert Users

Best compression for log files?

I have been doing some investigation into a log file from one of my systems, and the means which I currently use to compress and rotate it. I am looking for something smarter than gzip, faster than bzip2, and that can match or beat "my script" (which is slow as heck, but WAY better compression... (2 Replies)
Discussion started by: jjinno
2 Replies

3. UNIX for Dummies Questions & Answers

Difference between two huge files

Hi, As per my requirement, I need to take difference between two big files(around 6.5 GB) and get the difference to a output file without any line numbers or '<' or '>' in front of each new line. As DIFF command wont work for big files, i tried to use BDIFF instead. I am getting incorrect... (13 Replies)
Discussion started by: pyaranoid
13 Replies

4. UNIX for Dummies Questions & Answers

list the files but exclude the files in subdirectories

If I execute the command "ls -l /export/home/abcde/dev/proj/code/* | awk -F' ' '{print $9}' | cut -d'/' -f6-8" it will list all the files in /export/home/abcde/dev/proj/code/ directory as well as the files in subdirectories also proj/code/test.sh proj/code/test1.c proj/code/unix... (8 Replies)
Discussion started by: shyjuezy
8 Replies

5. UNIX for Advanced & Expert Users

Huge files manipulation

Hi , i need a fast way to delete duplicates entrys from very huge files ( >2 Gbs ) , these files are in plain text. I tried all the usual methods ( awk / sort /uniq / sed /grep .. ) but it always ended with the same result (memory core dump) In using HP-UX large servers. Any advice will... (8 Replies)
Discussion started by: Klashxx
8 Replies

6. Shell Programming and Scripting

Compare 2 folders to find several missing files among huge amounts of files.

Hi, all: I've got two folders, say, "folder1" and "folder2". Under each, there are thousands of files. It's quite obvious that there are some files missing in each. I just would like to find them. I believe this can be done by "diff" command. However, if I change the above question a... (1 Reply)
Discussion started by: jiapei100
1 Replies

7. Shell Programming and Scripting

Need to exclude .NFSxxx files in clear old files batch script

I am new to Shell Scripting and need some help. The following batch job has been failing for me due to the .nfsxxx files in use. I need to know how to modify the following script to exclude the .nfsxxx files so this batch job will not fail on me. I have done lots of googling and keep coming back... (2 Replies)
Discussion started by: kimberlyg2007
2 Replies

8. Shell Programming and Scripting

Modification of MySQLDump-files before compression needed

Hi @all! In my MySQL-backup-script I backup and compress every single table with this command: /usr/bin/mysqldump --opt database_x table_y | /usr/bin/pbzip2 -c > "/media/BackUpDrive/Backup/table_x.gz"Unfortunately these files need modification - they have to start with the following line(s):... (7 Replies)
Discussion started by: gogo555
7 Replies

9. Shell Programming and Scripting

Exclude files in ls

Hi, I need to exlucde the files which are present in exclude.txt from a directory exlcude.txt AUZ.txt AUZ.chk NZ.txt NZ.chk tried with below code but not working ls -ltr | grep -v `cat exclude.lst` (9 Replies)
Discussion started by: rohit_shinez
9 Replies
SHTOOL-TARBALL.TMP(1)					      GNU Portable Shell Tool					     SHTOOL-TARBALL.TMP(1)

NAME
shtool-tarball - GNU shtool command for rolling standardized tarballs SYNOPSIS
shtool tarball [-t|--trace] [-v|--verbose] [-o|--output tarball] [-c|--compress prog] [-d|--directory directory] [-u|--user user] [-g|--group group] [-e|--exclude pattern] path [path ...] DESCRIPTION
This command is for rolling input files under path into a distribution tarballs which can be extracted by tar(1). The four important aspects of good open source software tarballs are: (1) unpack into a single top-level directory, (2) top-level directory corresponds to the tarball filename, (3) tarball files should be sorted and (4) arbitrary names for file owner and group. OPTIONS
The following command line options are available. -v, --verbose Display some processing information. -t, --trace Enable the output of the essential shell commands which are executed. -o, --output tarball Output tarball to file tarball. -c, --compress prog Pipe resulting tarball through compression program prog. -d, --directory directory Sets the top-level directory into which the tarball unpacks. By default it is tarball without the trailing ".tar.*" extension. -u, --user user The user (owner) of files and directories in the tarball to user. -g, --group group The group of files and directories in the tarball to group. -e, --exclude pattern Exclude files and directories matching comma-separated list of regex pattern from the tarball. Directories are expanded before the filtering takes place. The default filter pattern is ""CVS,\.cvsignore,\.svn,\.[oa]$"". EXAMPLE
# Makefile.in dist: ... V=`shtool version -d short ...`; shtool tarball -o foobar-$$V.tar.gz -c 'gzip -9' -u bar -g gnu -e 'CVS,.cvsignore' . HISTORY
The GNU shtool tarball command was originally written by Ralf S. Engelschall <rse@engelschall.com> in 1999 for GNU shtool. SEE ALSO
shtool(1), tar(1), compress(1). 18-Jul-2008 shtool 2.0.8 SHTOOL-TARBALL.TMP(1)
All times are GMT -4. The time now is 09:36 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy