Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Compression - Exclude huge files

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Compression - Exclude huge files
# 1  
Old 06-02-2011
Compression - Exclude huge files
I have a DB folder which sizes to 60GB approx. It has logs which size from 500MB - 1GB. I have an Installation which would update the DB. I need to backup this DB folder, just incase my Installation FAILS. But I do not need the logs in my backup. How do I exclude them during compression (tar)?

Current command: tar cvfz <DB_Folder>.tar.gz <DB_Folder>
# 2  
Old 06-02-2011
If you have your log files names following a pattern (for example they all are ending in .log), then you can use this command:

Code:
tar zcf DB_Folder.tar.gz DB_Folder/ --exclude=*.log

If you cannot have a pattern that matches all but you have a list with all your log files, you can save that list to a file (say exclude.list) or you can even add patterns to that file and run the following command:

Code:
tar zcf DB_Folder.tar.gz DB_Folder/ --exclude-from=exclude.list

The contents of exclude.list could be:

Code:
*.log
some_unintuitive_log_file.ext
mylogs

etc
# 3  
Old 06-03-2011
Thanks a lot ciupinet! That worked exactly as expected.
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Exclude files in ls

Hi, I need to exlucde the files which are present in exclude.txt from a directory exlcude.txt AUZ.txt AUZ.chk NZ.txt NZ.chk tried with below code but not working ls -ltr | grep -v `cat exclude.lst` (9 Replies)
Discussion started by: rohit_shinez
9 Replies

2. Shell Programming and Scripting

Modification of MySQLDump-files before compression needed

Hi @all! In my MySQL-backup-script I backup and compress every single table with this command: /usr/bin/mysqldump --opt database_x table_y | /usr/bin/pbzip2 -c > "/media/BackUpDrive/Backup/table_x.gz"Unfortunately these files need modification - they have to start with the following line(s):... (7 Replies)
Discussion started by: gogo555
7 Replies

3. Shell Programming and Scripting

Need to exclude .NFSxxx files in clear old files batch script

I am new to Shell Scripting and need some help. The following batch job has been failing for me due to the .nfsxxx files in use. I need to know how to modify the following script to exclude the .nfsxxx files so this batch job will not fail on me. I have done lots of googling and keep coming back... (2 Replies)
Discussion started by: kimberlyg2007
2 Replies

4. Shell Programming and Scripting

Compare 2 folders to find several missing files among huge amounts of files.

Hi, all: I've got two folders, say, "folder1" and "folder2". Under each, there are thousands of files. It's quite obvious that there are some files missing in each. I just would like to find them. I believe this can be done by "diff" command. However, if I change the above question a... (1 Reply)
Discussion started by: jiapei100
1 Replies

5. UNIX for Advanced & Expert Users

Huge files manipulation

Hi , i need a fast way to delete duplicates entrys from very huge files ( >2 Gbs ) , these files are in plain text. I tried all the usual methods ( awk / sort /uniq / sed /grep .. ) but it always ended with the same result (memory core dump) In using HP-UX large servers. Any advice will... (8 Replies)
Discussion started by: Klashxx
8 Replies

6. UNIX for Dummies Questions & Answers

list the files but exclude the files in subdirectories

If I execute the command "ls -l /export/home/abcde/dev/proj/code/* | awk -F' ' '{print $9}' | cut -d'/' -f6-8" it will list all the files in /export/home/abcde/dev/proj/code/ directory as well as the files in subdirectories also proj/code/test.sh proj/code/test1.c proj/code/unix... (8 Replies)
Discussion started by: shyjuezy
8 Replies

7. UNIX for Dummies Questions & Answers

Difference between two huge files

Hi, As per my requirement, I need to take difference between two big files(around 6.5 GB) and get the difference to a output file without any line numbers or '<' or '>' in front of each new line. As DIFF command wont work for big files, i tried to use BDIFF instead. I am getting incorrect... (13 Replies)
Discussion started by: pyaranoid
13 Replies

8. UNIX for Advanced & Expert Users

Best compression for log files?

I have been doing some investigation into a log file from one of my systems, and the means which I currently use to compress and rotate it. I am looking for something smarter than gzip, faster than bzip2, and that can match or beat "my script" (which is slow as heck, but WAY better compression... (2 Replies)
Discussion started by: jjinno
2 Replies

9. UNIX for Advanced & Expert Users

du (exclude files)

Hi, I want to get the disk usage of a directory. But I want it to ignore a particular directory within it. Lets say I want disk usage of all files/dirs within dir1 except those that are named .snapshot Does du have the option of excluding a particular directory. (1 Reply)
Discussion started by: the_learner
1 Replies
Login or Register to Ask a Question