Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Archiving a directory that has data ~150GB
Top Forums UNIX for Advanced & Expert Users Archiving a directory that has data ~150GB Post 302604434 by erlanq on Monday 5th of March 2012 04:18:28 AM
Old 03-05-2012
Archiving a directory that has data ~150GB
I have a directory that need to backup, its size is about 150GB consist of multiple files and directories. I try to compress it become a single archive file using these commands:
Code:
tar cjf this_archive.tar.bz2 this_archive/

or
Code:
tar cf - this_archive/ | 7z a -si -t7z -m0=lzma -mx=9 -mfb=64 -md=32m -ms=on this_archive.tar.7z

both commands always take a long time to process compressing the directory, even if it can finish the process, the output file archive always corrupt/error.

I want to know if there's any efficient compressing method/technique for large size directory that capable to do archiving directory like in my case?
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

archiving a directory using tar

How do i archive a directory and subdirectories under it using tar? (2 Replies)
Discussion started by: harminder
2 Replies

2. SCO

/data directory not mounting

Dear sir, In my SCO unix system while running an isql because of the size of the files created the ./data directory become full and now I cannot boot the system in normal mode. I am botting the machine in single user mode but i cannot delete the files from /data directory cos it is not... (2 Replies)
Discussion started by: khelen
2 Replies

3. Solaris

Deletion of Data from Lost+Found Directory

Hie I am running a sun solaris server of about 300 gigabytes disk capacity. The problem is that the machine has been having problems over the past year and at times the machine would just freeze or hang and had to be re-booted. Consequently there are too many entries in the lost+found... (1 Reply)
Discussion started by: Ranganai
1 Replies

4. Solaris

Full Directory without data....

$ df -hl Filesystem size used avail capacity Mounted on /dev/md/dsk/d0 9.8G 5.0G 4.7G 52% / /proc 0K 0K 0K 0% /proc mnttab 0K 0K 0K 0% /etc/mnttab fd 0K 0K 0K ... (2 Replies)
Discussion started by: adel8483
2 Replies

5. Shell Programming and Scripting

Automatically Load data from all files in directory

I'm new in Unix shell scripting and i need someone to help me to make Script that run all time watching my directory that files uploaded to it via FTP (/mydir/incoming_files), if any files exists in it then (if many files exists, then sort files and load them ascending) it‘ll checks the size of the... (1 Reply)
Discussion started by: m_fighter
1 Replies

6. Shell Programming and Scripting

Compress the contents of a directory while copying data into it

Hi guys I have a need to compress the contents of a directory while I am copying data into it. I was able to do this when it was only one file by doing as below: STEP1: mknod myfile p STEP2: chmod 777 myfile STEP3: compress -v < myfile > myfile.Z & STEP4: cp -p xyz_file myfile... (2 Replies)
Discussion started by: user1602
2 Replies

7. Shell Programming and Scripting

Script for Archiving data folderwise

Hi, I want to archive more than 15 days old files from /var/spool/ directory which contains 67 folders. I have done a script for a single folder which is as follows : cd /data1/ctron80/var/spool/bmaprt/ find . -mtime +15 | awk '{print "mv " $1 " /back/spool/bmaprt"}' | sh cd /back/spool/... (1 Reply)
Discussion started by: tuxian
1 Replies

8. UNIX for Dummies Questions & Answers

Moving all files in a directory to another directory and archiving them

Hi All, i need to move all files in a directory to some other directory and need to archive them,,, Ex.. Source_Path/my_directory/ files in it are... acw.csv 123.txt bge.dat etc ..and we dont know how many files does my_directory contains and all are with different extensions ..so i need... (6 Replies)
Discussion started by: dssyadav
6 Replies

9. Shell Programming and Scripting

Archiving files keeping the same structure directory

Hello Team, We would like to backup a lot of files inside of a structure of directories, four, five or more levels in some Ubuntu, Mac and Solaris systems. For instance: /home/chuck/sales/virgin/rent-quote.pdf /home/chuck/sales/marriott/vacation-quote.pdf... (2 Replies)
Discussion started by: csierra
2 Replies

10. Shell Programming and Scripting

Archiving or removing few data from log file in real time

Hi, I have a log file that gets updated every second. Currently the size has grown to 20+ GB. I need to have a command/script, that will try to get the actual size of the file and will remove 50% of the data that are in the log file. I don't mind removing the data as the size has grown to huge... (8 Replies)
Discussion started by: Souvik Patra
8 Replies

LEARN ABOUT DEBIAN

rdiff-backup-fs

RDIFF-BACKUP-FS(1)					      General Commands Manual						RDIFF-BACKUP-FS(1)

NAME

       rdiff-backup-fs - Filesystem for accessing rdiff-backup archives.

SYNOPSIS

       rdiff-backup-fs <mount_point> <repository> [repositories ...] [-option ...]

DESCRIPTION

       rdiff-backup-fs is a filesystem in userspace that reads rdiff-backup archives and provides convenient access.

OPTIONS

       --debug <0-4>
	      Run rdiff-backup-fs in foreground with given verbosity of debug messages.

       -f, --full
	      Store information about all revisions in memory. CAUTION: this may take a lot of memory if your archive contains many revisions.

       -l, --last
	      Displays	files from the most recent increment as directories, each holding every version of the file. CAUTION: this stores informa-
	      tion about all revisions in memory and therefore may take a lot of memory if archive contains many revisions.

       -c <n>, --caching <n>
	      How many files retrieved from the rdiff-backup archive may be cached by filesystem. By default rdiff-backup-fs will cache up  to	10
	      files. If this switch is set to 0, no caching will be done.

       -r <n>, --revisions <n>
	      How many revisions should be stored in memory for on demand revision retrieval. By default rdiff-backup-fs will store up to 10 revi-
	      sions in memory.

       -d, --directory <path>
	      Set directory for directory with temporary files. By default rdiff-backup-fs uses /tmp.

       -v, --version
	      Print version of rdiff-backup-fs and exit.

SEE ALSO

       rdiff-backup(1)

COPYRIGHT

       rdiff-backup-fs is Copyright (c) 2007-2011 Filip Gruszczyski.

       rdiff-backup-fs is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License  as  published
       by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

       This  program  is  distributed  in  the	hope  that  it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MER-
       CHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for more details.

       You should have received a copy of the GNU General Public License along with this program.  If not, see <http://www.gnu.org/licenses/>.

AUTHORS

       Filip Gruszczyski <gruszczy@gmail.com>

																RDIFF-BACKUP-FS(1)
All times are GMT -4. The time now is 01:34 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy