[QUOTE=methyl;302622215]I don't think that tar or cp are the right commands.
To make a straight copy to another mounted filesystem and preserve permissions:
Ps. I have never used tar to back up anything. It is sometimes useful for moving files to alien systems.
Thanks for the input, but the goal is to move the 500Gb of data from the external drive to an offsite compute cluster. I believe the only way I can do this is ftp, and ftp only supports moving single files, not directories. GUIs like Filezilla don't work as they prompt for a new password every time the token-generated one expires.
I don't think it's possible to mount the external hard drive from a cluster that's behind a firewall - I can only connect to the cluster, not from it
---------- Post updated at 05:25 AM ---------- Previous update was at 05:20 AM ----------
Quote:
Originally Posted by Corona688
What bus speeds would you expect from your disks, omnisppot? Could you be having southbridge issues -- perhaps the bus is saturated?
Sorry, I don't know how to precisely answer that question! I know I can (if I had enough internal hard drive space) "cp -r" all the data down the SATA cable in a few hours without any issues. It certainly seems I/O on the external drive is the bottleneck with tar. Hopefully this is at the external disk end and not the mobo bus end. Hopefully (I'll find out next week) doing it on a 4-drive RAID0 will overcome that!
---------- Post updated at 05:28 AM ---------- Previous update was at 05:25 AM ----------
Quote:
Originally Posted by methyl
I agree with Corona688. Probably Hardware problem.
However I have seen a modern tar (i.e. one which can deal with files larger than 2Gb) crawl when it demands more memory.
My linux box has 16Gb RAM, but whilst doing this system usage didn't exceed 3Gb (including running the O/S and everything else).
I'm trying to tar a bunch of files off to a tape, but for one specific file (it is fairly large, roughly 10Gb) I get the error:
too large to archive
Does tar have a limit of the size of file it can write off to tape? I'm using SunOS 5.8.
Thanks!
-Fred (6 Replies)
If I have a directory /directory1 and want to tar and zip everything in it into a file new_tar.tar.gz on disk (not tape)
How can I do it?
I tried tar -cv /new_tar.tar /directory1/*
But I got an error: tar: /dev/rmt/0: No such device or address (4 Replies)
Recently we brought up a Spectralogic 2K Tape Library that had been out of service for about 3 years to replace a DDS-4 tape drive unit as our main backup device.
Everything seemed to go fine but now I have run into a little problem.
System details:
FBSD 6.1
SpectraLogic 2K library with a... (1 Reply)
Hi,
How do I tar all but a specific set of files in a directory? Is it possible to use regular expressions in the tar command? I want to tar all files except those beginning with D. I tried this
tar -cvf files.tar ^
but this didn't work. Anyone any ideas.
Thanks (2 Replies)
Hi all, my directory structure is as follows /a/b/c.
I would like to tar the /a directory including the subdirectories b and c.
i intend to use the command tar -cvfz a.tgz a/ My question is where do i execute the command? do i execute it at the '/' prompt or at '/a' prompt ? My concern at... (1 Reply)
Hi all. I was able to set up an IBM Ultrium LTO 4 tape drive to use iSCSI (using open-iscsi drivers) to communicate with Red Hat, but it's going really slow, maxing out in tar and dd tests at like 16 MB/s (using a block size of 128k). The thing is rated for 30MB/s. I feel like even though I have... (1 Reply)
I have a file that is 20 - 80+ MB in size that is a certain type of log file.
It logs one of our processes and this process is multi-threaded. Therefore the log file is kind of a mess. Here's an example:
The logfile looks like: "DATE TIME - THREAD ID - Details", and a new file is created... (4 Replies)
HI,
if I have a tarfile called pmapdata.tar that contains
tar -tvf pmapdata.tar
-rw-r--r-- 0/0 21 Oct 15 11:00 2009 /var/tmp/pmapdata/pmap4628.txt
-rw-r--r-- 0/0 21 Oct 14 20:00 2009 /var/tmp/pmapdata/pmap23752.txt
-rw-r--r-- 0/0 1625 Oct 13 20:00 2009... (1 Reply)
How do I create individual tars of a all the directories in a directory? I have a directory called 'patients', each patient has a directory in the patients directory. I want to create tars such that each patient has their own tar file.
Thanks! (5 Replies)
The below bash will untar each tar.bz2 folder in the directory, then remove the tar.bz2.
Each of the tar.bz2 folders ranges from 40-75GB and currently takes ~2 hours to extract. Is there a way to speed up the extraction process?
I am using a xeon processor with 12 cores. Thank you :).
... (7 Replies)
Discussion started by: cmccabe
7 Replies
LEARN ABOUT CENTOS
shell-quote
SHELL-QUOTE(1) User Contributed Perl Documentation SHELL-QUOTE(1)NAME
shell-quote - quote arguments for safe use, unmodified in a shell command
SYNOPSIS
shell-quote [switch]... arg...
DESCRIPTION
shell-quote lets you pass arbitrary strings through the shell so that they won't be changed by the shell. This lets you process commands
or files with embedded white space or shell globbing characters safely. Here are a few examples.
EXAMPLES
ssh preserving args
When running a remote command with ssh, ssh doesn't preserve the separate arguments it receives. It just joins them with spaces and
passes them to "$SHELL -c". This doesn't work as intended:
ssh host touch 'hi there' # fails
It creates 2 files, hi and there. Instead, do this:
cmd=`shell-quote touch 'hi there'`
ssh host "$cmd"
This gives you just 1 file, hi there.
process find output
It's not ordinarily possible to process an arbitrary list of files output by find with a shell script. Anything you put in $IFS to
split up the output could legitimately be in a file's name. Here's how you can do it using shell-quote:
eval set -- `find -type f -print0 | xargs -0 shell-quote --`
debug shell scripts
shell-quote is better than echo for debugging shell scripts.
debug() {
[ -z "$debug" ] || shell-quote "debug:" "$@"
}
With echo you can't tell the difference between "debug 'foo bar'" and "debug foo bar", but with shell-quote you can.
save a command for later
shell-quote can be used to build up a shell command to run later. Say you want the user to be able to give you switches for a command
you're going to run. If you don't want the switches to be re-evaluated by the shell (which is usually a good idea, else there are
things the user can't pass through), you can do something like this:
user_switches=
while [ $# != 0 ]
do
case x$1 in
x--pass-through)
[ $# -gt 1 ] || die "need an argument for $1"
user_switches="$user_switches "`shell-quote -- "$2"`
shift;;
# process other switches
esac
shift
done
# later
eval "shell-quote some-command $user_switches my args"
OPTIONS --debug
Turn debugging on.
--help
Show the usage message and die.
--version
Show the version number and exit.
AVAILABILITY
The code is licensed under the GNU GPL. Check http://www.argon.org/~roderick/ or CPAN for updated versions.
AUTHOR
Roderick Schertler <roderick@argon.org>
perl v5.16.3 2010-06-11 SHELL-QUOTE(1)