02-06-2012
Quote:
we have 30 GB files on our filesystem which we need to copy daily to 25 location on the same machine (but different filesystem).
This is ambiguous.
What version of HP-UX do you have?
How many files are you copying?
How often are you copying these files?
How big is the largest file? Is every file always smaller than 2 Gigabytes?
What is the filesystem type? Is NFS involved or are they all local discs?
Are you actually copying all of the files 25 times, or just copying each file once to one of 25 different directories?
What does "location" mean in unix terms?
Are these files from a recognised package (e.g. Oracle Archive logs)?
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
Hi all you enlightened unix people,
I've been trying to execute a perl script that contains the following line within backticks:
`grep -f patternfile.txt otherfile.txt`;It takes normally 2 minutes to execute this command from the bash shell by hand.
I noticed that when i run this command... (2 Replies)
Discussion started by: silverlocket
2 Replies
2. Shell Programming and Scripting
I am trying to copy a folder which contains a list of C executables.
It takes 2 mins for completion,where as the entire script takes only 3 more minutes for other process.
Is there a way to copy the folder faster so that the performance of the script will improve? (2 Replies)
Discussion started by: prasperl
2 Replies
3. UNIX for Dummies Questions & Answers
i)wc -c/etc/passwd|awk'{print $1}'
ii)ls -al/etc/passwd|awk'{print $5}' (4 Replies)
Discussion started by: karthi_g
4 Replies
4. Shell Programming and Scripting
Hi,
A datafile containing lines such as below needs to be split:
500000000000932491683600000000000000000000000000016800000GS0000000000932491683600*HOME
I need to get the 2-5, 11-20, and 35-40 characters and I can do it via cut command.
cut -c 2-5 file > temp1.txt
cut -c 11-20 file >... (9 Replies)
Discussion started by: daytripper1021
9 Replies
5. Shell Programming and Scripting
I'm sorting files from a source directory by size into 4 categories then copying them into 4 corresponding folders, just wondering if there's a faster/better/more_elegant way to do this:
find /home/user/sourcefiles -type f -size -400000k -exec /bin/cp -uv {} /home/user/medfiles/ \;
find... (0 Replies)
Discussion started by: unclecameron
0 Replies
6. Shell Programming and Scripting
Hi,
I have a script below for extracting xml from a file.
for i in *.txt
do
echo $i
awk '/<.*/ , /.*<\/.*>/' "$i" | tr -d '\n'
echo -ne '\n'
done
.
I read about using multi threading to speed up the script.
I do not know much about it but read it on this forum.
Is it a... (21 Replies)
Discussion started by: chetan.c
21 Replies
7. Shell Programming and Scripting
awk "/May 23, 2012 /,0" /var/tmp/datafile
the above command pulls out information in the datafile. the information it pulls is from the date specified to the end of the file.
now, how can i make this faster if the datafile is huge? even if it wasn't huge, i feel there's a better/faster way to... (8 Replies)
Discussion started by: SkySmart
8 Replies
8. UNIX for Dummies Questions & Answers
Hello guys,
I'm cleaning out big XML files (we're talking about 1GB at least), most of them contain words written in a non-latin alphabet.
The command I'm using is so slow it's not even funny:
cat $1 | sed -e :a -e 's/<*>//g;/</N;//ba;s/</ /g;s/>/... (4 Replies)
Discussion started by: bobylapointe
4 Replies
9. Shell Programming and Scripting
Hi,
I have a large number of input files with two columns of numbers.
For example:
83 1453
99 3255
99 8482
99 7372
83 175
I only wish to retain lines where the numbers fullfil two requirements. E.g:
=83
1000<=<=2000
To do this I use the following... (10 Replies)
Discussion started by: s052866
10 Replies
10. Shell Programming and Scripting
I have the below command which is referring a large file and it is taking 3 hours to run. Can something be done to make this command faster.
awk -F ',' '{OFS=","}{ if ($13 == "9999") print $1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12 }' ${NLAP_TEMP}/hist1.out|sort -T ${NLAP_TEMP} |uniq>... (13 Replies)
Discussion started by: Peu Mukherjee
13 Replies
LEARN ABOUT SUSE
xfs_copy
xfs_copy(8) System Manager's Manual xfs_copy(8)
NAME
xfs_copy - copy the contents of an XFS filesystem
SYNOPSIS
xfs_copy [ -bd ] [ -L log ] source target1 [ target2 ... ]
DESCRIPTION
xfs_copy copies an XFS filesystem to one or more targets in parallel (see xfs(5)). The first (source) argument must be the pathname of the
device or file containing the XFS filesystem. The remaining arguments specify one or more target devices or file names. If the pathnames
specify devices, a copy of the source XFS filesystem is created on each device. The target can also be the name of a regular file, in which
case an image of the source XFS filesystem is created in that file. If the file does not exist, xfs_copy creates the file. The length of
the resulting file is equal to the size of the source filesystem. However, if the file is created on an XFS filesystem, the file consumes
roughly the amount of space actually used in the source filesystem by the filesystem and the XFS log. The space saving is because xfs_copy
seeks over free blocks instead of copying them and the XFS filesystem supports sparse files efficiently.
xfs_copy should only be used to copy unmounted filesystems, read-only mounted filesystems, or frozen filesystems (see xfs_freeze(8)). Oth-
erwise, the generated filesystem(s) would be inconsistent or corrupt.
xfs_copy does not alter the source filesystem in any way. Each new (target) filesystem is identical to the original filesystem except that
new filesystems each have a new unique filesystem identifier (UUID). Therefore, if both the old and new filesystems will be used as sepa-
rate distinct filesystems, xfs_copy or xfsdump(8)/xfsrestore(8) should be used to generate the new filesystem(s) instead of dd(1) or other
programs that do block-by-block disk copying.
xfs_copy uses synchronous writes to ensure that write errors are detected.
xfs_copy uses pthreads(7) to perform simultaneous parallel writes. xfs_copy creates one additional thread for each target to be written.
All threads die if xfs_copy terminates or aborts.
OPTIONS
-d Create a duplicate (true clone) filesystem. This should be done only if the new filesystem will be used as a replacement for the
original filesystem (such as in the case of disk replacement).
-b The buffered option can be used to ensure direct IO is not attempted to any of the target files. This is useful when the filesystem
holding the target file does not support direct IO.
-L log Specifies the location of the log if the default location of /var/tmp/xfs_copy.log.XXXXXX is not desired.
DIAGNOSTICS
xfs_copy reports errors to both stderr and in more detailed form to a generated log file whose name is of the form
/var/tmp/xfs_copy.log.XXXXXX or a log file specified by the -L option. If xfs_copy detects a write error on a target, the copy of that one
target is aborted and an error message is issued to both stderr and the log file, but the rest of the copies continue. When xfs_copy termi-
nates, all aborted targets are reported to both stderr and the log file.
If all targets abort or if there is an error reading the source filesystem, xfs_copy immediately aborts.
xfs_copy returns an exit code of 0 if all targets are successfully copied and an exit code of 1 if any target fails.
NOTES
When moving filesystems from one disk to another, if the original filesystem is significantly smaller than the new filesystem, and will be
made larger, we recommend that mkfs.xfs(8) and xfsdump(8)/xfsrestore(8) be used instead of using xfs_copy and xfs_growfs(8). The filesys-
tem layout resulting from using xfs_copy/xfs_growfs is almost always worse than the result of using mkfs.xfs/xfsdump/xfsrestore but in the
case of small filesystems, the differences can have a significant performance impact. This is due to the way xfs_growfs(8) works, and not
due to any shortcoming in xfs_copy itself.
CAVEATS
xfs_copy does not copy XFS filesystems that have a real-time section or XFS filesystems with external logs. In both cases, xfs_copy aborts
with an error message.
SEE ALSO
mkfs.xfs(8), xfsdump(8), xfsrestore(8), xfs_freeze(8), xfs_growfs(8), xfs(5).
xfs_copy(8)