Sponsored Content
Operating Systems HP-UX Faster command for file copy than cp ? Post 302596004 by methyl on Monday 6th of February 2012 08:10:23 AM
Old 02-06-2012
Quote:
we have 30 GB files on our filesystem which we need to copy daily to 25 location on the same machine (but different filesystem).
This is ambiguous.

What version of HP-UX do you have?
How many files are you copying?
How often are you copying these files?
How big is the largest file? Is every file always smaller than 2 Gigabytes?
What is the filesystem type? Is NFS involved or are they all local discs?
Are you actually copying all of the files 25 times, or just copying each file once to one of 25 different directories?
What does "location" mean in unix terms?
Are these files from a recognised package (e.g. Oracle Archive logs)?
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

command faster in crontab..

Hi all you enlightened unix people, I've been trying to execute a perl script that contains the following line within backticks: `grep -f patternfile.txt otherfile.txt`;It takes normally 2 minutes to execute this command from the bash shell by hand. I noticed that when i run this command... (2 Replies)
Discussion started by: silverlocket
2 Replies

2. Shell Programming and Scripting

How to make copy work faster

I am trying to copy a folder which contains a list of C executables. It takes 2 mins for completion,where as the entire script takes only 3 more minutes for other process. Is there a way to copy the folder faster so that the performance of the script will improve? (2 Replies)
Discussion started by: prasperl
2 Replies

3. UNIX for Dummies Questions & Answers

Which command will be faster? y?

i)wc -c/etc/passwd|awk'{print $1}' ii)ls -al/etc/passwd|awk'{print $5}' (4 Replies)
Discussion started by: karthi_g
4 Replies

4. Shell Programming and Scripting

**HELP** need to split this line faster than cut-command

Hi, A datafile containing lines such as below needs to be split: 500000000000932491683600000000000000000000000000016800000GS0000000000932491683600*HOME I need to get the 2-5, 11-20, and 35-40 characters and I can do it via cut command. cut -c 2-5 file > temp1.txt cut -c 11-20 file >... (9 Replies)
Discussion started by: daytripper1021
9 Replies

5. Shell Programming and Scripting

faster command than find for sorting?

I'm sorting files from a source directory by size into 4 categories then copying them into 4 corresponding folders, just wondering if there's a faster/better/more_elegant way to do this: find /home/user/sourcefiles -type f -size -400000k -exec /bin/cp -uv {} /home/user/medfiles/ \; find... (0 Replies)
Discussion started by: unclecameron
0 Replies

6. Shell Programming and Scripting

Multi thread awk command for faster performance

Hi, I have a script below for extracting xml from a file. for i in *.txt do echo $i awk '/<.*/ , /.*<\/.*>/' "$i" | tr -d '\n' echo -ne '\n' done . I read about using multi threading to speed up the script. I do not know much about it but read it on this forum. Is it a... (21 Replies)
Discussion started by: chetan.c
21 Replies

7. Shell Programming and Scripting

Faster way to use this awk command

awk "/May 23, 2012 /,0" /var/tmp/datafile the above command pulls out information in the datafile. the information it pulls is from the date specified to the end of the file. now, how can i make this faster if the datafile is huge? even if it wasn't huge, i feel there's a better/faster way to... (8 Replies)
Discussion started by: SkySmart
8 Replies

8. UNIX for Dummies Questions & Answers

A faster equivalent for this sed command

Hello guys, I'm cleaning out big XML files (we're talking about 1GB at least), most of them contain words written in a non-latin alphabet. The command I'm using is so slow it's not even funny: cat $1 | sed -e :a -e 's/&lt;*&gt;//g;/&lt;/N;//ba;s/</ /g;s/>/... (4 Replies)
Discussion started by: bobylapointe
4 Replies

9. Shell Programming and Scripting

Making a faster alternative to a slow awk command

Hi, I have a large number of input files with two columns of numbers. For example: 83 1453 99 3255 99 8482 99 7372 83 175 I only wish to retain lines where the numbers fullfil two requirements. E.g: =83 1000<=<=2000 To do this I use the following... (10 Replies)
Discussion started by: s052866
10 Replies

10. Shell Programming and Scripting

How to make awk command faster?

I have the below command which is referring a large file and it is taking 3 hours to run. Can something be done to make this command faster. awk -F ',' '{OFS=","}{ if ($13 == "9999") print $1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12 }' ${NLAP_TEMP}/hist1.out|sort -T ${NLAP_TEMP} |uniq>... (13 Replies)
Discussion started by: Peu Mukherjee
13 Replies
xfs_copy(8)						      System Manager's Manual						       xfs_copy(8)

NAME
xfs_copy - copy the contents of an XFS filesystem SYNOPSIS
xfs_copy [ -bd ] [ -L log ] source target1 [ target2 ... ] DESCRIPTION
xfs_copy copies an XFS filesystem to one or more targets in parallel (see xfs(5)). The first (source) argument must be the pathname of the device or file containing the XFS filesystem. The remaining arguments specify one or more target devices or file names. If the pathnames specify devices, a copy of the source XFS filesystem is created on each device. The target can also be the name of a regular file, in which case an image of the source XFS filesystem is created in that file. If the file does not exist, xfs_copy creates the file. The length of the resulting file is equal to the size of the source filesystem. However, if the file is created on an XFS filesystem, the file consumes roughly the amount of space actually used in the source filesystem by the filesystem and the XFS log. The space saving is because xfs_copy seeks over free blocks instead of copying them and the XFS filesystem supports sparse files efficiently. xfs_copy should only be used to copy unmounted filesystems, read-only mounted filesystems, or frozen filesystems (see xfs_freeze(8)). Oth- erwise, the generated filesystem(s) would be inconsistent or corrupt. xfs_copy does not alter the source filesystem in any way. Each new (target) filesystem is identical to the original filesystem except that new filesystems each have a new unique filesystem identifier (UUID). Therefore, if both the old and new filesystems will be used as sepa- rate distinct filesystems, xfs_copy or xfsdump(8)/xfsrestore(8) should be used to generate the new filesystem(s) instead of dd(1) or other programs that do block-by-block disk copying. xfs_copy uses synchronous writes to ensure that write errors are detected. xfs_copy uses pthreads(7) to perform simultaneous parallel writes. xfs_copy creates one additional thread for each target to be written. All threads die if xfs_copy terminates or aborts. OPTIONS
-d Create a duplicate (true clone) filesystem. This should be done only if the new filesystem will be used as a replacement for the original filesystem (such as in the case of disk replacement). -b The buffered option can be used to ensure direct IO is not attempted to any of the target files. This is useful when the filesystem holding the target file does not support direct IO. -L log Specifies the location of the log if the default location of /var/tmp/xfs_copy.log.XXXXXX is not desired. DIAGNOSTICS
xfs_copy reports errors to both stderr and in more detailed form to a generated log file whose name is of the form /var/tmp/xfs_copy.log.XXXXXX or a log file specified by the -L option. If xfs_copy detects a write error on a target, the copy of that one target is aborted and an error message is issued to both stderr and the log file, but the rest of the copies continue. When xfs_copy termi- nates, all aborted targets are reported to both stderr and the log file. If all targets abort or if there is an error reading the source filesystem, xfs_copy immediately aborts. xfs_copy returns an exit code of 0 if all targets are successfully copied and an exit code of 1 if any target fails. NOTES
When moving filesystems from one disk to another, if the original filesystem is significantly smaller than the new filesystem, and will be made larger, we recommend that mkfs.xfs(8) and xfsdump(8)/xfsrestore(8) be used instead of using xfs_copy and xfs_growfs(8). The filesys- tem layout resulting from using xfs_copy/xfs_growfs is almost always worse than the result of using mkfs.xfs/xfsdump/xfsrestore but in the case of small filesystems, the differences can have a significant performance impact. This is due to the way xfs_growfs(8) works, and not due to any shortcoming in xfs_copy itself. CAVEATS
xfs_copy does not copy XFS filesystems that have a real-time section or XFS filesystems with external logs. In both cases, xfs_copy aborts with an error message. SEE ALSO
mkfs.xfs(8), xfsdump(8), xfsrestore(8), xfs_freeze(8), xfs_growfs(8), xfs(5). xfs_copy(8)
All times are GMT -4. The time now is 02:47 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy