01-06-2012
Faster command for file copy than cp ?
we have 30 GB files on our filesystem which we need to copy daily to 25 location on the same machine (but different filesystem).
cp is taking 20 min to do the copy and we have 5 different thread doing the copy.
so in all its taking around 2 hr and we need to reduce it.
Is there any other command which will copy the files faster.
Or any other way in which we can copy the files.
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
Hi all you enlightened unix people,
I've been trying to execute a perl script that contains the following line within backticks:
`grep -f patternfile.txt otherfile.txt`;It takes normally 2 minutes to execute this command from the bash shell by hand.
I noticed that when i run this command... (2 Replies)
Discussion started by: silverlocket
2 Replies
2. Shell Programming and Scripting
I am trying to copy a folder which contains a list of C executables.
It takes 2 mins for completion,where as the entire script takes only 3 more minutes for other process.
Is there a way to copy the folder faster so that the performance of the script will improve? (2 Replies)
Discussion started by: prasperl
2 Replies
3. UNIX for Dummies Questions & Answers
i)wc -c/etc/passwd|awk'{print $1}'
ii)ls -al/etc/passwd|awk'{print $5}' (4 Replies)
Discussion started by: karthi_g
4 Replies
4. Shell Programming and Scripting
Hi,
A datafile containing lines such as below needs to be split:
500000000000932491683600000000000000000000000000016800000GS0000000000932491683600*HOME
I need to get the 2-5, 11-20, and 35-40 characters and I can do it via cut command.
cut -c 2-5 file > temp1.txt
cut -c 11-20 file >... (9 Replies)
Discussion started by: daytripper1021
9 Replies
5. Shell Programming and Scripting
I'm sorting files from a source directory by size into 4 categories then copying them into 4 corresponding folders, just wondering if there's a faster/better/more_elegant way to do this:
find /home/user/sourcefiles -type f -size -400000k -exec /bin/cp -uv {} /home/user/medfiles/ \;
find... (0 Replies)
Discussion started by: unclecameron
0 Replies
6. Shell Programming and Scripting
Hi,
I have a script below for extracting xml from a file.
for i in *.txt
do
echo $i
awk '/<.*/ , /.*<\/.*>/' "$i" | tr -d '\n'
echo -ne '\n'
done
.
I read about using multi threading to speed up the script.
I do not know much about it but read it on this forum.
Is it a... (21 Replies)
Discussion started by: chetan.c
21 Replies
7. Shell Programming and Scripting
awk "/May 23, 2012 /,0" /var/tmp/datafile
the above command pulls out information in the datafile. the information it pulls is from the date specified to the end of the file.
now, how can i make this faster if the datafile is huge? even if it wasn't huge, i feel there's a better/faster way to... (8 Replies)
Discussion started by: SkySmart
8 Replies
8. UNIX for Dummies Questions & Answers
Hello guys,
I'm cleaning out big XML files (we're talking about 1GB at least), most of them contain words written in a non-latin alphabet.
The command I'm using is so slow it's not even funny:
cat $1 | sed -e :a -e 's/<*>//g;/</N;//ba;s/</ /g;s/>/... (4 Replies)
Discussion started by: bobylapointe
4 Replies
9. Shell Programming and Scripting
Hi,
I have a large number of input files with two columns of numbers.
For example:
83 1453
99 3255
99 8482
99 7372
83 175
I only wish to retain lines where the numbers fullfil two requirements. E.g:
=83
1000<=<=2000
To do this I use the following... (10 Replies)
Discussion started by: s052866
10 Replies
10. Shell Programming and Scripting
I have the below command which is referring a large file and it is taking 3 hours to run. Can something be done to make this command faster.
awk -F ',' '{OFS=","}{ if ($13 == "9999") print $1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12 }' ${NLAP_TEMP}/hist1.out|sort -T ${NLAP_TEMP} |uniq>... (13 Replies)
Discussion started by: Peu Mukherjee
13 Replies
NCOPY(1) ncopy NCOPY(1)
NAME
ncopy - NetWare file copy
SYNOPSIS
ncopy -V
ncopy [ -vmMnpptu ] [ -s amount ] file destinationfile|directory
ncopy [ -vmMnpptu ] [ -s amount ] file1 [ file2 ... ] directory
ncopy -r [ -vmMnpptu ] [ -s amount ] srcdir dstdir
DESCRIPTION
With ncopy you can copy files to different locations on a single NetWare file server without generating excess network traffic. The pro-
gram uses a NetWare function to do the copy rather than transferring the file across the network for both the read and write.
If the last argument is a directory, ncopy will copy the source file(s) into the directory. If only two files are given and the last argu-
ment is not a directory, it will copy the source file to the destination file.
If the source and destination files are not on the same NetWare server (or are not on NetWare servers at all), ncopy will do a normal file
copy.
OPTIONS
-V
Show version number and exit
-v
Verbose copy. Will show current file and percentage completion.
-m
Copy MAC resource fork. Copies MAC resource fork together with data fork.
-M
Copy MAC resource fork to/from non-MAC filesystem. It expects/creates resource forks in subdirectory .rsrc of each directory copied.
If you want to copy files from MAC volume to .rsrc scheme, you must specify both options, -mM. It is not possible to create .rsrc direc-
tory on MAC-aware volume in one step, you must first copy data to non-MAC media using ncopy -mM and then copy them back using ncopy -M.
If you want to copy files from .rsrc scheme on MAC volume to real MAC multiple-forks file, you must first copy data to non-MAC filesys-
tem using ncopy -M and then copy them back using ncopy -mM.
-n
Nice NetWare copy. Will sleep for a second between copying blocks on the NetWare server. Gives other people a chance to do some work
on the NetWare server when you are copying large files. This has no effect if you are not copying on a NetWare server.
-s amount
Nice time slice factor. Used in conjunction with the -n option, this specifies the number of 100K blocks to copy before sleeping.
Default is 10. (1 Megabyte)
-p
Preserve file attributes and date/time during copy.
-pp
Preserve file attributes, date/time and owner during copy. Name of owner is preserved, not owner ID.
-t
Preserve trustees during copy. Trustee name is preserved, not ID.
-r
Perform recursive copy.
-u
Perform copy only if mtime or size differs.
BUGS
ncopy does not preserve long (MAC, NFS, FTAM, OS2) names during copy.
SEE ALSO
ncpmount(8), ncpumount(8)
CREDITS
ncopy was written by Brian G. Reid (breid@tim.com) and Tom C. Henderson (thenderson@tim.com). Many thanks to Volker Lendecke
(lendecke@math.uni-goettingen.de) for the ncpfs and ncplib which made ncopy possible. Some further work was done by Petr Vandrovec (van-
drove@vc.cvut.cz).
ncopy 17/03/1996 NCOPY(1)