Move Large files


 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers Move Large files
# 1  
Old 12-14-2006
Move Large files

Hi,

Im running (on AIX 5.2) a script which moves 16 large files (up to 1 Go each).
It uses the 'mv' command and runs during 30 minutes. I would like to improve this by using something else faster. Any ideas ?
Thx in advance !

T
# 2  
Old 12-14-2006
If there was a faster way to move files, wouldn't mv do it the faster way? Maybe I'm missing something...
# 3  
Old 12-14-2006
Code:
for i in f1 f2 f3 f4 f5 f6 f7 f8 f9 f10 f11 f12 f13 f14 f15 f16
do
   mv $i /destination/"$i"  &
done
wait

# 4  
Old 12-14-2006
Thx for your replies. Actually, I am looking for sthing faster than mv command... I heard of mv large file aware.
# 5  
Old 12-15-2006
gzip the files before moving ?
# 6  
Old 12-15-2006
Computer

I agreed you can Gzip and move will improve performance and again you can gunzip or gzip -d option will do to uncrompress
# 7  
Old 12-15-2006
Er, won't the gzip/compress take time to zip the file in the first place? And largefile aware mv only means that it can handle files larger than 2GB in size. I doubt if it means that it speeds up the actual move step of files.
 
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Move files with a certain suffix based on how many files are in another folder

Hello, First time poster. I am looking for a way to script or program the process of moving files from one folder to another, automatically, based on the count of files in the destination folder. I was thinking a shell script would work, but am open to the suggestions of the experts... (6 Replies)
Discussion started by: comtech
6 Replies

2. Shell Programming and Scripting

SBATCH trinity for multiple files and rename/move the output files

Hey guys, I have wrote the following script to apply a module named "trinity" on my files. (it takes two input files and spit a trinity.fasta as output) #!/bin/bash -l #SBATCH -p node #SBATCH -A <projectID> #SBATCH -n 16 #SBATCH -t 7-00:00:00 #SBATCH --mem=128GB #SBATCH --mail-type=ALL... (1 Reply)
Discussion started by: @man
1 Replies

3. Shell Programming and Scripting

Move all files except sys date (today) files in Solaris 10

I want to move all files from one directory to another directory excluding today (sysdate files) on daily basis. file name is in pattern file_2013031801, file_2013031802 etc (2 Replies)
Discussion started by: khattak
2 Replies

4. Shell Programming and Scripting

Recursively move directories along with files/specific files

I would like to transfer all files ending with .log from /tmp and to /tmp/archive (using find ) The directory structure looks like :- /tmp a.log b.log c.log /abcd d.log e.log When I tried the following command , it movies all the log files... (8 Replies)
Discussion started by: frintocf
8 Replies

5. UNIX for Dummies Questions & Answers

Move same files and issue ls -al command on remaining files

I know I can use an ls -l junk1 command to get a listing of all files in the directory junk1, but I was wondering how I'd go about going through the files in junk1 in a for-in loop and issuing the ls -l command on them one by one. This is what I have so far: for file in $(ls -a $1) do ls... (1 Reply)
Discussion started by: Trinimini
1 Replies

6. AIX

Large Data move from Win to AIX

Hi All, I have a requirement to move a large amount of data *approx 160 GB* from a windows formatted SAN to AIX. The data is made up of several hundred thousand small files. Does anyone have a recommendation for the best tool or best procedure for this? Regards, Neil (2 Replies)
Discussion started by: stewartna77
2 Replies

7. Solaris

How to safely copy full filesystems with large files (10Gb files)

Hello everyone. Need some help copying a filesystem. The situation is this: I have an oracle DB mounted on /u01 and need to copy it to /u02. /u01 is 500 Gb and /u02 is 300 Gb. The size used on /u01 is 187 Gb. This is running on solaris 9 and both filesystems are UFS. I have tried to do it using:... (14 Replies)
Discussion started by: dragonov7
14 Replies

8. Shell Programming and Scripting

Divide large data files into smaller files

Hello everyone! I have 2 types of files in the following format: 1) *.fa >1234 ...some text... >2345 ...some text... >3456 ...some text... . . . . 2) *.info >1234 (7 Replies)
Discussion started by: ad23
7 Replies

9. AIX

Large file move from one server to another

I Have two AIX 5.3 servers. I want to move a 45GB file from Server B to Server A. Can I achieve this with using Tivoli? Or how could I achieve this, by mounting a directory from server B to server A? Any help apreciated! (14 Replies)
Discussion started by: tfort73
14 Replies
Login or Register to Ask a Question