Copy a number of files to a directory, then more to another


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Copy a number of files to a directory, then more to another
# 1  
Old 07-13-2012
Copy a number of files to a directory, then more to another

I can't find how to do this.

I want to take a bulk of files, and copy/move a specific number of them (say 1000) to a newly created directory. Once that directory is full, I want to create a new folder and copy/move another batch of files, and so on.

Seems like there should be an easy way to do this, but it eludes me.
# 2  
Old 07-13-2012
How will you quantify to be able to say directory full?
# 3  
Old 07-13-2012
When it reaches a certain number of files, like 1000.

Sorry if I wasn't clear enough, and thanks for the reply.
# 4  
Old 07-13-2012
Code:
# you need a list of the file names you want to move, there are limits to using *
cd /path/to/bulk/files
ls > /tmp/bulk.lis
cnt=0
fcnt=0
newdir=/path/to/new/files${cnt}
mkdir $newdir
while read fname
do
     fcnt=$(( $fcnt + 1 ))
     t=$(( $fcnt % 1000 ))
     if [ $t -eq 0 ] ; then
        cnt=$(( $cnt + 1 ))
        newdir=/path/to/new/files${cnt}
        mkdir $newdir
     fi
     mv $fname $newdir/$(basename $fname)
done <  /tmp/bulk.lis

Test that and see if it meets your needs.
# 5  
Old 07-13-2012
Thank you, it worked pretty good.

It didn't move some of the files though. There are about 400 some files left over. A simple cut and paste fixed that.

I think it might be that they had a space in the name. How would I fix that?
# 6  
Old 07-13-2012
Code:
mv "$fname" "$newdir/$(basename "$fname")"

# 7  
Old 07-16-2012
That seemed to fix it. Many thanks!

Last edited by twjolson; 07-17-2012 at 01:08 PM..
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Copy the files in directory and sub folders as it is to another directory.

How to copy files from one directory to another directory with the subfolders copied. If i have folder1/sub1/sub2/* it needs to copy files to folder2/sub1/sub2/*. I do not want to create sub folders in folder2. Can copy command create them automatically? I tried cp -a and cp -R but did... (4 Replies)
Discussion started by: santosh2626
4 Replies

2. Shell Programming and Scripting

How to count number of files in directory and write to new file with number of files and their name?

Hi! I just want to count number of files in a directory, and write to new text file, with number of files and their name output should look like this,, assume that below one is a new file created by script Number of files in directory = 25 1. a.txt 2. abc.txt 3. asd.dat... (20 Replies)
Discussion started by: Akshay Hegde
20 Replies

3. Red Hat

Unable to copy files due to many files in directory

I have directory that has some billion file inside , i tried copy some files for specific date but it's always did not respond for long time and did not give any result.. i tried everything with find command and also with xargs.. even this command find . -mtime -2 -print | xargs ls -d did not... (2 Replies)
Discussion started by: before4
2 Replies

4. UNIX for Dummies Questions & Answers

Copy files into another directory

I have a folder will a lot of documents (pdf, xls, doc etc.) which users have uploaded but only 20% of them are currently linking from my html files. So my goal is to copy only the files which are linked in my html files from my Document directory into another directory. Eg: My documents exist... (5 Replies)
Discussion started by: ankitha
5 Replies

5. UNIX for Dummies Questions & Answers

How to copy all files into the same directory

Dear All, Again I have another simple question. :confused: I want to write a csh which can copy all files of a current directory with a new name in the same directory, I mean: If I have tree bird apple as files in a directory I want to give ,say number 007 as argument to my csh and it copies... (3 Replies)
Discussion started by: dreamer0085
3 Replies

6. UNIX for Dummies Questions & Answers

Copy directory tree with files

Iam in the process of copying a directory with thousands of directories and files into a new directory. I need to preserve permissions, owner, group, date and timestamps, everything. Iam using AIX and would need help of writing the command whether it is cp-RP or cpio. Apprecaite your... (3 Replies)
Discussion started by: baanprog
3 Replies

7. UNIX for Dummies Questions & Answers

How to find and copy files from one directory to another

Ok i have three directories Destination - /u/dir1 (has subdirectories dir2 which also has subdirectory dir3) Source1 - /u/test/files/dir1/dir2/dir3 Source2 - /u/out/images/dir1/dir2/dir3 What i would like to do is copy everything from Source1 and Source2 into the Destination directory.... (3 Replies)
Discussion started by: ziggy25
3 Replies

8. Solaris

Copy files from the file to another directory

I have created a file that has list of all the files I want to copy into another directory.Is there a way to do it? Thanks In advance (4 Replies)
Discussion started by: shreethik
4 Replies

9. Shell Programming and Scripting

Copy files from one directory to another

Hi when copy the files from one directory to another as like below,it is tried to copy *. as a file. cp /home/rha/*. My objective is to copy all the files (don't care about case sensitive), Thanks in advance for your valuable reply. (1 Reply)
Discussion started by: HAA
1 Replies

10. Shell Programming and Scripting

Copy files from one directory to another

I need to copy about 13 Tb of data from one directory and subdirectories to the other (another mount point). If I run this as a cron, say between 10 pm and 7 am, not all of the files will be copied over. Is there a way of 'resuming' the copy the following evenings until all files are copied over? (0 Replies)
Discussion started by: hd2006
0 Replies
Login or Register to Ask a Question