Directory Split


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Directory Split
# 1  
Old 09-18-2013
Error Directory Split

Hi. I am trying to come up with a korn shell script to perform a directory split. My directory is on AIX and its 32GB. The folder contents belong to only three files types: pdf, word, tiff and jpeg. I am trying to come up with a script that would create four sub folders each of about 8GB in size, and then tar each of these sub folders created. any help in automating this procedure using a shell script would be much appreciated.
Thank you.
# 2  
Old 09-18-2013
Do the resulting directories need to contain a specific file type or can they be selected randomly as long as each directory does not exceed 8 GBs?
This User Gave Thanks to verdepollo For This Post:
# 3  
Old 09-18-2013
the resulting directories, can contain the files randomly.
# 4  
Old 09-18-2013
A simple approach would be to create the 4 subdirectories in advance and then set an 8 GB quota for each of them.

If we can assume that the four file types are equally distributed in the source directory then we can filter them out by extension. Something like:
Code:
mkdir PDF JPG TIFF WORD
for file in *.pdf; do mv "$file" PDF 2>/dev/null; done && tar -cf PDF.tar PDF
for file in *.doc; do mv "$file" WORD 2>/dev/null; done  && tar -cf WORD.tar WORD
for file in *.tiff; do mv "$file" TIFF 2>/dev/null; done && tar -cf TIFF.tar TIFF
for file in *.jpeg; do mv "$file" JPEG 2>/dev/null; done && tar -cf JPEG.tar JPEG

The above is not really "random", though.

There's however a much simpler option by using dirsplit which comes bundled with cdrkit:
Code:
dirsplit -m -s 8G -f '/\.jpeg/' -p jpeg_ source_directory/ && tar -cf jpeg_files.tar jpeg_*

Unfortunately dirsplit is a Linux thing but you may be able to compile it under AIX (I have had some success with it in Solaris).
# 5  
Old 09-18-2013
Thank you. Can you give me some instructions on how to compile dirsplit under AIX.
# 6  
Old 09-18-2013
In lieu of the for ... loops above you could also use e.g. mv *.pdf PDF (see man mv).
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Split and Rename Split Files

Hello, I need to split a file by number of records and rename each split file with actual filename pre-pended with 3 digit split number. What I have tried is the below command with 2 digit numeric value split -l 3 -d abc.txt F (# Will Produce split Files as F00 F01 F02) How to produce... (19 Replies)
Discussion started by: techedipro
19 Replies

2. Shell Programming and Scripting

List files with date, create directory, move to the created directory

Hi all, i have a folder, with tons of files containing as following, on /my/folder/jobs/ some_name_2016-01-17-22-38-58_some name_0_0.zip.done some_name_2016-01-17-22-40-30_some name_0_0.zip.done some_name_2016-01-17-22-48-50_some name_0_0.zip.done and these can be lots of similar files,... (6 Replies)
Discussion started by: charli1
6 Replies

3. Shell Programming and Scripting

Shell scripting-I need a script which should watch a directory for a file with specific directory

I need a script which should watch a directory for a file with specific directory. If it finds a file in directory, it should search for few specific keyword in the file. if the keyword exists, it should trim string from specific column. The file should be moved to another directory and the a... (8 Replies)
Discussion started by: akashdeepak
8 Replies

4. UNIX for Dummies Questions & Answers

Extract directory name from the full directory path in UNIX using shell scripting

My input is as below : /splunk/scrubbed/rebate/IFIND.REBTE.WROC.txt /splunk/scrubbed/rebate/IFIND.REBTE.WROC.txt /splunk/scrubbed/loyal/IFIND.HELLO.WROC.txt /splunk/scrubbed/triumph/ifind.triumph.txt From the above input I want to extract the file names only . Basically I want to... (5 Replies)
Discussion started by: IshuGupta
5 Replies

5. Web Development

Directory index forbidden by Options directive error on specific directory with indexing disabled

I am seeing the following error appear numerous times in my Apache error log: I have my Apache config configured as below, so I would expect indexing not to occur on this directory as it falls under the parent /web directory. Strangely all the IP address, including this example, all... (5 Replies)
Discussion started by: crmpicco
5 Replies

6. Shell Programming and Scripting

awk to split one field and print the last two fields within the split part.

Hello; I have a file consists of 4 columns separated by tab. The problem is the third fields. Some of the them are very long but can be split by the vertical bar "|". Also some of them do not contain the string "UniProt", but I could ignore it at this moment, and sort the file afterwards. Here is... (5 Replies)
Discussion started by: yifangt
5 Replies

7. Shell Programming and Scripting

Grepping file names, comparing them to a directory of files, and moving them into a new directory

got it figured out :) (1 Reply)
Discussion started by: sHockz
1 Replies

8. UNIX for Dummies Questions & Answers

Directory Split

Hello again unix.com. How can I split a directory in 100 other ones. I have the directory "example.com" and I want to make 100 like it with names 1.example.com, 2.example.com, 3.example.com ... to 100.example.com. Thanks. (6 Replies)
Discussion started by: galford
6 Replies

9. UNIX for Dummies Questions & Answers

Split a file with no pattern -- Split, Csplit, Awk

I have gone through all the threads in the forum and tested out different things. I am trying to split a 3GB file into multiple files. Some files are even larger than this. For example: split -l 3000000 filename.txt This is very slow and it splits the file with 3 million records in each... (10 Replies)
Discussion started by: madhunk
10 Replies
Login or Register to Ask a Question