Shell Programming and Scripting

Good morning.

I am trying to write a script on SunOS that will tar up files from a set of directories. This is no problem, but what I need to do is to create 3 different archives each containing a third of the files from a particular sub-directory.

So if I have a directory structure:

masterdir

-0
--dir
---dir
----0
----1
----2
----3

-1
--dir
---dir
----0
----1
----2
----3

-2
--dir
---dir
----0
----1
----2
----3

So I have a directory with 10 subdirectories labeled 0-9 and in each one a series of subdirectories labeled 0-1000 containing a series of files.

I need to create archives with identical directory structures and each one containing approximately 1/3 of the files from each leaf directory.

Any suggestions on the best way to accomplish this?

Thanks in advance

Is the list of files stable, or might new files appear while you are processing the files you have so far?

I would try to come up with a system for dividing the files (maybe just a simple grep expression) and once you are satisified with that, feed it to | xargs tar cf -

If you have numeric file names, you could simply sort by the last digit modulo three. If the list of files doesn't change, you could even just categorize by line number within the list, but then you won't get the same files next time you run this on a set of files which is almost, but not quite identical.

Code:

awk 'NR % 3 == 0'  list>set1
awk 'NR % 3 == 1' list >set2
awk 'NR % 3 == 2' list >set3

Here, I suppose you could have run a find command to generate the file list -- if the list of files will not change while you are doing this, you could re-run find three times, if you want to avoid using a temporary file.

Code:

for m in 0 1 2 ; do
  find -what -ever | awk -v m=$m 'NR % 3 == m' | xargs tar cf tar$m.tar
done

Instead of tar, you could run it with wc to see that the number of files is roughly similar for all three runs, before you proceed to actually tar them up.

There may be architectures where xargs or tar are too crippled for this, but if you can use this, it's simple and straightforward.