Shell Programming and Scripting

I have 4 directory

Dir1
file1 file2 file3 file4
Dir2
file3 file5 file6 file8
Dir3
file1 file2 file6 file9 file10
Dir4
file3 file6 file12 file15

and all the 4 dirs are having couple of files.
Few of the files are common to other directory/ies and few of them are brand new.

I need a finaloutput as
dir1 having all the files from all other 3 directory but not having its original files same for other dirs. And i need to change the file name in dir1. Actually all the files are having a ext as .text so i need to remove the .text from all the files.

eg
dir1 file5 file6 file8 file9 file10 file12 file15
dir2 file1 file2 file4 file9 file10 file12 file15


I am doing like this

newdir1 - copying all the files from dir2 dir3 and dir4
then delete the files from newdir1 which are there in dir1
so this way newdir1 is my final result.
Doing the same for other directory

>mkdir newdir1
> cp dir2/* ./newdir1
> cp dir3/* ./newdir1
> cp dir4/* ./newdir1

ls dir1 | while read file; do
rm ./newdir1/${file}
done
cd newdir1

ls | while read file; do
mv ${file} `echo ${file} | sed 's/\.txt//'`
done



Is there any other best and fast way to do? Actually i need some way which can be much more efficient than this instead of using 2 while loops as i have more than 5000 files in each folder and most of them are common across each folder only 20 are new files.

can i do this through file manipulation means instead of working on actual file if i make a file with all the filename and then doing some commands on names instead of actual file and then finally copying the actual file

This is untested, but it should be close:

Code:

#! /usr/bin/ksh
for newdir in newdir1 newdir2 newdir3 newdir4 ; do
      olddir=${newdir#new}
      mkdir $newdir
      for dir in dir1 dir2 dir3 dir3 ; do
               if [[ $dir != $olddir ]] ; then
                    cd $dir
                     ls | while read file ; do
                            newfile=${file%.txt}
                            [[ -f ../${newdir}/$newfile || -f ../${olddir}/$file ]] || cp $file ../${newdir}/${newfile}
                     done
                     cd ..
              fi
       done
done
exit 0

This way you copy the file to $newdir only if it is not already there. Also the presense of the file in the old directory will inhibit the copy. From what you said, this will save copying thousands of files. Now no rm step is needed. The files renamed as they were copied, no separate rename step needed either.