UNIX for Dummies Questions & Answers

Hi,

How to loop through all the subdirectories in a directory, merge the files present in it to a single file in that subdirectory itself and remove the original files?

Please advise.

hmm... lemme give it a shot:
you can combine the files like this

ls -1R|grep :$|sed 's/:$//g'| awk '{print $1"/*"}'|xargs cat > bigfile

edit:
changed output method

as for removing, i'd probably suggest a different location for the bigfile and to rm the full path.

ie if you have:

/jobname/logs/(lots of dated directories)

do:
ls -1R /jobname/logs|grep :$|sed 's/:$//g'| awk '{print $1"/*"}'|xargs cat > /jobname/bigfile

then rm -r /jobname/logs/*
or something similar

Thanks NYankz for your quick reply but I didn't get it & also didn't work.Can you please let me know what are you doing there.

And, here is what I have done.

Code:

origloc=/original/path/location
destloc=/destination/path/location
cd ${origloc}
find . -type f -name "*.log" | \
while read file
do
   destdir=${destloc}/`dirname ${file}`
   destfile=`basename ${file}`_OK
   [ ! -d ${destdir} ] && mkdir ${destdir}
   cp ${file} ${destdir}/${destfile}
   cat ${destdir}/${destfile} | tr -d '\n' >> ${destdir}/merged.txt
   rm -rf ${destdir}/${destfile}
   done

It works fine merging all the log files in each directory to a single one called merged.txt & then removing those files. But now, I am again stuck with other problem.
The above script looks into each directory and merges the files but how to check only for certain subdirectories and not all of them.

e.g. root has 9 subdir with names like subdir-1-subdir-9.I want to do this only for subdir 2,4,6.How should I go for that.

I'll like if soemone posts different view on my original problem so that I can learn soemthing new.

Thanks
er_ashu

ya i keep forgeting you don't get output from awk '{system("cat"w/e)}' i edited my post

ls -1R|grep :$|sed 's/:$//g'| awk '{print $1"/*"}'|xargs cat > bigfile


ls -1R #recursively searches all files in all subdirectories

grep :$ #will show all lines which end in :, this is a list of all the sub directories.

sed 's/:$//g' #removes the : from the end so you only get the full path of the directories, at this point if you want to limit which sub directories you can put in a grep statement.

awk '{print $1"/*"}' #formats the directory path to include all files in the sub directory you can change that to: awk '{print $1/"*.log"}' if you want log files only.

xargs cat #will run all of the arguments through cat which are piped into it, at this point you'd be running cat on all the files in all your sub directories

then obviously > will pipe it to a file


*edit* and yes i'm very bored at work.

Many thanks........