How to loop through all the subdirectories in a directory, merge the files present in it to a single file in that subdirectory itself and remove the original files?
hmm... lemme give it a shot:
you can combine the files like this
ls -1R|grep :$|sed 's/:$//g'| awk '{system("cat "$1"/*)}' > bigfile
Thanks NYankz for your quick reply but I didn't get it & also didn't work.Can you please let me know what are you doing there.
And, here is what I have done.
It works fine merging all the log files in each directory to a single one called merged.txt & then removing those files. But now, I am again stuck with other problem.
The above script looks into each directory and merges the files but how to check only for certain subdirectories and not all of them.
e.g. root has 9 subdir with names like subdir-1-subdir-9.I want to do this only for subdir 2,4,6.How should I go for that.
I'll like if soemone posts different view on my original problem so that I can learn soemthing new.
ls -1R|grep :$|sed 's/:$//g'| awk '{print $1"/*"}'|xargs cat > bigfile
ls -1R #recursively searches all files in all subdirectories
grep :$ #will show all lines which end in :, this is a list of all the sub directories.
sed 's/:$//g' #removes the : from the end so you only get the full path of the directories, at this point if you want to limit which sub directories you can put in a grep statement.
awk '{print $1"/*"}' #formats the directory path to include all files in the sub directory you can change that to: awk '{print $1/"*.log"}' if you want log files only.
xargs cat #will run all of the arguments through cat which are piped into it, at this point you'd be running cat on all the files in all your sub directories
ls -1R|grep :$|sed 's/:$//g'| awk '{print $1"/*"}'|xargs cat > bigfile
ls -1R #recursively searches all files in all subdirectories
grep :$ #will show all lines which end in :, this is a list of all the sub directories.
sed 's/:$//g' #removes the : from the end so you only get the full path of the directories, at this point if you want to limit which sub directories you can put in a grep statement.
awk '{print $1"/*"}' #formats the directory path to include all files in the sub directory you can change that to: awk '{print $1/"*.log"}' if you want log files only.
xargs cat #will run all of the arguments through cat which are piped into it, at this point you'd be running cat on all the files in all your sub directories
I found this awesome sed script here:
https://www.unix.com/shell-programming-scripting/48228-perl-search-string-line-then-search-replace-text.html
sed -i '/MatchText/ s/ReplaceMe/REPLACED/' filename
Question though to save me manually doing this.
How do I do this from a root directory... (6 Replies)
Please help me with a shell script to get the stats on many subdirectories (sub1), (sub2) etc under a mother directory (big)
/big
|
|_______sub1
|_______sub2
|_______sub3
---------
I want to know
1. What is the last file accessed in each subdirectory with date and by whom
2.... (2 Replies)
I have 2 directories a/ and b/,
they have different subdirectories, how to diff with missing file, or missing subdirectory
and if i have in a/ directory "a/ACD/DEF" DEF is a file, but in b/ directory "b/ACD/DEF is a SUBDIRECTORY, how to diff it, thanks
my solution for directories,
... (7 Replies)
Hello,
Can anyone help me figure out how to test if the item in the directory is a subdirectory?
I'm writing a code to copy all the contents of directory1 to directory2, but I want to skip all the subdirectories.
Thanks! (4 Replies)
Here is my question in bash
for f in f1 f2
do
cd $f
cd ??? # i need to enter the two layers of sub folders then find the folder named "abcde" ?
cd .. # how to get out two layers subdirectories?
cd ..
done (3 Replies)
Here is my question in bash
for f in f1 f2
do
cd $f
cd ??? # i need to enter the two layers of sub folders then find the folder named "abcde" ?
cd .. # how to get out two layers subdirectories?
cd ..
done (2 Replies)
Suppose I have two directories a and b. Each directory has a few subdirectories, a1 a2 a3 and b1, b2, b3 respectively.
Using ls, I can see a and b. Then I need cd a, ls, cd ../b, ls to see all the subdirectories.
How to see all the directories and subdirectories easily, say using just one... (2 Replies)
Hello:
Could someone please explain to me how to create a subdirectory in the /home directory. I have tried creating a new user but the default path for a new user is /export/home. I am running Unix 5.8 on a Sun Blade 100. Thanks. (8 Replies)
Dear All,
I want to write the Unix command that searches through all subdirectories, finds the files named ''core'' and deletes them.
I will very much appreciate your help.
David (4 Replies)