I use the below script that will find n record the entire directory structure for a given folder.
The problem is the folder i m running this script under where i want it to record all the directory structure has millions of files. This causes the below command to run for 4 hrs now with no results.
Apart from the slowness issue I see that my script above does not record all but only some of the folders structure under the directory it runs
Can you help fix this script ?
Last edited by mohtashims; 08-22-2015 at 06:56 AM..
If you want to record a list of the directories in the file hierarchy rooted in your current working directory, the command you want would be:
If you are trying to change the mode of every file (including all of the directories and all of the regular files) in the file hierarch rooted in your current working directory, the command you want would be:
(but that would be a TERRIBLE thing to do from a security standpoint).
If you are planning to run the shell script created by you current script (moht.sh), then be aware that all of the mkdir commands will fail (since the directories already exist),
and each chmod -R directory after the first one (chmod -R .) is duplicating work done by the first one (and the 1st one will take a LONG time if there are millions of files and subsequent chmod commands (repeating the work already done by the first one) will waste time proportional to the number of files in the hierarchy rooted in that subdirectory. And if any of your existing directory names contain space, tab, or newline characters; your mkdir commands will create new (presumably unwanted) directories (because you aren't properly quoting the arguments you're passing to mkdir).
The requirement is to create a similar directory only structure on a different server and there are more than 1000 folders and million files.
So i run my script which gives me moht.sh. I then take to moht.sh to the second server and simply run it to create similar directory & sub-directory structure on the other box.
But like i said 1. my script takes unreasonable time to complete and 2. it is missing to record a few directories / sub-directories like i posted in the OP.
mohtashims,
Is this the same system you had problems with November last year (Find command takes too long [too] complete)?
Are there any symbolic links in the hierarchy? These can be found with:
Also, check the manpage for find on your system, the -print was not always enabled by default.
To "record" the directory structure, why not use:
To duplicate the structure use:
I have no means to try that with a really big filesystem but as a wild guess I'd say the reason why this is slow is because of the shell-loop to do the writing to the file.
I suggest you do it this way:
This writes a "mkdir -p $dir" for every dir found and then a "chmod -R 777 *" as the last command. You didn't say that but i supposed you start the directory recreation process in an otherwise empty directory.
When piped to a text processor like sed then the file name should be in quotes:
Same for the shell processor:
Two others that might restore faster:
If you want to copy the permissions, then cpio is the efficient solution, as was proposed by derekludwig.
Hi,
I am trying to search for a Directory called "mont" under a directory path "/opt/app/var/dumps"
Although "mont" is in the very parent directory called "dumps" i.e "/opt/app/var/dumps/mont" and it can never be inside any Sub-Directory of "dumps"; my below find command which also checks... (5 Replies)
Hi,
Below is my find command
find /opt/app/websphere -name myfolder -perm -600 | wc -l
At time it even takes 20 mins to complete.
my OS is : SunOS mypc 5.10 Generic_150400-09 sun4v sparc SUNW,T5440 (10 Replies)
find /tmp -type f -mtime +180
I have this script get the list to clean up files older than 180 days under /tmp.
But, I want to make sure to grep only a type of files, which have only 6 character long.
....
LT3hqa dRMoya ... (16 Replies)
I was trying to do some experiment with "sed".
I want to find the filenames which are three characters. So, this is what I have done to search for it, using sed.
sed -n -e '/^\{3\}$/p' test
This returns the correct output for characters. But if I make change, let's say i create 2 more... (4 Replies)
Dear All,
OS = Solaris 5.10
Hardware Sun Fire T2000 with 1 Ghz quode core
We have oracle application 11i with 10g database. When ever i am trying to take cold backup of database with 55GB size its taking long time to finish. As the application is down nobody is using the server at all... (8 Replies)
Hi Guys,
I need a help. I have 1130 zip files. Each one of them has files including 1 html file with long file name (includes special charactors, Alphabetic and numbers).
I have copied all 1130 zip files to my linux system and extracted using below command.
Find . -name "*.zip" -exec... (7 Replies)
Hi ,
I am trying to find some files on a remote machine using the find command.
>ssh -q atukuri@remotehostname find /home/atukuri/ -name abc.txt
/home/atukuri/abc.txt
The above command works fine and lists the file, but if I want to do a long listing of files (ls -l) its not working . ... (2 Replies)
The purpose of those comands are to find the newest file in a directory acvrdind to system date, and it has to be recursively found in each directory.
The problem is that i want to list in a long format every found file, but the commands i use produce unexpected results ,so the output lists in a... (5 Replies)
If I have very large text file
***************
***************
***************
***************
***************
ABC-sdfsdf
BBB-xk[ptr';
CCC-sdfolb
ABC-dltg'fl;l
My aim:
-> tail last 10 lines from large text. ***** Ok *****
-> Change first 3 charactors which begin with "ABC" to "abc".... (1 Reply)
Hello,
I create a file touch 1201093003 fichcomp
and inside a repertory (which hava a lot of files) I want to list all files created before this file :
find *.* \! -maxdepth 1 - newer fichcomp but this command returned bash: /usr/bin/find: Argument list too long
but i make a filter all... (1 Reply)