|
|
Sponsored Content
Operating Systems
Linux
Debian
Problem with maximum files and directories
Post 302952897 by ganon551 on Monday 24th of August 2015 07:07:51 AM
08-24-2015
Problem with maximum files and directories
Hi,
I'm using Kali linux, I think it's a debian linux ?
I'm trying to create a folder in which there'll be 256 folders, and in each of this folders there will also be 256 folders.
Then in each terminate folders I want to create 4096 files.
It will look like /dir/aa/aa/aaa.txt, /dir/aa/aa/aab.txt, and so on.
Problem is when I try to fill the files, I'm using php (which may not be the best idea, but I need some functions such as sha512), so I open a big text file and read it to dispatch every line in its specific file. When I only have the 4096 files (ie. without all the directories before), there's no problem reading and writing, it's going quite fast.
When I try with all the directories, the read/write process takes super long time, like one minute to print one line into a file.
Do I reached the limit of files ? For what I thought, I could create up to 2^32 files in a directory.
Is php struggling ? Will perl or bash be better for that ? If so I need hashing functions that are not in bash.
If you guys have any idea, I'll be glad to hear it.
Thank you in advance
I'm using Kali linux, I think it's a debian linux ?
I'm trying to create a folder in which there'll be 256 folders, and in each of this folders there will also be 256 folders.
Then in each terminate folders I want to create 4096 files.
It will look like /dir/aa/aa/aaa.txt, /dir/aa/aa/aab.txt, and so on.
Problem is when I try to fill the files, I'm using php (which may not be the best idea, but I need some functions such as sha512), so I open a big text file and read it to dispatch every line in its specific file. When I only have the 4096 files (ie. without all the directories before), there's no problem reading and writing, it's going quite fast.
When I try with all the directories, the read/write process takes super long time, like one minute to print one line into a file.
Do I reached the limit of files ? For what I thought, I could create up to 2^32 files in a directory.
Is php struggling ? Will perl or bash be better for that ? If so I need hashing functions that are not in bash.
If you guys have any idea, I'll be glad to hear it.
Thank you in advance
Last edited by rbatte1; 08-25-2015 at 01:01 PM..
|
|
9 More Discussions You Might Find Interesting
1. UNIX for Dummies Questions & Answers
Does Solaris impose limits on :
- the maximum number of files a directory can have,
- total file size in a directory
If there is such limits, how can I can check for each?
Thanks...:confused: (1 Reply)
Discussion started by: deaniyoer
1 Replies
2. UNIX for Dummies Questions & Answers
what is the maximum number ls can list down (6 Replies)
Discussion started by: karnan
6 Replies
3. Linux
Dear friends,
How many maximum number of files and directories can be created in linux system.
Thanks.. (2 Replies)
Discussion started by: karthigayan
2 Replies
4. UNIX for Dummies Questions & Answers
Hi,
Please help me, how to get all the direcotries, its sub directories and its sub directories recursively, need to exclude all the files in the process.
I wanted to disply using a unix command all the directories recursively excluding files.
I tried 'ls -FR' but that display files as... (3 Replies)
Discussion started by: pointers
3 Replies
5. Shell Programming and Scripting
Hi All,
I have this command coded in C Shell to get the top ten sub directories in the order of number of files they contain.
find $parent_dir -type d -exec filecount {} \; | sort -nr | head -10
But it does not seem to show any output. Can someone please help me out in correcting this... (5 Replies)
Discussion started by: adurga
5 Replies
6. Shell Programming and Scripting
How to list all the files, directories and sub-directories in the current path except one directory?
Can anyone come up with a unix command that lists
all the files, directories and sub-directories in the current directory
except a folder called log.?
Thank you in advance. (7 Replies)
Discussion started by: Manjunath B
7 Replies
7. Shell Programming and Scripting
Hi all,
I have more than 12000 files in 46 different directories and each directory has 2 sub-directories named “dat” or “gridded”. Dat sub-directories have files with extension “jpg.dat” and gridded sub-directories have files with extension “.jpg”.
I need to... (1 Reply)
Discussion started by: AshwaniSharma09
1 Replies
8. Shell Programming and Scripting
Hi ,
I'm using Ksh 88
I've the following files in a directory with YearMonthDate (Ex:20130601)
YDT:FILE1:20130601
YDT:FILE1:20130615
YDT:FILE2:20130601
YDT:FILE2:20130615
YDT:FILE3:20130601
YDT:FILE3:20130615
And i need the files having maximum timestamp , Means i need to display ... (8 Replies)
Discussion started by: smile689
8 Replies
9. Shell Programming and Scripting
how can i move "dataName".sql.gz into a folder called 'database' and then move "$fileName".tar.gz * .htaccess into a folder called 'www' with the entire gzipped file being "$fileName".tar.gz? Is this doable or overly complex.
so
mydemo--2015-03-23-1500.tar.gz
> database
-... (5 Replies)
Discussion started by: wyclef
5 Replies
LEARN ABOUT MOJAVE
new
NEW(1) [nmh-1.5] NEW(1) NAME
new - report on folders with new messages fnext - set current folder to next folder with new messages fprev - set current folder to previous folder with new messages unseen - scan new messages in all folders with new messages SYNOPSIS
new [sequences] [-mode mode] [-folders foldersfile] [-version] [-help] fnext is equivalent to new -mode fnext fprev is equivalent to new -mode fprev unseen is equivalent to new -mode unseen DESCRIPTION
New in its default mode produces a one-line-per-folder listing of all folders containing messages in the listed sequences or in the sequences listed in the profile entry "Unseen-Sequence". Each line contains the folder, the number of messages in the desired sequences, and the message lists from the .mh_sequences file. For example: foo 11.* 40-50 bar 380. 760-772 824-828 total 391. The `*' on foo indicates that it is the current folder. The last line shows the total number of messages in the desired sequences. New crawls the folder hierarchy recursively to find all folders, and prints them in lexicographic order. Override this behavior by provid- ing foldersfile containing the pre-sorted list of folders new should check, one per line. In fnext and fprev modes, new instead changes to the next or previous matching folder, respectively. In unseen mode, new executes scan sequences for each matching folder. FILES
$HOME/.mh_profile The user profile PROFILE COMPONENTS
Path: To determine the user's nmh directory Current-Folder: To find the default current folder Unseen-Sequence: The name of the unseen message sequence SEE ALSO
scan(1), mh-format(5) HISTORY
Based on Luke Mewburn's new (http://www.mewburn.net/luke/src/new). MH.6.8 11 June 2012 NEW(1)