maximum number of times - a file can be opened


 
Thread Tools Search this Thread
Top Forums Programming maximum number of times - a file can be opened
# 1  
Old 03-20-2006
maximum number of times - a file can be opened

Hi All,

We can find the maximum of open file descriptors in hold with respect to a process.
As default size was 256 (with getrlimit) and the hard limt was 65536
I tried changing the limit to 1024(with setrlimit)
successfully changed the limit

but still I couldnt have as many open file descriptors as I had mentioned in the setrlimit, I am just pushed off with 256 file descriptors itself.

And one more,
Though the process has limitations on the number of file descriptors it can have, is there any limit imposed indirectly on a file that it can have only 'n' number of active open(s).
(virtually, I think it is not possible). say, File dum.c can be opened only 'Z' no of times.

Any ptrs for the above!!!

Thanks.
# 2  
Old 03-20-2006
I can't explain why your attempt to raise your softlimit failed.

Each open of a file creates a file table entry. The size of the file table is a kernel tunable parameter called nfile. You need to look at your own kernel to see what that is. You won't be able to use all of the file table entries repeatedly opening some file. You will need to open a few other files just to set up your attempt. Each file table entry for a file increments a counter in the vnode for that file. If you increment it to far, it would wrap around. This might not stop you from opening the file some more, but it would not be a good thing. The size of the integer that holds nfile and the size of the reference count should be the same size integer thus preventing this problem. It may be cheating, but you can have an opened file without opening a file. There are various ways to "dup" a file descriptor. The dup'ed fd points to the same file table entry as the original fd. Unix traditionally limits pid's to 32,000 (but your kernel may have a different limit). So I guess the limit is 32000 * max-fd-per-process. But again, minus a few for the overhead of reaching the point where you could try this stunt.
# 3  
Old 03-21-2006
Quote:
I can't explain why your attempt to raise your softlimit failed.
Probably I would try to research more on that.
Quote:
The size of the file table is a kernel tunable parameter called nfile.
You need to look at your own kernel to see what that is.
I tried locating it under /dev/kvm but couldn't where should I look for, probably i cannot run System Administration Manager, i dont have root perm.
Quote:
Each file table entry for a file increments a counter in the vnode for that file. If you increment it to far, it would wrap around.
Exactly here is the point, I am trying to exploit the counter for the specific file. Wrapping around - (are the index ptrs to the file table wrapped )
Does that mean mapping a newly created file descriptor to an previously
existing one ? ( I think I am wrong, tht cannot be the case)
Quote:
It may be cheating, but you can have an opened file without opening a file. There are various ways to "dup" a file descriptor. The dup'ed fd points to the same file table entry as the original fd.
I dont agree here, dup function works only on a open file descriptor else it would fail. Though it points to the same file table entry as the original fd, file descriptor entry maintained on a per-process basis would have its available lowest free descriptor entry being used.
Quote:
Unix traditionally limits pid's to 32,000 (but your kernel may have a different limit).
In SunOS, maximum PID no possible is 999, 999 and from sysdef maximum no of processes allowed is 17722
Quote:
So I guess the limit is 32000 * max-fd-per-process.
If that is the case of maximum no of processes possible * 256 ( max-fd-per-process )
literally, I cannot fork that many process to exploit vnode counter for any particular file
# 4  
Old 03-21-2006
If you don't have root, ask your system administrator what nfile is. Be sure to explain that you plan to exploit a possible os weakness. We sysadms just love to hear that. I decline to comment on what result would ensue if you wrap the counter. (Which I doubt is possible.)

If I execute a single open(), followed by a dup(), I now have two open files but only one file table entry. (I did say this might be cheating.) If you doubt that this represents 2 open files, try it and run lsof on your program. Or suppose I do this and then I call exit(). All opened files are closed by exit(). How many times will exit invoke close()? Or leave your limit at 256 and open a file and dup it 256 times...do you succeed? And dup() is not the only way to duplicate an fd. There is even a way to "dup" an fd from one process to another unrelated process. Also only the original dup is constrained to the lowest available fd. It is better to control the desired fd. The original dup is really still around for compatibility. On most OS's, dup is just a wrapper around fcntl(). I know that duped fd's all link to the same file table entry...that is the whole point. It circumvents the nfile limit. And I point out one more time that I did say it might be cheating.

Max pid may or may not relate strongly to max number of processes. Max processes is another kernel tunable called NPROC but there may be a limit how high it can be set. If you want to achieve the max number of times that a file can be opened, you will need to take extraordinary steps and this includes tuning the kernel to max out various limits. (Sadly, few kernels have been tuned for this particular activity...) This in turn may cause you to need a lot of physical memory. Setting records is never easy.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

File exists, but cannot be opened.How to check- whether it could be opened to read when it exists

Hi #Testing for file existence if ; then echo 'SCHOOL data is available for processing' else echo 'SCHOOL DATA IS NOT AVAILABLE FOR PROCESSING' : i wrote a script, where it begins by checking if file exists or not. If it exists, it truncates the database... (2 Replies)
Discussion started by: rxg
2 Replies

2. UNIX for Dummies Questions & Answers

how to count number of times each word exist in a file

I'm trying to count the number of times each word in the file exist for example if the file has: today I have a lot to write, but I will not go for it. The main thing is that today I am looking for a way to get each word in this file with a word count after it specifying that this word has... (4 Replies)
Discussion started by: shnkool
4 Replies

3. UNIX for Advanced & Expert Users

Number of files currently opened in linux

Hello, How do i check number of files currently opening in the linux server? Your help is highly appreciated. Thank you ---------- Post updated at 02:43 PM ---------- Previous update was at 02:19 PM ---------- never mind!! I got it. ---------- Post updated at 02:44 PM ---------- Previous... (3 Replies)
Discussion started by: govindts
3 Replies

4. UNIX for Dummies Questions & Answers

ls - maximum number of files

what is the maximum number ls can list down (6 Replies)
Discussion started by: karnan
6 Replies

5. UNIX for Dummies Questions & Answers

maximum number of arguments

Hi, What is the maximum number of arguments that could be passed to zsh ? To find out that I tried a simple script. And the maximum number of arguments that could be passed turned out to be 23394 #! /bin/zsh arg=1 i=1 subIndex=23000 while do arg=$arg" "$i i=$(($i + 1))... (9 Replies)
Discussion started by: matrixmadhan
9 Replies

6. UNIX and Linux Applications

handling maximum number characters in an input file

Hi, Can anyone help me? An input file has three lines. Each line should only be 2098 as number of characters however line 2 of the input file has more than the maximum number of characters, it exceeded up to 4098. What should I do so that could handle maximum number of characters? that it could... (1 Reply)
Discussion started by: chrysSty
1 Replies

7. Shell Programming and Scripting

TO find the word which occurs maximum number of times

Hi Folks !!!!!!!!!!!!!!!!!!! My Requirement is............. i have a input file: 501,501.chan 502,502.anand 503,503.biji 504,504.raja 505,505.chan 506,506.anand 507,507.chan and my o/p should be chan->3 i.e. the word which occurs maximum number of times in a file should be... (5 Replies)
Discussion started by: aajan
5 Replies

8. Shell Programming and Scripting

Will the output file be opened and closed several times?

Hi, there, I wrote a script like this: #!/bin/bash #put something into a LIST for item in $LIST do cat $item >> /tmp/output done My question is that if I have 5 items in that LIST, should it be opened and closed every time when the ">>" works? So that file will be opened and... (7 Replies)
Discussion started by: koifans
7 Replies

9. Shell Programming and Scripting

Counting Number of times a File is accessed

Hi, I need to count the number of times a script is accessed from within the script. Is it possible ? Example: I have a script called lo.sh and i execute the script for the first time, then the counter variable declared inside the lo.sh should increment by 1. For every execution the... (1 Reply)
Discussion started by: pathanjalireddy
1 Replies

10. UNIX for Dummies Questions & Answers

maximum number of input on solaris

Hi, Can anyone tell me what the maximum amount of input characters is on solaris command line? (standard ksh I think) (1 Reply)
Discussion started by: marcello
1 Replies
Login or Register to Ask a Question