The UNIX and Linux Forums  

Go Back   The UNIX and Linux Forums > Top Forums > High Level Programming
Google UNIX.COM


High Level Programming Post questions about C, C++, Java, SQL, and other programming languages here.

More UNIX and Linux Forum Topics You Might Find Helpful
Thread Thread Starter Forum Replies Last Post
handling maximum number characters in an input file chrysSty UNIX and Linux Applications 1 05-12-2008 07:19 AM
TO find the word which occurs maximum number of times aajan Shell Programming and Scripting 5 01-11-2008 12:11 AM
Will the output file be opened and closed several times? koifans Shell Programming and Scripting 7 12-28-2007 02:57 AM
ls - maximum number of files karnan UNIX for Dummies Questions & Answers 5 12-10-2007 07:09 AM
Counting Number of times a File is accessed pathanjalireddy Shell Programming and Scripting 1 04-11-2005 06:49 AM

Reply
 
Submit Tools LinkBack Thread Tools Display Modes
  #1 (permalink)  
Old 03-20-2006
Technorati Master
 

Join Date: Mar 2005
Location: Large scale systems...
Posts: 2,572
Stumble this Post!
maximum number of times - a file can be opened

Hi All,

We can find the maximum of open file descriptors in hold with respect to a process.
As default size was 256 (with getrlimit) and the hard limt was 65536
I tried changing the limit to 1024(with setrlimit)
successfully changed the limit

but still I couldnt have as many open file descriptors as I had mentioned in the setrlimit, I am just pushed off with 256 file descriptors itself.

And one more,
Though the process has limitations on the number of file descriptors it can have, is there any limit imposed indirectly on a file that it can have only 'n' number of active open(s).
(virtually, I think it is not possible). say, File dum.c can be opened only 'Z' no of times.

Any ptrs for the above!!!

Thanks.
Reply With Quote
Forum Sponsor
  #2 (permalink)  
Old 03-20-2006
Perderabo's Avatar
Unix Daemon
 

Join Date: Aug 2001
Location: Washington DC Area
Posts: 8,444
Stumble this Post!
I can't explain why your attempt to raise your softlimit failed.

Each open of a file creates a file table entry. The size of the file table is a kernel tunable parameter called nfile. You need to look at your own kernel to see what that is. You won't be able to use all of the file table entries repeatedly opening some file. You will need to open a few other files just to set up your attempt. Each file table entry for a file increments a counter in the vnode for that file. If you increment it to far, it would wrap around. This might not stop you from opening the file some more, but it would not be a good thing. The size of the integer that holds nfile and the size of the reference count should be the same size integer thus preventing this problem. It may be cheating, but you can have an opened file without opening a file. There are various ways to "dup" a file descriptor. The dup'ed fd points to the same file table entry as the original fd. Unix traditionally limits pid's to 32,000 (but your kernel may have a different limit). So I guess the limit is 32000 * max-fd-per-process. But again, minus a few for the overhead of reaching the point where you could try this stunt.
Reply With Quote
  #3 (permalink)  
Old 03-21-2006
Technorati Master
 

Join Date: Mar 2005
Location: Large scale systems...
Posts: 2,572
Stumble this Post!
Quote:
I can't explain why your attempt to raise your softlimit failed.
Probably I would try to research more on that.
Quote:
The size of the file table is a kernel tunable parameter called nfile.
You need to look at your own kernel to see what that is.
I tried locating it under /dev/kvm but couldn't where should I look for, probably i cannot run System Administration Manager, i dont have root perm.
Quote:
Each file table entry for a file increments a counter in the vnode for that file. If you increment it to far, it would wrap around.
Exactly here is the point, I am trying to exploit the counter for the specific file. Wrapping around - (are the index ptrs to the file table wrapped )
Does that mean mapping a newly created file descriptor to an previously
existing one ? ( I think I am wrong, tht cannot be the case)
Quote:
It may be cheating, but you can have an opened file without opening a file. There are various ways to "dup" a file descriptor. The dup'ed fd points to the same file table entry as the original fd.
I dont agree here, dup function works only on a open file descriptor else it would fail. Though it points to the same file table entry as the original fd, file descriptor entry maintained on a per-process basis would have its available lowest free descriptor entry being used.
Quote:
Unix traditionally limits pid's to 32,000 (but your kernel may have a different limit).
In SunOS, maximum PID no possible is 999, 999 and from sysdef maximum no of processes allowed is 17722
Quote:
So I guess the limit is 32000 * max-fd-per-process.
If that is the case of maximum no of processes possible * 256 ( max-fd-per-process )
literally, I cannot fork that many process to exploit vnode counter for any particular file
Reply With Quote
  #4 (permalink)  
Old 03-21-2006
Perderabo's Avatar
Unix Daemon
 

Join Date: Aug 2001
Location: Washington DC Area
Posts: 8,444
Stumble this Post!
If you don't have root, ask your system administrator what nfile is. Be sure to explain that you plan to exploit a possible os weakness. We sysadms just love to hear that. I decline to comment on what result would ensue if you wrap the counter. (Which I doubt is possible.)

If I execute a single open(), followed by a dup(), I now have two open files but only one file table entry. (I did say this might be cheating.) If you doubt that this represents 2 open files, try it and run lsof on your program. Or suppose I do this and then I call exit(). All opened files are closed by exit(). How many times will exit invoke close()? Or leave your limit at 256 and open a file and dup it 256 times...do you succeed? And dup() is not the only way to duplicate an fd. There is even a way to "dup" an fd from one process to another unrelated process. Also only the original dup is constrained to the lowest available fd. It is better to control the desired fd. The original dup is really still around for compatibility. On most OS's, dup is just a wrapper around fcntl(). I know that duped fd's all link to the same file table entry...that is the whole point. It circumvents the nfile limit. And I point out one more time that I did say it might be cheating.

Max pid may or may not relate strongly to max number of processes. Max processes is another kernel tunable called NPROC but there may be a limit how high it can be set. If you want to achieve the max number of times that a file can be opened, you will need to take extraordinary steps and this includes tuning the kernel to max out various limits. (Sadly, few kernels have been tuned for this particular activity...) This in turn may cause you to need a lot of physical memory. Setting records is never easy.
Reply With Quote
Google The UNIX and Linux Forums
Reply

Thread Tools
Display Modes




All times are GMT -7. The time now is 08:37 PM.


Powered by: vBulletin, Copyright ©2000 - 2006, Jelsoft Enterprises Limited.
The UNIX and Linux Forums Content Copyright ©1993-2008 The CEP Blog All Rights Reserved -Ad Management by RedTyger Visit The Global Fact Book

Content Relevant URLs by vBSEO 3.2.0