Sponsored Content
Special Forums Hardware Filesystems, Disks and Memory Performance Hit With Many Files Post 302294978 by MarkSeger on Friday 6th of March 2009 08:59:24 AM
Old 03-06-2009
I too have done similar testing but slightly different. Rather than simply time the creation of 100 files, which can be very misleading OR timing the creation of 1M files which is no better, I prefer to look at what is happening across the system resources during the entire event.

I run collectl with a monitoring interval of 1 second, logging to a file or simply watching the system in real time. When I create a million files I can watch the cpu periodically increase. In fact, when getting in the higher ends of files I can actually see spike in cpu load. This is something you can't see when just doing end-to-end numbers.

Another interesting test is to set up an alarm in your script to write out the number of files created every 10th (or even hundredth) of a second. You'll be amazed to see how linearly the number of files created/second drops over time as well as how things periodically slow down but are not visible when only looking at second-level samples.

You can also run collectl at a monitoring interval of 0.1 seconds and see micro-spikes in CPU load as well. This is something most people miss because none of the existing tools can deal with sub-second reporting.

-mark
 

9 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Anyone else see a performance hit from ext3

I reinstalled my Linux box with RedHat 7.2 and used the ext3 journaling file system. This thing is a pig now. There isn't much running on the box, and performance is sad. (1 Reply)
Discussion started by: 98_1LE
1 Replies

2. Shell Programming and Scripting

Shell Script to hit a url

Hi all, I have a php file that grabs xml, parses it and updates my db accordingly. I want to automate the execution of this process, rather than having to hit the url manually. I have been looking into using cron to execute a script to do this, however i'm not exactly sure what command i would... (1 Reply)
Discussion started by: restivz77
1 Replies

3. Programming

why multiple SIGINT raises when i hit C-c

hi, in my application, i have set up to capture SIGINT and execute a handler.the problem is whenever i hit C-c, multiple SIGINT are sent to the application.I have blocked the SIGINT right after catching the first one but it is unsuccessful.Here is what i do : jmp_buf main_loop; int... (1 Reply)
Discussion started by: Sedighzadeh
1 Replies

4. Shell Programming and Scripting

Hit count on a shell script

I have a unix shell script (ex.sh) written. How to find out how many users (incl. myself) have run this .sh ? I can insert code snipet at top of script if need be. - Ravi (2 Replies)
Discussion started by: ravi368
2 Replies

5. Shell Programming and Scripting

Getting Next Best Hit..

Hi.. I need to get the following output from the input file like this INPUT GRM1 GRM1 0 GRM1 ABC1 1 GRM1 FEQ1 2 GRM1 SED1 3 ABC2 GRM1 0 ABC2 ABC2 1 ABC2 FEQ1 2 ABC2 BED1 3 SED1 SED1 0 SED1 SED1 1 SED1 SED1 2 SED1 ABC1 3 OUTPUT: (7 Replies)
Discussion started by: empyrean
7 Replies

6. SuSE

Java hit

Hello, I'm having trouble looking for info for SUSIE on this CVE-2012-4681. This is basically the newest Java hit. It is mostly a web browser issue but I would like to see if the versions on our servers are vulnerable. I already found the pages/info for Solaris and RHEL. Any help would be... (4 Replies)
Discussion started by: bitlord
4 Replies

7. Cybersecurity

vnc password hit from Retina

Hello, I'm having an issue with VNC. Security at work says that they scanned my servers (Solaris, RHEL, SLES) and found that you don't need a password to access a VNC session. I have tested this and you can't login to the VNC session without a password. Can someone tell what the Retina scanner... (1 Reply)
Discussion started by: bitlord
1 Replies

8. Shell Programming and Scripting

Curl to hit the submit button

Hello, I am looking to hit a URL using curl and click on submit button so that I can get the results. The below is the code <input name="tos_accepted" id="tos_accepted" class="button" type="submit" value="Yes, I Agree"/> <input name="tos_discarded" id="tos_discarded"... (1 Reply)
Discussion started by: Kochappa
1 Replies

9. Shell Programming and Scripting

Best performance to merge two files

Hi Gurus, I need to merge two files. file1 (small file, only one line) this is first linefile2 (large file) abc def ghi ... I use below command to merge the file, since the file2 is really large file, the command read whole file2, the performance is not good. cat file1 > file3... (7 Replies)
Discussion started by: green_k
7 Replies
krsd(1M)																  krsd(1M)

NAME
krsd - kernel registry services daemon SYNOPSIS
seconds] DESCRIPTION
saves data, flagged as persistent in the core kernel KRS tree, to files on disk. These files are read when the system is booted, in order to restore the aforementioned persistent data. When running in daemon mode, sleeps, waking periodically to check if any persistent data have changed since the last save. If persistent data have changed, the new state of the data is saved to disk. Otherwise, sleeps for another period of time and then repeats the process. When running in one time (non-daemon) mode, unconditionally saves persistent data to disk and then exits. Normally, is executed in daemon mode by init(1M). The entry for will cause to be respawned automatically if it is terminated. uses the message logging facility to log all activity. Persistent KRS data are maintained in files based on the class of the data in question. Currently, the following classes are recognized: Data that apply to the system in general, regardless of what configuration is booted. This data is saved in the file Data specific to a given bootable configuration. This data is saved in the file only saves the class; the class is managed by the kernel configuration tools (see kconfig(5)). Options recognizes the following options: One time mode, unconditionally save persistent data to disk, then exit. Run from parent does not spawn a child nor exit. Required if from inittab is to work properly. Set the delay time to seconds seconds. This is the time interval between attempted saves of persistent data. The default interval is 300 seconds. AUTHOR
was developed by Hewlett-Packard Company. SEE ALSO
krs_flush(1M), kconfig(5), krs(5). krsd(1M)
All times are GMT -4. The time now is 01:35 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy