nhfsstone - Network File System benchmark program
nhfsstone [ -v ] [[ -t secs ] | [ -c calls ]] [ -l load ] [ -p nprocs ] [ -m mixfile ] [
nhfsstone (pronounced n-f-s-stone, the "h" is silent) is used on a NFS client to generate
an artificial load with a particular mix of NFS operations. It reports the average
response time of the server in milliseconds per call and the load in calls per second.
The program adjusts its calling patterns based on the client's kernel NFS statistics and
the elapsed time. Load can be generated over a given time or number of NFS calls.
Because it uses the kernel NFS statistics to monitor its progress, nhfsstone cannot be
used to measure the performance of non-NFS filesystems.
The nhfsstone program uses file and directory manipulation in an attempt to generate par-
ticular NFS operations in response to particular system calls. To do this it uses several
tricks that are based on a knowledge of the implementation of the NFS client side refer-
ence port. For example, it uses long file names to circumvent the kernel name lookup
cache so that a stat(2) system call generates an NFS lookup operation.
The mix of NFS operations can be set with a mix file, which is the output of the nfs-
stat(8C) command (see the "-m" option below). The percentages taken from the mix file are
calculated based on the number of NFS calls, not on the percentages printed by nfsstat.
Operations with 0% in the mix will never get called by nhfsstone. In a real server load
mix, even though the percentage of call for a particular NFS operation may be zero, the
number of calls is often nonzero. Nhfsstone makes the assumption that the number of calls
to these 0 percent operations will have an insignificant effect on server response.
Normally nhfsstone should be given a list of two or more test directories to use (default
is to use the current directory). The test directories used should be located on differ-
ent disks and partitions on the server to realistically simulate typical server loads.
Each nhfsstone process looks for a directory <dir>/testdir<n> (where <n> is a number from
0 to nprocs - 1). If a process directory name already exists, it is checked for the cor-
rect set of test files. Otherwise the directory is created and populated.
-v Verbose output.
-t secs Sets calls based on the given running time (in seconds) and the load.
-c calls Total number of NFS calls to generate (default is 5000).
-l load Load to generate in NFS calls per second (default is 30).
-p nprocs Number of load generating sub-processes to fork (default is 7). This can be
used to maximize the amount of load a single machine can generate. On a small
client machine (slow CPU or small amount of memory) fewer processes might be
used to avoid swapping.
-m mixfile Mix of NFS operations to generate. The format of mixfile is the same as the
output of the nfsstat(8C) program. A mix file can be created on a server by
typing "nfsstat -s > mixfile". The default mix of operations is: null 0%,
getattr 13%, setattr 1%, root 0%, lookup 34%, readlink 8%, read 22%, wrcache
0%, write 15%, create 2%, remove 1%, rename 0%, link 0%, symlink 0%, mkdir 0%,
rmdir 0%, readdir 3%, fsstat 1%.
As with all benchmarks, nhfsstone can only provide numbers that are useful if experiments
that use it are set up carefully. Since it is measuring servers, it should be run on a
client that will not limit the generation of NFS requests. This means it should have a
fast CPU, a good ethernet interface and the machine should not be used for anything else
during testing. A Sun-3/50 can generate about 60 NFS calls per second before it runs out
Nhfsstone assumes that all NFS calls generated on the client are going to a single server,
and that all of the NFS load on that server is due to this client. To make this assump-
tion hold, both the client and server should be as quiescent as possible during tests.
If the network is heavily utilized the delays due to collisions may hide any changes in
server performance. High error rates on either the client or server can also cause delays
due to retransmissions of lost or damaged packets. netstat(8C) -i can be used to measure
the error and collision rates on the client and server.
To best simulate the effects of NFS clients on the server, the test directories should be
set up so that they are on at least two of the disk partitions that the server exports and
the partitions should be as far apart as possible. The dkinfo(8) command can be used to
find the physical geometry of disk on BSD based systems. NFS operations tend to randomize
access the whole disk so putting all of the nhfsstone test directories on a single parti-
tion or on two partitions that are close together will not show realistic results.
On all tests it is a good idea to run the tests repeatedly and compare results. The num-
ber of calls can be increased (with the -c option) until the variance in milliseconds per
call is acceptably small. If increasing the number of calls does not help there may be
something wrong with the experimental setup. One common problem is too much memory on the
client test machine. With too much memory, nhfsstone is not able to defeat the client
caches and the NFS operations do not end up going to the server at all. If you suspect
that there is a caching problem you can use the -p option to increase the number of pro-
The numbers generated by nhfsstone are most useful for comparison if the test setup on the
client machine is the same between different server configurations. Changing nhfsstone
parameters between runs will produce numbers that can not be meaningfully compared. For
example, changing the number of generator processes may affect the measured response time
due to context switching or other delays on the client machine, while changing the mix of
NFS operations will change the whole nature of the experiment. Other changes to the
client configuration may also effect the comparability of results. While nhfsstone tries
to compensate for differences in client configurations by sampling the actual NFS statis-
tics and adjusting both the load and mix of operations, some changes are not reflected in
either the load or the mix. For example, installing a faster CPU or mounting different NFS
filesystems may effect the response time without changing either the load or the mix.
To do a comparison of different server configurations, first set up the client test direc-
tories and do nhfsstone runs at different loads to be sure that the variability is reason-
ably low. Second, run nhfsstone at different loads of interest and save the results.
Third, change the server configuration (for example, add more memory, replace a disk con-
troller, etc.). Finally, run the same nhfsstone loads again and compare the results.
The nhfsstone.c source file has comments that describe in detail the operation of of the
illegal calls value
The calls argument following the -c flag on the command line is not a positive num-
illegal load value
The load argument following the -l flag on the command line is not a positive num-
illegal time value
The time argument following the -t flag on the command line is not a positive num-
bad mix file
The mixfile file argument following the -m flag on the command line could not be
can't find current directory
The parent process couldn't find the pathname of the current directory. This usu-
ally indicates a permission problem.
The parent couldn't fork the child processes. This usually results from lack of
resources, such as memory or swap space.
can't open log file
can't stat log
can't truncate log
can't write sync file
can't write log
can't read log
A problem occurred during the creation, truncation, reading or writing of the syn-
chronization log file. The parent process creates the log file in /tmp and uses it
to synchronize and communicate with its children.
can't open test directory
can't create test directory
can't cd to test directory
wrong permissions on test dir
can't stat testfile
wrong permissions on testfile
can't create rename file
can't create subdir
A child process had problems creating or checking the contents of its test direc-
tory. This is usually due to a permission problem (for example the test directory
was created by a different user) or a full filesystem.
bad mix format: unexpected EOF after 'nfs:'
bad mix format: can't find 'calls' value
bad mix format: unexpected EOF after 'calls'
bad mix format: can't find %d op values
bad mix format: unexpected EOF
A problem occurred while parsing the mix file. The expected format of the file is
the same as the output of the nfsstat(8C) command when run with the "-s" option.
One of the internal pseudo-NFS operations failed. The name of the operation, e.g.
read, write, lookup, will be printed along with an indication of the nature of the
The select system call returned an unexpected error.
Running nhfsstone on a non-NFS filesystem can cause the program to run forever because it
uses the kernel NFS statistics to determine when enough calls have been made.
Nhfsstone uses many file descriptors. The kernel on the client may have to be reconfigured
to increase the number of available file table entries.
Shell scripts that used nhfsstone will have to catch and ignore SIGUSR1 (see signal(3)).
This signal is used to synchronize the test processes. If the signal is not caught the
shell that is running the script will be killed.
/vmunix system namelist
/dev/kmem kernel virtual memory
./testdir* per process test directory
/tmp/nhfsstone%d process synchronization log file
4 October 1989 NHFSSTONE(8)