cvadmin(1) cvadmin(1)
NAME
cvadmin - Administer an Xsan File System
SYNOPSIS
cvadmin [-H FSMHostName] [-F VolumeName] [-M] [-f filename] [-e com-
mand1 -e command2 ...] [-x]
DESCRIPTION
cvadmin is an interactive command used for general purpose administra-
tion of an Xsan volume including:
1. displaying file system and client status
2. activating a file system currently in stand-by mode
3. viewing and modifying storage pool attributes
4. administering user, group, and directory quotas
5. enabling File System Manager (FSM) tracing
6. displaying disk and path information for the local system
7. forcing FSM failover
8. fetching FSM usage and performance statistics
9. temporarily enabling or disabling global file locking
10. generating a report of open files
11. listing currently held file locks 13. starting, restarting and
stopping of daemon processes
14. resetting RPL information
USAGE
Invoke cvadmin to start the interactive session and list the running
File System Managers (FSMs). (Note: Xsan system services must be
started prior to running cvadmin. In particular, the local fsmpm(8)
process must be active.)
Then (optionally) use the select command described below to pick an FSM
to connect to. Once connected, the command will display basic informa-
tion about the selected volume and prompt for further commands.
Note that a few commands such as paths, disks, start, and stop require
information obtained from the local fsmpm(8) only, so there is is no
need to select an FSM prior to using them.
OPTIONS
-H FSMHostName
Connect to the FSM located on the machine FSMHostName. By
default cvadmin will attempt to connect to an FSM located on the
local machine.
-F VolumeName
Automatically set the volume VolumeName as the active volume in
cvadmin.
-M When listing file systems with the select command, display [man-
aged] next to each file system with DataMigration enabled. This
option is currently only intended for use by support personnel.
-f filename
Read commands from filename
-e command
Execute command(s) and exit
-x Enable extended commands.
COMMANDS
The cvadmin commands can be used to display and modify the Xsan active
configuration. When a modification is made, it exists only as long as
the FSM is running. More permanent changes can be made in the configu-
ration file. Refer to the snfs_config(5) man page for details. The fol-
lowing commands are supported.
activate volume_name [hostname_or_IP_address]
Activate a volume volume_name. This command may cause an FSM to
activate. If the FSM is already active, no action is taken.
activate volume_name number_of_votes
Apple Internal only. Bypass the election system and attempt to
activate the fsm on this node.
debug [[+/-] value]
View or set the File System Manager's debugging flags. Entering
the command with no value will return current settings, the
location of the FSM log file and a legend describing what each
setting does. By entering the command with a numeric value, the
FSM Debugging Flags will be set accordingly. Use a standard dec-
imal or hexadecimal (0x) value of up to 32 bits. Using the '+'
or '-' flags enable ('+') or disable ('-') only the selected
flags, leaving all other flags unchanged.
NOTE - Setting Debugging Flags will severely impact the FSM's
performance! Do this only when directed by an Apple specialist.
dirquotas {create|destroy|mark} path
Create creates a Directory Quota Name Space (DQNS) on the sup-
plied directory. Destroy destroys the DQNS. Mark creates an
unintialized DQNS. See snquota(1) for an explanation of the
trade-offs between create and mark.
disks [refresh]
Display the Xsan disk volumes local to the system that cvadmin
is attached to. Using the optional refresh argument will force
the fsmpm to re-scan all volumes before responding.
disks [refresh] fsm
Display the Xsan meta-data disk volumes in use by the fsm. If
the optional refresh argument is used, additional paths to these
volumes may be added by the fsm.
down poolname
Down the storage pool poolname. This will down any access to the
storage pool. fail {volume_name|index_number} Initiate an FSM
Failover of volume volume_name. This command may cause a stand-
by FSM to activate. If an FSM is already active, the FSM will
shut down. A stand-by FSM will then take over. If a stand-by FSM
is not available the primary FSM will re-activate after failover
processing is complete.
fsmlist [volume_name] [ on hostname_or_IP_address]
Display the state of FSM processes, running or not. Optionally
specify a single volume_name to display. Optionally specify the
host name or IP address of the system on which to list the FSM
processes.
filelocks
Query cluster-wide file/record lock enforcement. Currently
cluster-wide file locks are automatically used on Unix. Windows
file/record locks are optional.
help (?)
The help or ? command will display a command usage summary.
latency-test [index_number|all] [seconds]
Run an I/O latency test between the FSM process and one client
or all clients. The default test duration is 2 seconds.
multipath groupname {balance|cycle|rotate|static|sticky}
Xsan has the capability of utilizing multiple paths from a sys-
tem to the SAN disks.
This capability is referred to as "multi-pathing", or sometimes
"multi-HBA support". (HBA := Host Based Adaptor).
At "disk discovery" time, for each physical path (HBA), a scan
of all of the SAN disks visible to that path is initiated, accu-
mulating information such as the Xsan label, and where possible,
the disk (or LUN) serial number.
At mount time, the visible set of Xsan labeled disks is matched
against the requested disks for the volume to be mounted.
If the requested disk label appears more than once, then a
"multi-path" table entry is built for each available path.
If the disk (or LUN) device is capable of returning a serial
number, then that serial number is used to further verify that
all of the paths to that Xsan labeled device share the same
serial number.
If the disk (or LUN) device is not capable of returning a serial
number then the device will be used, but Xsan will not be able
to discern the difference between a multi-path accessible
device, and two or more unique devices that have been assigned
duplicate Xsan labels.
The presence of serial numbers can be validated by using the
"cvlabel -ls" command. The "-s" option requests the displaying
of the serial number along with the normal label information.
There are five modes of multi-path usage which can also be spec-
ified in the filesystem config file. In cases where there are
multiple paths and an error has been detected, the algorithm
falls back to the rotate method. The balance and cycle methods
will provide the best aggregate throughput for a cluster of
hosts sharing storage.
balance
The balance mode provides load balancing across all the
available, active, paths to a device. At I/O submission
time, the least used HBA/controller port combination is
used as the preferred path. All Xsan File System I/O in
progress at the time is taken into account.
cycle The cycle mode rotates I/O to a LUN across all the avail-
able, active, paths to it. As each new I/O is submitted,
the next path is selected.
rotate The rotate mode is the default for configurations where
the operating system presents multiple paths to a device.
In this mode, as an I/O is initiated, an HBA controller
pair to use for this I/O is selected based on a load bal-
ance method calculation.
If an I/O terminates in error, a "time penalty" is
assessed against that path, and another "Active" path is
used. If there are not any "Active" paths that are not
already in the "error penalty" state, then a search for
an available "Passive" path will occur, possibly trigger-
ing an Automatic Volume Transfer to occur in the Raid
Controller.
static The "default" mode for all disks other than Dual Raid
controller configurations that are operating in
Active/Active mode with AVT enabled.
As disks (or LUNs) are recognized at mount time, they are
statically associated with an HBA in rotation.
i.e. given 2 HBA's, and for disks/LUNs:
disk 0 -> HBA 0
disk 1 -> HBA 1
disk 2 -> HBA 0
disk 3 -> HBA 1
and so on...
sticky In this mode, the path to use for an I/O is based on the
identity of the target file. This mode will better uti-
lize the controller cache, but will not take advantage of
multiple paths for a single file.
The current mode employed by a storage pool can be viewed via
the "cvadmin" command "show long", and modified via the "cvad-
min" command "multipath".
Permanent modifications may be made by incorporating a "Multi-
PathMethod" configuration statement in the configuration file
for a storage pool.
In the case of an I/O error, that HBA is assessed an "error
penalty", and will not be used for a period of time, after which
another attempt to use it will occur.
The first "hard" failure of an HBA often results in a fairly
long time-out period (anywhere from 30 seconds to a couple of
minutes).
With most HBA's, once a "hard" failure (e.g. unplugged cable)
has been recognized, the HBA immediately returns failure status
without a time-out, minimizing the impact of attempting to re-
use the HBA periodically after a failure. If the link is
restored, most HBA's will return to operational state on the
next I/O request.
paths Display the Xsan disk volumes visible to the local system. The
display is grouped by <controller> identity, and will indicate
the "Active" or "Passive" nature of the path if the Raid Con-
troller has been recognized as configured in Active/Active mode
with AVAT enabled.
proxy [long]
Display Disk Proxy servers and optionally display the disks they
serve for this volume.
proxy who hostname
The "who" option displays all proxy connections for the speci-
fied host.
qos Display per-stripe group QOS statistics. Per-client QoS statis-
tics are also displayed under each qos-configured stripe group.
quit This command will disconnect cvadmin from the FSM and exit.
quotas Check to see if quotas are currently enabled on the selected
volume. When quotas is enabled, the meta-data controller must
stay on either Windows or a non-Windows machine.
quotas get {user|group} name
Get current quota parameters for user or group name. The name
field can also be an integer such as -2 or 512 and will be
interpreted as a uid (for user) or gid (for group). The values
for hard limit and soft limit are expressed in bytes. The value
for time limit is expressed in minutes.
quotas get {dir|dirfiles} path
Get current quota parameters for the DQNS whose root is path.
The path is relative to the root of the currently selected vol-
ume. For dir, the values for hard limit and soft limit are
expressed in bytes. For dirfiles, the values for hard and soft
limit are expressed in number of files. For both, the value for
time limit is expressed in minutes. The path must have already
been made into a DQNS using the dirquotas command.
quotas set {user|group} name hardlim softlim timelim
Set current quota parameters for user or group name. The name
field can also be an integer such as -2 or 512 and will be
interpreted as a uid (for user) or gid (for group). Setting the
hard limit (hardlim), soft limit (softlim), and time limit
(timelim) to 0 disables quota enforcement for that user or
group. The values for hardlim and softlim are expressed in
bytes. The value for timelim is expressed in minutes.
quotas set {dir|dirfiles} path hardlim softlim timelim
Set current quota parameters for the DQNSs whose root is path.
Setting the hard limit (hardlim), soft limit (softlim), and time
limit (timelim) to 0 disables quota enforcement for that DQNS.
For dir, the values for hard limit and soft limit are expressed
in bytes. For dirfiles, the values for hard and soft limit are
expressed in number of files. For both, the value for time
limit is expressed in minutes. The path must have already been
made into a DQNS using the dirquotas command.
quotacheck
Recalculate the amount of space consumed by each user and group
on the file system. This command can be run on an active volume
although file updates (writes, truncates, etc.) will be delayed
until quotacheck has completed.
quotareset
Like quotacheck, but deletes the quota database before perform-
ing the check. All limits will be zeroed and all Directory
Quota Name Spaces (DQNSs) will be destroyed. Before using this
command, it's best to be sure there is a recent quota_regen.in
file in /Library/Logs/Xsan/data/<volume_name>. This file can be
fed to cvadmin to restore the limits and DQNSs. Use with
extreme caution.
ras enq event "detail string"
Generate an SNFS RAS event. For internal use only.
ras enq event reporting_FRU violating_FRU "detail string"
Generate a generic RAS event. For internal use only.
repquota
Generate quota reports for all users and groups in the volume.
Three files are generated:
1. quota_report.txt - a "pretty" text file report.
2. quota_report.csv - a comma delimited report suitable
for Excel spreadsheets.
3. quota_regen.in - a list of cvadmin commands that can
be used to set up an identical quota database on
another Xsan volume.
repfl Generate a report that displays the file locks currently held.
Note: this command is only intended for debugging purposes by
support personnel. In future releases, the format of the report
may change or the command may be removed entirely. Running the
repfl command will write out a report file and display the out-
put filename.
repof Generate a report that displays all files that are currently
open on each Xsan client. Only file inode numbers and stat
information are displayed, filenames are not displayed. Running
the repof command will write out a report file and display the
output filename. In future releases, the format of the report
may change.
resetrpl [clear]
Repopulate Reverse Path Lookup (RPL) information. The optional
clear argument causes existing RPL data to be cleared before
starting repopulation. Note: resetrpl is only available when
cvadmin is invoked with the -x option. Running resetrpl may
significantly delay FSM activation. This command is not
intended for general use. Only run resetrpl when recommended by
Technical Support.
restartd daemon [once]
Restart the daemon process. For internal use only.
select [volume_name|N]
Select an active FSM to view and modify. If no argument is
specified, a numbered list of FSMs and running utilities will be
displayed. If there is only one active volume in the list, it
will automatically be selected.
When a running utility is displayed by the select command, it
will show the following information. First the name of the file
system is displayed. Following that, in brackets "[]", is the
name of the utility that is running. Third, a letter indicating
the access type of the operation. The options here are (W) for
read-write access, (R) for read-only access and (U) for unique
access. Finally, the location and process id of the running
utility is displayed.
If volume_name is specified, then cvadmin will connect to the
current active FSM for that volume. If N (a number) is speci-
fied, cvadmin will connect to the Nth FSM in the list. However,
only active FSMs may be selected in this form.
show [poolname] [long]
Display information about the storage pools associated with the
selected volume. If a storage pool name poolname is given only
that storage pool's information will be given. Omitting the
poolname argument will display all storage pools associated with
the active file system. Using the long modifier will addition-
ally display detailed information about the disk units associ-
ated with displayed storage pools.
start volume_name [on hostname_or_IP_address]
Start a File System Manager for the volume volume_name. When
the command is running on an MDC of an HA cluster, the local FSM
is started, and then an attempt is made to start the FSM on the
peer MDC as identified by the /Library/Preferences/Xsan/ha_peer
file. When the optional hostname_or_IP_address is specified,
the FSM is started on that MDC only. The volume's configuration
file must be operational and placed in /Library/Prefer-
ences/Xsan/<volume_name>.cfg before invoking this command. See
snfs_config(5) for information on how to create a configuration
file for an Xsan volume.
startd daemon [once]
Start the daemon process. For internal use only.
stat Display the general status of the volume. The output will show
the number of clients connected to the volume. This count
includes any administrative programs, such as cvadmin. Also
shown are some of the static file-system-wide values such as the
block size, number of storage pools, number of mirrored storage
pools and number of disk devices. The output also shows total
blocks and free blocks for the entire volume.
stats client_IP_address [clear]
Display read/write statistics for the selected file system.
This command connects to the host FSMPM who then collects sta-
tistics from the file system client. The ten most active files
by bytes read and written and by the number of read/write
requests are displayed. If the file system is configured for
reverse path lookup (RPL), then the file path is displayed. Oth-
erwise, the files are identified by the inode number. If clear
is specified, zero the stats after printing.
stop volume_name [on hostname_or_IP_address]
Stop the File System Manager for volume_name. This will shut
down the FSM for the specified volume on every MDC. When the
optional hostname or IP address is specified, the FSM is stopped
on that MDC only. Further operations to the volume will be
blocked in clients until an FSM for the volume is activated.
stopd daemon
Start the daemon process. For internal use only.
up poolname
Up the storage pool poolname. This will restore access to the
storage pool.
who Query client list for the active volume. The output will show
the following information for each client.
SNFS I.D. - Client identifier
Type - Type of connection. The client types are:
FSM - File System Manager(FSM) process
ADM - Administrative(cvadmin) connection
CLI - Volume client connection. May be followed by a CLI
type character:
S - Disk Proxy Server
C - Disk Proxy Client
H - Disk Proxy Hybrid Client. This is a client that has
been configured as a proxy client but is operating
as a SAN client.
Location - The clients hostname or IP address
Up Time - The time since the client connection was initiated
License Expires - The date that the current client license will
expire
EXAMPLES
Invoke the cvadmin command for FSM host cornice, volume named default.
spaceghost% cvadmin -H k4 -F xsan1
Xsan File System Administrator
Enter command(s)
For command help, enter "help" or "?".
List FSS
File System Services (* indicates service is in control of FS):
1>*xsan1[0] located on k4:32823 (pid 3988)
Select FSM "xsan1"
Created : Fri Jul 25 16:41:44 2003
Active Connections: 3
Fs Block Size : 4K
Msg Buffer Size : 4K
Disk Devices : 1
Stripe Groups : 1
Mirror Groups : 0
Fs Blocks : 8959424 (34.18 GB)
Fs Blocks Free : 8952568 (34.15 GB) (99%)
Show all the storage pools in the volume;
snadmin (xsan1) > show
Show stripe group(s) (File System "xsan1")
Stripe Group 0 [StripeGroup1] Status:Up,MetaData,Journal
Total Blocks:8959424 (34.18 GB) Free:8952568 (34.15 GB) (99%)
MultiPath Method:Rotate
Primary Stripe 0 [StripeGroup1] Read:Enabled Write:Enabled
Display the long version of the RegularFiles storage pool;
snadmin (xsan1) > show StripeGroup1 long
Show stripe group "StripeGroup1" (File System "xsan1")
Stripe Group 0 [StripeGroup1] Status:Up,MetaData,Journal
Total Blocks:8959424 (34.18 GB) Free:8952568 (34.15 GB) (99%)
MultiPath Method:Rotate
Stripe Depth:1 Stripe Breadth:16 blocks (64.00 KB)
Affinity Set:
Realtime limit IO/sec:0 (~0 mb/sec) Non-Realtime reserve IO/sec:0
Committed RTIO/sec:0 Non-RTIO clients:0 Non-RTIO hint IO/sec:0
Disk stripes:
Primary Stripe 0 [StripeGroup1] Read:Enabled Write:Enabled
Node 0 [disk002]
Down the storage pool named stripe1;
snadmin (xsan1) > down stripe1
Down Stripe Group "stripe1" (File System "xsan1")
Stripe Group 0 [stripe1] Status:Down,MetaData,Journal
Total Blocks:2222592 (8682 Mb) Free:2221144 (8676 Mb) (99%)
Mirrored Stripes:1 Read Method:Sticky
Primary Stripe 0 [stripe1] Read:Enabled Write:Enabled
Disable reads on the mirrored storage pool stripe1m.
snadmin (xsan1) > disable stripe1m read
Disable Stripe Group "stripe1m" (File System "xsan1")
Stripe Group 0 [stripe1] Status:Down,MetaData,Journal
Total Blocks:2222592 (8682 Mb) Free:2221144 (8676 Mb) (99%)
Mirrored Stripes:1 Read Method:Sticky
Primary Stripe 0 [stripe1] Read:Enabled Write:Enabled
FILES
/Library/Preferences/Xsan/*.cfg
SEE ALSO
cvfs(1), snfs_config(5), fsmpm(8), fsm(8), mount_acfs(1)
Xsan File System December 2011 cvadmin(1)