pmlogger_check, pmlogger_daily, pmlogger_merge - administration of Performance Co-Pilot
archive log files
$PCP_BINADM_DIR/pmlogger_check [-CNsTV] [-c control]
$PCP_BINADM_DIR/pmlogger_daily [-NorV] [-c control] [-k discard] [-m addresses] [-s size]
[-t want] [-x compress] [-X program] [-Y regex]
$PCP_BINADM_DIR/pmlogger_merge [-fNV] [input-basename ... output-name]
This series of shell scripts and associated control files may be used to create a custom-
ized regime of administration and management for Performance Co-Pilot (see PCPintro(1))
archive log files.
pmlogger_daily is intended to be run once per day, preferably in the early morning, as
soon after midnight as practicable. Its task is to aggregate and rotate one or more sets
of PCP archives. After some period, old PCP archives are discarded. This period is 14
days by default, but may be changed using the -k option. Two special values are recognized
for the period (discard), namely 0 to keep no archives beyond the current one, and forever
to prevent any archives being discarded.
Archive data files can optionally be compressed after some period (compress), to conserve
disk space. This is particularly useful for large numbers of pmlogger processes under the
control of pmlogger_check. The -x option specifies the number of days after which to com-
press archive data files, and the -X option specifies the program to use for compression -
by default this is bzip2(1). Use of the -Y option allows a regular expression to be spec-
ified causing files in the set of files matched for compression to be omitted - this
allows only the data file to be compressed, and also prevents the program from attempting
to compress it more than once. The default regex is ".meta$|.index$|.Z$|.gz$|.bz2|.zip$"
- such files are filtered using the -v option to egrep(1).
In addition, if the PCP ``notices'' file ($PCP_LOG_DIR/NOTICES) is larger than 20480
bytes, pmlogger_daily will rename the file with a ``.old'' suffix, and start a new
``notices'' file. The rotate threshold may be changed from 20480 to size bytes using the
Use of the -m option causes pmlogger_daily to construct a summary of the ``notices'' file
entries which were generated in the last 24 hours, and e-mail that summary to the set of
space-separated addresses. This daily summary is stored in the file
$PCP_LOG_DIR/NOTICES.daily, which will be empty when no new ``notices'' entries were made
in the previous 24 hour period.
pmlogger_check may be run at any time, and is intended to check that the desired set of
pmlogger(1) processes are running, and if not to re-launch any failed loggers. Use of the
-s option provides the reverse functionality, allowing the set of pmlogger processes to be
cleanly shutdown. Use of the -C option queries the system service runlevel information
for pmlogger, and uses that to determine whether to start or stop processes.
pmlogger_merge is a wrapper script for pmlogmerge(1) that merges all of the archive logs
matching the input-basename arguments, and creates a new archive using output-name as the
base name for the physical files that constitute an archive log. The input-basename argu-
ments may contain meta characters in the style of sh(1). If specified, the -f option
causes all of the input files to be removed once the output archive has been created.
pmlogger_merge is used by pmlogger_daily.
To assist with debugging or diagnosing intermittent failures the -t option may be used.
This will turn on very verbose tracing (-VV) and capture the trace output in a file named
$PCP_LOG_DIR/pmlogger/daily.datestamp.trace, where datestamp is the time pmlogger_daily
was run in the format YYYYMMDD.HH.MM. In addition, the want argument will ensure that
trace files created with -t will be kept for want days and then discarded.
Both pmlogger_daily and pmlogger_check are controlled by a PCP logger control file that
specifies the pmlogger instances to be managed. The default control file is $PCP_PMLOG-
GERCONTROL_PATH, but an alternate may be specified using the -c option.
The control file should be customized according to the following rules that define for the
current version (1.1) of the control file format.
1. Lines beginning with a ``#'' are comments.
2. Lines beginning with a ``$'' are assumed to be assignments to environment variables
in the style of sh(1), and all text following the ``$'' will be eval'ed by the
script reading the control file, and the corresponding variable exported into the
environment. This is particularly useful to set and export variables into the
environment of the administrative scripts, e.g.
Warning: The $PCP_PMLOGGERCONTROL_PATH file must not be writable by any user other
3. There must be a version line of the form:
4. There should be one line in the control file for each pmlogger instance of the
host y|n y|n directory args
5. Fields within a line of the control file are separated by one or more spaces or
6. The first field is the name of the host that is the source of the performance met-
rics for this pmlogger instance.
7. The second field indicates if this is a primary pmlogger instance (y) or not (n).
Since the primary logger must run on the local host, and there may be at most one
primary logger for a particular host, this field can be y for at most one pmlogger
instance, in which case the host name must be the name of the local host.
8. The third field indicates if this pmlogger instance needs to be started under the
control of pmsocks(1) to connect to a pmcd through a firewall (y or n).
9. The fourth field is a directory name. All files associated with this pmlogger
instance will be created in this directory, and this will be the current directory
for the execution of any programs required in the maintenance of those archives. A
useful convention is that primary logger archives for the local host with hostname
myhost are maintained in the directory $PCP_LOG_DIR/pmlogger/myhost (this is where
the default pmlogger start-up script in $PCP_RC_DIR/pcp will create the archives),
while archives for the remote host mumble are maintained in $PCP_LOG_DIR/pmlog-
10. All other fields are interpreted as arguments to be passed to pmlogger(1) and/or
pmnewlog(1). Most typically this would be the -c option.
The following sample control lines specify a primary logger on the local host (bozo), and
a non-primary logger to collect and log performance metrics from the host boing.
bozo y n $PCP_LOG_DIR/pmlogger/bozo -c config.default
boing n n $PCP_LOG_DIR/pmlogger/boing -c ./pmlogger.config
Typical crontab(5) entries for periodic execution of pmlogger_daily and pmlogger_check are
given in $PCP_SYSCONF_DIR/pmlogger/crontab (unless installed by default in /etc/cron.d
already) and shown below.
# daily processing of archive logs
14 0 * * * $PCP_BINADM_DIR/pmlogger_daily
# every 30 minutes, check pmlogger instances are running
25,55 * * * * $PCP_BINADM_DIR/pmlogger_check
The output from the cron(1) execution of the scripts may be extended using the -V option
to the scripts which will enable verbose tracing of their activity. By default the
scripts generate no output unless some error or warning condition is encountered.
The -N option enables a ``show me'' mode, where the actions are echoed, but not executed,
in the style of ``make -n''. Using -N in conjunction with -V maximizes the diagnostic
capabilities for debugging.
By default all possible archives will be merged. The -o option reinstates the old behav-
iour in which only yesterday's archives will be considered as merge candidates.
The -T option provides a terser form of output for pmlogger_check that is most suitable
for a pmlogger ``farm'' where many instances of pmlogger are expected to be running.
To accommodate the evolution of PMDAs and changes in production logging environments,
pmlogger_daily is integrated with pmlogrewrite(1) to allow optional and automatic rewrit-
ing of archives before merging. If there are global rewriting rules to be applied across
all archives mentioned in the control file, then create the directory
$PCP_SYSCONF_DIR/pmlogrewrite and place any pmlogrewrite(1) rewriting rules in this direc-
tory. For rewriting rules that are specific to only one family of archives, use the
directory name from the control file (the fourth field) and create a file, or a directory,
or a symbolic link named pmlogrewrite within this directory and place the required rewrit-
ing rule(s) in the pmlogrewrite file or in files within the pmlogrewrite subdirectory.
pmlogger_daily will choose rewriting rules from the archive directory if they exist, else
rewriting rules from $PCP_SYSCONF_DIR/pmlogrewrite if that directory exists, else no
rewriting is attempted.
The -r command line option acts as an over-ride and prevents all archive rewriting with
pmlogrewrite(1) independent of the presence of any rewriting rule files or directories.
The script $PCP_BINADM_DIR/pmlogger_daily could be copied and modified to implement a
site-specific procedure for end-of-week and/or end-of-month management for a set of PCP
the PCP logger control file
Warning: this file must not be writable by any user other than root.
sample crontab for automated script execution by $PCP_USER (or root). Exists
only if the platform does not support the /etc/cron.d mechanism.
default pmlogger configuration file location for the local primary logger, typi-
cally generated automatically by pmlogconf(1).
default location for archives of performance information collected from the host
transient lock file to guarantee mutual exclusion during pmlogger administration
for the host hostname - if present, can be safely removed if neither pmlog-
ger_daily nor pmlogger_check are running
PCP archive folio created by mkaf(1) for the most recently launched archive con-
taining performance metrics from the host hostname
PCP ``notices'' file used by pmie(1) and friends
Environment variables with the prefix PCP_ are used to parameterize the file and directory
names used by PCP. On each installation, the file /etc/pcp.conf contains the local values
for these variables. The $PCP_CONF variable may be used to specify an alternative config-
uration file, as described in pcp.conf(5).
bzip2(1), cron(1), egrep(1), PCP(1), pmlc(1), pmlogconf(1), pmlogger(1), pmlogmerge(1),
pmlogrewrite(1), pmnewlog(1) and pmsocks(1).
Performance Co-Pilot PCP PMLOGGER_CHECK(1)