Performance Co-Pilot

provides a systems-level suite of tools that cooperate to deliver distributed, integrated performance management services. PCP is designed for the in-depth analysis and sophisticated control mechanisms that are needed to understand and manage the hardest performance problems in our most complex systems.

Performance data may be collected and exported from multiple sources, most notably the Irix Kernel, DBMS products, layered services, and end-user applications.

Performance Co-Pilot is targeted at the performance analyst, benchmarker, engineering developer, database administrator, capacity planner or system administrator with an interest in overall system performance. It provides unparalleled power to quickly isolate and understand performance behavior, resource utilization, activity levels and performance bottlenecks.

Performance Co-Pilot (PCP) includes the following tools and capabilities:

Shipped support for domains of performance metrics include:

• all Irix 5.3 (or later) kernel instrumentation
• process-level resource utilization
• Web server activity and quality of service
• activity, resource utilization and query response time for the Informix, ORACLE and Sybase DBMS
• SGI platform-specific instrumentation, e.g. R10K, FailSafe and Array
• Cisco router statistics
• environmental monitors for high end server systems
• support for Origin 200 and Origin 2000 servers

Features

Customized Visualization of Performance Data

• A generalized 3-D Inventor application (pmview) that supports dynamic displays of clusters of related performance metrics as utilization blocks (or towers) on a common base plane.
• Assorted visualization tools for processor visualization (mpvis), disk device utilization (dkvis), Web server activity and response time (webvis, weblogvis and webpingvis), and NFS request traffic (nfsvis).
• A graphical strip-chart monitor (pmchart) to display trends over time for arbitrarily selected performance metrics from one or more hosts, and one or more domains of performance metrics.

Intelligent Filtering of Data

• The inference engine (pmie) evaluates predicate-action rules based on performance metrics.
• Performance alarms may be set to enable actions when a problem is detected; it is possible to automatically launch a performance visualization tool, or to page the system admin, or to run a script to dynamically tune, etc.
• The inference engine enables adaptive logging which uses performance feedback to dynamically control detail in archive logs.
• Live or archived performance data can be run through the engine to identify limiting resources for a performance audit or for capacity planning.
• Intelligent filtering reduces the volume of performance data without loss of critical information.

Expandability of Performance Metrics

• An exported Performance Metrics API (PMAPI) for building site-specific or application-specific performance-related tools.
• This PMAPI provides access to all of the services of the underlying PCP infrastructure.
• Source programs and scripts, as well as the programming guide, demonstrate how new PMAPI clients and new sources of metrics can be implemented.
• The guiding principle is "if it is important for monitoring system performance and you can measure it, you can easily integrate it into the Performance Co-Pilot tools framework."

Logging and Retrospective Analysis

• A range of tools are provided to support adaptive and flexible logging of performance metrics for archival, play-back, remote-diagnosis and capacity planning. Archive logs may be accumulated either at the host being monitored, or at a monitoring workstation, or both.
• A universal replay mechanism, modelled on a VCR paradigm, supports "stop, rewind and replay at variable speed" processing of performance information for both archived and real-time data.
• A host-based security model provides optional control over the execution of PCP service requests from designated remote hosts and/or workstations.

Additional Features

• Text-based display
  • Text-based display of arbitrary groups of performance metric values or rates (pmval and pmkstat), suitable for ASCII logs or enquiry over a modem.
• Distributed Operation
  • With Co-Pilot tools, a single workstation may concurrently monitor the performance of multiple remote hosts. At the same time, a single host may be monitored from multiple remote workstations. Performance Co-Pilot uses a classical "client-server" architecture to provide seamless and concurrent access to performance metrics, independent of their host location.
• Muliple Performance Metrics Domains
  • At the lowest level, performance metrics are collected and managed in autonomous performance domains, e.g. IRIX, a DBMS, or an end-user application. These domains support a variety of access-control policies, access methods, data semantics, and multiversion support. All of this detail is irrelevant to the developer and user of a performance monitoring tool, and is hidden by the Performance Co-Pilot infrastructure.
• opsview
  • The power of performance visualization is demonstrated by a sample Inventor application (opsview) that provides a high-level view of the performance of two nodes in an ORACLE Parallel Server (OPS) configuration, with "drill-down" navigational links to other visualization tools. Although a demonstration, users with a real OPS configuration can use the interactive configuration script, and tailor opsview to monitor a production OPS system.
• Uniform Naming and Access to Performance Metrics
  • Usability and extensibility of performance management tools mandate a single scheme for naming performance metrics. The set of defined names constitutes a Performance Metric Name Space (PMNS). Within Performance Co-Pilot, the PMNS is adaptive so that it can be extended and reshaped to meet the needs of particular applications and users. A single interface is provided to retrieve the values for all performance metrics.
• Dynamic Adaptation to Change
  • Complex systems are subject to continual changes as network connections fail and are re-established, nodes are taken out of service and rebooted, hardware is added and removed, and software (IRIX, DBMS, and application) is upgraded, installed, or removed. Variations in the available performance metrics as a consequence of configuration changes are handled automatically and become visible to all clients as soon as the reconfigured host is rebooted or the responsible agent is restarted. The tools also detect loss of client-server connections, and support subsequent automated client reconnection.

Optional Performance Co-Pilot Agents

The following performance agents are available for PCP running in a variety of environments. These agents allow you to correlate their respective performance metrics into the the PCP framework.

PCPWEB: Performance Co-Pilot for Web servers

PCPWEB augments the capabilities of the base Performance Co-Pilot (PCP) product with additional services and tools tailored for the monitoring and management of a system running one or more Web servers. Specifically, PCPWEB assists the performance-related tasks by providing:

• Performance metrics for Web server activity, workload and quality of service,
• Performance visualization customized for the Web server environment,
• Infrastructure tools to manage archival logs to record Web server performance,
• Rule sets to provide automated reasoning and alarm conditions for web serving.

PCPORA7: Performance Co-Pilot for Oracle 7

The Performance Co-Pilot Oracle 7 Agent allows internal performance metrics from Oracle 7 RDBMS instances to be integrated into the Performance Co-Pilot framework.

Two PCP agents:

• pmdaoracle7 agent for collecting Oracle database dynamic performance data
• oraping agent for monitoring Oracle database application end-user response times.

PCPINFMX7: Performance Co-Pilot for Informix 7

The Informix Performance Metrics Domain Agent obtains performance metrics from an Informix OnLine database server and makes them available to users of Performance Co-Pilot. PCPINFMX7 retrieves information from the database server by querying the Informix System Monitoring Interface (SMI). Queries are performed only when metrics are requested from the PMDA to minimize impact on the database.

PCPSYB10: Performance Co-Pilot for Sybase 10

PCPSYB10 provides additional support for Sybase System-10 and System-11 SQL Servers.

• "sybvis" gives you a 3D visualization scene of Sybase SQL Server performance.
• PCPSYB agents:
  • syb10 agent for monitoring metrics from the Sybase SQL Server system tables (for both System-10 and System-11)
  • sybping"agent for monitoring Sybase SQL end-user response times.
• Three Pmchart "view" configurations: Monitor, Processes, and Ping.

PCPARRAY: Performance Co-Pilot for POWER CHALLENEGEArray

Contains a set of tools to assist in monitoring and managing an Array system resources across the systems

• arraytop - display resource usage of the top sessions across all hosts in an array,
• ashtop - display resource usage of the top processes in a particular array session,
• ash2pids - enumerates the processes belonging in a particular array session,
• procvis - a 3D tool to visualize the resource usage of a set of processes in an array,
• arrayvis - a 3D tool to visualize the summary performance and status of each host in an array, and of the whole array in aggregate.

PCPFailSafe: Performance Co-Pilot for FailSafe High Availability

PCPFailSafe provides a 3-D visualization of activity and resource utilization at the level of a FailSafe cluster.

Application Programming Interface