A Instrumenting Applications with DMS

A.1.1 Instrumenting Applications with DMS

DMS Instrumentation refers to the process you use when you insert DMS calls into application code. By using the DMS API, you can collect and analyze performance information for your application.

To create DMS metrics you add DMS API calls that notify DMS when events occur, when important intervals begin and end, or when pre-computed values change their state. At run time, DMS stores metrics in memory and enables you to save or view the metrics.

Oracle Fusion Middleware includes built-in DMS metrics. By adding DMS calls to your applications you expand the set of built-in metrics. When you instrument your applications with DMS calls, you use the same API that the built-in metrics use. In addition, to save and display your metrics, you use the same monitoring tools that you use with built-in metrics.

Note that oracle.dms.trace.triggered.enable defaults to TRUE in Oracle Fusion Middleware 11g release 1 (11.1.1) and defaults to FALSE in 11g release 1 (11.1.1.2.0).

See Section A.2, "Adding DMS Instrumentation to Java Applications" for details.

A.1.2 Monitoring DMS Metrics

Monitoring DMS metrics refers to the process of retrieving performance metrics. When an application runs, DMS stores metrics in memory; you can view the metrics on the console or using a web browser.

Oracle Fusion Middleware provides several run time tools for viewing and saving DMS metrics, including the WLST command (with the displayMetricTableNames, displayMetricTables, and dumpMetrics subcommands) and the Spy servlet.

Example A-1 shows the output of wlst dumpMetrics.

/dmsDemo [type=n/a]
  /dmsDemo/BasicBinomial [type=MathSeries]
   computeSeries.active: n threads
   computeSeries.avg: n msecs
   computeSeries.completed: n ops
   computeSeries.maxActive: n threads
   computeSeries.maxTime: n msecs
   computeSeries.minTime: n msecs
   computeSeries.time: n msecs
   lastComputed.value: n    loops.count: n ops

A.1.3 Understanding DMS Terminology (Nouns and Sensors)

This section introduces terminology related to DMS. This section contains the following subsections:

• Section A.1.3.1, "DMS Metrics"

• Section A.1.3.2, "DMS Sensors"

• Section A.1.3.3, "DMS Nouns"

• Section A.1.3.4, "DMS Roll-up Nouns"

• Section A.1.3.5, "DMS Object Relationships"

Figure A-1 illustrates the organization of a set of DMS metrics that correspond to the metrics in the demo application described in this chapter and the metrics shown in Example A-1.

Figure A-1 Organization of Sample Metrics for dmsDemo Application

Description of "Figure A-1 Organization of Sample Metrics for dmsDemo Application"

A.1.4 DMS Naming Conventions

When you create DMS nouns and sensors, you should follow the guidelines in this section so that users can easily understand DMS metrics across applications and across Oracle Fusion Middleware components.

This section includes the following subsections:

• Section A.1.4.1, "General DMS Naming"

• Section A.1.4.2, "General DMS Naming Conventions and Character Sets"

• Section A.1.4.3, "Noun and Noun Type Naming Conventions"

• Section A.1.4.4, "Sensor Naming Conventions"

A.2.1 Including DMS Imports

To use DMS you need to add DMS imports. The following example shows the imports that the sample application BasicBinomial.java requires.

import oracle.dms.instrument.DMSConsole;
import oracle.dms.instrument.Event;
import oracle.dms.instrument.Noun;
import oracle.dms.instrument.PhaseEvent;
import oracle.dms.instrument.State;
import oracle.dms.instrument.Sensor;

A.2.2 Organizing Performance Data

In the sample application with the BasicBinomial and ImprovedBinomial servlets, the Nouns are organized as follows:

Example A-2 shows a section of code from BasicBinomial.java. It shows calling Noun.create to create the /dmsDemo noun root and the BasicBinomial noun, which is a child of /dmsDemo.

private Noun binRoot;                // Container for Binomial series DMS metrics
Noun base = Noun.create("/dmsDemo");
binRoot   = Noun.create(base, "BasicBinomial", "MathSeries");

/dmsDemo is the noun root for the binomial sample application. The initial slash character marks it as the root.

The BasicBinomial noun is created with a noun type of MathSeries (the third parameter for the create method). The Noun type is a name that reflects the set of metrics being collected. In this case, MathSeries represents the metrics collected for the sample binomial application.

In the ImprovedBinomial servlet, located in the same binomial application, there is a similar set of lines (Example A-3). The only difference is that it creates a child noun called ImprovedBinomial instead of BasicBinomial.

private Noun binRoot;                // Container for Binomial series DMS metrics
Noun base = Noun.create("/dmsDemo");
binRoot   = Noun.create(base, "ImprovedBinomial", "MathSeries");

Note that the dmsDemo noun was not created with a noun type because its children are all nouns. It does not have any child sensors.

The Spy servlet displays all nouns of the same type as rows in a table; the name of the table is the noun-type. Metrics are represented as columns in the table.

It is good practice to only use Noun types for Nouns that directly contain Sensors. When a Noun contains only other Nouns, as in dmsDemo, and does not directly contain Sensors, the Spy servlet displays the Noun type as a metric table, with no metrics.

The dmsDemo Noun includes a Noun (BasicBinomial), but no direct Sensors. When the Noun type is not included for such a Noun, the Spy servlet does not display a metric table associated with the Noun.

A.2.3 Defining and Using Metrics for Timing

To create metrics that measure the duration of a segment of code, you use PhaseEvent Sensors:

• Section A.2.3.1, "Defining PhaseEvent Sensors"

• Section A.2.3.2, "Using PhaseEvent Sensors"

private PhaseEvent binComp; // Time to compute Binomial series
...
binComp = PhaseEvent.create(binRoot, "computeSeries", "Time to compute a Binomial series");
binComp.deriveMetric(Sensor.all);

long token = 0; // DMS
try {
    token = binComp.start(); // DMS
    BigInteger bins[] = bin(length);
    out.println("<H2>Binomial series for " +  length + "</H2>");
    for (int i = 0; i < length; i++)
        out.println("<br>" + bins[i]);
}
finally {
    binComp.stop(token); // DMS
    out.close();
}

PhaseEvent pe = heavyPhase(param);
long token1 = 0;
long token2 = 0;
boolean stopped = false;
try {
      token1 = binComp.start();
      if (pe != null) token2 = pe.start();
      BigInteger bins[] = bin(length);
      out.println("<H2>ImprovedBinomial series for " +  length + "</H2>");
      for (int i = 0; i < length; i++)
        out.println("<br>" + bins[i]);
      if (pe != null) pe.stop(token2);
      binComp.stop(token1);
      stopped = true;
}
finally {
      if (!stopped) {
        if (pe != null) pe.abort(token2);
        binComp.abort(token1);
      }
}

A.2.4 Defining and Using Metrics for Counting

To create metrics that count the occurrences of an event, define and use an Event Sensor:

• Section A.2.4.1, "Defining Event Sensors"

• Section A.2.4.2, "Using Event Sensors"

private Event binLoop;   // Loops needed for Binomial series.
...
binLoop = Event.create(binRoot, "loops", "Iterations to compute series");

binLoop.occurred();

A.2.5 Defining and Using Metrics for Recording Status Information (State Sensors)

DMS captures status information with State Sensors. State Sensors track the value of Java primitives or the content of a Java Object. The supported types include integer, double, long, and object, as specified in the third argument to the create method. When a Java primitive State Sensor is updated with the wrong type, DMS attempts to convert the supplied value to the correct type. For object type State Sensors, DMS stores a reference to the object and by default calls toString on the object when the DMS value is sampled.

To create metrics that record status information, define and use State Sensors:

• Section A.2.5.1, "Defining State Sensors"

• Section A.2.5.2, "Using State Sensors"

private State binLast;   // Value of the last computed element in series.
...
binLast = State.create(binRoot, "lastComputed", State.OBJECT, "",
                           "Value of last computed series element");

binLast.update(bins[k-1].toString());

A.3.1 Validating DMS Metrics

Use wlst and other available DMS monitoring tools to verify and test new metrics.

Try to validate the following for new metrics:

• Do expected metrics appear in the display? Test this by examining the code to ensure that all the metric names added using DMS instrumentation appear in your display or saved set of metrics.

• Do unexpected metrics appear in the display? Verify that you have only added the metrics that you planned to add.

• Are the metric values you see within reasonable ranges? Usually, upper and lower bounds for metrics can be established. You then test that the reported values for metrics do not exceed the expected bounds.

  For example, a "size of pool" metric should never report a negative value.

• Ensure the new metrics are needed. For example, if you add a PhaseEvent that always measures an event of very short duration, consider changing the metric to an Event metric, or remove the metric.

• Ensure the new metrics are accurate. For most applications using DMS metrics, accuracy is more important than the performance cost of adding the DMS instrumentation. New DMS metrics should provide reliable and useful information.

  Testing for accuracy can be difficult. However, if an alternate means of measuring a particular metric is available then use it to verify metric values. For example, if you submit a known number of requests to a server and measure total time for the experiment, then you predict correct values for the relevant metrics and compare them with the actual monitored values. As another example, you can verify an Event Sensor count metric by examining records that you write to a log file or to the console.

  Check for timing inaccuracies that may apply for the metrics. Timing inaccuracies may be caused when low-resolution clocks time metrics for an interval of short duration. For example on Windows systems, the default Java clock advances only once every 15 milliseconds. DMS metrics reported for brief events on these systems must be analyzed with care. Consider using the high resolution clock to address this issue.

A.3.2 Testing DMS Metrics For Efficiency

The use of DMS metrics has some influence on application performance. When adding metrics, note the following:

• The processing required for computing and storing metrics can slow down the execution of an application. DMS is fast, but it may have some required performance cost. In addition, DMS cannot prevent developers from using the DMS API inefficiently. Therefore, before adding DMS instrumentation, establish reasonable expectations. After completing the implementation, measure the actual costs and compare them to your expectations. Be prepared to make changes to the instrumentation to reduce performance impacts until the measurements agree with expectations.

• DMS provides the DMSConsole.getSensorWeight method to help you control the use of metrics. The central setting is an advisory measurement level that DMS does not enforce. To control which metrics to include, at run time, the code must test the value for SensorWeight to determine whether to make DMS calls. See Section A.5, "Conditional Instrumentation Using DMS Sensor Weight" for details.

• When integrating DMS instrumentation with an existing package or when implementing a new feature, you should consider insulating a previously working system. For example, you could include an option to enable and disable new DMS metrics.

• You should run your performance tests with and without DMS enabled. If your tests show unacceptable results with DMS enabled, then you may want to re-design or re-implement metrics.

A.8.1 Isolating Expensive Intervals Using PhaseEvent Metrics

Carefully consider the requirements for new metrics when you add DMS instrumentation. It is important to add a sufficient number of metrics to validate that your code is working as expected.

Try to observe the following guidelines when you add DMS metrics:

1. Add PhaseEvent Sensors only to provide an overview of the time the system spends in your block of code or module. You do not need to collect performance data for every method call, or for every distinct phase of your code or module.

2. When your code calls external code that you do not control, and that you expect could take a significant amount of time, add a PhaseEvent Sensor to track the start and the completion of the external code.

Following these guidelines for adding PhaseEvent metrics provides the following benefits:

• Helps limit the amount of information that DMS collects.

• Enables those analyzing the system to prove that a module gives the expected run time performance.

• Ensures that people viewing DMS metrics can validate run time performance without seeing an overwhelming amount of data.

• Enables those analyzing system performance to separate and track your module from other system modules that are either expensive or failure prone.

A.9.1 Configuring DMS Clocks for Reporting Time for Java

Selecting the high resolution clock changes clocks for all applications running on the process where the clock is changed. You set the DMS clock and the reporting values globally using the oracle.dms.clock and oracle.dms.clock.units properties, which control process startup options.

For example, to use the high resolution clock with the default values, set the following property on the Java command line:

-Doracle.dms.clock=highres

-Doracle.dms.clock.units=msecs

Table A-6 shows supported values for the oracle.dms.clock property.

Table A-7 shows supported values for the oracle.dms.clock.units property.

Note the following when using the high resolution DMS clock:

• When you set the oracle.dms.clock and the oracle.dms.clock.units properties, any combination of upper and lower case characters is valid for the value that you select (case is not significant). For example, any of the following values are valid to select the high resolution clock: highres, HIGHRES, HighRes.

• DMS checks the property values at startup. When you set the clock with a value that does not match those listed in Table A-6, then DMS uses the default clock. If the oracle.dms.clock property is not set, DMS also uses the default clock.

• If the specified clock units property value does not match those listed in Table A-7, then DMS uses the default units for the specified clock. If the oracle.dms.clock.units property is not set, DMS uses the default units for the specified clock.

A.9.2 Configuring DMS Clocks for Reporting Time for Oracle HTTP Server

The default clock for measuring Oracle HTTP Server performance has a resolution of microseconds (usecs). You can optionally select a higher resolution clock to monitor C processes running under Oracle HTTP Server. To use the high resolution clock under Oracle HTTP Server, you need to set configuration options in httpd.conf, or specify environment variables on the command line.

Table A-8 lists the environment variables that control the Oracle HTTP Server DMS clock. Table A-9 describes the httpd.conf configuration options that control the Oracle HTTP Server DMS clock. If you set both the command line options and the httpd.conf configuration options, the configuration options override the values set on the command line.

With these options DMS uses a high resolution clock for all the Oracle HTTP Server processes that it monitors, and DMS reports values using the milliseconds units (msecs).

DmsClock=HIGHRES
DmsClockUnits=USECS