Contents

Title and Copyright Information

Preface

Audience

Documentation Accessibility

Related Documents

Conventions

1 Introduction to Decision Center Reports

1.1 Accessing Decision Center

1.2 About the Inline Service Navigator

1.3 About the Decision Center Toolbar

1.4 About Perspectives

1.4.1 About the Explore Perspective

1.4.2 About the Design Perspective

1.4.3 About the At a Glance Perspective

1.5 About the CrossSell Example

1.6 Time Window Reporting

1.7 General Report Information

2 About Decision Center Reports

2.1 About Application Reports

2.1.1 Application Definition Overview

2.1.2 Application Definition Integration Map

2.1.3 Application Definition Interactive Integration Map

2.1.4 Application Performance Trends

2.2 About Strategic Objectives Reports

2.2.1 Performance Goals Definition Overview

2.2.2 Performance Goal Definition Overview for Each Performance Goal

2.2.3 Decisions Definition Overview

2.2.4 Decision Definition Overview for Each Decision

2.2.5 Decision Definition Priorities for Each Decision

2.3 About Decision Process Reports

2.3.1 Filtering Rule Definition Overview

2.3.2 Scoring Rule Definition Overview

2.3.3 Choice/Choice Group Definition Overview

2.3.3.1 Choice Group Definition Overview

2.3.3.2 Choice Definition Overview

2.3.4 Choice Definition Eligibility

2.3.5 Choice/Choice Group Performance Counts

2.3.5.1 Choice Group Performance Counts

2.3.5.2 Choice Performance Counts

2.3.6 Choice Performance History

2.3.7 Choice/Choice Group Analysis Drivers

2.3.7.1 Choice Analysis Drivers

2.3.7.2 Choice Group Analysis Drivers

2.3.8 Choice/Choice Group Analysis Trends

2.3.8.1 Choice Group Analysis Trends

2.3.8.2 Choice Analysis Trends

2.3.9 Choice/Choice Group Analysis Best-Fit

2.3.9.1 Choice Group Analysis Best-Fit

2.3.9.2 Choice Analysis Best-Fit

2.3.10 Choice/Choice Group Quality

2.3.10.1 Model Quality Concepts

2.3.10.2 Choice Group Quality

2.3.10.3 Choice Quality

2.4 About Integration Point Reports

2.4.1 Informant Reports

2.4.2 Advisor Reports

2.4.2.1 Advisor Definition Report

2.4.2.2 Advisor Measure Report

2.5 About Entity Catalog Reports

2.5.1 Entity Definition Overview

2.5.2 Entity Analysis Counts

2.5.3 Entity Analysis Statistics

2.5.4 Entity Analysis Drivers

2.5.5 Entity Analysis Trends

3 Working with Choice Groups and Choices

3.1 About Modifying and Redeploying Inline Services

3.2 Adding Choice Groups or Choices

3.2.1 Adding Choice Groups

3.2.2 Adding Choices

3.3 Modifying Choice Groups or Choices

3.4 Cloning Choice Groups or Choices

3.5 Deleting Choice Groups or Choices

3.6 Moving Choices

4 About Rules

4.1 About Eligibility Rules

4.1.1 Adding Eligibility Rules

4.1.2 Evaluating Choice Group Rules and Choice Rules

4.1.3 Determining Eligibility

4.2 About Filtering Rules

4.2.1 Using Filtering Rules to Segment Population

4.3 Adjusting the Priorities of Business Goals by Population Segment

4.4 About Scoring Rules

4.5 Working with Rules

4.5.1 About the Rule Editor Toolbar

4.5.2 Oracle RTD Rule Terms and Statements

4.5.3 Adding Statements to Rules

4.5.4 Selecting an Operator

4.5.5 Editing Boolean Statements

4.5.5.1 Using Type-Restricted Objects in Rules

4.5.6 Editing Rule Properties

4.5.7 Inverting Rule Elements

4.5.8 Moving Rules and Rule Sets

Oracle® Fusion Middleware

Decision Center User's Guide for Oracle Real-Time Decisions

11g Release 1 (11.1.1)

E16631-03

December 2011

Explains how to use the Oracle Real-Time Decisions Decision Center tool to analyze and adjust deployed Inline Services.

Oracle Fusion Middleware Decision Center User's Guide for Oracle Real-Time Decisions, 11g Release 1 (11.1.1)

E16631-03

Copyright © 2010, 2011, Oracle and/or its affiliates. All rights reserved.

Primary Author: Oracle Corporation

Contributors: Oracle Real-Time Decisions development, product management, and quality assurance teams.

This software and related documentation are provided under a license agreement containing restrictions on use and disclosure and are protected by intellectual property laws. Except as expressly permitted in your license agreement or allowed by law, you may not use, copy, reproduce, translate, broadcast, modify, license, transmit, distribute, exhibit, perform, publish, or display any part, in any form, or by any means. Reverse engineering, disassembly, or decompilation of this software, unless required by law for interoperability, is prohibited.

The information contained herein is subject to change without notice and is not warranted to be error-free. If you find any errors, please report them to us in writing.

If this is software or related documentation that is delivered to the U.S. Government or anyone licensing it on behalf of the U.S. Government, the following notice is applicable:

U.S. GOVERNMENT RIGHTS Programs, software, databases, and related documentation and technical data delivered to U.S. Government customers are "commercial computer software" or "commercial technical data" pursuant to the applicable Federal Acquisition Regulation and agency-specific supplemental regulations. As such, the use, duplication, disclosure, modification, and adaptation shall be subject to the restrictions and license terms set forth in the applicable Government contract, and, to the extent applicable by the terms of the Government contract, the additional rights set forth in FAR 52.227-19, Commercial Computer Software License (December 2007). Oracle America, Inc., 500 Oracle Parkway, Redwood City, CA 94065.

This software or hardware is developed for general use in a variety of information management applications. It is not developed or intended for use in any inherently dangerous applications, including applications that may create a risk of personal injury. If you use this software or hardware in dangerous applications, then you shall be responsible to take all appropriate fail-safe, backup, redundancy, and other measures to ensure its safe use. Oracle Corporation and its affiliates disclaim any liability for any damages caused by use of this software or hardware in dangerous applications.

Oracle and Java are registered trademarks of Oracle and/or its affiliates. Other names may be trademarks of their respective owners.

Intel and Intel Xeon are trademarks or registered trademarks of Intel Corporation. All SPARC trademarks are used under license and are trademarks or registered trademarks of SPARC International, Inc. AMD, Opteron, the AMD logo, and the AMD Opteron logo are trademarks or registered trademarks of Advanced Micro Devices. UNIX is a registered trademark of The Open Group.

This software or hardware and documentation may provide access to or information on content, products, and services from third parties. Oracle Corporation and its affiliates are not responsible for and expressly disclaim all warranties of any kind with respect to third-party content, products, and services. Oracle Corporation and its affiliates will not be responsible for any loss, costs, or damages incurred due to your access to or use of third-party content, products, or services.

Preface

Oracle Real-Time Decisions (Oracle RTD) comes with Decision Center, a tool used to analyze and adjust deployed Inline Services. The target user of Decision Center is business analyst who can benefit from structured presentation of analyzed data.

This guide gives an overview of the information presented by Decision Center and the means for adjusting and redeploying Inline Services.

Audience

This document is intended for business users of Decision Center. Users should have a working knowledge using browser-based applications.

Documentation Accessibility

For information about Oracle's commitment to accessibility, visit the Oracle Accessibility Program website at http://www.oracle.com/pls/topic/lookup?ctx=acc&id=docacc.

Access to Oracle Support

Oracle customers have access to electronic support through My Oracle Support. For information, visit http://www.oracle.com/pls/topic/lookup?ctx=acc&id=info or visit http://www.oracle.com/pls/topic/lookup?ctx=acc&id=trs if you are hearing impaired.

Related Documents

For more information, see the following documents in the Oracle Fusion Middleware 11g Release 1 (11.1.1) documentation set:

Oracle Fusion Middleware Release Notes for your platform

Oracle Fusion Middleware Administrator's Guide for Oracle Real-Time Decisions

Oracle Fusion Middleware Platform Developer's Guide for Oracle Real-Time Decisions

Conventions

The following text conventions are used in this document:

Convention
Meaning

boldface

Boldface type indicates graphical user interface elements associated with an action, or terms defined in text or the glossary.

italic

Italic type indicates book titles, emphasis, or placeholder variables for which you supply particular values.

monospace

Monospace type indicates commands within a paragraph, URLs, code in examples, text that appears on the screen, or text that you enter.

1 Introduction to Decision Center Reports

Decision Center is a thin client tool for business users to explore, modify, and examine the structure and data gathered by a deployed Inline Service. Decision Center provides a number of reports and editors to allow users to make changes to the Inline Service and redeploy it to Real-Time Decision Server.

Decision Center offers three ways to navigate: the Inline Service Navigator, the toolbar, and the tabs and subtabs.

This chapter contains the following topics:

Section 1.1, "Accessing Decision Center"

Section 1.2, "About the Inline Service Navigator"

Section 1.3, "About the Decision Center Toolbar"

Section 1.4, "About Perspectives"

Section 1.5, "About the CrossSell Example"

Section 1.6, "Time Window Reporting"

Section 1.7, "General Report Information"

1.1 Accessing Decision Center

To access Decision Center from any computer, open a Web browser and go to http://server_name:port/ui.

The Oracle Real-Time Decisions application port is typically 9704.

In the Sign In window, enter your User Name and Password, then click Sign In.

Then, assuming that you have deployed at least one Inline Service, open up an Inline Service.

1.2 About the Inline Service Navigator

The left pane of Decision Center is the Inline Service Navigator, a tree structure showing the elements that comprise the Inline Service.

Clicking on any element in the Inline Service Navigator will bring the corresponding reports for that element into the right-hand viewing pane.

Figure 1-1 shows the Inline Service Navigator.

Figure 1-1 Inline Service Navigator

1.3 About the Decision Center Toolbar

Many tasks can be accomplished using the Decision Center toolbar. Table 1-1 lists the Decision Center toolbar icons.

Table 1-1 Decision Center Toolbar Icons

Icon
Click this icon to...

Select previous object

Select next object

Select an Inline Service

Select perspective

Promote an Inline Service to a different deployment state

Redeploy the Inline Service

Discard changes

Print the current report page

Create new filtering rule

Create new scoring rule

Delete filtering rule (appears when filtering rule is accessed)

Delete scoring rule (appears when scoring rule is accessed)

Note:

Do not use browser printing options for printing a Decision Center report, including browser print icons or menu print options.

1.4 About Perspectives

Decision Center lets you work with an Inline Service from several perspectives. A perspective defines the initial layout in Decision Center. Each perspective provides a set of functionality aimed at accomplishing a specific type of task, and works with specific types of resources. Perspectives control which elements of an Inline Service are visible, and which reports associated with those elements are visible.

Decision Center has three perspectives: Explore, Design, and At a Glance. The Inline Service Navigator changes according to the perspective you are using.

Custom perspectives can be defined to further control which elements and reports are visible.

This section contains the following topics:

Section 1.4.1, "About the Explore Perspective"

Section 1.4.2, "About the Design Perspective"

Section 1.4.3, "About the At a Glance Perspective"

1.4.1 About the Explore Perspective

The Explore perspective provides a reporting view into an Inline Service. It enables a business analyst to explore the key elements of Oracle RTD Decisions.

1.4.2 About the Design Perspective

The Design perspective enables a business analyst not only to analyze an Inline Service, but also to make modifications to an Inline Service.

These modifications include:

Adjust and create scoring and filtering rules.

Adjust weighting for Performance Metrics on Decisions.

Add and remove segments on Decisions.

Modify the eligibility rules for choices and choice groups

Modify the filtering rules of choices and choice groups.

Deploy an Inline Service to a different server.

Redeploy an Inline Service after it has been changed.

Modifications can only be made when an Inline Service is opened for editing (in the Select Inline Service window) and with the Design perspective.

1.4.3 About the At a Glance Perspective

The At a Glance perspective provides a managerial view into an Inline Service.

The At a Glance perspective gives a high-level summary of Inline Service elements and reports contained in the Inline Service.

1.5 About the CrossSell Example

To illustrate Decision Center features and reports, this guide uses the cross selling example that is included with Oracle Real-Time Decisions. Ask your system administrator to deploy this example to help you explore Decision Center.

The CrossSell Inline Service example is a simple implementation of cross-selling and retention decisions for a financial services contact center. As calls come into the center, information about the customer and the channel of the contact is captured.

Based on what we know of this customer, a cross selling offer is selected that is extended to the customer. The success or failure of that offer is tracked and sent back to the server, so that the underlying decision model has the feedback that helps to refine its ability to make a better cross selling recommendation.

We will use the CrossSell example to explore the reports in Decision Center.

1.6 Time Window Reporting

Users can choose the time period for many of their Decision Center reports by selecting values for the Time Window and Show Incomplete Time Window parameters in the report header.

Decision Center users can report on the information learned in:

Any specific time window - by selecting it from the Time Window list of values

Only the current time window - by selecting it from the Time Window list of values and selecting Show Incomplete Time Window

The special case of the current time window + previous time window - by selecting the current time window from the Time Window list of values and leaving Show Incomplete Time Window unchecked

1.7 General Report Information

This section contains supplementary report information about the values that may appear in various reports.

In many Decision Center reports, you can filter the results by choosing a partition attribute value in the report header, as in the following Performance Counts example.

For these reports, a value of "(missing)" in the filtering list of values for the partitioning attribute indicates that a null value for that attribute was presented at least once.

2 About Decision Center Reports

Each element in the Inline Service Navigator represents a portion of the running Inline Service. Each of these elements gathers and displays useful information for analysis. Decision Center includes an extensive set of reports that let users explore all the aspects of the learning and statistics to any desired depth. Some of this information can also be exported to other BI systems for further use in analysis. This allows individuals to understand how Oracle Real-Time Decisions works and to verify its analysis.

The reports shown in this chapter are included with the CrossSell example Inline Service. Some of the data in your reports will differ slightly from the report examples in this chapter, due to the random nature of the data used in this simulated use case.

The following considerations apply in general to Decision Center reports:

While all the reports shown in this chapter are built out of the box, the labels (such as descriptions, object names, event names, and partitions) are defined at configuration time in Decision Studio.

Key data fields are represented as blue text links. You can drill down on these links for more information.

This chapter contains the following topics:

Section 2.1, "About Application Reports"

Section 2.2, "About Strategic Objectives Reports"

Section 2.3, "About Decision Process Reports"

Section 2.4, "About Integration Point Reports"

Section 2.5, "About Entity Catalog Reports"

2.1 About Application Reports

This section describes the reports that are available at the application level. The application element in the Inline Service Explorer is identified by your Inline Service name. The Inline Service name is followed by its Deployment State. For example:

This section contains the following topics:

Section 2.1.1, "Application Definition Overview"

Section 2.1.2, "Application Definition Integration Map"

Section 2.1.3, "Application Definition Interactive Integration Map"

Section 2.1.4, "Application Performance Trends"

2.1.1 Application Definition Overview

The Application Definition Overview report provides general information about the Inline Service, including:

Description: A description of the Inline Service.

Definition: Any application parameters that are used by the Inline Service.

Goals: The performance goals that the Inline Service supports.

Figure 2-1 shows an example of the Application Definition Overview report.

Figure 2-1 Application Definition Overview Report

2.1.2 Application Definition Integration Map

The Application Definition Integration Map report provides a swim lane view of the overall Inline Service. Each system to which the Inline Service integrates is shown as a swim lane, with the name along the left hand side. Integration Points are represented along the blue swim lane. There are two types of Integration Points: Informants and Advisors.

Informants are collectors of information and events from the various operational systems; they show data flowing in one direction towards the Inline Service. For instance, the CallStart Informant shown in Figure 2-2 is collecting data about an incoming call from an IVR system.

Advisors provide advice back to the operational systems at the request of that system; they show data flowing both to the Inline Service (making a request for advice) and back to the operational system with the advice. For instance, in Figure 2-2, the OfferRequest Advisor shows a request coming in to the Inline Service for a cross selling offer, and the advice for cross sell being returned to the CRM system.

Figure 2-2 shows an example of the Application Definition Integration Map report.

Figure 2-2 Application Definition Integration Map Report

2.1.3 Application Definition Interactive Integration Map

The Application Definition Interactive Integration Map report displays the same swim lane view of the Inline Service as the Application Definition Interactive Integration Map report, but it also contains features that enable users to interact with the Integration Points appearing on the report. Users can trigger Informant and Advisor calls, and examine the internal steps performed by Oracle RTD when executing the calls.

Figure 2-3 shows an example of the Application Definition Interactive Integration Map report, as it is launched.

Figure 2-3 Application Definition Interactive Integration Map Report

Note:

This section concentrates more on Advisors than Informants, to describe the more detailed information that Advisors generate.

Every Advisor is associated with a Decision. In general, the main high-level stages of the Decision process logic for an Advisor request can be summarized as follows:

Eligibility

First, the Decision checks Choices for eligibility in a downward sequence starting from the Choice Group to which the Decision is attached and continuing until all eligibility conditions for all Choices within this Choice Group are assessed.

Scoring for Each Goal

Then, Oracle RTD computes a score (the total score) for each eligible Choice. The total score for each Choice is a function of the decision priorities, weights and scoring functions associated with each of the Performance Goals.

Ordering the Choices

Finally, Oracle RTD compares each of the Choices and orders them by decreasing total score to identify the "winner(s)" of this selection process.

To illustrate this, take an example of the Advisor OfferRequest - the Integration point that will trigger the Oracle Real-Time Decisions Decision logic.

In this example, the customer, whose ID is 1000, interacts with the Call channel (both 1000 and Call are parameters to the Advisor OfferRequest). The net result of the Oracle Real-Time Decisions Decision logic for the OfferRequest Advisor is as follows:

Oracle Real-Time Decisions returns an offer - in this example, Credit Protection

Additional information is also passed back to the client application (such as the likelihood of purchase and a message), so that it can properly render the recommendation to the customer.

In general, clicking any of the Integration Points on the Integration Map causes the Event area to display the session keys and parameters for the Integration Point. For example, when the Advisor Offer Request is clicked, the following appears in the Event area:

After you enter key and parameter values and click Send Event, Oracle RTD performs the logic to action the Integration Point, and then makes available the following information:

The Decision Process Trace area, when expanded, contains a trace of the decision process steps

For Advisor calls, the Decisions area, when expanded, shows the recommended choice and choice attribute values.

Note:

The specific results you will get when clicking the OfferRequest Advisor will not necessarily match the values in the following output. This is due to the fact that the learning data associated with your CrossSell example has been built with a data generation script using the LoadGen tool, which simulates customer responses with a level of randomness. As a result, the Advisor may return an Offer other than Credit Protection, and the likelihood value will be different from the value shown in the following output.

Continuing the OfferRequest call example, sending an event after entering 1000 for Customer ID and Call (the default value) for channel produced the following attribute values for the Credit Protection choice in the expanded Decisions area:

likelihoodOfPurchase = 0.0089383116255807877

message = Some things in life are expected....

shouldRespondPositively = false

The expanded Decision Process Trace area contained the following timed trace entries (annotated here to show the three main Advisor processing stages of Eligibility, Scoring, and Ordering):

Eligibility

10:36:59,906 Entering integration point OfferRequest
10:36:59,906 Decision  will return 1 choice(s).
10:36:59,906 Decision  will use choicegroups: Offers
10:36:59,921 Checking eligibility for group Offers.
10:36:59,921 Group Offers is eligible.
10:36:59,921 Checking eligibility for group Loans.
10:36:59,921 Group Loans is eligible.
10:36:59,921 Checking eligibility for group Loans.
10:36:59,921 Group Loans is eligible.
10:36:59,921 Checking eligibility for group Loan Services.
10:36:59,921 Group Loan Services is eligible.
10:36:59,921 Checking eligibility for group Loan Services.
10:36:59,921 Group Loan Services is eligible.
10:36:59,921 Checking eligibility for choice Waive Fees
10:36:59,921 Choice Waive Fees is eligible
10:36:59,921 Checking eligibility for choice Reduced Interest Rate
10:36:59,921 Choice Reduced Interest Rate is eligible
10:36:59,921 Checking eligibility for group Loan Products.
10:36:59,921 Group Loan Products is eligible.
10:36:59,921 Checking eligibility for group Loan Products.
10:36:59,921 Group Loan Products is eligible.
10:36:59,921 Checking eligibility for choice Home Mortgage
10:36:59,921 Choice Home Mortgage is eligible
10:36:59,921 Checking eligibility for choice Student Loan
10:36:59,921 Choice Student Loan is eligible
10:36:59,921 Checking eligibility for group Credit Cards.
10:36:59,921 Group Credit Cards is eligible.
10:36:59,921 Checking eligibility for group Credit Cards.
10:36:59,921 Group Credit Cards is eligible.
10:36:59,921 Checking eligibility for group Credit Services.
10:36:59,921 Group Credit Services is eligible.
10:36:59,921 Checking eligibility for group Credit Services.
10:36:59,921 Group Credit Services is eligible.
10:36:59,921 Checking eligibility for choice Electronic Payments
10:36:59,921 Choice Electronic Payments is eligible
10:36:59,921 Checking eligibility for choice Credit Protection
10:36:59,921 Choice Credit Protection is eligible
10:36:59,921 Checking eligibility for group Credit Products.
10:36:59,921 Group Credit Products is eligible.
10:36:59,921 Checking eligibility for group Credit Products.
10:36:59,921 Group Credit Products is eligible.
10:36:59,921 Checking eligibility for choice Platinum Card
10:36:59,921 Choice Platinum Card is eligible
10:36:59,921 Checking eligibility for choice Gold Card
10:36:59,921 Choice Gold Card is eligible
10:36:59,921 Checking eligibility for choice Miles Card
10:36:59,921 Choice Miles Card is eligible
10:36:59,921 Eligible choices: (Waive Fees,Reduced Interest Rate,Home Mortgage,Student Loan,Electronic Payments,Credit Protection,Platinum Card,Gold Card,Miles Card)

Scoring for Each Goal

10:36:59,921 Goals Normalization Factors Applied: (Customer Retention: 1.0, Revenue: 1.0)
10:36:59,921 Goals Optimizations Applied: (Customer Retention: Maximize , Revenue: Maximize )
10:36:59,921 Goal Weights Received: (CustomerRetention: 0.30000000000000004, Revenue: 0.7 )
10:36:59,921 Weighted, normalized score for performance goal Customer Retention for choice Waive Fees: 2.175
10:36:59,921 Weighted, normalized score for performance goal Revenue for choice Waive Fees: 0
10:36:59,921 Total score for choice Waive Fees: 2.175
10:36:59,921 Weighted, normalized score for performance goal Customer Retention for choice Reduced Interest Rate: 2.175
10:36:59,921 Weighted, normalized score for performance goal Revenue for choice Reduced Interest Rate: 0
10:36:59,921 Total score for choice Reduced Interest Rate: 2.175
10:36:59,921 Weighted, normalized score for performance goal Customer Retention for choice Home Mortgage: 0.3
10:36:59,921 Weighted, normalized score for performance goal Revenue for choice Home Mortgage: 0.02
10:36:59,921 Total score for choice Home Mortgage: 0.32
10:36:59,921 Weighted, normalized score for performance goal Customer Retention for choice Student Loan: 0.3
10:36:59,921 Weighted, normalized score for performance goal Revenue for choice Student Loan: 0.02
10:36:59,921 Total score for choice Student Loan: 0.32
10:36:59,921 Weighted, normalized score for performance goal Customer Retention for choice Electronic Payments: 1.5
10:36:59,921 Weighted, normalized score for performance goal Revenue for choice Electronic Payments: 0.024
10:36:59,921 Total score for choice Electronic Payments: 1.524
10:36:59,921 Weighted, normalized score for performance goal Customer Retention for choice Credit Protection: 2.1
10:36:59,921 Weighted, normalized score for performance goal Revenue for choice Credit Protection: 0.125
10:36:59,921 Total score for choice Credit Protection: 2.225
10:36:59,921 Weighted, normalized score for performance goal Customer Retention for choice Platinum Card: 0.6
10:36:59,921 Weighted, normalized score for performance goal Revenue for choice Platinum Card: 0.991
10:36:59,921 Total score for choice Platinum Card: 1.591
10:36:59,921 Weighted, normalized score for performance goal Customer Retention for choice Gold Card: 0.6
10:36:59,921 Weighted, normalized score for performance goal Revenue for choice Gold Card: 0.419
10:36:59,921 Total score for choice Gold Card: 1.019
10:36:59,921 Weighted, normalized score for performance goal Customer Retention for choice Miles Card: 0.6
10:36:59,921 Weighted, normalized score for performance goal Revenue for choice Miles Card: 0.207
10:36:59,921 Total score for choice Miles Card: 0.807

Ordering the Choices

10:36:59,921 Comparing choice Waive Fees random value 0.207 with choice Reduced Interest Rate random value 0.818
10:36:59,921 Comparing choice Reduced Interest Rate total score 2.175 with choice Home Mortgage total score 0.32
10:36:59,921 Comparing choice Reduced Interest Rate total score 2.175 with choice Student Loan total score 0.32
10:36:59,921 Comparing choice Reduced Interest Rate total score 2.175 with choice Electronic Payments total score 1.524
10:36:59,921 Comparing choice Reduced Interest Rate total score 2.175 with choice Credit Protection total score 2.225
10:36:59,921 Comparing choice Credit Protection total score 2.225 with choice Platinum Card total score 1.591
10:36:59,921 Comparing choice Credit Protection total score 2.225 with choice Gold Card total score 1.019
10:36:59,921 Comparing choice Credit Protection total score 2.225 with choice Miles Card total score 0.807
10:36:59,921 Winner number 0 is Credit Protection
10:36:59,921 Selected choices: Credit Protection

2.1.4 Application Performance Trends

The Application Performance Trends report shows a summary of important trends related to events associated with choices defined in the Inline Service.

The Application Performance Trends report shows the relative percentage of change in the number of offers extended across all models in the Inline Service for a particular time period. Each offer (Choice column) is shown for each event that occurred during the time period.

In the CrossSell example:

There are three events defined by the Inline Service for cross sell offers: Delivered, Interested, and Purchased. Delivered is the baseline event.

The % Count Change is the amount of change in Interested or Purchased events from time period to time period.

The % Relative Change is the amount of change in Interest or Purchase rates over baseline.

Figure 2-4 shows an example of the Application Performance Trends report.

Figure 2-4 Application Performance Trends Report

2.2 About Strategic Objectives Reports

This section describes the reports that are available at the Strategic Objectives level. Strategic Objectives reports are identified under a branch titled Strategic Objectives in the Inline Service Navigator. For example:

This section contains the following topics:

Section 2.2.1, "Performance Goals Definition Overview"

Section 2.2.2, "Performance Goal Definition Overview for Each Performance Goal"

Section 2.2.3, "Decisions Definition Overview"

Section 2.2.4, "Decision Definition Overview for Each Decision"

Section 2.2.5, "Decision Definition Priorities for Each Decision"

2.2.1 Performance Goals Definition Overview

The Performance Goals Definition Overview report lists all performance goals defined in the Inline Service. In the example shown in Figure 2-5, the advice given by the Inline Service is optimized to both maximize customer retention and to maximize revenue.

Figure 2-5 shows an example of the Performance Goals Definition Overview report.

Figure 2-5 Performance Goals Definition Overview Report

2.2.2 Performance Goal Definition Overview for Each Performance Goal

Each performance goal also has a Performance Goal Definition Overview report. This report shows details about each performance goal, including whether it is required, a normalization factor, and which direction in which to optimize.

Note:

If you have more than one performance metric, and they use different units, a Normalization Factor has been set to normalize score values. For instance, if you had another metric called "Minimize hold time" measured in seconds, an example of the normalization factor would be how many minimized seconds are worth a dollar (revenue) to your organization.

Figure 2-6 shows the Definition Overview report for the Customer Retention performance goal.

Figure 2-6 Performance Goal Definition Overview Report

The Required flag indicates whether a score is needed for each choice when using this performance goal. When Required is true, this indicates that random value might be used for this performance goal, when associated with a non-convergent predictive model.

2.2.3 Decisions Definition Overview

The Decisions Definition Overview report lists all Decisions that are defined in the Inline Service. There are two decisions in our example: OfferDecision and RandomDecision. A Decision is used by an Advisor to determine eligible Choices, score those Choices dynamically, weight the scoring according to segments of the population, and select the best outcome of the choice. The Choices that a Decision selects represent the advice that is given to a requesting operational system, such as the CRM system in our example.

Each Decision shows a description, and the Performance Goals associated with this Decision.

Figure 2-7 shows an example of the Decisions Definition Overview report.

Figure 2-7 Decisions Definition Overview Report

2.2.4 Decision Definition Overview for Each Decision

Each Decision also has a Decision Definition Overview report. This report shows a description of the Decision, the number of Choices (advice) returned, and the Choice Group from which it picks.

The OfferDecision definition overview report shows the following description:

The OfferDecision decides on the offer to suggest to the customer. Offers that support the customer retention performance metric will be more heavily weighted for the segment of the population that is eligible for retention.

Figure 2-8 shows the Definition Overview report for the Decision called "OfferDecision."

Figure 2-8 Decision Definition Overview Report

Note that you can navigate to the Offers Choice Group by clicking the Offers link.

2.2.5 Decision Definition Priorities for Each Decision

Decision priorities indicate the relative importance of performance goals. Different priorities can be defined for different situations or customer segments.

For instance, in the example shown in Figure 2-9, for the decision OfferDecision, we have segmented a group of customers that it is important to retain, in the context of deciding which offer to present. You can see that the Customer Retention performance goal is heavily weighted for this segment of population.

For the remaining population, we choose to weight our Maximize Revenue performance goal more heavily.

See Section 4.2.1, "Using Filtering Rules to Segment Population" for information on segmenting population.

Note:

Changing priorities in this report is possible only when the Inline Service is opened is in edit mode.

Figure 2-9 shows the Definition Priorities report for the Decision called "OfferDecision." The CrossSell Inline Service was opened in edit mode.

Figure 2-9 Decision Definition Priorities Report

2.3 About Decision Process Reports

This section describes the reports that are available at the Decision Process level. Decision Process reports are identified under a branch titled Decision Process in the Inline Service Navigator. For example:

Choices and Choice Groups are hierarchical. Choice Groups are useful for organizing choices. Choice Groups can be used in one of two ways: they provide a way to organize the observations that are collected and analyzed; they are also a way to organize the feedback we will give to the business process through the Advisor Integration Points.

Impact of Dynamic Choices on Decision Center Reports

Choices can be either static or dynamic:

Static choices are explicitly defined in an Inline Service

Dynamic choices are retrieved at run time from external data sources or can be generated programmatically at run time

Note:

In Decision Center, whereas all static choices are displayed in the tree regardless of whether events associated with them have occurred, only those dynamic choices that have been added to model learning data appear in the Inline Service Navigator.

As a result, dynamic choices in the tree and associated reports will only appear in Decision Center with the first occurrences of the associated events.

During the definition of dynamic choices in an Inline Service, another factor influences how dynamic choices appear in the Decision Center, namely the parameter Maximum number of choices within one choice group folder on decision center, also referred to in this section as the Maximum choices parameter. You specify this parameter when you define the Choice Group that includes dynamic choices.

In Decision Center reports, each static choice is preceded by a check mark icon:

and each dynamic choice is preceded by an icon, that contains a check mark and an asterisk:

The Choice Group folder under which the dynamic choices appear can be represented in one of two ways, that depends on the Maximum choices parameter:

If the total number of choices exceeds the maximum defined for the Choice Group folder, the choices appear in system-created "range groups" or subfolders

Otherwise all the choices appear directly under the Choice Group name

The name of each system-created folder is made up of the names of the first and last choices in the folder, with the string "..." separating the two choices. System-created folders are also known as range folders. The icon representing the range folder contains a check mark and an asterisk.

The example Inline Service Navigator menu in Figure 2-10 shows the following:

Five dynamic choices were recommended and added to model learning data.

Each dynamic choice appears under one of the two system-created range folder names.

For this example, the maximum number of choices per choice group for the Insurance Services choice group was set to 3.

Figure 2-10 Example of Dynamic Choice Layout in the Inline Service Navigator

Note:

There is an important difference in how you use standard and range folders in the Decision Center.

Whereas you can click the name of a standard folder to open reports for choices within the folder, range folders are for presentation only.

In other words, clicking the name of a range folder will not open any reports. To see reports for a dynamic choice, you must click the dynamic choice itself.

This section contains the following topics:

Section 2.3.1, "Filtering Rule Definition Overview"

Section 2.3.2, "Scoring Rule Definition Overview"

Section 2.3.3, "Choice/Choice Group Definition Overview"

Section 2.3.4, "Choice Definition Eligibility"

Section 2.3.5, "Choice/Choice Group Performance Counts"

Section 2.3.6, "Choice Performance History"

Section 2.3.7, "Choice/Choice Group Analysis Drivers"

Section 2.3.8, "Choice/Choice Group Analysis Trends"

Section 2.3.9, "Choice/Choice Group Analysis Best-Fit"

Section 2.3.10, "Choice/Choice Group Quality"

2.3.1 Filtering Rule Definition Overview

The Filtering Rule Definition Overview report is a graphical representation of a particular filtering rule defined in the Inline Service.

Filtering rules enable business users to define reusable logic that can be referenced in multiple places such as eligibility rules, scoring rules, and decision priorities.

See Chapter 4 for more information about creating new rules.

The example shown in Figure 2-11 defines the population segment that we want to retain. In this example, the segment is defined to be when the following two conditions are met:

The number of calls abandoned by this customer is greater than or equal to six

The customer tenure is greater than or equal to two years

Figure 2-11 shows the Definition Overview report for the Segment to Retain filtering rule. The CrossSell Inline Service was opened in edit mode.

Figure 2-11 Filtering Rule Definition Overview Report

2.3.2 Scoring Rule Definition Overview

The Scoring Rule Definition Overview report is a graphical representation of a particular scoring rules defined in the Inline Service. Scoring rules can be used for assigning scores to Choices (Offers in our example). For instance, the rule shown in Figure 2-12 assigns a different score according to the level of the customer's credit line amount.

Scoring rules enable business users to define reusable scoring logic that can be associated to choice attributes and performance goals.

See Chapter 4 for more information about creating new rules.

Figure 2-12 shows the Definition Overview report for the Reduced Interest Rate Retention scoring rule. The CrossSell Inline Service was opened in edit mode.

Figure 2-12 Scoring Rule Definition Overview Report

2.3.3 Choice/Choice Group Definition Overview

Choice Groups are created for grouping Choices into logical categories and can be set up in nested hierarchies. Choices are always members of a particular Choice Group.

In Decision Center, definition reports are available for Choice Groups and for static Choices only - there are no definition reports for dynamic Choices.

Note:

Unless stated otherwise, in the rest of this section, all references to Choices apply to static Choices only.

Groups that contain other groups and choices are arranged in a nested manner in the Inline Service Navigator, as in the following example which contains Choice Groups and static Choices:

In the examples shown in the following two sections, the Gold Card Choice is part of the Credit Cards Choice Group.

This section contains the following topics

Section 2.3.3.1, "Choice Group Definition Overview"

Section 2.3.3.2, "Choice Definition Overview"

2.3.3.1 Choice Group Definition Overview

The Definition Overview report for Choice Groups provides a summary of the Choice Group. In the example shown in Figure 2-13, Credit Cards contains two Choice Groups: Credit Products and Credit Services.

Figure 2-13 shows the Definition Overview report for the Credit Cards choice group.

Figure 2-13 Choice Group Definition Overview Report

2.3.3.2 Choice Definition Overview

The Definition Overview report for Choices describes the different attributes of the Choice and their values. For each attribute, the report indicates its value, or how that value is inherited from a value or function defined at a higher level of the choice group hierarchy.

These Choice attributes are organized into different categories. In the example shown in Figure 2-14, the categories include Presentation Attributes, Targeting Attributes, and Scoring Attributes.

Figure 2-14 shows the Definition Overview report for the Gold Card choice.

Figure 2-14 Choice Definition Overview Report

2.3.4 Choice Definition Eligibility

Note:

While eligibility rules can be associated with both static and dynamic choices, choice definition eligibility reports are available for choice groups and static choices only, but not for dynamic choices.

Both Choices and Choice Groups can use rules to define when they are eligible to participate in the Decision. In the example shown in Figure 2-15, the Credit Products choice group is eligible for any Decision that uses it.

Eligibility rules on Choices and Choice Groups are additive. So, if a choice that belongs to this Choice Group (for example, Gold Card) has an Eligibility rule, it is executed after this Rule is executed.

Note:

When not explicitly defined, all choices and choice groups are considered eligible for all decisions.

Figure 2-15 shows the Definition Eligibility report for the Credit Products choice group. The CrossSell Inline Service was opened in edit mode.

Figure 2-15 Choice Definition Eligibility Report

2.3.5 Choice/Choice Group Performance Counts

The Choice Performance Counts report shows how each individual Choice is performing within the system. The Choice Group Performance Counts report is a rollup of the Choices contained in the group. Both types of report allow you to set a time frame for the data reported.

This section contains the following topics:

Section 2.3.5.1, "Choice Group Performance Counts"

Section 2.3.5.2, "Choice Performance Counts"

2.3.5.1 Choice Group Performance Counts

In the Choice Group Performance Counts report, counts for all the contained Choices, Base Event, and Outcomes are displayed.

In the example shown in Figure 2-16, the Miles Card was presented (base event Delivered) 5028 times. For the positive outcomes, Interested and Purchased, customers showed interest 292 times (6%), and purchased 41 times (1%).

Figure 2-16 shows the Performance Counts report for the Credit Products choice group.

Figure 2-16 Choice Group Performance Counts Report

Additionally, a Pareto chart is shown for the Choices for each of the outcomes and the base event. A Pareto chart is used to display the relative counts for each choice in decreasing order.

Figure 2-17 shows the Pareto chart of the Performance Counts for the Interested outcome.

Figure 2-17 Performance Counts Pareto Chart

Rollup

You can use the Rollup check box to control the level of detail for the Choice Group Performance Counts report.

With the Rollup check box unset, the report shows counts for the Choices in all the lower-level Choice Groups.

If the Rollup check box is set, the report contents depend on the level of the Choice Group within the Choice Group hierarchy:

For a Choice Group at the lowest level of the hierarchy, the report shows counts for all the Choices in the Choice Group

For a Choice Group at any other level of the hierarchy (the top level or an intermediate level), the report shows summary counts for the next lower level Choice Groups

Note:

Rollup does not apply to the dynamic choice folder names, whose format is <dynamic_choice_1>...<dynamic_choice_2>.

Figure 2-18 shows the Performance Counts report for the intermediate level Credit Cards choice group, with the Rollup check box not set.

Figure 2-18 Intermediate Level Choice Group Performance Counts Report without Rollup

Figure 2-19 shows the Performance Counts report for the intermediate level Credit Cards choice group, with the Rollup check box set.

Figure 2-19 Intermediate Level Choice Group Performance Counts Report with Rollup

2.3.5.2 Choice Performance Counts

In the Choice Performance Counts report, counts for the individual choice are provided for the chosen time period.

Figure 2-20 shows the Performance Counts report for the Miles Card choice.

Figure 2-20 Choice Performance Counts Report

2.3.6 Choice Performance History

The Choice Performance History report is a historical report that shows how the counts of a particular Choice change over time. The graph shows the outcome of a particular choice over the time period selected, and an associated report shows the detailed outcome counts as tabular data.

You can choose to display counts for a "success" event only (such as Interested for an Offer), a "base" event only (such as Delivered), or both a success event and a base event.

The granularity of the data depends on the time period defined for the associated model. Each model time period that lies within the time period req uested for the report is represented in the report, and appears under the appropriate graph bar or pair of bars (and in the Detailed counts section).

If the time period requested for the report spans a time when the model time period was changed, the time periods in the report reflect the different model time period lengths. For example, if the model time period changed from Quarter to Month, the report could show both quarter and month date ranges.

Figure 2-21 shows the Performance History report for the Credit Protection choice, with the associated Detailed counts section appearing under the graphical section. The time period for the associated model was Week, and each pair of bars represents a week's data.

Figure 2-21 Choice Performance History Report with Detailed Counts Section

2.3.7 Choice/Choice Group Analysis Drivers

Choice/Choice Group Analysis reports provide insights into why choices are associated with positive outcomes, such as Interest and Purchase. Exploring these reports enable business analysts to gain a deep understanding of data that is correlated with each of these events.

Notes:

If the reports show a high correlation between the output and input of a prediction, it may be because the output is represented in the input. For example, when the output is a product category and the input is a product of that product category.

In this case, a more accurate set of correlations will be achieved with Premise Noise Reduction selected for the predictive model associated with the Choice Group. For more information, see the topic "Premise Noise Reduction" in Oracle Real-Time Decisions Platform Developer's Guide.

An attribute value of OTHER indicates that more than the maximum number of discrete values (default is 500) was found. Counts and statistics for all attribute values found beyond the maximum number are accumulated and appear for the OTHER attribute.

This section contains the following topics:

Section 2.3.7.1, "Choice Analysis Drivers"

Section 2.3.7.2, "Choice Group Analysis Drivers"

2.3.7.1 Choice Analysis Drivers

The Choice Analysis Drivers report identifies the attributes that are influential as drivers of predictiveness for each of the choices. The outcome and time frame can be adjusted at the top of the report.

Oracle Real-Time Decisions automatically creates the ordered list of most predictive attributes from the complete list of attributes defined as Entities in the Inline Service.

In the Choice Analysis Drivers report, by clicking an attribute name, you generate several charts associated with that attribute, as shown in Figure 2-22.

Figure 2-22 Choice Analysis Drivers Reports Overview

The rest of this section describes the "base" analysis drivers report and each of the charts associated with a particular attribute.

Figure 2-23 shows the Analysis Drivers report for the Credit Protection choice.

In the examples shown in this section, Credit Line Amount, Marital Status, and AvailableCreditAsPercentOfCreditLine are the three highest predictive factors associated with customer interest in a cross sell offer for credit protection.

Note:

For numeric attributes such as CreditLineAmount and AvailableCreditAsPercentOfCreditLine, Oracle RTD automatically divides the continuum of values into significant ranges, by empirical observation from the data.

For example, for AvailableCreditAsPercentOfCreditLine, the ranges in the example reports are 0, 1 to 21, 22 to 69, 70 to 80, and 81 to 100.

Figure 2-23 Choice Analysis Drivers Report

Figure 2-24 shows additional information about the customer AvailableCreditAsPercentOfCreditLine attribute. The graph and data show that the higher the percentage of available credit, the less likely the customer is to be interested in a cross sell offer for credit protection.

Figure 2-24 Most Important Analysis Drivers Graph and Data

Note:

Input and Output counts in the graph reflect significant correlations, and do not include the cases where an attribute either had no value for a session or a low correlation.

Other graphs are also available. For example, Figure 2-25 shows a relative distribution as compared to the overall population.

This graph shows, for example, that there are proportionally more Interested customers in the 1 to 21 range for AvailableCreditAsPercentOfCreditLine in the Credit Protection offer than there are in the overall population. In the graph, these two values are represented by the second areas from the bottom in the two columns.

The relative area sizes are derived from the values displayed in Figure 2-24, as follows:

For the 1 to 21 range for AvailableCreditAsPercentOfCreditLine, the Actual Count of customers that were interested in the Credit Protection choice is 291, and the total across all ranges is 1112 - a ratio of 0.26.

For the 1 to 21 range for AvailableCreditAsPercentOfCreditLine, the Input Count of all customers is 2468, out of a total across all ranges of 16975 - a ratio of 0.145.

The difference between the two ratios (0.26 and 0.145) is an 81% difference between the two populations, that is, there are 81% more people with the 1 to 21 range for AvailableCreditAsPercentOfCreditLine in the "Interested" audience than in the overall population.

Figure 2-25 Analysis Drivers Relative Distribution Graph

Figure 2-26 shows a Pareto graph of the credit protection customers.

Figure 2-26 Analysis Drivers Pareto Graph: Credit Protection Customers

Figure 2-27 shows a Pareto graph of the entire population.

Figure 2-27 Analysis Drivers Pareto Graph: Entire Population

2.3.7.2 Choice Group Analysis Drivers

The Choice Group Analysis Drivers report summarizes the predictiveness information for the Choices in the Choice Group. The outcome and time frame can be adjusted at the top of the report.

There are two types of Choice Group Analysis Drivers report - with and without rollup:

The Choice Group Analysis Drivers report without rollup shows the average and maximum predictiveness for the most predictive attributes, as well as the Choice within the Choice Group with the maximum predictiveness.

This report also displays the Average Model Quality, which is the arithmetic mean of model qualities at each descendant Choice of the Choice Group.

In this report, you can click the "maximum predictiveness" Choice to display the Choice Analysis Drivers report for that Choice. For more information, see Section 2.3.7.1, "Choice Analysis Drivers."

The Choice Group Analysis Drivers report with rollup shows the most predictive attributes across the combination of all descendant Choices of the Choice Group.

The submodels at each choice are combined for a particular attribute, and then the predictiveness of the attribute is calculated across this composite model.

In this report, you can click an attribute to enable the additional attribute-related graphs (such as the Most Important Values and Pareto graphs) that are available from the Choice Analysis Drivers report. For more information, see Section 2.3.7.1, "Choice Analysis Drivers."

Figure 2-28 shows the Analysis Drivers report for the Loan Services choice group without rollup.

Figure 2-28 Choice Group Analysis Drivers Report without Rollup

Figure 2-29 shows the Analysis Drivers report for the Loan Services choice group with rollup.

In Figure 2-29, DaysToDueDate and AvailableCreditAsPercentOfCreditLine are the top two most predictive factors associated with customer interest in Loan Services. This is an aggregate predictiveness report across all choices within the Loan Service choice group.

Figure 2-29 Choice Group Analysis Drivers Report with Rollup

2.3.8 Choice/Choice Group Analysis Trends

You can see trend reports at both the Choice Group and Choice levels.

This section contains the following topics:

Section 2.3.8.1, "Choice Group Analysis Trends"

Section 2.3.8.2, "Choice Analysis Trends"

2.3.8.1 Choice Group Analysis Trends

At the Choice Group level, the Analysis Trends report shows the change of counts for each of the Choices and outcomes over a period of time. The time frame can be adjusted at the top of the report.

In the CrossSell example:

There are three events recorded by the Inline Service for cross sell offers: Delivered, Interested, and Purchased. Delivered is the baseline event.

The % Count Change is the amount of change in Interested, Delivered, or Purchased events from time period to time period.

The % Relative Change is the amount of change in Interest or Purchase rates over baseline.

Figure 2-30 shows the Analysis Trends report for the Credit Services choice group.

Figure 2-30 Choice Group Analysis Trends Report

2.3.8.2 Choice Analysis Trends

At the Choice level, the Analysis Trends report shows the change of predictiveness for each of the Choices and attributes over a period of time. The time frame can be adjusted at the top of the report. If you have multiple channels, you can see the predictiveness across each channel independently.

Note:

An attribute value of OTHER indicates that more than the maximum number of discrete values (default is 500) was found. Counts and statistics for all attribute values found beyond the maximum number are accumulated and appear for the OTHER attribute.

Figure 2-31 shows the Analysis Trends report for the Credit Protection choice. As you can see from this example, some attributes become more predictive over the time period and some less so.

Figure 2-31 Choice Analysis Trends Report

2.3.9 Choice/Choice Group Analysis Best-Fit

You can see analysis Best-Fit reports at both the Choice Group and Choice levels.

The Best-Fit reports list attribute values that are the most correlated with the selected positive outcome.

Note that there might not exist a record that has all those attribute values at once. In that regard, the Best-Fit report is a hypothetical profile.

Notes:

For a discrete, non-numeric attribute, an attribute value of the highest importance on a Drivers report will also appear as the highest correlating attribute value in the corresponding Best-Fit report, provided that the Actual Count for that driving value (as displayed in the Most Important attribute values table in the Drivers report) is greater than or equal to the Significance Threshold (a parameter defined for the model from which the report is derived).

For a numeric attribute, the attribute values displayed on a Drivers report depend on the Number of bins selected for that report, and will not necessarily match up with attribute values displayed in the corresponding Best-Fit report.

An attribute value of OTHER indicates that more than the maximum number of discrete values (default is 500) was found. Counts and statistics for all attribute values found beyond the maximum number are accumulated and appear for the OTHER attribute.

This section contains the following topics:

Section 2.3.9.1, "Choice Group Analysis Best-Fit"

Section 2.3.9.2, "Choice Analysis Best-Fit"

2.3.9.1 Choice Group Analysis Best-Fit

The Choice Group Analysis Best-Fit report shows all of the attributes and values that are most likely to predict all the outcomes in the Choice Group.

For the example shown in Figure 2-32, the following attributes and values have the highest correlation to a customer being interested in the choices in the Credit Services choice group:

customer CreditLineAmount - 8000

customer Age - 47

customer MaritalStatus - Divorced

Figure 2-32 shows the Analysis Best-Fit Report for the Credit Services choice group.

Figure 2-32 Choice Group Analysis Best Fit Report

2.3.9.2 Choice Analysis Best-Fit

The Choice Analysis Best-Fit report shows all of the attributes and hypothetical values that are most likely to predict the outcome that is specified.

For the example shown in Figure 2-33, the following attributes and values have the highest correlation to the Platinum Card choice:

customer CreditLineAmount - 0

customer Occupation - Homemaker

customer DayOfWeek - Tuesday

The Best-Fit report gives you a snapshot of the most predictive behavior for this cross sell offer.

Figure 2-33 shows the Analysis Best Fit Report for the Platinum Card choice.

Figure 2-33 Choice Analysis Best Fit Report

2.3.10 Choice/Choice Group Quality

Quality reports show how effective models are at predicting positive outcomes in a particular time window.

This section contains the following topics:

Section 2.3.10.1, "Model Quality Concepts"

Section 2.3.10.2, "Choice Group Quality"

Section 2.3.10.3, "Choice Quality"

2.3.10.1 Model Quality Concepts

Predictive models are functions built from an observable data sample and used to estimate an outcome that is unknown at the point of decision. For example:

Based on one year's weather data for an area, what will tomorrow's weather be?

Based on 5 years data of marketing campaign responses, will a given individual respond positively to the next campaign?

The difference between the estimated and actual values occurs for a variety of reasons, for example:

Because of randomness, as the underlying phenomenon is probabilistic in nature

Because the estimator does not account for information that could produce a more accurate estimate

Because the estimator was built on stale data

By the very nature of most data and inter-data associations, estimates of future events and values are always subject to error.

The Oracle RTD quality reports provide the following set of metrics, at both choice and choice group level, to characterize the quality of predictive models:

Model Quality

Cumulative Gains Chart

Model Errors

Model Quality

Model quality is a numeric value, from 0 to 100, that indicates how reliable a model is at predicting an outcome.

Model quality is computed as follows.

In Oracle RTD, model quality is computed incrementally as more records are seen. During the learning phase, there is a period where there is not yet enough data in the model for the model to be reliable. During this period, Decision Center reports show the model quality as 0, until a number of positive counts have been recorded, for example, 50. After the model becomes reliable, Decision Center reports will display model quality.

While the model quality metric distills model evaluation into a single number, it should be understood as a relative number rather than an absolute value.

For example:

A model quality of 80 indicates that in 80% of the cases, the choice that Oracle RTD considers best is actually the best case, regardless of the actual predicted value.

A model quality below 50 is generally not very useful. This may occur for a variety of reasons, such as:

Input data not sufficiently correlated to output

Too much noise introduced by input variables with too many values

Modeling the wrong data

A model quality above 95 should be examined, because some inputs may be self-fulfilling, that is, they are always present when the outcome is present.

Note:

In certain Decision Center reports, this often appears as very high values for the correlation of certain values and very high values for predictiveness of attributes.

Oracle RTD models can be set up to attempt to automatically identify the cases where some attributes have a suspiciously high degree of predictiveness by enabling Premise Noise Reduction in the Model definition. For more information, see the topic "Premise Noise Reduction" in Oracle Real-Time Decisions Platform Developer's Guide.

Cumulative Gains Chart

Cumulative gains charts measure the degree to which the predictions of a model are better than randomly generated predictions.

A cumulative gains chart shows graphically the variation and comparison between the following two sets of data, which form the basis for the axes of the chart:

The total number of positive event predictions made by the model

The accumulated number of base events recorded

To illustrate the structure and interpretation of a cumulative gains chart, consider the following annotated example.

Figure 2-34 Cumulative Gains Chart Example

For the Credit Protection choice, Figure 2-34 shows the effectiveness of the predictions for the positive outcome Interested as compared to the base event Delivered.

To build the data for the chart, the data is first divided into quantiles after it is scored. A typical number of quantiles is 10.

The data is ranked by probability of the positive class from highest to lowest, so that the highest concentration of positive predictions is in the top quantiles.

In Figure 2-34, the X-axis shows the cumulated counts of the base event Delivered for the choice Credit Protection. The first quantile contains the count of the records with the highest likelihood of the positive outcome Interested. The counts for each successive quantile include the aggregated counts of all previous quantiles.

The Y-axis shows the counts of the actual positive outcomes Interested for the appropriate number of accumulated records in the X-axis.

The Cumulative Gains Chart displays three graphs:

The diagonal (base graph) is a straight line, that displays the expected success rate (number of true positives predicted for a sample) which would be obtained for random samples.

The line connects the zero point of the chart to the point represented by (total base event counts, total positive outcome counts).

Figure 2-34 shows the following:

1285 responders out of 6424 (20%) if selected randomly (Point A)

[This means that 20% of the responses would come from the first two quantiles if they were selected randomly.]

3213 responders out of 6424 (50%) if selected randomly (Point B)

The cumulative gains graph indicates the cumulative number of positive classification identified by a model at a given quantile, for example:

57 responders out of 157 (36%) if selected from top 20% highest likelihoods (Point C)

[This means that 36% of the responses would come from the first two quantiles if they were selected using the Oracle Real-Time Decisions model.]

108 responders out of 157 (68%) if selected from top 50% highest likelihoods (Point D)

Note:

The gain provided by a predictive model at a given quantile reflects how good the model is. For example, the gain at the second quantile is 36% (Point C) versus 20% (Point A).

The maximal graph indicates how an ideal model would classify the data, where the base events are ordered such that all the highest likelihood base events lead to positive outcomes.

The first part of the maximal graph is the straight line connecting the zero point to the point represented by (total positive outcome counts, total positive outcome counts). The maximal graph then continues as a horizontal straight line to the top right end point.

Graphically, the more effective the model is at predicting positive outcomes, the higher the elevation is of the cumulative gains graph above the base diagonal. In general, model quality is the area between the cumulative gains graph and the base diagonal.

Model Errors

Model error metrics are various statistical metrics characterizing errors, where errors are the amounts by which predictions differ from the actual values of the quantity being estimated.

For example, Mean Square Error is a metric that incorporates the variance of the estimator, and normalizes positive and negative deviations of the predicted from the actual values.

The basic components used in the calculation of the model errors are actual likelihoods and predicted likelihoods. The average of the actual likelihoods is also used as a component in the model errors.

Actual likelihoods are represented as subscripted "p" elements:

Predicted likelihoods are represented as subscripted "p*" elements:

The average actual likelihood is the average of the actual likelihoods:

The model errors shown on the quality reports are:

Mean Square Error (MSE)

Relative MSE

Root Mean Square Error (RMSE)

Relative RMSE

Mean Absolute Error (MAE)

Relative MAE

Mean Error (ME)

Relative ME

2.3.10.2 Choice Group Quality

Each choice group quality report contains the following report sections:

Cumulative gains chart

Cumulative gains table, which shows the cumulative gains chart data values in table form

Model errors

Glossary describing the model errors

For more information on model quality and cumulative gains charts, see Section 2.3.10.1, "Model Quality Concepts."

Choice Group Quality Cumulative Gains Chart

The Cumulative Gains Chart has the following properties:

The X-axis shows the cumulated counts of all the Choices in a Choice Group.

The Y-axis shows the counts of the actual positive outcomes for the appropriate amount of records in the X-axis.

There are three graphs on each cumulative gains chart:

Base

The Base graph is a straight line, that reflects what you would expect without model predictions.

Cumulative Gains

The Cumulative Gains graph shows the "lift" provided by the model predictions. The size of the area between the Base graph and the Cumulative Gains graph that is above the Base graph reflects the effectiveness or quality of the model.

Maximal

The Maximal graph shows the hypothetical "ideal" situation, where the base events are ordered such that all the first base events lead to positive outcomes.

Figure 2-35 shows the Cumulative Gains Chart for the Choice Group Credit Services for the positive outcome Interested.

Figure 2-35 Choice Group Cumulative Gains Chart

Choice Group Quality Cumulative Gains Table

The Cumulative Gains Table report section shows the values that are plotted and which appear in the Cumulative Gains Chart. The first column shows the X-axis values, the second column shows the Y-axis values.

Choice Group Quality Model Errors

At the Choice Group level, the Model Errors report section shows model errors of predicted likelihoods for the total counts of all the Choices in the Choice Group.

Figure 2-36 shows the Model Errors for the Choice Group Credit Services for the positive outcome Interested.

Figure 2-36 Choice Group Model Errors Report

2.3.10.3 Choice Quality

Each choice quality report contains the following report sections:

Cumulative gains chart

Cumulative gains table, which shows the cumulative gains chart data values in table form

Model errors

Glossary describing the model errors

For more information on model quality and cumulative gains charts, see Section 2.3.10.1, "Model Quality Concepts."

Choice Quality Cumulative Gains Chart

The Cumulative Gains Chart has the following properties:

The X-axis shows the cumulated counts of a particular Choice.

The Y-axis shows the counts of the actual positive outcomes for the appropriate amount of records in the X-axis.

There are three graphs on each cumulative gains chart:

Base

The Base graph is a straight line, that reflects what you would expect without model predictions.

Cumulative Gains

The Cumulative Gains graph shows the "lift" provided by the model predictions. The size of the area between the Base graph and the Cumulative Gains graph that is above the Base graph reflects the effectiveness or quality of the model.

Maximal

The Maximal graph shows the hypothetical "ideal" situation, where the base events are ordered such that all the first base events lead to positive outcomes.

Figure 2-37 shows the Cumulative Gains Chart for the Choice Reduced Interest Rate in the Choice Group Loan Services for the positive outcome Interested.

Figure 2-37 Choice Cumulative Gains Chart

Choice Quality Cumulative Gains Table

The Cumulative Gains Table report section shows the values that are plotted and which appear in the Cumulative Gains Chart. The first column shows the X-axis values, the second column shows the Y-axis values.

Choice Quality Model Errors

At the Choice level, the Model Errors report section shows model errors of predicted likelihoods for the Choice that is the subject of the quality report.

Figure 2-38 shows the Model Errors for the Choice Reduced Interest Rate for the positive outcome Interested.

Figure 2-38 Choice Model Errors Report

2.4 About Integration Point Reports

This section describes the reports that are available at the Integration Point level. Integration Point reports are identified under a branch titled Integration Points in the Inline Service Navigator.

There are two types of Integration Point report, one for Informants, the other for Advisors. For example:

Each integration point has a set of keys and a set of attributes.

This section contains the following topics:

Section 2.4.1, "Informant Reports"

Section 2.4.2, "Advisor Reports"

2.4.1 Informant Reports

The Informant Definition report describes the keys and input parameters for the Informant.

Figure 2-39 shows the Definition report for the Callinfo informant.

Figure 2-39 Informant Definition Report

2.4.2 Advisor Reports

There are two types of Advisor report, Definition and Measure.

This section contains the following topics:

Section 2.4.2.1, "Advisor Definition Report"

Section 2.4.2.2, "Advisor Measure Report"

2.4.2.1 Advisor Definition Report

The Advisor Definition report describes the types, keys, and input parameters for the Advisor.

Figure 2-40 shows the Definition report for the OfferRequest advisor.

Figure 2-40 Advisor Definition Report

2.4.2.2 Advisor Measure Report

The Advisor Measure report describes the statistics for an advisor for a particular time window. The statistics are:

The number of times that the advisor was called

The number of times that the advisor used the Control Group decision

The percentage of Control Group calls within the total number of calls

Figure 2-41 shows the Measure report for the OfferRequest advisor.

Figure 2-41 Advisor Measure Report

2.5 About Entity Catalog Reports

This section describes the reports that are available at the Entity Catalog level. Entity Catalog reports are identified under a branch titled Entity Catalog in the Inline Service Navigator. For example:

The Entity / Analysis / Counts report and Entity / Analysis / Statistics reports typically draw their information from a learning model called "Statistics" that is automatically added to any Inline Service you create. If the reports are unable to access the Statistics model, you may choose any other model from which to examine the counts by entity. The advantage of having this separate Statistics model is that its only job is to record the objective characteristics of the population, acting as a historian.

This section contains the following topics:

Section 2.5.1, "Entity Definition Overview"

Section 2.5.2, "Entity Analysis Counts"

Section 2.5.3, "Entity Analysis Statistics"

Section 2.5.4, "Entity Analysis Drivers"

Section 2.5.5, "Entity Analysis Trends"

2.5.1 Entity Definition Overview

The Entity Definition Overview report describes all of the attributes for the Entity.

Figure 2-42 shows the Definition Overview report for the Customer entity.

Figure 2-42 Entity Definition Overview Report

2.5.2 Entity Analysis Counts

The Entity Analysis Counts shows the total count of how many times a particular attribute has been part of a transaction in the Inline Service. The Entity Analysis Count report allows you to set a time frame and choose an attribute for the data reported.

Note:

An attribute value of OTHER indicates that more than the maximum number of discrete values (default is 500) was found. Counts and statistics for all attribute values found beyond the maximum number are accumulated and appear for the OTHER attribute.

Figure 2-43 shows the Analysis Counts report for the Customer entity. This example shows the distribution of the reason for calls into the call center.

Figure 2-43 Entity Analysis Counts Report

In addition, a pie chart is provided, as shown in Figure 2-44.

Figure 2-44 Entity Analysis Counts Pie Chart

2.5.3 Entity Analysis Statistics

The Entity Analysis Statistics reports shows the minimum, maximum, median, average, and sigma values for each of the numeric attributes of the entity. For instance, in the example shown in Figure 2-45, the median age of customers is 48.

Note:

An attribute value of OTHER indicates that more than the maximum number of discrete values (default is 500) was found. Counts and statistics for all attribute values found beyond the maximum number are accumulated and appear for the OTHER attribute.

Figure 2-45 shows the Analysis Statistics report for the Customer entity.

Figure 2-45 Entity Analysis Statistics Report

2.5.4 Entity Analysis Drivers

The Entity Analysis Drivers report identifies the attributes that are influential as drivers of predictiveness for the Entity. The outcome and time frame can be adjusted at the top of the report.

Additionally, a best fit report for the Entity is included. This shows the importance of specific values on predictiveness of an attribute.

The Entity Analysis Drivers report helps business users to analyze how a specific attribute correlates with any of the predictive models defined in the Inline Service. Unlike the Choice Analysis reports that only look at the correlation between one choice and attributes, the Entity Analysis Drivers report helps users understand the choices that specific attribute values are most successful at predicting.

Note:

An attribute value of OTHER indicates that more than the maximum number of discrete values (default is 500) was found. Counts and statistics for all attribute values found beyond the maximum number are accumulated and appear for the OTHER attribute.

Figure 2-46 shows the Analysis Drivers report for the Customer entity.

Figure 2-46 Entity Analysis Drivers Report

With reference to the example shown in Figure 2-46, the following observation can be made:

Looking at the Customer entity, and the particular attribute Amount of Pending Transactions, the most likely prediction is that customers will be interested in an offer to reduce their interest rate, especially if the customer has between 901 to 1200 pending transactions.

2.5.5 Entity Analysis Trends

The Entity Analysis Trends report shows the change of predictiveness for an attribute of an Entity over a period of time. The time frame can be adjusted at the top of the report.

Note:

An attribute value of OTHER indicates that more than the maximum number of discrete values (default is 500) was found. Counts and statistics for all attribute values found beyond the maximum number are accumulated and appear for the OTHER attribute.

Figure 2-47 shows the Analysis Trends report for the Customer entity.

Figure 2-47 Entity Analysis Trends Report

A trend chart is also provided, as shown in Figure 2-48.

Figure 2-48 Entity Analysis Trends Chart

3 Working with Choice Groups and Choices

Choices can be organized into groups of choices, in other words, Choice Groups.

This enables the following:

Rollup performance and analysis reports are available at the group level

Eligibility and scoring logic shared across multiple Choices can be defined at the group level

Choice Groups and Choices are structured in a hierarchical fashion. Both attributes and rules of Choice Groups and Choices are inherited from parents to children. This allows Oracle Real-Time Decisions to have Choice Groups configured in a way that ensures uniformity when the Inline Service is first deployed, but also allows the flexibility to add additional individual Choices and Choice Groups through the Decision Center interface after the Inline Service is deployed.

This chapter contains the following topics:

Section 3.1, "About Modifying and Redeploying Inline Services"

Section 3.2, "Adding Choice Groups or Choices"

Section 3.3, "Modifying Choice Groups or Choices"

Section 3.4, "Cloning Choice Groups or Choices"

Section 3.5, "Deleting Choice Groups or Choices"

Section 3.6, "Moving Choices"

3.1 About Modifying and Redeploying Inline Services

There are three areas where permissions can be set for users of Decision Center:

Editing permission for the Inline Service with which you are working.

Permission to deploy to a server, potentially in a cluster.

Permission to use a perspective in Decision Center.

In order to edit and redeploy an Inline Service, you must have permission to edit, permission to deploy to a server, and permission to use the Design perspective.

If you receive an error when trying to open the Inline Service for editing, you may not have permission on Real-Time Decision Server or with the Inline Service to edit. Contact your system administrator to check your rights.

In general, to make changes to the Inline Service:

When opening the Inline Service, select Open the Inline Service for editing. This gives you exclusive editing rights to the Inline Service.

Change to the Design perspective.

Select the element that you would like to modify, and use the editing pane to make changes.

Redeploy to Real-Time Decision Server using the Redeploy button on the toolbar. The Inline Service is redeployed to the deployment state in which it was opened for editing. By default, deployment states are: Development, QA, and Production. Your system administrator may have added custom states.

To promote an Inline Service to a new state, use the Promote button on the toolbar. Choose the deployment state to which you want to promote the Inline Service.

As you redeploy or promote an Inline Service, you have the option to keep the Inline Service open for editing. This retains your exclusive editing rights for the Inline Service. If you choose to redeploy without keeping it open for editing, others will be able to edit the Inline Service.

Certain elements of deployed Inline Services can be modified by the business user in Decision Center. The following sections outline the process for modifying Inline Services.

3.2 Adding Choice Groups or Choices

Adding a Choice Group and adding a Choice are different procedures.

This section contains the following topics:

Section 3.2.1, "Adding Choice Groups"

Section 3.2.2, "Adding Choices"

3.2.1 Adding Choice Groups

To add a Choice Group:

Select the Choice Group under which you want the new group to appear, then select the Configuration tab.

You cannot create a new top-level Choice Group.

Use the New Group subtab to create a new Choice Group object.

Name the Group and click Create. The Group is created under the Choice Group you originally selected.

3.2.2 Adding Choices

To add a Choice:

Select the Choice Group in which you want the new Choices to appear, then select the Configuration tab.

Use the New Choices subtab to create new Choice objects.

Enter up to ten Names for the Choices, then click Create. The Choices are created under the Choice Group you originally selected. They inherit the attributes of the Choice Group.

See Section 3.3, "Modifying Choice Groups or Choices" for information about setting choice attributes.

3.3 Modifying Choice Groups or Choices

To modify a Choice Group or Choice:

Select a Choice Group or Choice to modify, then select the Configuration tab.

Use the Basic subtab to modify the Name or Description.

Use the Attributes subtab to modify the value of the attributes. Attributes may only be edited at the Choice level.

3.4 Cloning Choice Groups or Choices

To clone a Choice Group or Choice:

Select the Choice Group or Choice to modify, then select the Configuration tab.

Use the Clone subtab to clone the object. Cloning an object produces an identical object, with the same values for all attributes. The only difference is the name given to the new object.

To change attribute values on Choices, use the Attribute tab.

3.5 Deleting Choice Groups or Choices

To delete a Choice or Choice Group:

Select the Choice Group or Choice you want to delete, then select the Configuration tab.

Use the Delete subtab to delete Choice Groups or Choices.

Confirm the deletion, then click Delete.

3.6 Moving Choices

To move a Choice:

Select the Choice Group that contains the Choices to move, then select the Configuration tab.

Use the Move subtab to move Choices.

Select the Choices to move and the Choice Group in which to move them, then click Save.

4 About Rules

There are three types of Rules:

Eligibility Rules are used to determine whether Choice Groups and Choices are eligible to participate in a Decision.

Filtering Rules are reusable components of decisions and other rules. They are used to segment population, so that performance metrics can be weighted more heavily for one segment than another.

Scoring Rules are used for scoring Choices Groups and Choices. The score is used by the Decision to determine the proper Choice.

This chapter contains the following topics:

Section 4.1, "About Eligibility Rules"

Section 4.2, "About Filtering Rules"

Section 4.3, "Adjusting the Priorities of Business Goals by Population Segment"

Section 4.4, "About Scoring Rules"

Section 4.5, "Working with Rules"

4.1 About Eligibility Rules

As part of a Choice or Choice Group, the Eligibility Rule determines whether the Choice is eligible to participate in the Decision or logic that makes the Choice selection. Choice Groups and Choices are available in the Inline Service Navigator under Decision Process.

Choice Groups are identified by the following icon:

Choices are identified by the following icon:

An eligibility rule determines whether the sub-tree headed by a Choice Group is eligible for a decision. Note that even if Choices themselves are eligible, they will not be eligible unless all their parents are eligible.

Choice Group Eligibility Rules are accessed by selecting the Choice Group and then selecting Definition > Group Eligibility or Choice Eligibility.

Choice Eligibility Rules are accessed by selecting the Choice and then selecting Definition > Eligibility.

This section contains the following topics:

Section 4.1.1, "Adding Eligibility Rules"

Section 4.1.2, "Evaluating Choice Group Rules and Choice Rules"

Section 4.1.3, "Determining Eligibility"

4.1.1 Adding Eligibility Rules

To add an eligibility rule, go to the Group Eligibility, Choice Eligibility, or Eligibility tab of the Choice or Choice Group, then add a new rule. See Section 4.5.3, "Adding Statements to Rules" for details of how to create rules and to add statements to them.

Note:

Rule modification is possible only if the Inline Service is opened for editing.

4.1.2 Evaluating Choice Group Rules and Choice Rules

Choice Group rules and Choice rules are inherited and additive. That is, if there are rules at the Choice Group (Group and Choice rule) and rules at the Choice level, it is as if there is a logical AND extending the rules. The inherited rules are shown in an expandable section at the top of the rule labeled Inherited eligibility conditions. Use the up arrow and down arrow buttons to expand and collapse the sections.

Example 4-1 shows an example of how Choice Group rules and Choice rules are evaluated.

Example 4-1 Evaluating Choice Group Rules and Choice Rules

Group1 has rules GroupRule1 and ChoiceRule1
    Group2 is a child of Group1 and has rules GroupRule2 and ChoiceRule2
        Group2 has a Choice, Choice1, and it has a rule, Rule1

In this example, when evaluating the rules for Choice1, the following rules are invoked:

GroupRule1 AND GroupRule2 AND ChoiceRule1 AND ChoiceRule2 AND Rule1

4.1.3 Determining Eligibility

When determining eligibility for a Choice, parent eligibility is tested first, to avoid the unnecessary evaluation of eligibility rules on Choices.

Example 4-2 shows an example of how eligibility is determined for a Choice.

Example 4-2 Determining Eligibility for a Choice

Group1 has rule GroupRule1
Group2 is a child of Group1 and has rule GroupRule2
Group2 has a Choice, Choice1, and it has a rule, Rule1

In this example, eligibility is determined in the following manner:

If Choice1 is eligible with Rule1 test with GroupRule2 if eligible test with GroupRule1

4.2 About Filtering Rules

As standalone rules, filtering rules can be used to segment population or be used as components of other rules. Standalone rules are reusable by many different elements.Standalone filtering rules are accessed under Decision Process > Rules Library > Filtering Rules.

4.2.1 Using Filtering Rules to Segment Population

Filtering rules can be used by any other rule and also for decision priorities. In the latter case, filtering rules are assigned to a Decision to apply performance metrics to different segments of a population. Figure 4-1 shows a typical filtering rule used to segment population.

Figure 4-1 Filtering Rule

The scoring rule shown in Figure 4-1 targets customers over the age of 18 with a credit line amount over $8000. Filtering rules are used by Decisions.

To use a Filtering rule to segment population:

To add a new rule, click the filtering rule icon on the toolbar above the navigator. The filtering rule icon appears as follows:

Create a standalone filtering rule for segmentation.

See Section 4.5.3, "Adding Statements to Rules" for details of how to create rules and to add statements to them.

Select the Decision for which you want to segment population. Select Definition > Priorities.

In the editor, using Segment, select the filtering rule that defines the segment of population.

Optionally, you can adjust the weighting of the performance metrics for this segment.

Click the Save icon to save your changes.

4.3 Adjusting the Priorities of Business Goals by Population Segment

One of the changes you can make to an Inline Service is to adjust the priorities of your business goals. The outcome of a decision is made based on the performance goals that are defined for your organization, the population segments that have been defined, and the prioritization of those goals for each segment of the population. Priorities can be adjusted on the Decision > Definition > Priorities subtab.

Note:

Rule modification is possible only if the Inline Service is opened for editing.

Figure 4-2 shows an example of priorities for business goals.

Figure 4-2 Priorities for Business Goals

In the example shown in Figure 4-2, the population segment called Segment to Retain has the Customer Retention business goal heavily weighted in its favor (71%), while the remaining population has maximizing revenue as a priority (29%).

If there is a special circumstance or a change in population, you can adjust this prioritization by sliding the weighting bar. Once adjusted, you must redeploy your Inline Service and log in to Decision Center again to see your changes.

4.4 About Scoring Rules

As opposed to eligibility rules that return Boolean values, scoring rules return numeric values. These values can be used throughout the Oracle RTD Decision logic. Typical use cases are:

Setting the score of a choice for a given performance goal

Setting the value for a choice attribute

Figure 4-3 shows an example of a scoring rule.

Figure 4-3 Scoring Rule

In the example shown in Figure 4-3, scores are assigned based on a customer's credit line amount. If a customer does not fit any of the credit line range categories, the score defaults to 3.

4.5 Working with Rules

Rules are used for several purposes within Decision Studio and Decision Center, namely:

For determining the eligibility of Choice Groups and Choices to take part in a Decision

As standalone, reusable rules for filtering population segments

As standalone, reusable rules for scoring Choices

The editors that are used to create rules are very similar. The following sections describe how to create rules using these editors and the Rule Editor toolbar.

Note:

Rule modification is possible only if the Inline Service is opened for editing.

This section contains the following topics:

Section 4.5.1, "About the Rule Editor Toolbar"

Section 4.5.2, "Oracle RTD Rule Terms and Statements"

Section 4.5.3, "Adding Statements to Rules"

Section 4.5.4, "Selecting an Operator"

Section 4.5.5, "Editing Boolean Statements"

Section 4.5.6, "Editing Rule Properties"

Section 4.5.7, "Inverting Rule Elements"

Section 4.5.8, "Moving Rules and Rule Sets"

4.5.1 About the Rule Editor Toolbar

The Rule Editor toolbar provides access to features used to edit rules. This toolbar is sensitive to the context of the task you are performing.

Figure 4-4 Rule Editor Toolbar

From left to right, the toolbar functions are as follows:

Edit rule properties

Add conditional value

Add Rule

Add Rule Set

Delete

Invert

Move up

Move down

Copy

Cut

Paste

4.5.2 Oracle RTD Rule Terms and Statements

Oracle RTD has three types of rules:

Filtering rules

Scoring rules

Eligibility rules

An Oracle RTD rule consists of one of more rule conditions, with logical operators governing how the conditions are combined. These conditions are expressed in the form of rule statements, as described in the section.

Note:

This section describes rules that have conditions, that is, they are not always true.

The following is an example of a simple filtering rule, called Select List Rule, used to illustrate rule statements in this section:

Select List Rule is true when
All of the following
1. session / customer / Age > 21
2. session / customer / MaritalStatus = "MARRIED"

Table 4-1 shows a more formal representation of the Oracle RTD rule grammar, using BNF (Backus-Naur Form)-type conventions, which describes the terms used in Oracle RTD rule statements.

Table 4-1 Oracle RTD Rule Grammar

Term
Term Component
Notes

<rtd rule>

<header line> <logical operator> <rule entry>+ |

<header line> <array operator> <logical operator> <rule entry>+

None.

<header line>

<rule name> is true when

Provides the name for the rule.

After the rule is created, the header line is not editable.

<logical operator>

All of the following | Any of the following |

None of the following | Not all of the following

The logical operator controls the <rule entry> immediately following. The logical operator All of the following is the default logical operator when you initially create a rule.

<rule entry>

<boolean statement> | <rule set>

Rule entries are always numbered. They may contain boolean statements or other rule entries.

<rule set>

<logical operator> <rule entry>+ |

<array operator> <logical operator> <rule entry>+

The second <rule set> component option is known as an array-processing rule set.

<boolean statement>

<boolean> |

<boolean function> |

<left operand> <relational operator> <right operand>

The <boolean statement> is the lowest level of a rule set - it cannot be decomposed further.

<array operator>

<quantifier> <array variable> in <array name>,

Used only for array-processing rule sets. See "Quantifiers and Array-Processing Rules" for details.

<quantifier>

For all | There exists

None.

The Select List Rule example can be categorized as follows:

Select List Rule is true when = <header line>

All of the following = <logical operator>

The remainder of the rule consists of a <rule entry> made up of the following two statements:

session / customer / Age > 21 = <boolean statement>

session / customer / MaritalStatus = "MARRIED" = <boolean statement>

Rule Sets and Boolean Statements

A rule set (as denoted by <rule set> in Table 4-1) is a composite statement, that consists of one or more numbered rule entries, each of which is either a boolean statement or another rule set.

A boolean statement (as denoted by <boolean statement> in Table 4-1) contains a condition that evaluates to true or false when the rule is processed. The boolean statement is the lowest-level element of a rule set - it cannot be decomposed further.

You can optionally name the rule sets defined within higher-level rule sets.

Note:

Each Oracle RTD rule has an implicit, unnamed, top-level rule set.

Each rule set is qualified by a logical operator, which controls the processing of the rule set statements. See "Logical Operators" for more information.

The following Exclusion Rule example shows a rule set within a rule set.

In the example:

The top-level unnamed rule set contains the logical operator All of the following and two rule entries. The first rule entry is a boolean statement, the second rule entry is a rule set.

The rule set inside the top-level rule set contains the logical operator None of the following and three rule entries, each of which is a boolean statement.

Note:

Oracle RTD also supports rule sets that depend on values in an array. See "Quantifiers and Array-Processing Rules" for more details.

For rule sets which depend on values in an array, the rule set has an additional, overall "quantifier" expression (either For all or There exists). This quantifier qualifies the logical operator of the rule set. See "Quantifiers and Array-Processing Rules" for more information.

Logical Operators

The Oracle RTD logical operators are as follows:

All of the following (logical and). The rule set evaluates to true when all of its boolean statements and lower-level rule sets are satisfied.

Any of the following (logical or). The rule set evaluates to true when any one of its boolean statements or lower-level rule sets is true.

None of the following (logical not and). The rule set evaluates to true when all of its boolean statements and lower-level rule sets are false.

Not all of the following (logical not or). The rule set evaluates to true when any one of its boolean statements or lower-level rule sets is false.

Quantifiers and Array-Processing Rules

Rule sets may depend on values that occur in arrays, as follows:

Rule sets can evaluate elements of an array

An expression within a rule set can reference elements of an array

These types of rules are referred to as array-processing rules.

For these cases, there are rule sets where a "quantifier" expression - also referred to as a quantifier - qualifies the logical operator of the rule set. Either the statements of the rule set must be fulfilled for all array elements, or they must be fulfilled for at least one array element.

In the following example, a rule has been created which examines all of the agents contained in an array attribute, session/agents. In this example, the rule evaluates to true when all the agents are at least 30 years old and have a status of "Qualified."

Agent Rule is true when
For all people in session/agents, All of the following
1. people / Age >= 30
2. people / Status = "Qualified"

Full details of the syntax of array-processing rules appear in the section following. In the preceding example, the term people is an arbitrary term that is defined by the user and is used to identify individual array elements in the rule set.

Array-Processing Rule Qualification

The general formula for an array-processing rule qualification is:

<quantifier> <array_variable> in <array_name>, <logical_operator>

where:

quantifier is one of the following: For all, There exists

For all. This quantifier, together with the rule set logical operator, specifies that each array element must be examined, and for each array element, all the boolean statements and lower-level rule sets in the qualified rule set must be fulfilled.

Rule evaluation will stop processing the array as soon as one array element is found where all the boolean statements and lower-level rule sets in the qualified rule set are not fulfilled. The rest of the elements in the array will be skipped.

There exists. This quantifier, together with the rule set logical operator, specifies that all the boolean statements and lower-level rule sets in the qualified rule set must be fulfilled for at least one array element.

Rule evaluation will stop processing the array as soon as one array element is found where all the boolean statements and lower-level rule sets in the qualified rule set are fulfilled. The rest of the elements in the array will be skipped.

array_variable is an arbitrary name to identify individual array elements of the array array_name in the boolean statements and lower-level rule sets in the qualified rule set

Notes:

The array_variable can still be referenced in the boolean statements of lower-level rule sets that have their own specific array variables.

The array_variable must be unique within the scope of the rule, that is, you cannot use the same array_variable name as the array variable for a lower-level array-processing rule.

array_name is the array to be examined

logical_operator is one of the following: All of the following, Any of the following, None of the following, Not all of the following

As an example of an array-processing rule set, consider the two entities session and customer.

The session entity contains the following attributes:

The Integer attribute AgentDept

The array attribute CustInfo of type customer

The customer entity contains the following attributes:

The Integer attribute CompSize

The String attribute Region

You require a filtering rule to satisfy both of the following conditions:

The value of AgentDept must be 42.

For at least one customer in the CustInfo array, the CompSize must be > 100, and the Region must be "West."

The filtering rule could then be defined as follows (the array qualification expression is highlighted):

Cust Rule is true when
All of the following
1. session / AgentDept = 42
2. There exists some_customer in session / CustInfo, All of the following
   1. some_customer / CompSize > 100
   2. some_customer / Region = "West"

4.5.3 Adding Statements to Rules

Add Rule Set

To add a rule set, click the Add Rule Set icon:

If this is the first element to be created in the rule, the following statements appear in the rule:

The default logical operator All of the following

A rule set entry as the numbered first line in the rule, which itself contains the default logical operator All of the following

An empty two-operand boolean statement within the newly-defined rule set

Otherwise, a new rule set entry is added as the next entry in the current rule set, containing an empty two-operand boolean statement. For example, when you add a rule set to an existing rule set that already contains one boolean statement, the added rule set entry appears beside the line number 2, as in the following:

You can name the rule set, by clicking the top right corner. When a rule set is named, you can collapse or expand the rule set, by clicking the appropriate chevron icon in the top right corner of the rule set box.

Add (Boolean Statement to) Rule

To add a boolean statement, click the Add Rule icon:

If this is the first element to be created in the rule, the default logical operator All of the following appears, followed by an empty two-operand boolean statement, as in the following:

Otherwise, an empty two-operand boolean statement is added to the current rule set, as in the following example, where one boolean statement already exists:

By default, boolean statements have two operands, with an intervening operator.

To switch between single and double operands in boolean statements, click the line number of the boolean statement, then click the arrowhead icon in the lower-right corner of the boolean statement box, as in the following example:

Single operands always evaluate to a Boolean.

4.5.4 Selecting an Operator

Click the operator, then click the lower-right corner to select an operator:

Table 4-2 lists the available operators.

Table 4-2 Rule Operators

Operator
Description

none

A simple expression that has only one operand

=

Left is equal to Right

<>

Left is not equal to Right

<

Left is less than Right

<=

Left is less than or equal to Right

>

Left is greater than Right

>=

Left is greater than or equal to Right

in

Left value is contained in a List on the Right side

not in

Left value is not contained in a List on the Right side

includes all of

Left list includes all the values of the Right list

excludes all of

Left list contains none of the values of the Right list

includes any of

Left list includes any one of the values of the Right list

does not include all of

Left list does not include all of the values of the list on the Right

4.5.5 Editing Boolean Statements

To edit the boolean statements of a rule, click the left side, then click the ellipsis. You can choose from a constant, attribute, or function call. Select Array at the top of the page to specify an array value.

Note:

Use of the Esc key at any time in an operand dialog will close the dialog.

If you choose Constant, provide the Data type and a Value for the item. If you selected Array, add as many items to the array as needed. Then, for each item, choose a Data Type and provide a Value.

If you choose Attribute, provide one of the following:

Group attribute: Attributes that are part of the Choice Group or its Choices that is selected in the Properties of the rule.

Session attribute: Attributes that are part of the Session entity.

Application attribute: Attributes that are a member of the Application element.

Array variable: Names used to identify individual array elements for rule sets within a quantified logical operator expression. See "Quantifiers and Array-Processing Rules" for more information.

Optionally, select Apply filter type and choose a Data type to filter the attributes by type. If you have selected Array, add as many items to the array as needed, then assign an attribute value for each.

If you choose Function call, provide one of the following:

Filtering rules: Standalone filtering rules defined for the Inline Service.

Scoring rules: Standalone scoring rules defined for the Inline Service.

Function calls: Standalone functions defined for the Inline Service.

Optionally, select Apply filter type and choose a Data type to filter the attributes by type. If you have selected Array, add as many items to the array as needed, then assign a function or rule for each.

4.5.5.1 Using Type-Restricted Objects in Rules

When you select a type-restricted object for an operand or part of an operand, you may view values from a dropdown list of values for the other operand. You may select a value from this dropdown list, but you do not have to.

For more information about type restrictions and type-restricted objects, see the topic "About Type Restrictions" in Oracle Fusion Middleware Platform Developer's Guide for Oracle Real-Time Decisions.

4.5.6 Editing Rule Properties

Both Filtering and Scoring rules have rule properties that can be set. To edit rule properties, click the Rule properties icon:

Edit rule properties appears.

Rule properties include call templates and negative call templates. Call templates provide a user-friendly way to describe how to call a rule from another rule.

To define a call template, add the number of parameters for the rule by clicking the Add button under Parameters. Using {0}, {1}, and so on as arguments, and phrasing to describe the rule, define the template for call. It is important to use good phrasing, as this is what will be shown when using the rule.

For instance, a rule that checks if there were at least x calls from the user in the last y days could be phrased as follows:

There were at least {0} calls in the last {1} days

The negative call template is used when a rule is inverted, and should express the opposite. For example:

There were less than {0} calls in the last {1} days

Rule properties also let you assign which Choice Group to use with the rule. By selecting Use with choice group, you can specify which Choice Group or its Choices will provide the Choice attributes for use by parameters. These attributes will be available when you edit the value of an operand.

4.5.7 Inverting Rule Elements

Use the Invert icon to invert different elements of a rule. By selecting the number of a boolean statement, you can invert the operator of the boolean statement. For instance, if the operand was =, it will be inverted to <>.

Logical operators for a rule set can also be inverted. To do this, select the logical operator and click Invert. For instance, All of the following becomes Not all of the following.

The final use for Invert is to invert a Boolean, or single operand, rule. When this type of rule is inverted, it is transformed to the negative call template of the function that defines the rule.

4.5.8 Moving Rules and Rule Sets

To change the order of rules and rule sets within a rule:

Select the rule entry by clicking its number.

Use the following operations from the Rule Editor toolbar to move rule entries:

Delete

Copy selection to clipboard

Cut selection to clipboard

Move up

Move down

See Section 4.5.1, "About the Rule Editor Toolbar" for information about the Rule Editor toolbar icons.

Click the Save icon to save your changes.

This image shows a screen capture of the Definition Integration Map report for the CrossSell example Inline Service. The tab "Definition" and the subtab "Integration Map" are selected.

In the upper portion of the screen, three white swim lanes are shown, with names on the left: IVR, CRM, and Fulfilment. One blue swim lane, RTD, is shown in the lower portion of the screen.

The following Integration Points are shown in the blue swim lane:

CallStart

CallInfo

OfferRequest

OfferResponse

CallResolution

This image shows a screen capture of the Choice Group Analysis Drivers report for the Loan Services choice group. The tab "Analysis" and the subtab "Drivers" are selected. Report Settings appear as follows:

Time Window: Since Jan 1, 2004

Offer Acceptance Level: Interested

Channel: All

Show Incomplete Time Window: Selected

Rollup: Selected

Minimum Predictiveness: 0

Maximum Number of Rows: 10

Count: 231

Model Quality: 74

The lower portion of the screen shows a report titled "Predictiveness of Entity Attributes for Loan Services." Attributes shown include:

customer DaysToDueDate

customer AvailableCreditAsPercentOfCreditLine

customer Amount Of Pending Transactions

For each attribute, the Predictiveness values shown.

This image shows a screen capture of the Entity Analysis Counts pie chart for the customer CallReason attribute. Call reasons shown include Balance & payment due questions, Credit limit related questions, and Payment dispute request.

This image shows a screen capture of the Choice Analysis Drivers report for the Credit Protection choice, together with the five report names associated with the top attribute.

The left-hand side of the screen shows the Decision Center menu for the CrossSell (Development) Inline Service, with the entry Credit Protection highlighted under Credit Services, which is under Credit Cards, which is under Offers, which is under Decision Process.

On the right-hand side of the screen, the tab "Analysis" and the subtab "Drivers" are selected. Report Settings appear as follows:

Time Window: Since April 1, 2003

Offer Acceptance Level: Interested

Channel: All

Show Incomplete Time Window: Not selected

Minimum Predictiveness: 5

Number of bins: 3

Maximum Number of Rows: 10

Count: 435

Model Quality: 83

The middle portion of the screen shows a report titled "Predictiveness of Entity Attributes for Credit Protection." Attributes shown include:

customer CreditLineAmountcustomer MaritalStatuscustomer AvailableCreditAsPercentOfCreditLine

A box appears in the middle portion of the screen containing the words "Click an attribute name to generate associated reports", and an arrow from the box points to the lower portion of the screen.

The lower portion of the screen shows the names of the charts associated with the CreditLineAmount attribute :

Most Important customer CreditLineAmount Values for Credit Protection (graph)Most Important customer CreditLineAmount Values for Credit Protection (tabular data)

Relative Distribution of customer CreditLineAmount Values

Pareto of customer CreditLineAmount Values

Pareto of customer CreditLineAmount for the Whole Population

The Decision Process branch contains a Rules Library node (with subnodes for Filtering Rules and Scoring Rules) and an Offers node (with subnodes for Credit Cards and Loans).

This image shows the following text:

If

All of the following

session / customer / CreditLineAmount > 0

session / customer / CreditLineAmount <= 50000

Then 7.0

If

All of the following

session / customer / CreditLineAmount > 50000

session / customer / CreditLineAmount <= 60000

Then 6.0

If

All of the following

session / customer / CreditLineAmount > 60000

session / customer / CreditLineAmount <= 70000

Then 5.0

If

All of the following

session / customer / CreditLineAmount > 70000

session / customer / CreditLineAmount <= 80000

Then

4.0

Otherwise...

The value is: 3

This image shows a screen capture of the Entity Analysis Trends chart for the customer CallReason attribute. Call reasons shown include APR related questions, Balance & payment due questions, and Balance transfer.

This image shows a screen capture of the Choice Definition Overview report for the Gold Card choice. The tab "Definition" and the subtab "Overview" are selected. A description is shown for the choice: "Offer to get a Gold card."

Three additional sections are shown:

Presentation Attributes (with Message attribute shown)

Targeting Attributes (with Profit Margin and Likelihood of Purchase attributes shown)

Scoring Attributes (with Customer Retention and Revenue performance goals shown)

This image shows a screen capture of the Scoring Rule Definition Overview report for the Reduced Interest Rate Retention scoring rule. The tab "Definition" and the subtab "Overview" are selected.

The rule shows columns for Condition and Value. For example, the first row states:

"If

All of the following:

session / customer / CreditLineAmount > 0

session / customer / CreditLineAmount < = 50000

Then 7.25"

This image shows a screen capture of the Decision Definition Priorities report for the Decision "OfferDecision." The tab "Definition" and the subtab "Priorities" are selected.

Columns are shown for Segments and Selection Function. Two segments are shown: Segment to Retain, with selection functions Customer Retention (70%) and Revenue (30%), and otherwise, with selection functions Customer Retention (30%) and Revenue (70%).

This image shows a screen capture of the Entity Analysis Counts report for the Customer entity. The tab "Analysis" and the subtab "Counts" are selected. Report Settings appear as follows:

Time Window: January 2004

Attribute: customer CallReason

Show Incomplete Time Window: Selected

The lower portion of the screen shows a report titled "Counts for values of customer CallReason." Values shown include:

Balance & payment due questions

Electronic payment

Credit line increase requests

This image shows a screen capture of the Choice Analysis Best-Fit report for the Credit Services choice group. The tab "Analysis" and the subtab "Best-fit" are selected. Report Settings appear as follows:

Time Window: Oct 1, 2003 - Dec 31, 2003

Offer Acceptance Level: Interested

Channel: All

Show Incomplete Time Window: Selected

Maximum Number of Rows: 10

Count: 699

Model Quality: 82

The lower portion of the screen shows a report titled "Highest correlating attribute values for Credit Services." Attributes shown include:

customer CreditLineAmount

customer Age

customer MaritalStatus

This image shows a screen capture of the Model Errors report for the Choice Group Credit Services for the positive outcome Interested. The tab "Analysis" and the subtab "Quality" are selected. Report Settings appear as follows:

Time Window: Jul 1, 2003 - Sep 30, 2003

Offer Acceptance Level: Interested

Channel: All

Show Incomplete Time Window: Not selected

Count: 1287

Model Quality: 84

The lower portion of the screen shows a report titled "Model Errors." The model errors shown are Mean Square Error (MSE), Relative MSE, Root Mean Square Error (RMSE), Relative RMSE, Mean Absolute Error (MAE), Relative MAE, Mean Error (ME), Relative ME.

This image shows the formula for the Relative ME. It is calculated as the ME, divided by "p bar", the average actual likelihood, where the average actual likelihood is not zero.

This image shows the following text:

People to sell to

All of the following

customer / Age >= 18

customer / CreditLineAmount >= 8000

This image shows a screen capture of a Choice Group Performance Counts report. The tab "Performance" and the subtab "Counts" are selected.

Report settings appear as follows:

Time Window: Since Jan1, 2010

Real Estate: ConfirmationPage

A green arrow points to an open list of values box. The values visible are All, (missing), ConfirmationPage, CoverageSummary, Payment Processing, and SiteRegistration. The value (missing) is surrounded by a green highlighting box.

The Distribution of PI section shows the PI values of Light_CostSaving, Light_General, and Light_PeaceOfMind.

This image shows the formula for the Mean Error, also known as ME. First the sum is taken, for values from 1 to n, of the exact, sign-significant differences between each actual likelihood and the corresponding predicted likelihood. Then the sum is divided by n.

This image shows a screen capture of the Choice Group Performance Counts report for the Credit Products choice group. The tab "Performance" and the subtab "Counts" are selected.

Report settings appear as follows:

Time Window: Oct 1, 2003 - Dec 31, 2003

Channel: All

Show Incomplete Time Window: Selected

Rollup: Not selected

The Distribution of Credit Products section shows the counts for the Gold Card, Miles Card, and Platinum Card credit products.

This image shows a screen capture of the Performance Goals Definition Overview report. The tab "Definition" and the subtab "Overview" are selected. The Customer Retention section shows the text "Maximize customer retention." The Revenue section shows the text "Maximize revenue from cross sell offers."

This image shows a screen capture of the Definition Interactive Integration Map report for the CrossSell example Inline Service. The tab "Definition" and the subtab "Interactive Integration Map" are selected.

In the upper portion of the screen, three white swim lanes are shown, with names on the left: IVR, CRM, and Fulfilment. One blue swim lane, RTD, is shown in the lower portion of the screen.

The following Integration Points are shown in the blue swim lane:

CallStart

CallInfo

OfferRequest

OfferResponse

CallResolution

The lower portion of the screen shows three screen areas Event, Decisions, and Decision Process Trace.The following italicized command appears in the Event area:

Click on an Integration Point to send an event

This image shows a screen capture of the Choice Group Performance Counts report for the Credit Cards choice group. The tab "Performance" and the subtab "Counts" are selected.

Report settings appear as follows:

Time Window: Jul 1, 2003 - Sep 30, 2003

Channel: All

Show Incomplete Time Window: Selected

Rollup: Selected

The Distribution of Credit Cards section shows the total counts for the Credit Services and Credit Products choice groups.

This image shows a screen capture of the Choice Analysis Best-Fit report for the Platinum Card choice. The tab "Analysis" and the subtab "Best-fit" are selected. Report Settings appear as follows:

Time Window: Oct 1, 2003 - Dec 31, 2003

Offer Acceptance Level: Interested

Channel: All

Show Incomplete Time Window: Selected

Maximum Number of Rows: 10

Count: 227

Model Quality: 38

The lower portion of the screen shows a report titled "Highest correlating attribute values for Platinum Card." Attributes shown include:

customer CreditLineAmount

customer Occupation

customer DayOfWeek

This image shows a screen capture of the Entity Definition Overview report for the Customer entity. The tab "Definition" and the subtab "Overview" are selected. Entity Attributes shown include Age, Amount Of Pending Transactions, and AvailableCreditAsPercentOfCreditLine. For each attribute, values are shown for Type (such as integer, double, and string) and Description (such as "The age of the customer").

This image shows a screen capture of the Decision Definition Overview report for the Decision "OfferDecision." The tab "Definition" and the subtab "Overview" are selected.

The following description is provided: "The OfferDecision decides on the offer to suggest to the customer. Offers that support the customer retention performance metric will be more heavily weighted for the segment of the population that is eligible for retention."

Columns are shown for Choice Group and Description. A single item is shown in the report: "Offers: Offers to customers for cross selling and retention."

This image shows a screen capture of the Choice Group Performance Counts Pareto Chart for the Interested outcome. Counts for Miles Card, Platinum Card, and Gold Card are graphically represented.

This image shows a screen capture of the Informant Definition Overview report for the Callinfo informant. The tab "Definition" and the subtab "Overview" are selected. Informant keys and inputs are shown, each with Name and Description.

This image shows the representation of the predicted likelihoods, as "p asterisk subscript i", followed by "i = 1, ...., n."

This image shows a screen capture of the Choice Group Performance Counts report for the Credit Cards choice group. The tab "Performance" and the subtab "Counts" are selected.

Report settings appear as follows:

Time Window: Jul 1, 2003 - Sep 30, 2003

Channel: All

Show Incomplete Time Window: Selected

Rollup: Not selected

The Distribution of Credit Cards section shows the counts for the Credit Protection, Electronic Payments, Gold Card, Miles Card, and Platinum Card choices.

This image shows a screen capture of an Analysis Drivers pareto graph for the Credit Protection choice. This graph shows the customer AvailableCreditAsPercentOfCreditLineValues attribute for the Credit Protection customer group.

This image shows a screen capture of the Choice Analysis Drivers report for the Credit Protection choice. The tab "Analysis" and the subtab "Drivers" are selected. Report Settings appear as follows:

Time Window: Oct 1, 2003 - Dec 31, 2003

Offer Acceptance Level: Interested

Channel: All

Show Incomplete Time Window: Selected

Minimum Predictiveness: 0

Number of bins: 3

Maximum Number of Rows: 10

Count: 1112

Model Quality: 81

The lower portion of the screen shows a report titled "Predictiveness of Entity Attributes for Credit Protection." Attributes shown include:

customer CreditLineAmountcustomer MaritalStatuscustomer AvailableCreditAsPercentOfCreditLine

This image shows a screen capture of the Cumulative Gains Chart for the Choice Reduced Interest Rate for the positive outcome Interested. The tab "Analysis" and the subtab "Quality" are selected. Report Settings appear as follows:

Time Window: Jul 1, 2003 - Sep 30, 2003

Offer Acceptance Level: Interested

Channel: All

Show Incomplete Time Window: Not selected

Count: 152

Model Quality: 73

The lower portion of the screen shows the Cumulative Gains Chart. The X-axis shows aggregated counts of the base events (Delivered) for the Choice Reduced Interest Rate. The Y-axis shows counts of the positive outcomes for the appropriate amount of records in the X-axis.

There are three graphs on the chart:

Maximal

This graph has two connected lines: first, the straight line connecting the zero point to the point whose coordinate is (total positive outcome counts, total positive outcome counts), then a horizontal line to the top right hand point of the chart.

Cumulative Gains

This graph shows the exact values plotted for the positive outcome counts against the total aggregated base event counts.

Base

This graph is a straight line connecting the zero point to the point whose coordinate is (total base event counts, total positive outcome counts).

This image shows a screen capture of the Choice Performance Counts report for the Miles Card choice. The tab "Performance" and the subtab "Counts" are selected.

Report settings appear as follows:

Time Window: Oct 1, 2003 - Dec 31, 2003

Channel: All

Show Incomplete Time Window: Selected

The Counts section shows the Miles Card credit product.

This image shows the formula for Mean Square Error, also known as MSE. For each value from i to n, the difference beween the actual likelihood and the corresponding predicted likelihood is squared, then divided by n. The n results are then accumulated, which gives the Mean Square Error.

This image shows the formula for the Relative MSE. It is calculated as the MSE, divided by the square of "p bar", the average actual likelihood, where the average actual likelihood is not zero.

This image shows the Inline Service Navigator. A tree view is shown, with the root level labeled CrossSell (Development). Secondary nodes in the tree are shown as follows:

Strategic Objectives (with subnodes Performance Goals and Decisions)

Decision Process (with subnodes Rules Library and Offers)

Integration Points (with subnodes Informants and Advisors)

Entity Catalog (with subnodes Session and Customers

This image shows a screen capture of the Performance Goal Definition Overview report for the Customer Retention performance goal. The tab "Definition" and the subtab "Overview" are selected. Columns are shown for Settings and Values. The following items are shown in the report:

Description: Maximize customer retention

Required: true

Normalization Factor: 1

Optimize For: maximize

This image shows the representation of the actual likelihoods, as "p subscript i", followed by "i = 1, ...., n."

This image shows a screen capture of the Entity Analysis Drivers report for the Customer entity. The tab "Analysis" and the subtab "Drivers" are selected. Report Settings appear as follows:

Time Window: Oct 1, 2003 - Dec 31, 2003

Offer Acceptance Level: Interested

Channel: All

Attribute: customer Amount Of Pending Transactions

Show Incomplete Time Window: Selected

Minimum Predictiveness: 1

The lower portion of the screen shows two reports: one titled "Predictiveness of customer Amount Of Pending Transactions on choices," showing the choices Reduced Interest Rate and Platinum Card, and one titled "Importance of specific values of customer Amount Of Pending Transactions," with items including:

Reduced Interest Rate

Platinum Card

Credit Protection

This image shows a screen capture of the Choice Group Definition Overview report for the Credit Cards choice group. The tab "Definition" and the subtab "Overview" are selected.

Four sections are shown:

Summary of Credit Cards (with Credit Protection and Credit Services choice groups shown)

Presentation Attributes (with Message attribute shown)

Targeting Attributes (with Profit Margin and Likelihood of Purchase attributes shown)

Scoring Attributes (with Revenue performance goal shown)

This image shows a screen capture of the Analysis Drivers Relative Distribution graph for the Credit Protection choice. The graph compares the distribution of the customer AvailableCreditAsPercentOfCreditLineValues autoboot for the Credit Protection customer group and the Overall Population.

This image shows a screen capture of the Choice Group Analysis Drivers report for the Loan Services choice group. The tab "Analysis" and the subtab "Drivers" are selected. Report Settings appear as follows:

Time Window: Since Jan 1, 2004

Offer Acceptance Level: Interested

Channel: All

Show Incomplete Time Window: Selected

Rollup: Not selected

Minimum Predictiveness: 0

Maximum Number of Rows: 10

Maximum Model Quality: 74

Average Model Quality: 37

The lower portion of the screen shows a report titled "Predictiveness of attributes across Loan Services." Attributes shown include:

customer DaysToDueDate

customer AvailableCreditAsPercentOfCreditLine

customer Amount Of Pending Transactions

For each attribute, the values shown are Average Predictiveness, Max Predictiveness, and Loan Services with Max Predictiveness (which shows the choice within the Loan Services choice group that had the Max Predictiveness value).

This image shows a screen capture of the Advisor Definition Overview report for the OfferRequest advisor. The tab "Definition" and the subtab "Overview" are selected. The decisions for the advisor are shown, with Type, Name and Description, Also shown are the keys and inputs for the advisor, each with Name and Description.

This image shows a screen capture of the Most Important Analysis Drivers graph and data for the Credit Protection choice. The graph shows Negative Importance and Positive Importance for the customer AvailableCreditAsPercentOfCreditLineValues attribute. The data includes Actual Count, Expected Count, Difference, Importance, Input Count, Output Count, and Total Count.

This image shows a screen capture of the Choice Definition Eligibility report for the Credit Products choice group. The tab "Definition" and the subtab "Choice Eligibility" are selected. The report shows the following two lines: "Inherited eligibility conditions" with a double down-chevron icon, followed by "Credit Products choices are eligible."

The Strategic Objectives branch contains a Performance Goals node (with subnodes for Customer Retention and Revenue) and a Decisions node (with subnodes for OfferDecision and RandomDecision).

This image shows the formula for the Mean Absolute Error, also known as MAE. First the sum is taken, for values from 1 to n, of the absolute, signless differences between each actual likelihood and the corresponding predicted likelihood. Then the sum is divided by n.

This image shows a screen capture of the Choice Performance History report for the Miles Card choice. The tab "Performance" and the subtab "History" are selected.

At the top of the screen, report settings appear as follows:

Time Window 1: Apr 25, 2010

Time Window 2: May 22, 2010

Offer Acceptance Level: Interested

Channel: All

Graph Options: Show Delivered and Interested

Show Incomplete Time Window: Selected

The Graph of counts for Credit Protection section shows a graphical representation of the Delivered and Interested outcomes.

The lower part of the screen shows the Detailed counts section, with the data shown in table form for the Interested and Delivered outcomes of the Credit Protection choice. Columns are shown for When, Offer Acceptance Level, and Count.

This image shows a screen capture of the Filtering Rule Definition Overview report for the Segment to Retain filtering rule. The tab "Definition" and the subtab "Overview" are selected.

The rule states:

"Segment to Retain is true when

All of the following:

session / customer / CallsAbandoned > = 6

session / customer / Tenure > = 2"

This image shows a screen capture of an Analysis Drivers pareto graph for the Credit Protection choice. This graph shows the customer AvailableCreditAsPercentOfCreditLineValues attribute for the whole population.

This image shows the formula for the Relative RMSE. It is calculated as the RMSE, divided by "p bar", the average actual likelihood, where the average actual likelihood is not zero.

This image shows the Rule Editor toolbar. The functions of the icons shown on the toolbar are described in the surrounding text.

This image shows a screen capture of the Advisor Measure report for the OfferRequest advisor. The tab "Measure" and the subtab "Overview" are selected. Statistics for the advisor are shown, with Event, Count, Control Group Count, and Control Group %.

This image shows a screen capture of the Definition Overview report for the CrossSell example Inline Service. The tab "Definition" and the subtab "Overview" are selected. Screen text appears as follows: “The cross selling example application segments customers to whom we want to offer retention offers from those we will offer cross sell offers."

The Definition section shows columns for Parameter and Value, with nothing to display.

The Goals section shows columns for Goal and Description. The following items appear:

Customer Retention: Maximize customer retention

Revenue: Maximize revenue from cross sell offers

This image shows a screen capture of the Entity Analysis Statistics report for the Customer entity. The tab "Analysis" and the subtab "Statistics" are selected. Report Settings appear as follows:

Time Window: February 2004

Show Incomplete Time Window: Selected

The lower portion of the screen shows a report titled "Statistics on continuous attributes of Customer." Attributes shown include:

customer Age

customer Amount Of Pending Transactions

customer AvailableCreditAsPercentOfCreditLine

This image shows a screen capture of the Performance Trends report for the CrossSell example Inline Service. The tab "Performance" and the subtab "Trends" are selected. Report Settings appear as follows:

Time Window 1: Jul 1, 2003 - Sep 30, 2003

Time Window 2: Since Oct 1, 2003

Show Incomplete Time Window: Not selected

Minimum Count: 15

Minimum Absolute Percent Change: 5

Minimum Number of Rows: 10

The lower portion of the screen shows a report titled "Changes in counts relative to the baseline." Choice Groups and Choices shown include:

Loan Products: Student Loan

Credit Products: Miles Card

Credit Services: Credit Protection

Loan Services: Waive Fees

This image shows a screen capture of the Choice Analysis Trends report for the Credit Protection choice. The tab "Analysis" and the subtab "Trends" are selected. Report Settings appear as follows:

Time Window 1: Oct 1, 2003 - Dec 31, 2003

Time Window 2: Since Jan 1, 2004

Offer Acceptance Level: Interested

Channel: All

Show Incomplete Time Window: Selected

Minimum Predictiveness: 1

Minimum Absolute Percent Change: 1

The lower portion of the screen shows a report titled "Changes in Predictiveness of Entity Attributes for Credit Protection." Attributes shown include:

customer Age

customer Amount Of Pending Transactions

customer CallReason

This image shows the formula for the average actual likelihoods, as "p bar", calculated as the sum of all of the n actual likelihoods divided by n.

This image shows the formula for Root Mean Square Error, also known as RMSE. First, for each value from i to n, the difference between the actual likelihood and the corresponding predicted likelihood is squared, then divided by n. The n results are then accumulated, and the square root of the final total gives the Root Mean Square Error.

This image shows a screen capture of the Model Errors report for the Choice Reduced Interest Rate for the positive outcome Interested. The tab "Analysis" and the subtab "Quality" are selected. Report Settings appear as follows:

Time Window: Jul 1, 2003 - Sep 30, 2003

Offer Acceptance Level: Interested

Channel: All

Show Incomplete Time Window: Not selected

Count: 152

Model Quality: 73

The lower portion of the screen shows a report titled "Model Errors." The model errors shown are Mean Square Error (MSE), Relative MSE, Root Mean Square Error (RMSE), Relative RMSE, Mean Absolute Error (MAE), Relative MAE, Mean Error (ME), Relative ME.

This image shows a screen capture of the Cumulative Gains Chart for the Choice Group Credit Services for the positive outcome Interested. The tab "Analysis" and the subtab "Quality" are selected. Report Settings appear as follows:

Time Window: Jul 1, 2003 - Sep 30, 2003

Offer Acceptance Level: Interested

Channel: All

Show Incomplete Time Window: Not selected

Count: 1287

Model Quality: 84

The lower portion of the screen shows the Cumulative Gains Chart. The X-axis shows aggregated counts of the base events (Delivered) for the Choice Group Credit Services. The Y-axis shows counts of the positive outcomes for the appropriate amount of records in the X-axis.

There are three graphs on the chart:

Maximal

This graph has two connected lines: first, the straight line connecting the zero point to the point whose coordinate is (total positive outcome counts, total positive outcome counts), then a horizontal line to the top right hand point of the chart.

Cumulative Gains

This graph shows the exact values plotted for the positive outcome counts against the total aggregated base event counts.

Base

This graph is a straight line connecting the zero point to the point whose coordinate is (total base event counts, total positive outcome counts).

This image shows a table with two columns: Segments and Selection Function. The table has two rows.

In the Segments column, the first row shows a drop-down list with the option "Segment to Retain" selected. The second row shows "otherwise."

In the Selection Function column, the first row shows "Customer Retention - 71%" and "Revenue - 29%". Graphical slider bars are shown next to both of these items. The second row shows "Customer Retention - 30%" and Revenue - 70%". Graphical slider bars are also shown next to both of these items.

This image shows a screen capture of the Decisions Definition Overview report. The tab "Definition" and the subtab "Overview" are selected. Columns are shown for Decisions, Customer Retention, Revenue, and Description. The Decisions OfferDecision and RandomDecision are shown in the report.

This image shows the formula for the Relative Mean Absolute Error, or Relative MAE. It is calculated as the MAE, divided by "p bar", the average actual likelihood, where the average actual likelihood is not zero.

The Choices branch (Offers) contains a Choice Group node called Credit Cards (with subnodes for Credit Products and Credit Services) and a Choice Group node called Loans (with subnodes for Loan Products and Loan Services).

This image shows a screen capture of the Entity Analysis Trends report for the Customer entity. The tab "Analysis" and the subtab "Trends" are selected. Report Settings appear as follows:

Time Window 1: February 2004

Time Window 2: Since Mar 1, 2004

Attribute: customer CallReason

Show Incomplete Time Window: Selected

Minimum Count: 0

Minimum Absolute Percent Change: 1

The lower portion of the screen shows a report titled "Changes in counts for Customer." It shows customer CallReasons, including Rewards related questions, Grace period request, and Card stolen.

This image shows a screen capture of the Choice Group Analysis Trends report for the Credit Services choice group. The tab "Analysis" and the subtab "Trends" are selected. Report Settings appear as follows:

Time Window 1: Oct 1, 2003 - Dec 31, 2003

Time Window 2: Since Jan 1, 2004

Show Incomplete Time Window: Selected

Minimum Count: 0

Minimum Absolute Percent Change: 0

Maximum Number of Rows: 10

The lower portion of the screen shows a report titled "Changes in choice counts relative to the baseline." Choices (including Credit Protection and Electronic Payments) and Event Names (including Purchased, Delivered, and Interested) are shown, along with Counts and Relative Change.

Oracle Legal Notices

Copyright Notice

Copyright © 1994-2010, Oracle and/or its affiliates. All rights reserved.

License Restrictions Warranty/Consequential Damages Disclaimer

This software and related documentation are provided under a license agreement containing restrictions on use and disclosure and are protected by intellectual property laws. Except as expressly permitted in your license agreement or allowed by law, you may not use, copy, reproduce, translate, broadcast, modify, license, transmit, distribute, exhibit, perform, publish, or display any part, in any form, or by any means. Reverse engineering, disassembly, or decompilation of this software, unless required by law for interoperability, is prohibited.

Warranty Disclaimer

The information contained herein is subject to change without notice and is not warranted to be error-free. If you find any errors, please report them to us in writing.

Restricted Rights Notice

If this software or related documentation is delivered to the U.S. Government or anyone licensing it on behalf of the U.S. Government, the following notice is applicable:

U.S. GOVERNMENT RIGHTS Programs, software, databases, and related documentation and technical data delivered to U.S. Government customers are "commercial computer software" or "commercial technical data" pursuant to the applicable Federal Acquisition Regulation and agency-specific supplemental regulations. As such, the use, duplication, disclosure, modification, and adaptation shall be subject to the restrictions and license terms set forth in the applicable Government contract, and, to the extent applicable by the terms of the Government contract, the additional rights set forth in FAR 52.227-19, Commercial Computer Software License (December 2007). Oracle USA, Inc., 500 Oracle Parkway, Redwood City, CA 94065.

Hazardous Applications Notice

This software is developed for general use in a variety of information management applications. It is not developed or intended for use in any inherently dangerous applications, including applications which may create a risk of personal injury. If you use this software in dangerous applications, then you shall be responsible to take all appropriate fail-safe, backup, redundancy, and other measures to ensure the safe use of this software. Oracle Corporation and its affiliates disclaim any liability for any damages caused by use of this software in dangerous applications.

Trademark Notice

Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners.

Third-Party Content, Products, and Services Disclaimer

This software and documentation may provide access to or information on content, products, and services from third parties. Oracle Corporation and its affiliates are not responsible for and expressly disclaim all warranties of any kind with respect to third-party content, products, and services. Oracle Corporation and its affiliates will not be responsible for any loss, costs, or damages incurred due to your access to or use of third-party content, products, or services.