10 Validating Connectivity and Policies Using the Access Tester

About OAM Agent and Server Interoperability

The two primary types of actors in the OAM architecture are the policy servers (OAM Servers) and OAM policy enforcement agents (WebGates or AccessGates). In the security world, Agents represent the policy enforcement point (PEP), while OAM Servers represent the policy decision point (PDP):

• The Agent plays the role of a gatekeeper to secure resources such as http-based applications and manage all interactions with the user who is trying to access that resource. This is accomplished according to access control policies maintained on the policy server (OAM Server).

• The role of the OAM Server is to provide policy, identity, and session services to the Agent to properly secure application resources, authenticate and authorize users, and manage user sessions.

This core OAM product architecture revolves around the following exchanges, which drive the interaction between the Agent and OAM Server. To expose interoperability and the key decision points, Figure 10-1 illustrates a typical OAM Agent and OAM Server interaction during a user's request for a resource.

Figure 10-1 OAM Agent (PEP) and OAM Server (PDP) Interoperability

The following overview outlines the processing that occurs between OAM Agents and OAM Servers. During testing, the Access Tester emulates the Agent and communicates with the OAM Server while the administrator emulates the end user.

Process overview: Interoperability between OAM Agents and OAM Servers

1. Establish server connectivity: The registered OAM Agent connects to the OAM Server.

2. The user requests accesses to a resource.

3. Validate resource protection: The Agent forwards the request to the OAM Server to determine if the resource is protected.

   Protected: The OAM Server responds with the type of credentials required.

4. User credentials: Establishing the user identity enables tracking for Audit and SSO purposes, and conveyance to the application. For this, the Agent prompts the user for his credentials.

5. Authenticate user credentials: The Agent forwards the supplied user credentials to the OAM Server for validation.

   Authentication Success: The Agent forwards the resource request to the OAM Server.

6. Authorize user access to a resource: The Agents must first determine if the user is allowed to access the resource by forwarding the request for access to the OAM Server for authorization policy evaluation.

7. The Agent grants or denies access based on the policy response.

About Access Tester Security and Processing

The Access Tester supports only Open and Simple connection modes for communication with the OAM Server.

The Access Tester encrypts all password-type values that it saves to configuration files and test cases. All network connectivity inherits the NetPoint Access Protocol (NAP) limit of a single connection pool (one primary or secondary connection pool).

Persistence: The Access Tester manages a number of data structures that require persistent storage between Access Tester invocations. XML-file-based storage is provided for the following types of information:

• Configuration data to minimize data entry between invocations of the application (OamTestConfiguration)

• Test scripts consisting of captured test cases (OamTestScriptCase)

• Statistical data representing execution metric from a test run (OamTestStats)

XML Files for Input, Logging, and Analysis: The following XML files are produced when you run the Access Tester to process test scripts:

• Configuration Script: config.xml is the output file generated using the Save Configuration command within the Access Tester. The name of this document is used within the input script to provide proper connection information to the Access Tester running in command line mode. For details, see "About the Saved Connection Configuration File".

• Input Script: script.xml represents a script that is generated by the Access Tester after capturing one or more test cases. For details, see "About the Generated Input Test Script".

• Target Output Script: oamtest_target.xml is generated by running the Access Tester in command line mode and specifying the input script. For details, see "About the Target Output File Containing Test Run Results". For example: -Dscript.scriptfile="script.xml" -jar oamtest.jar

• Statistics: oamtest_stats.xml is generated together with the output script. For details, see "About the Statistics Document".

• Execution Log: lamtest_log.log is generated together with the output script. For details, see "About the Execution Log".

For more information, see "About Access Tester Modes and Administrator Interactions".

About Access Tester Modes and Administrator Interactions

In Console mode, the Access Tester provides a single window for interactions with the user. All Access Tester operations are available in the main window, which performs as a central dashboard where users can submit specific details for the test case and view responses.

Alternatively, you can use the Access Tester in command line mode and develop test scripts, which you can run interactively or in batches for computerized execution to maximize productivity and minimize costs and resources.

Run-Time: The Access Tester requires nap-api.jar in the same directory as the main jar oamtest.jar. Starting the application requires oamtest.jar.

Regardless of the mode you choose for running the Access Tester, your primary interactions with the Access Tester include:

• Issuing Requests and Reviewing Results

  You use the Access Tester to issue requests to the OAM Server to validate resource protection, policy configuration, user authentication, and user authorization. You can immediately analyze test case results and also retain the data for longer-term analysis, if needed.

• Managing Test Scripts

  You can build test scripts by capturing the data generated by test execution, which is available as stand-alone documents. You can run the test script for manual or automated analysis. The Access Tester provides for some automated analysis after each test run, while collecting full set of statistics to enable analysis after the fact.

• Managing OAM Server Connectivity

  You can manage application settings that include server connection information.

Figure 10-2 depicts the flow of information during operations in both Console and command-line modes. Details follow the figure. Advanced operations include building and executing test scripts.

Figure 10-2 User Interactions with the Access Tester

Table 10-1 describes the process flow of information during both Console mode operations and command-line mode operations.

The following overview outlines the tasks involved with using the Access Tester, and the topics where more information can be found in this chapter.

Task overview: Testing OAM 11g connections and policies includes

1. Review the following topics:

   • Installing and Starting the Access Tester

   • Introduction to the Access Tester Console and Navigation

2. Perform and capture tests using the Access Tester Console as described in "Testing Connectivity and Policies from the Access Tester Console":

3. Proceed to "Creating and Managing Test Cases and Scripts"

Installing the Access Tester

This topic describes how to install the Access Tester for use on any computer. Following installation, the Access Tester is ready to use. No additional setup is required.

To install the Access Tester

1. Ensure that the computer from which the tester will be run includes JDK/JRE 6. For example, you can test for Java as follows:

   java -version
   

   The previous command returns the following information:

   java version "1.6.0_18"
   Java(TM) SE Runtime Environment (build 1.6.0_18-b07)
   Java HotSpot(TM) Client VM (build 16.0-b13, mixed mode)
   

2. On a computer hosting the OAM Server, locate and copy the Access Tester Jar files. For example:

   Oracle_HOME/oam/server/tester/oamtest.jar 
   Oracle_HOME/oam/server/tester/nap-api.jar 
   
   

3. Store the jar file copies together in the same directory on any computer from which you want to run the Access Tester.

4. Proceed as follows, depending on your environment and requirements:

   • Starting the Access Tester Without System Properties For Use in Console Mode enables you to manually drive requests.

   • Starting the Access Tester with System Properties For Use in Command Line Mode

   • Executing a Test Script enables you to use a test script that has been created against a "Known Good" policy configuration and marked as "Known Good"

About Access Tester Supported System Properties

The Access Tester supports a number of configuration options that are used for presentation or during certain aspects of testing. These options are specified at startup using the Java-D mechanism, as shown in Table 10-2, which describes all supported system properties.

Starting the Access Tester Without System Properties For Use in Console Mode

To manually drive (and capture) requests and view real-time response through the graphical user interface, start the tester in Console mode. This procedure omits all system properties, even though several can be used with Console mode.

The jar file defines the class to be started by default; no class name need be specified. Ensure that the nap-api.jar is present in the same directory as oamtest.jar.

To start the Access Tester in console mode without system properties

1. From the directory containing the Access Tester jar files, enter the following command:

   java -jar oamtest.jar
   

2. Proceed to one of the following topics for more information:

   • Introduction to the Access Tester Console and Navigation

   • Testing Connectivity and Policies from the Access Tester Console

   • Creating and Managing Test Cases and Scripts

Starting the Access Tester with System Properties For Use in Command Line Mode

This section is divided into the following topics:

• About the Access Tester Command Line Mode

• Starting the Access Tester Without System Properties For Use in Console Mode

Access Tester Menus and Command Buttons

Table 10-5 identifies additional Access Tester Console buttons and their use. All command buttons provide a tip when the cursor is on the button.

Table 10-5 Additional Access Tester Buttons

Command Buttons	Description
	Loads connection configuration details that were saved to an XML file (config.xml, by default). You can refresh the information in the Console by clicking this button.
	Saves connection configuration details to a file (default name, config.xml). You can add the name of this document to the input script to provide proper connection information to the Access Tester running in command line mode. The Save command button at the bottom of the Console saves the content of the Status Message panel to a log file.
	Clears fields on a panel containing the icon. Tool bar action clears all fields except connection fields if the connection has already been established.
	Captures the last named request to the capture queue with the corresponding response received from the OAM Server. Together, the request and response create a test case. The capture queue status at the bottom of the Console is updated to reflect the number of test cases in the queue. You can save the contents of the capture queue to create a test script containing multiple test cases using the Generate Script command on the Test menu or a command button.
	Generates a test script that includes every test case currently in the capture queue, and asks if the queue should be cleared. Do not clear the queue until all your test cases have been captured and saved to a test script.
	Runs a test script against the current OAM Server. The Status message window is populated with the execution status as the script progresses through each test case.
	Imports a copied URI from the clipboard after parsing it to populate fields in the URI panel.
	Displays a dialog showing the password in clear text

The Access Tester provides the menus described in Table 10-6. All menu items have mnemonics that are exposed by holding down the ALT key (on Windows systems). There are also command accelerators (keyboard activation) available using the CTRL-<KEY> combination defined for each menu command.

Establishing a Connection Between the Access Tester and the OAM Server

Before you can send a request to the OAM Server you must establish a connection between the Access Tester and the server. This section describes how to establish that connectivity.

• About the Connection Panel

• Connecting the Access Tester with the OAM Server

About the Connection Panel

You enter required information for the OAM Server and the Agent you are emulating in the Access Tester Connection panel and then click the Connect button. The Tester initiates the connection, and displays the status in the Status Messages panel. Once the connection is established, it is used for all further operations.

Caution:

Once the connection is established, it cannot be changed until you restart the Access Tester Console.

Figure 10-4 illustrates the Server Connection panel and controls.

Figure 10-4 Server Connection Panel in the Access Tester

Table 10-7 describes the information needed to establish the connection. The source of your values is the OAM Administration Console, System Configuration tab.

Table 10-7 Connection Panel Information

Fields	Description
IP Address	The IP Address of the Primary and Secondary OAM Proxy listens on for this set of tests. Note: Oracle recommends that you enter values for only the Primary OAM Proxy. The Secondary OAM Proxy is needed only if you want to test failover between the primary and secondary OAM Server. However, a more practical use of the Secondary Server is reserved for later use, when the OAP API supports load balancing between Primary and Secondary OAM Server.
Port	Enter the port number of the Primary and Secondary OAM Server.
Max Conn	The maximum number of physical connection (TCP) sockets the Access Tester will use. Access Tester emulates a single threaded Agent. Note: Oracle recommends that you accept the default value, 1.
Min Conn	The minimum number of physical connection (TCP) sockets the Access Tester will use. The Access Tester emulates a single threaded Agent. Note: Oracle recommends that you accept the default value, 1.
Timeout	The number of milliseconds the Access Tester should wait for the connection to be established or to receive a response from the OAM Server. Note: Oracle recommends that you accept the default value.
Mode	The level of communication security that is designated for the Agent to be emulated. Open--No communication security set between the Agent and OAM Server. Simple--The physical connection is encrypted using built-in certificates. Choosing Simple presents a field for the global pass phrase set the OAM Server. Cert--The physical connection is encrypted using customer-supplied certificates. Choosing Cert presents a button that opens a dialog for the certificate information.
Agent ID	Enter the identity of the OAM Agent the Tester is simulating.
Agent Password	Enter the password for the OAM Agent the Tester is simulating, if there is one configured.
	The green check mark beside the Connect button indicates a "Yes" response; the connection is made. The Status Messages panel also indicates a "Yes" response for the connection.
	The red circle beside the Connect button indicates a "No" response; no connection exists. The Status Messages panel also indicates a "No" response for the connection.

After entering information and establishing a connection, you can save details to a configuration file that can be re-used later.

See Also:

"Establishing a Connection Between the Access Tester and the OAM Server"

Validating Resource Protection from the Access Tester Console

Before a user can access a resource, the Agent must first validate that the resource is protected. Using the Access Tester, you can act as the Agent to have the OAM Server validate whether or not the given URI is protected and communicate the response to the Access Tester, as described here.

• About the Protected Resource URI Panel

• Validating Resource Protection

About the Protected Resource URI Panel

You must enter required information for the resource you want to validate in the Access Tester Protected Resource URI panel, and then click the Validate button.

To minimize data entry, you can import long URIs that you have copied from a browser and then click the Import URI command button. The Tester parses the URI saved to the clipboard and populates the URI fields in the Access Tester.

Figure 10-5 illustrates the panel where you enter the URI details to validate that the resource is protected. When combined, the URI fields follow RFC notation. For example: http://oam_server1:7777/index.html.

Figure 10-5 Protected Resource URI Panel in the Access Tester

Table 10-8 describes the information needed to perform this validation.

Table 10-8 Protected Resource URI Panel Fields and Controls

Field or Control	Description
Scheme	Enter http or https, depending on the communication security specified for the resource. Note: The Access Tester supports only http or https resources. You cannot use the Access Tester to test policies that protect custom non-http resources.
Host	Enter a valid host name for the resource. Note: Your <host:port> combination specified in the Access Tester must match one of the Host Identifiers defined in the OAM Administration Console. If the host identifier is not recognized, OAM cannot validate resource protection.
Port	Enter a valid port for the URI. Note: The <host:port> combination specified in the Access Tester must match one of the Host Identifiers as defined in the OAM Server. If the host identifier is not recognized, OAM cannot validate resource protection.
Resource	Enter the Resource component of the URI (/index.htm in the example). This resource should match a resource defined for an authentication and authorization policy in the OAM Administration Console. Note: If protected, the resource identifier that you provide here must match the one specified in an authorization policy in the OAM Administration Console.
	Click this button to parse and import a URI that is saved on a clipboard.
Operation	Select the operational component of the URI from the list provided in the Access Tester. The OAM Server does not distinguish between different actions, however. Therefore, leaving this set to Get should suffice.
Get Auth Scheme	Check this box to request the OAM Server to return details about the Authentication Scheme that is used to secure the protected resource. If the URI is protected, this information is displayed in the Status Messages panel.
Validate	Click the Validate button to submit the request to the OAM Server. When the response is received, the Access Tester displays it in the Status Messages panel.
	A green check mark appearing beside the Validate button indicates a "Yes" response; the resource is protected. The Status Messages panel provides the redirect URL for the resource and that credentials are expected. Note: If you checked the Get Auth Scheme box, the name and level of the Authentication Scheme that protects this resource are also provided in the Status Messages panel.
	A red circle appearing beside the Validate button indicates that the resource is not protected. A No response will also appear in the Status Messages.

You can capture each request and response pair to create a test case, and save multiple test cases to a script file that can be run later.

See Also:

• "Validating Resource Protection from the Access Tester Console"

• "Creating and Managing Test Cases and Scripts"

Testing User Authentication from the Access Tester Console

This topic provides the following information:

• About the User Identity Panel

• Testing User Credential Authentication

About the User Identity Panel

Before a user can access a resource, the Agent must validate the user's identity based on the defined authentication policy on the OAM Server. Using the Access Tester, you can act as the Agent to have the OAM Server authenticate a specific userID for the protected resource. All relevant authentication responses are considered during this policy evaluation.

Figure 10-6 illustrates the Access Tester panel where you enter the information needed to test authentication.

Figure 10-6 Access Tester User Identity Panel

Table 10-9 describes the information you must provide.

Table 10-9 Access Tester User Identity Panel Fields and Controls

Field or Control	Description
IP Address	Enter the IP Address of the user whose credentials are being validated. All Agents communicating with the OAM Server send the IP address of the end user. Default: The IP address that is filled in belongs to the computer from which the Access Tester is run. To test a policy that requires a real user IP address, replace the default IP address with the real IP address.
User Name	Enter the userID of the individual whose credentials are being validated. Note: The Access Tester enables (or disables) the user name and password fields if the resource is protected by an authentication scheme that requires those credentials. Otherwise, this field is disabled.
Password	Enter the password of the individual whose credentials are being validated.
?	Click this button to display the password in clear text within a popup window.
User Certificate Store	The file (in PEM format) containing the X.509 certificate of the user whose credentials should be authenticated. If the URI is protected by the X.509 Authentication Scheme, the Access Tester uses the PEM-formatted X.509 certificate as a credential instead of (or in addition to) the username/password. If the the Authentication Scheme does not require an X.509 certificate, this field is disabled. Note: For certificate-based authentication to work, the OAM Server must be properly configured with root CA certificates and SSL keystore certificates. See Appendix E for details about securing communication between OAM 11g Servers and WebGates.
...	Click this button to browse the file system for the user certificate store path.
Authenticate	Click the Authenticate button to submit the request to the OAM Server and look for a response in the Status Messages panel. Note: The type of credentials supplied (username/password or X.509 certificate) must match the requirements of the authentication scheme that protects the URI.
Authorize	After the user's credentials are validated, you can click the Authorize button to submit the request for the resource to the OAM Server. Check the Status Messages panel for a response.
	A green check mark appearing beside the Authenticate button indicates authentication success; The Status Messages panel also indicates "yes" authentication was successful, and provides the user DN and session id. A green check mark appearing beside the Authorize button indicates authorization success; The Status Messages panel also indicates "yes" authorization was successful, and provides application domain details.
	A red circle appearing beside the Authenticate button indicates authentication failure; The Status Messages panel also indicates "no" authentication was not successful. A red circle appearing beside the Authorize button indicates authorization failure; The Status Messages panel also indicates "no" authorization was not successful.

You can capture each request and response pair to create a test case, and save multiple test cases to a script file that can be run later.

See Also:

• "Testing User Authentication from the Access Tester Console"

• "Creating and Managing Test Cases and Scripts"

Testing User Authorization from the Access Tester Console

Before a user can access a resource, the Agent must validate the user's permissions based on defined policies on the OAM Server. Using the Access Tester, you can act as the Agent to have the OAM Server validate whether or not the authenticated user identity can be authorized to access the resource.

Use the following procedure to verify the authenticated end user's authorization for the resource. All relevant authorization constraints and responses are considered during this policy evaluation.

Prerequisites

Testing User Authentication from the Access Tester Console with all information retained in the Console

To test user authorization

1. In the Access Tester User Identity panel, confirm the user is authenticated (Table 10-9).

2. In the Access Tester User Identity panel, click the Authorization button.

3. Beside the Authorization button, look for the green check mark indicating the user is authorized.

   Not Successful: Confirm the authorization policy using the OAM Administration Console.

4. In the Status Messages panel (or execution log file), verify details about the test run.

5. Capture the request and response to create a test case for use later, as described in "Creating and Managing Test Cases and Scripts".

6. Proceed to:

   • Observing Request Latency

   • Creating and Managing Test Cases and Scripts

   • Evaluating Scripts, Log File, and Statistics

Observing Request Latency

To understand OAM Server performance you must know how well the OAM Server handles requests passed by the Agent. While there are many ways to expose a server's metrics, it is sometimes useful to expose server performance from the standpoint of the Agent. Using the Access Tester, you can do just that as described here.

Prerequisites

"Installing and Starting the Access Tester"

Task overview: Observing request latency includes

1. "Validating Resource Protection"

2. "Testing User Authentication from the Access Tester Console"

3. "Testing User Authorization from the Access Tester Console"

4. Check latency information in the execution logfile as shown here, as well as in other files generated during a test run. For example:

   ...
   [2/3/10 11:03 PM][info] Summary statistics
   [2/3/10 11:03 PM][info] Matched 4 of 4, avg latency 232ms vs 238ms
   [2/3/10 11:03 PM][info] Validate: matched 2 of 2, avg latency 570ms vs 578ms
   [2/3/10 11:03 PM][info] Authenticate: matched 1 of 1, avg latency 187ms vs 187ms
   [2/3/10 11:03 PM][info] Authorize: matched 1 of 1, avg latency 172ms vs 188ms
   ...
   

5. Proceed to:

   • Creating and Managing Test Cases and Scripts

   • Evaluating Scripts, Log File, and Statistics

About Test Cases and Test Scripts

A test case is created from the request sent to, and response data received from, the OAM Server using the Access Tester. Among other data elements, a test case includes request latency and other identifying information that enables analysis and comparison of old and new test cases.

Once captured, the test case can be replayed without new input, and then new results can be compared with old results. If the old results are marked as "known good" then deviations from those results constitute failed test cases.

The test case workflow is illustrated by Figure 10-7.

Figure 10-7 Test Case Workflow

Task overview: Creating and managing a test case

From the Access Tester Console, you can connect to the OAM Server and manually conduct individual tests. You can save the request to the capture queue after a request is sent and the response is received from the OAM Server. You can continue capturing additional test cases before generating a test script and clearing the capture queue. If you exit the Access Tester before saving the capture queue, you are asked if the test cases should be saved to a script before exiting. Oracle recommends that you do not clear the queue until all your test cases have been captured.

Once you have the test script, you can run it from either the Access Tester Console or from the command line.

Capturing Test Cases

You can save each test case to a capture queue after sending the request from the Access Tester to the OAM Server and receiving the response. You can capture as many individual test cases as you need before generating a test script that will automate running the group of test cases. For instance, the following outlines three test cases that must be captured individually:

• A validation request and response

• An authentication request and response

• An authorization request and response

Table 10-10 describes the location of the capture options.

Table 10-10 Access Tester Capture Request Options

Location	Description
Test menu Capture last "..." request	Select this command from the Test menu to add the last request issued and results received to the capture queue (for inclusion in a test script later).
	Select this command button from the tool bar to add the last request issued and results received to the capture queue (for inclusion in a test script later).

If you exit the Access Tester before saving the capture queue, you are asked if the test cases should be saved to a script before exiting. Do not clear the Access Tester capture queue until all your test cases have been captured.

To capture one or more test cases

1. Initiate a request from the Access Tester Console, as described in "Testing Connectivity and Policies from the Access Tester Console".

2. After receiving the response, click the Capture last "..." request command button in the tool bar (or choose it from the Test menu).

3. Confirm the capture in the Status Messages panel and note the Capture Queue test case count at the bottom of the Console, as shown here.

   
   

4. Repeat steps 1, 2, and 3 to capture in the queue each test case that you need for your test script.

5. Proceed to "Generating an Input Test Script".

Generating an Input Test Script

A test script is a collection of individual test cases that were captured using the Access Tester Console. When individual test cases are grouped together, it becomes possible to automate test coverage to validate policy configuration for a specific application or site.

You can create a test script to be used as input to the Access Tester and drive automated processing of multiple test cases. The Generate Script option enables you to create an XML file test script and clear the capture queue. If you exit the Access Tester before saving the capture queue, you are asked if the test cases should be saved to a script before exiting.

About Generating an Input Test Script

You can create a test script to be used as input to the Access Tester and drive automated processing of multiple test cases. Such a script must follow these rules:

• Allows possible replay by a person or system

• Allows possible replay against different policy servers w/o changing the script, to enable sharing of test scripts to drive different Policy Servers

• Allows comparison of test execution results against "Known Good" results

Following are the locations of the Generate Script command.

Table 10-11 Generate Script Command

Location of the Command	Description
Test menu Generate Script	Select Generate Script from the Test menu to initiate creation of the script containing your captured test cases.
	Select the Generate Script command button from the tool bar to initiate creation of the script containing your captured test cases. After you specify or select a name for your script, you are asked if the capture queue should be cleared. Do not clear the capture queue until all your test cases are saved to a script.

Personalizing an Input Test Script

This section describes how to personalize and customize a test script.

• About Customizing a Test Script

• Customizing a Test Script

Executing a Test Script

Once a test script has been created against a "Known Good" policy configuration and marked as "Known Good", it is important to drive the Access Tester using the script rather than specifying each test manually using the Console. This section provides the following topics:

• About Test Script Execution

• Running a Test Script

About Test Script Execution

You can interactively execute tests scripts from within the Access Tester Console, or use automated test runs performed by command scripts. Automated test runs can be scheduled by the operating system or a harness such as Apache JMeter, and executed without manual intervention. Other than lack of human input in command line mode, the two execution modes are identical.

Note:

A script such as .bat (Windows) or .sh (Unix) executes a test script in command line mode. Once a test script is created, it can be executed using either the Run Script menu command or the Access Tester command line.

Table 10-13 describes the commands to execute a test script.

Table 10-13 Run Test Script Commands

Location	Description
Test menu Run Script	Select the Run Script command from the Test menu to begin running a saved test script against the current policy server. The Status message panel is populated with the execution status as the script progresses.
	Select the Run Script command button from the tool bar to begin running a saved test script against the current policy server. The Status message panel is populated with the execution status as the script progresses.
Command line mode	A script such as .bat (Windows) or .sh (Unix) executes a test script in command line mode. Once a test script is created, it can be executed using either the Run Script menu command or the Access Tester command line.

The following overview describes how the Access Tester operates when running a test. Other than lack of human input in command line mode, the two execution modes are identical.

Process overview: Access Tester behavior when running a test script

1. The Access Tester loads the input xml file.

   In command line mode, the Access Tester opens the configuration XML file defined within the input test script's Control element.

2. The Access Tester connects to the primary and secondary OAM Proxy using information in the Server Connection panel of the Console.

   In command line mode, the Access Tester uses information in the Connection element of the configuration XML file.

3. In command line mode, the Access Tester checks the Control elements in the input script XML file to ensure none have been overwritten on the command line (command line values take precedence).

4. For each original test case defined in the script, the Access Tester:

   1. Creates a new target test case.

   2. Sends the original request to the OAM Server and collects the response.

   3. Makes the following comparisons:

      Compares the new response to the original response.

      Compares response codes and marks as "mismatched" any new target test case where response codes differ from the original test case. For instance, if the original Validate returned "Yes", and now returns "No", a mismatch is marked.

      When response codes are identical, and "the ignorecontent" control parameter is "false", the Access Tester compares Content (the name of the Authentication scheme or post authorization actions that are logged after each request). If Content sections differ, the new target test case is marked "mismatched".

   4. Collect new elapsed time and store it in the target use case.

   5. Build a new target test case containing the full state of the last server request and the same unique ID (UUID) as the original test case.

   6. Update the internal statistics table with statistics for the target test case (request type, elapsed time, mismatched, and so on).

5. After completing all the input test cases, the Access Tester:

   1. Displays summary results.

   2. Obtains and combines the testname and testnumber, and generates a name for the "results bundle" (three files whose names start with <testname>_<testnumber>.

      Note:

      Shell scripts can automate generating the bundle by providing testname and testnumber command line parameters.

      Obtain testname from the command line parameter. If not specified in the command line, use the testname element of the input script's Control block.

      Obtain testnumber from the command line parameter. If not specified, testnumber defaults to a 7-character numeric string based on the current local time: 2 character minutes, 2 character seconds, 3 character hundredths.

   3. Generates the "results bundle": three files whose names start with <testname>_<testnumber>:

      The target XML script contains the new test cases: <testname>_<testnumber_results.xml.

      The statistics XML file contains a summary and detailed statistics of the entire test run, plus those test cases marked as "mismatched": <testname>_<testnumber_stats.xml

      The execution log file contains information from the Status Message panel: <testname>_<testnumber_log.log.

   4. In command line mode, the Access Tester exits with the exit code as described in "About the Access Tester Command Line Mode".

About Evaluating Test Results

At the end of a test run a "results bundle" gets generated containing three documents:

• Target script: An XML document containing new test cases

• Execution log: A text file containing the messages displayed during script execution

• Execution statistics: An XML document containing test metrics and a list of mismatched elements

The matching pair of test cases in the original and target scripts shares the test case ID. This ID is represented by a UUID value, which makes it possible to compare individual test cases in the original script with those in the target script. For more information, see "About the Generated Input Test Script".

The statistics document contains the summary and detail statistics, as well as a list of test cases that did not match. The detailed statistics can be used for further analysis or to keep a historical trail of results. The summary statistics are the same statistics displayed at the end of the test run and can be used to quickly assess the state of a test run. The list of mismatched test cases as created in the statistics document contains test case IDs that have triggered mismatch and includes the reason for the mismatch, as seen in Table 10-14.

About the Saved Connection Configuration File

This is the output files that is saved using the Save Configuration command on the File menu; the default file name is config.xml. This connection configuration file includes details that were specified in the Access Tester Console, Server Connection panel.

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<oamtestconfig xmlns="http://xmlns.oracle.com/idm/oam/oamtest/schema" 
version="1.0">
    <connection timeout="30000" minnconn="1" mode="open">
        <agent password="00030d05101b050c42" name="agent1"/>
        <keystore rootstore="" keystore_password="" keystore="" 
global_passphrase=""/>
        <primary>
            <server maxconn="1" port="2100" addr="oam_server1"/>
        </primary>
        <secondary>
            <server maxconn="1" port="0" addr=""/>
        </secondary>
    </connection>
    <uri getauthscheme="true">
        <scheme>http</scheme>
        <host>oam_server1</host>
        <port>7777</port>
        <resource>/index.html</resource>
        <operation>Get</operation>
    </uri>
    <identity>
        <id>admin1</id>
        <password>00030d05101b050c42</password>
        <ipaddr>111.222.3.4</ipaddr>
    </identity>
</oamtestconfig>

About the Generated Input Test Script

The input test script is generated by using the Access Tester and capturing your own test cases. The "configfile" attribute of the "Control" element is updated after creation to specify the connection configuration file to be used in command line mode for establishing a connection to the OAM Server.

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<oamtestscript xmlns="http://xmlns.oracle.com/idm/oam/oamtest/schema" 
version="1.0">
    <history description="Manually generated using agent 'agent1'" 
createdon="2010-02-03T22:28:00.468-05:00" createdby="test_user"/>
    <control numthreads="1" numiterations="1" ignorecontent="false" 
testname="samplerun1" configfile="config.xml"/>
    <cases numcases="4">
        <case uuid="465a4fda-d814-4ab7-b81b-f3f1cd72bbc0">
            <request code="Validate">
                <uri getauthscheme="true">
                    <scheme>http</scheme>
                    <host>oam_server1</host>
                    <port>7777</port>
                    <resource>/index.html</resource>
                    <operation>Get</operation>
                </uri>
            </request>
            <response elapsed="984" code="Yes">
                <comment></comment>
                <status>Major code: 4(ResrcOpProtected) Minor code: 
2(NoCode)</status>
                <content>
                    <line type="auth.scheme.id">LDAPScheme</line>
                    <line type="auth.scheme.level">2</line>
                    <line type="auth.scheme.required.creds">2</line>
                    <line type="auth.scheme.redirect.url">http://dadvmh0172.us.oracle.com:14100/oam/server/</line>
                </content>
            </response>
        </case>
        <case uuid="009b44e3-1a94-4bfc-a0c3-84a38a9e0f2a">
            <request code="Authenticate">
                <uri getauthscheme="true">
                    <scheme>http</scheme>
                    <host>oam_server1</host>
                    <port>7777</port>
                    <resource>/index.html</resource>
                    <operation>Get</operation>
                </uri>
                <identity>
                    <id>weblogic</id>
                    <password>00030d05101b050c42</password>
                    <ipaddr>192.168.1.8</ipaddr>
                </identity>
            </request>
            <response elapsed="187" code="Yes">
                <comment></comment>
                <status>Major code: 10(CredentialsAccepted) Minor code: 
2(NoCode)</status>
                <content>
                    <line type="user.dn">cn=weblogic,dc=us,dc=oracle,dc=com</line>
                </content>
            </response>
        </case>
        <case uuid="84fe9b06-86d1-47df-a399-6311990743c3">
            <request code="Authorize">
                <uri getauthscheme="true">
                    <scheme>http</scheme>
                    <host>oam_server1</host>
                    <port>7777</port>
                    <resource>/index.html</resource>
                    <operation>Get</operation>
                </uri>
                <identity>
                    <id>weblogic</id>
                    <password>00030d05101b050c42</password>
                    <ipaddr>192.168.1.8</ipaddr>
                </identity>
            </request>
            <response elapsed="188" code="Yes">
                <comment></comment>
                <status>Major code: 8(Allow) Minor code: 2(NoCode)</status>
                <content/>
            </response>
        </case>
        <case uuid="61579e47-5532-42c3-bbc7-a00828256bf4">
            <request code="Validate">
                <uri getauthscheme="false">
                    <scheme>http</scheme>
                    <host>oam_server1</host>
                    <port>7777</port>
                    <resource>/index.html</resource>
                    <operation>Get</operation>
                </uri>
            </request>
            <response elapsed="172" code="Yes">
                <comment></comment>
                <status>Major code: 4(ResrcOpProtected) Minor code: 
2(NoCode)</status>
                <content/>
            </response>
        </case>
    </cases>
</oamtestscript>

About the Target Output File Containing Test Run Results

This example was generated by running the Access Tester in command line mode and specifying the script.xml file as input to execute the 4 captured test cases:

Dscript.scriptfile="script.xml" -jar oamtest.jar

Notice the various sections in Example 10-3. As shown in the execution log, this test run found no mismatches, and shows that 4 out of 4 requests matched.

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<oamtestscript xmlns="http://xmlns.oracle.com/idm/oam/oamtest/schema" 
version="1.0">
    <history description="Generated from script 'script.xml' using agent 'agent1'" 
createdon="2010-02-03T23:03:02.171-05:00" createdby="test_user"/>
    <control numthreads="1" numiterations="1" ignorecontent="false" 
testname="oamtest" configfile=""/>
    <cases numcases="4">
        <case uuid="465a4fda-d814-4ab7-b81b-f3f1cd72bbc0">
            <request code="Validate">
                <uri getauthscheme="true">
                    <scheme>http</scheme>
                    <host>oam_server1</host>
                    <port>7777</port>
                    <resource>/index.html</resource>
                    <operation>Get</operation>
                </uri>
            </request>
            <response elapsed="969" code="Yes">
                <comment></comment>
                <status>Major code: 4(ResrcOpProtected) Minor code: 
2(NoCode)</status>
                <content>
                    <line type="auth.scheme.id">LDAPScheme</line>
                    <line type="auth.scheme.level">2</line>
                    <line type="auth.scheme.required.creds">2</line>
                    <line type="auth.scheme.redirect.url">http://dadvmh0172.us.oracle.com:14100/oam/server/
</line>
                </content>
            </response>
        </case>
        <case uuid="009b44e3-1a94-4bfc-a0c3-84a38a9e0f2a">
            <request code="Authenticate">
                <uri getauthscheme="true">
                    <scheme>http</scheme>
                    <host>oam_server1</host>
                    <port>7777</port>
                    <resource>/index.html</resource>
                    <operation>Get</operation>
                </uri>
                <identity>
                    <id>weblogic</id>
                    <password>00030d05101b050c42</password>
                    <ipaddr>111.222.3.4</ipaddr>
                </identity>
            </request>
            <response elapsed="187" code="Yes">
                <comment></comment>
                <status>Major code: 10(CredentialsAccepted) Minor code: 
2(NoCode)</status>
                <content>    
                    <line type="user.dn">cn=weblogic,dc=us,dc=oracle,dc=com</line>
                </content>
            </response>
        </case>
        <case uuid="84fe9b06-86d1-47df-a399-6311990743c3">
            <request code="Authorize">
                <uri getauthscheme="true">
                    <scheme>http</scheme>
                    <host>oam_server1</host>
                    <port>7777</port>
                    <resource>/index.html</resource>
                    <operation>Get</operation>
                </uri>
                <identity>
                    <id>weblogic</id>
                    <password>00030d05101b050c42</password>
                    <ipaddr>111.222.3.4</ipaddr>
                </identity>
            </request>
            <response elapsed="172" code="Yes">
                <comment></comment>
                <status>Major code: 8(Allow) Minor code: 2(NoCode)</status>
                <content/>
            </response>
        </case>
        <case uuid="61579e47-5532-42c3-bbc7-a00828256bf4">
            <request code="Validate">
                <uri getauthscheme="false">
                    <scheme>http</scheme>
                    <host>oam_server1</host>
                    <port>7777</port>
                    <resource>/index.html</resource>
                    <operation>Get</operation>
                </uri>
            </request>
            <response elapsed="171" code="Yes">
                <comment></comment>
                <status>Major code: 4(ResrcOpProtected) Minor code: 
2(NoCode)</status>
                <content/>
            </response>
        </case>
    </cases>
</oamtestscript>

About the Statistics Document

The statistics file (_stats.xml) is generated together with the target output script during the test run identified in the Execution log. The script.xml file was used as input to execute the 4 captured test cases. The test run found no mismatches, and shows that 4 out of 4 requests matched.

A sample statistics document is shown in Example 10-4. The various sections that provide statistics for this run, which you can compare against statistics for an earlier "known good" run.

A sample statistics document is shown here. Notice, 
<oamteststats xmlns="http://xmlns.oracle.com/idm/oam/oamtest/schema" 
version="1.0">
     <history description="Generated from script 'script.xml' using agent 
       'agent1'" createdon="2010-02-03T23:03:02.171-05:00" createdby="test_user"/>
     <summary>
          <total>
              <nummatched>4</nummatched>
              <numtotal>4</numtotal>
              <avgelapsedsource>238</avgelapsedsource
              <avgelapsedtarget>232</avgelapsedtarget>
          </total>
          <validate>
              <nummatched>2</nummatched>
              <numtotal>2</numtotal>
              <avgelapsedsource>578</avgelapsedsource>
              <avgelapsedtarget>570</avgelapsedtarget>
          </validate>
          <authenticate>
              <nummatched>1</nummatched>
              <numtotal>1</numtotal>
              <avgelapsedsource>187</avgelapsedsource>
              <avgelapsedtarget>187</avgelapsedtarget>
          </authenticate>
          <authorize>
              <nummatched>1</nummatched>
              <numtotal>1</numtotal>
              <avgelapsedsource>188</avgelapsedsource>
              <avgelapsedtarget>172</avgelapsedtarget>
          </authorize>
          <summary>
          <detail>
               <source>
                    <validate>
                       <yes>2</yes>
                       <no>0</no>
                       <error>0</error>
                       <mismatch>0</mismatch>
                       <elapsed>1156</elapsed>
                    </validate>
                <authenticate>
                       <yes>1</yes>
                       <no>0</no>
                       <error>0</error>
                       <mismatch>0</mismatch>
                       <elapsed>187</elapsed>
               </authenticate>
               <authorize>
                       <yes>1</yes>
                       <no>0</no>
                       <error>0</error>
                       <mismatch>0</mismatch>
                       <elapsed>188</elapsed>
               </authorize>
          </source>
          <target>
               <validate>
                       <yes>2</yes>
                       <no>0</no>
                       <error>0</error>
                       <mismatch>0</mismatch>
                       <elapsed>1140</elapsed>
               </validate>
          <authenticate>
                       <yes>1</yes>
                       <no>0</no>
                       <error>0</error>
                       <mismatch>0</mismatch>
                       <elapsed>187</elapsed>
          </authenticate>
          <authorize>
                       <yes>1</yes>
                       <no>0</no>
                       <error>0</error>
                       <mismatch>0</mismatch>
                       <elapsed>172</elapsed>
          </authorize>
      <target>
      </detail>
    <mismatch numcases="0"/>
</oamteststats>

About the Execution Log

This sample execution log was generated together with the target output script during a test run using script.xml to execute 4 test cases. The test run found no mismatches, and shows that 4 out of 4 requests matched.

As you review this example, notice the information provided which is the same as the information you see in the Status Messages panel of the Access Tester. Notice the test cases, test name, connection configuration file, agent name, connection status, request validation status, authentication scheme, redirect URL, credentials expected, authentication status and user DN, session ID, authorization status, validation status, and summary statistics. Also notice that the target script and statistics document were generated by this run.

[2/3/10 11:02 PM][info] Setting up to run script 'script.xml'
[2/3/10 11:02 PM][info] Loading test cases and control parameters from script
[2/3/10 11:02 PM][info] Loaded 4 cases
[2/3/10 11:02 PM][info] Control data for this test run:
[2/3/10 11:02 PM][info] Test name : 'samplerun1'
[2/3/10 11:02 PM][info] Configuration file : 'config.xml'
[2/3/10 11:02 PM][info] Ignore content : 'false'
[2/3/10 11:02 PM][info] Loading server configuration from file
[2/3/10 11:02 PM][info] Loaded server configuration
[2/3/10 11:02 PM][info] Connecting to server as agent 'oam_agent1'
[2/3/10 11:03 PM][info][request] Connect : Yes
...
[2/3/10 11:03 PM][info] Test 'samplerun1' will process 4 cases
[2/3/10 11:03 PM][info][request] Validate : Yes
[2/3/10 11:03 PM][info] Authentication scheme : LDAPScheme, level : 2
[2/3/10 11:03 PM][info] Redirect URL : 
http://oam_server1.us.company.com:2100/server/
[2/3/10 11:03 PM][info] Credentials expected: 0x01 (password)
[2/3/10 11:03 PM][info][request] Authenticate : Yes
[2/3/10 11:03 PM][info] User DN : cn=admin1,dc=us,dc=company,dc=com
[2/3/10 11:03 PM][info] Session ID : -1
[2/3/10 11:03 PM][info][request] Authorize : Yes
[2/3/10 11:03 PM][info][request] Validate : Yes
[2/3/10 11:03 PM][info] Summary statistics
[2/3/10 11:03 PM][info] Matched 4 of 4, avg latency 232ms vs 238ms
[2/3/10 11:03 PM][info] Validate: matched 2 of 2, avg latency 570ms vs 578ms
[2/3/10 11:03 PM][info] Authenticate: matched 1 of 1, avg latency 187ms vs 187ms
[2/3/10 11:03 PM][info] Authorize: matched 1 of 1, avg latency 172ms vs 188ms
[2/3/10 11:03 PM][info] Generated target script 'samplerun1_0302171__target.xml'
[2/3/10 11:03 PM][info] Generated statistics log 'samplerun1_0302171__stats.xml'